Advertisement
Digestive Endoscopy| Volume 53, ISSUE 2, P216-223, February 2021

Intelligent detection endoscopic assistant: An artificial intelligence-based system for monitoring blind spots during esophagogastroduodenoscopy in real-time

  • Yan-Dong Li
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
  • Shu-Wen Zhu
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
  • Jiang-Ping Yu
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
  • Rong-Wei Ruan
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
  • Zhao Cui
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
  • Yi-Ting Li
    Affiliations
    Department of Internal Medicine, Seton Hall University School of Health and Medical Sciences, Saint Francis Medical Center, Trenton, NJ, United States
    Search for articles by this author
  • Mei-Chao Lv
    Affiliations
    Hithink RoyalFlush Information Network Co., Ltd, Hangzhou, China
    Search for articles by this author
  • Huo-Gen Wang
    Affiliations
    Hithink RoyalFlush Information Network Co., Ltd, Hangzhou, China
    Search for articles by this author
  • Ming Chen
    Affiliations
    Hithink RoyalFlush Information Network Co., Ltd, Hangzhou, China
    Search for articles by this author
  • Chao-Hui Jin
    Affiliations
    Hithink RoyalFlush Information Network Co., Ltd, Hangzhou, China
    Search for articles by this author
  • Shi Wang
    Correspondence
    Corresponding author.
    Affiliations
    Department of Endoscopy, The Cancer Hospital of the University of Chinese Academy of Sciences (Zhejiang Cancer Hospital), Institute of Basic Medicine and Cancer (IBMC), Chinese Academy of Sciences, Hangzhou, Zhejiang, China
    Search for articles by this author
Published:November 30, 2020DOI:https://doi.org/10.1016/j.dld.2020.11.017

      Abstract

      Background

      Observation of the entire stomach during esophagogastroduodenoscopy (EGD) is important; however, there is a lack of effective evaluation tools.

      Aims

      To develop an artificial intelligence (AI)-assisted EGD system able to automatically monitor blind spots in real-time.

      Methods

      An AI-based system, called the Intelligent Detection Endoscopic Assistant (IDEA), was developed using a deep convolutional neural network (DCNN) and long short-term memory (LSTM). The performance of IDEA for recognition of gastric sites in images and videos was evaluated. Primary outcomes included diagnostic accuracy, sensitivity, and specificity.

      Results

      A total of 170,297 images and 5779 endoscopic videos were collected to develop the system. As the test group, 3100 EGD images were acquired to evaluate the performance of DCNN in recognition of gastric sites in images. The sensitivity, specificity, and accuracy of DCNN were determined as 97.18%,99.91%, and 99.83%, respectively. To assess the performance of IDEA in recognition of gastric sites in EGD videos, 129 videos were used as the test group. The sensitivity, specificity, and accuracy of IDEA were 96.29%,93.32%, and 95.30%, respectively.

      Conclusions

      IDEA achieved high accuracy for recognition of gastric sites in real-time. The system can be applied as a powerful assistant tool for monitoring blind spots during EGD.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Digestive and Liver Disease
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Teh JL
        • Tan JR
        • Lau LJ
        • et al.
        Longer examination time improves detection of gastric cancer during diagnostic upper gastrointestinal endoscopy.
        Clin Gastroenterol Hepatol. 2015; 13: 480-487
        • Rutter MD
        • Rees CJ.
        Quality in gastrointestinal endoscopy.
        Endoscopy. 2014; 46: 526-528
        • Hosokawa O
        • Tsuda S
        • Kidani E
        • et al.
        Diagnosis of gastric cancer up to three years after negative upper gastrointestinal endoscopy.
        Endoscopy. 1998; 30: 669-674
        • Bisschops R
        • Areia M
        • Coron E
        • et al.
        Performance measures for upper gastrointestinal endoscopy: a European society of gastrointestinal endoscopy (ESGE) quality improvement initiative.
        Endoscopy. 2016; 48: 843-864
        • Yao K.
        The endoscopic diagnosis of early gastric cancer.
        Ann Gastroenterol. 2013; 26: 11-22
        • O'Mahony S
        • Naylor G
        • Axon A
        Quality assurance in gastrointestinal endoscopy.
        Endoscopy. 2000; 32: 483-488
        • Esteva A
        • Kuprel B
        • Novoa RA
        • et al.
        Dermatologist-level classification of skin cancer with deep neural networks.
        Nature. 2017; 542: 115-118
        • Hirasawa T
        • Aoyama K
        • Tanimoto T
        • et al.
        Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images.
        Gastric Cancer. 2018; 21: 653-660
        • Byrne MF
        • Chapados N
        • Soudan F
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. 2019; 68: 94-100
        • Hochreiter S
        • Schmidhuber J
        Long short-term memory.
        Neural Comput. 1997; 9: 1735-1780
        • Moon HS.
        Improving the endoscopic detection rate in patients with early gastric cancer.
        Clin Endosc. 2015; 48: 291-296
        • Yao K
        • Uedo N
        • Muto M
        • Ishikawa H
        Development of an e-learning system for teaching endoscopists how to diagnose early gastric cancer: basic principles for improving early detection.
        Gastric Cancer. 2017; 20: 28-38
        • Tang SJ
        • Raju G
        Endoscopic photography and image documentation.
        Gastrointest Endosc. 2015; 82: 925-931
        • Di L
        • Fu KI
        • Xie R
        • et al.
        A modified endoscopic submucosal dissection for a superficial hypopharyngeal cancer: a case report and technical discussion.
        BMC Cancer. 2017; 17: 712
        • Kuwabara T
        • Hiyama T
        • Oka S
        • et al.
        Clinical features of pharyngeal intraepithelial neoplasias and outcomes of treatment by endoscopic submucosal dissection.
        Gastrointest Endosc. 2012; 76: 1095-1103
        • Okada K
        • Tsuchida T
        • Ishiyama A
        • et al.
        Endoscopic mucosal resection and endoscopic submucosal dissection for en bloc resection of superficial pharyngeal carcinomas.
        Endoscopy. 2012; 44: 556-564
        • Muto M
        • Satake H
        • Yano T
        • et al.
        Long-term outcome of transoral organ-preserving pharyngeal endoscopic resection for superficial pharyngeal cancer.
        Gastrointest Endosc. 2011; 74: 477-484
        • Hosokawa O
        • Hattori M
        • Douden K
        • et al.
        Difference in accuracy between gastroscopy and colonoscopy for detection of cancer.
        Hepatogastroenterology. 2007; 54: 442-444
        • Yoon HJ
        • Kim S
        • Kim JH
        • et al.
        A lesion-based convolutional neural network improves endoscopic detection and depth prediction of early gastric cancer.
        J Clin Med. 2019; 8: 1310
        • Zhu Y
        • Wang QC
        • Xu MD
        • et al.
        Application of convolutional neural network in the diagnosis of the invasion depth of gastric cancer based on conventional endoscopy.
        Gastrointest Endosc. 2019; 89: 806-815
        • Horiuchi Y
        • Aoyama K
        • Tokai Y
        • et al.
        Convolutional neural network for differentiating gastric cancer from gastritis using magnified endoscopy with narrow band imaging.
        Dig Dis Sci. 2020; 65: 1355-1363
        • Li L
        • Chen Y
        • Shen Z
        • et al.
        Convolutional neural network for the diagnosis of early gastric cancer based on magnifying narrow band imaging.
        Gastric Cancer. 2020; 23: 126-132
        • Ohmori M
        • Ishihara R
        • Aoyama K
        • et al.
        Endoscopic detection and differentiation of esophageal lesions using a deep neural network.
        Gastrointest Endosc. 2019; 91: 301-309
        • Cai SL
        • Li B
        • Tan WM
        • et al.
        Using a deep learning system in endoscopy for screening of early esophageal squamous cell carcinoma (with video).
        Gastrointest Endosc. 2019; 90: 745-753
        • Owais M
        • Arsalan M
        • Choi J
        • Mahmood T
        • Park KR
        Artificial intelligence-based classification of multiple gastrointestinal diseases using endoscopy videos for clinical diagnosis.
        J Clin Med. 2019; 8: 986
        • Wu L
        • Zhou W
        • Wan X
        • et al.
        A deep neural network improves endoscopic detection of early gastric cancer without blind spots.
        Endoscopy. 2019; 51: 522-531
        • Chen D
        • Wu L
        • Li Y
        • et al.
        Comparing blind spots of unsedated ultrafine, sedated, and unsedated conventional gastroscopy with and without artificial intelligence: a prospective, single-blind, 3-parallel-group, randomized, single-center trial.
        Gastrointest Endosc. 2020; 91: 332-339
        • Wu L
        • Zhang J
        • Zhou W
        • et al.
        Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy.
        Gut. 2019; 68: 2161-2169
        • Chang P
        • Grinband J
        • Weinberg BD
        • et al.
        Deep-learning convolutional neural networks accurately classify genetic mutations in gliomas.
        AJNR Am J Neuroradiol. 2018; 39: 1201-1207