Read on Twitter
New in @Gut_BMJ from @MayoClinicGIHep and others: AI to enhance the diagnosis of Autoimmune Pancreatitis(AIP). /thread #AI #GItwitter #TensorFlow #keras" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="🔥" title="Feuer" aria-label="Emoji: Feuer">New in @Gut_BMJ from @MayoClinicGIHep and others: AI to enhance the diagnosis of Autoimmune Pancreatitis(AIP). /thread #AI #GItwitter #TensorFlow #keras" class="img-responsive" style="max-width:100%;"/>
1. Diffusely enlarged pancreas with hypo-attenuated rim -> only in 40% patients
2. IgG4 >2x ULN -> only 53% sensitive
3. EUS FNA/B sensitivity only 58.2% in recent study with as low as 7.9% in other previous studies. 2/n
1. Development of CNN (Convoluted neural network) that differentiated AIP from Pancreatic duct adenocarcinoma(PDAC)
2. AIP vs other benign conditions [chronic pancreatitis (CP) and normal pancreas (NP)] 3/n
All patients with HISORt- verified AIP who had undergone EUS since the introduction of EUS at @MayoClinicGIHep in 1996.
1. 50% from PDAC cohort
2. 50% to benign disorders (50% AIP, 25% CP, 25% NP) 4/n
1. All available physician captured still (PCS) image and video (all frames) were identified and extracted.
2. minimum resolution (448×448 pixels) chosen
3. potentially confounding image features and patient identifying information removed. 5/n
ResNet50V2 CNN implemented in Keras and TensorFlow 2.0 using ImageNet weights was created with a randomly initialised 4-class probability output layer.
The model was trained in two stages. 6/n
CNN model selection, training and validation.ResNet50V2 CNN implemented in Keras and TensorFlow 2.0 using ImageNet weights was created with a randomly initialised 4-class probability output layer.The model was trained in two stages. 6/n" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="📢" title="Lautsprecheranlage" aria-label="Emoji: Lautsprecheranlage">CNN model selection, training and validation.ResNet50V2 CNN implemented in Keras and TensorFlow 2.0 using ImageNet weights was created with a randomly initialised 4-class probability output layer.The model was trained in two stages. 6/n" class="img-responsive" style="max-width:100%;"/>
Representative images from each diagnostic classification were identified by selecting those with the highest predicted class probabilities and occlusion heat maps were produced for these. 7/n
Occlusion heat-map analysisRepresentative images from each diagnostic classification were identified by selecting those with the highest predicted class probabilities and occlusion heat maps were produced for these. 7/n" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="📢" title="Lautsprecheranlage" aria-label="Emoji: Lautsprecheranlage">Occlusion heat-map analysisRepresentative images from each diagnostic classification were identified by selecting those with the highest predicted class probabilities and occlusion heat maps were produced for these. 7/n" class="img-responsive" style="max-width:100%;"/>
When provided full video assets, the CNN model was 90% sensitive and 93% specific for differentiating AIP from PDAC and was 90% sensitive and 85% specific for differentiating AIP from all conditions that were studied. 8/n
The results were impressive! When provided full video assets, the CNN model was 90% sensitive and 93% specific for differentiating AIP from PDAC and was 90% sensitive and 85% specific for differentiating AIP from all conditions that were studied. 8/n" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="📢" title="Lautsprecheranlage" aria-label="Emoji: Lautsprecheranlage">The results were impressive! When provided full video assets, the CNN model was 90% sensitive and 93% specific for differentiating AIP from PDAC and was 90% sensitive and 85% specific for differentiating AIP from all conditions that were studied. 8/n">
The results were impressive! When provided full video assets, the CNN model was 90% sensitive and 93% specific for differentiating AIP from PDAC and was 90% sensitive and 85% specific for differentiating AIP from all conditions that were studied. 8/n" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="📢" title="Lautsprecheranlage" aria-label="Emoji: Lautsprecheranlage">The results were impressive! When provided full video assets, the CNN model was 90% sensitive and 93% specific for differentiating AIP from PDAC and was 90% sensitive and 85% specific for differentiating AIP from all conditions that were studied. 8/n">
Correctly identifying any of the four diagnoses, the diagnostic accuracy of the CNN was significantly higher than endosonographers (75.6% vs 61.6%, p=0.026). 9/n
Comparison of CNN model to human endosonographers. Correctly identifying any of the four diagnoses, the diagnostic accuracy of the CNN was significantly higher than endosonographers (75.6% vs 61.6%, p=0.026). 9/n" title="https://abs.twimg.com/emoji/v2/... draggable="false" alt="📢" title="Lautsprecheranlage" aria-label="Emoji: Lautsprecheranlage">Comparison of CNN model to human endosonographers. Correctly identifying any of the four diagnoses, the diagnostic accuracy of the CNN was significantly higher than endosonographers (75.6% vs 61.6%, p=0.026). 9/n" class="img-responsive" style="max-width:100%;"/>
Tested on: AMD Ryzen 3600 CPU, an Nvidia GeForce RTX 2080 Ti GPU, and 48GB of RAM.
Minimum speed required for real-time processing on EUS processors? 30 frames per second
Speed bench-marked? 955 frames per second! 10/n
1. single center retrospective data over two decades.
2. model was trained on images with meta-data and EUS overlays removed.
3. not tested for real-time analysis and prediction 11/n
1. Large dataset of PDAC, AIP, CP and NP patients used to build EUS-CNN model.
2. Highly accurate for differentiating AIP from PDAC and other benign conditions
3. Current criteria centred on imaging and serologic
analyses identify only 70% of patients with AIP. 12/n
link (paywalled): https://gut.bmj.com/content/early/2020/10/07/gutjnl-2020-322821">https://gut.bmj.com/content/e... @Gut_BMJ 13/n
Read more about deep neural networks:
1. He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks: 2016 computer-vision deep-learning microsoft, 2016. https://arxiv.org/pdf/1512.03385.pdf
2.">https://arxiv.org/pdf/1512.... Introduction to ResNets https://towardsdatascience.com/introduction-to-resnets-c0a830a288a4">https://towardsdatascience.com/introduct... 14/n
1. He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks: 2016 computer-vision deep-learning microsoft, 2016. https://arxiv.org/pdf/1512.03385.pdf
2.">https://arxiv.org/pdf/1512.... Introduction to ResNets https://towardsdatascience.com/introduction-to-resnets-c0a830a288a4">https://towardsdatascience.com/introduct... 14/n
