This project aims to create a machine learning challenge of predicting pneumonia from chest X-ray images.
The original dataset RSNA Pneumonia requires locating the lung opacity region in chest X-ray.
We adapt this dataset for a binary classification of Normal vs Lung Opacity.
We train a classifier, then test its performance on locating the anomaly region.
We study 2 protocols to assess systematically the attention heatmaps generated by GradCAM.
The 1st protocol assesses the trustworthiness of the attention heatmap with proportion of activation inside bounding box (following GradCAM).
Modified from https://jrzech.medium.com/what-are-radiological-deep-learning-models-actually-learning-f97a546c5b98
The 2nd protocol measures trustworthiness with Intersection Over Union of annotated and predicted bounding box.
The predicted bounding box is defined as the smallest rectangle holding at leats 95% of activation (following ProtoPNet).
Left: target bounding box in red. Middle: heatmap. Right: rectangle holding at least 95% of activation in orange. IOU = 0.65
The Intersection Over Union protocol appears to work better then proportion of activation inside box.
Nevertheless, this protocol still has some flaws.
The major problem to address is how to convert attention heatmap to predicted bounding box.
Left: target bounding box in red. Middle: heatmap. Right: rectangle holding at least 95% of activation in orange. IOU = 0.31
Failure with multiple non-contiguous regions
Code and experiment details:
https://www.kaggle.com/phananhvu/hygieia-evaluation/
Instructors: Isabelle Guyon, Kim Gerdes
Université Paris Saclay 2021
Hygieia cropped image: https://competitions.codalab.org/competitions/29560
Hygieia tabular: https://competitions.codalab.org/competitions/28158
References:
GradCAM: https://arxiv.org/abs/1610.02391
ProtoPNet: https://arxiv.org/abs/1806.10574
RSNA Pneumonia Data: https://www.rsna.org/education/ai-resources-and-training/ai-image-challenge/rsna-pneumonia-detection-challenge-2018