Chat
Physics and Astronomy · MapleScholar Plus

Looking Where Doctors Look: Gaze-Guided Graph Neural Networks for Chest X-Ray Diagnosis

Medical vision algorithms frequently focused on irrelevant image artifacts to classify lung pathologies; GazeGNN guides graph neural networks by modeling the eye-tracking fixation scanpaths of expert radiologists.

Author
Bin Wang et al.
Published
2024
Journal
2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Last updated
September 2026
Looking Where Doctors Look: Gaze-Guided Graph Neural Networks for Chest X-Ray Diagnosis

Deep learning models excel at classifying chest radiographs, but their black-box nature often conceals dangerous diagnostic shortcuts: algorithms frequently latch onto hospital bed markers, tube artifacts, or rib shadows rather than actual pulmonary lesions.

Radiologists do not scan X-rays uniformly; their eyes jump between anatomically meaningful landmarks in structured clinical scanpaths honed by decades of clinical training.

GazeGNN introduces a graph neural network that explicitly incorporates expert radiologists' eye-tracking gaze data as a topological attention prior. By modeling anatomical relationships as nodes and clinical fixation sequences as edges, the model forces the vision network to evaluate genuine radiographic pathology.

This gaze-anchored graph architecture dramatically boosts diagnostic accuracy for pneumothorax, cardiomegaly, and lung nodules, setting a new standard for interpretable, clinically aligned diagnostic AI in hospital emergency radiology.

Reference

Wang, B., Pan, H., Aboah, A., Zhang, Z., Keles, E., Torigian, D., Turkbey, B., Krupinski, E., Udupa, J., & Bagci, U. (2024). GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classification. 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2183–2192.

Title

GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classification

Abstract

Eye tracking research is important in computer vision because it can help us understand how humans interact with the visual world. Specifically for high-risk applications, such as in medical imaging, eye tracking can help us to comprehend how radiologists and other medical professionals search, analyze, and interpret images for diagnostic and clinical purposes. Hence, the application of eye tracking techniques in disease classification has become increasingly popular in recent years. Contemporary works usually transform gaze information collected by eye tracking devices into visual attention maps (VAMs) to supervise the learning process. However, this is a time-consuming preprocessing step, which stops us from applying eye tracking to radiologists’ daily work. To solve this problem, we propose a novel gaze-guided graph neural network (GNN), GazeGNN, to leverage raw eye-gaze data without being converted into VAMs. In GazeGNN, to directly integrate eye gaze into image classification, we create a unified representation graph that models both images and gaze pattern information. With this benefit, we develop a real-time, real-world, end-to-end disease classification algorithm for the first time in the literature. This achievement demonstrates the practicality and feasibility of integrating real-time eye tracking techniques into the daily work of radiologists. To our best knowledge, GazeGNN is the first work that adopts GNN to integrate image and eye-gaze data. Our experiments on the public chest X-ray dataset show that our proposed method exhibits the best classification performance compared to existing methods. The code is available at https://github.com/ukaukaaaa/GazeGNN.

Cited 20 times · View on doi.org

Continue

Continue Exploring

Ask this paper your own questions, or keep browsing the verified research catalogue.