Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts

Summary

This paper extends causal attribution frameworks to high-dimensional medical image segmentation, quantifying how acquisition protocols and annotation variability independently contribute to deep learning model performance degradation under distribution shifts. By modeling the data-generating process with a causal graph and employing Shapley values, the authors reveal that annotation shifts are the dominant cause of performance drops when models cross annotators, while acquisition shifts dominate when crossing imaging centers. This mechanism-specific quantification allows for targeted intervention strategies to improve model robustness.

Medical Relevance

This research is crucial for deploying reliable and robust AI in clinical settings by providing a principled way to understand *why* medical imaging models fail under real-world variability. It enables healthcare practitioners and AI developers to prioritize effective interventions, ultimately leading to safer and more trustworthy diagnostic and segmentation tools.

AI Health Application

This research aims to improve the robustness, interpretability, and reliability of deep learning models used for medical image segmentation. Specifically, it attributes performance gaps in medical AI models (e.g., for MS lesion segmentation) to causal factors like acquisition protocols and annotation variability. This understanding allows for targeted interventions to make medical AI models more dependable for diagnosis, monitoring, and treatment planning in clinical settings, thereby directly impacting patient care and healthcare efficiency.

Key Points

• Deep learning models for medical image segmentation suffer significant performance drops due to distribution shifts, with poorly understood causal mechanisms.
• The authors propose extending causal attribution frameworks to high-dimensional segmentation tasks to quantify independent contributions of acquisition protocols and annotation variability.
• A causal graph models the data-generating process, and Shapley values are employed to fairly attribute performance changes to individual causal mechanisms.
• The framework addresses specific challenges in medical imaging: high-dimensional outputs, limited sample sizes, and complex interactions between mechanisms.
• Validation on Multiple Sclerosis (MS) lesion segmentation across 4 centers and 7 annotators demonstrated context-dependent failure modes.
• Annotation protocol shifts were found to be the dominant factor contributing to performance degradation (7.4% ± 8.9% DSC attribution) when models were deployed across different annotators.
• Acquisition shifts were identified as the dominant factor (6.5% ± 9.1% DSC attribution) when models were deployed across different imaging centers, highlighting the need for context-specific interventions.

Clinical Impact

The clinical impact is significant as it provides a data-driven approach for identifying and addressing the root causes of AI model failures in medical imaging. Clinicians and developers can use this mechanism-specific quantification to prioritize interventions, such as investing in better annotation standardization training when annotator variability is the main issue, or focusing on image harmonization techniques when acquisition shifts dominate. This leads to more efficient resource allocation and the development of more robust, generalizable, and clinically reliable AI systems for tasks like disease segmentation and diagnosis.

Limitations

Not explicitly mentioned in the abstract.

Future Directions

The abstract implies future work in applying this framework to guide targeted interventions for improving model robustness, by enabling practitioners to prioritize efforts based on the identified dominant causal factors in specific deployment contexts.

Medical Domains

Keywords

Abstract

Deep learning models for medical image segmentation suffer significant performance drops due to distribution shifts, but the causal mechanisms behind these drops remain poorly understood. We extend causal attribution frameworks to high-dimensional segmentation tasks, quantifying how acquisition protocols and annotation variability independently contribute to performance degradation. We model the data-generating process through a causal graph and employ Shapley values to fairly attribute performance changes to individual mechanisms. Our framework addresses unique challenges in medical imaging: high-dimensional outputs, limited samples, and complex mechanism interactions. Validation on multiple sclerosis (MS) lesion segmentation across 4 centers and 7 annotators reveals context-dependent failure modes: annotation protocol shifts dominate when crossing annotators (7.4% $\pm$ 8.9% DSC attribution), while acquisition shifts dominate when crossing imaging centers (6.5% $\pm$ 9.1%). This mechanism-specific quantification enables practitioners to prioritize targeted interventions based on deployment context.

Comments

Medical Imaging meets EurIPS Workshop: MedEurIPS 2025