Table 2 Gaps and potential solutions for XAI applications in disease prediction

Model Scope

Limited datasets affecting model diversity and bias

Partner globally for diverse datasets; use synthetic data to mitigate bias

Data imbalance skewing predictive outcomes

Employ SMOTE/ADASYN techniques for balanced datasets

Inadequate development of explainable, transparent models

Adopt XAI frameworks, conduct audits, and provide training for healthcare providers

Modeling Approach

Struggle to balance complex models with user interpretability

Use SHAP and LIME for interpretability

Dependency on single data types of limits prediction scope

Support interdisciplinary innovation for data integration

Narrow performance metrics focus, overlooking comprehensive assessment

Tailor metrics to clinical outcomes and provider needs

Technology

“Black box” models obscure operational understanding

Build transparent XAI models

Single-data modality fails to offer a complete diagnostic picture

Create simulation tools for single-modal data insights

AI interpretability not aligned with clinical reasoning

Use AI coaching to enhance clinical reasoning

Implementation

AI interfaces lack accessibility for medical staff

Design user-centered AI interfaces with customizable options

Complex AI tools challenge clinical workflow integration

Create modular AI tools for seamless workflow integration and training