Table 12 Discourse on AI explainability and scientific discovery

IN FAVOUR of the necessity of explainability of cAI

“correlations uncovered by XAI might turn out to be real but previously unknown biomedical relationships, in which case XAI could be used as a tool for scientific discovery” [28]

“it has the potential to discover correlations that a human observer is totally ignorant of” [53]

“mechanistic explanations can lead to false conclusions, and mechanistic reasoning alone has been shown to have a high degree of fallibility. At times empirical results can be entirely contrary to mechanistic expectations, as in the case of prophylactic antiarrhythmic drugs actually acting to increase mortality from arrhythmia after recurrent acute myocardial infarction” [8]

“Interpretability may thus feed a misguided expectation that understanding a set of associations valuable for specific diagnostic or prediction tasks will increase our ability to perform additional tasks to which those associations are not well suited and for which their accuracy has not been validated.[…]

The long medical preference for radical mastectomy over less aggressive alternatives was driven by the pathophysiological theory that removing as much tissue from the breast as possible would reduce the probability of cancer recurrence. Only after a series of clinical trials was this theory shown to be false” [10]

OPPOSED to the necessity of explainability of cAI

“Ultimately, the primary goals of medicine are pragmatic: to relieve suffering and promote health. The elucidation of mechanisms comes secondary to this goal” [8]