As machine learning systems increasingly influence decisions with significant real-world consequences, there is growing demand for understanding how these systems arrive at their outputs. Explanations are often assumed to promote trust by enabling auditing and oversight, while also offering guidance for modifying input variables to achieve preferred outcomes. They are further expected to expose latent biases, identify preprocessing errors, and distinguish meaningful patterns from spurious correlations. In regulatory, ethical, and operational contexts, explainability is frequently positioned as a foundational requirement for responsible deployment of algorithmic systems.
Despite these expectations, current explainability techniques exhibit substantial limitations. Rather than delivering transparency, many methods introduce additional uncertainty, risk, and ambiguity. As a result, some researchers in the field of explainable artificial intelligence (XAI) have expressed serious reservations about their use, with a minority calling for their abandonment altogether. The gap between the promises of explainability—such as fairness, safety, and transparency—and its actual capabilities remains wide.
This paper argues that existing explainability approaches fail to meet the standards required of empirical scientific tools and are not yet suitable for supporting high-stakes decision-making. It identifies foundational challenges that must be addressed before such methods can be reliably applied, organizing these challenges into three interrelated dimensions: meaning, interpretation, and actionability. Until these conceptual and methodological issues are resolved, the paper advises practitioners to refrain from relying on explainability techniques in contexts where decisions carry significant consequences.
Citation
TY - JOURAU - Gervais, NicolasPY - 2026/01/14SP - T1 - Explainable AI: An Idea that Badly Needs GroundworkVL - ER -
Link to full paper:
https://www.researchgate.net/publication/399763923_Explainable_AI_An_Idea_that_Badly_Needs_Groundwork
