Beyond Accuracy: A Quality Framework for AI/ML in Official Statistics
Conference
66th ISI World Statistics Congress
Format: WSC 2027: IPS Abstract
Keywords: machine learning, quality assurance, quality frameworks;
Abstract
The increasing use of AI/ML in official statistics calls for clear and practical quality standards that ensure trustworthy and fit-for-purpose outputs. This talk proposes a model-centric quality framework aligned with the European Statistics Code of Practice, built around four key dimensions: accuracy & robustness, explainability, reproducibility, and effectiveness. These dimensions extend the traditional quality framework for statistical production to explicitly account for AI/ML-based methods and serve as a bridge between established quality standards in official statistics and the methodological perspective of the Total Machine Learning Error framework (Puts et al., 2025), translating machine learning error sources into assessable quality requirements.
To bridge the gap between conceptual principles and practical implementation, the framework introduces a structured set of quality guidelines functioning as a checklist, translating each dimension into concrete, verifiable actions across the AI/ML lifecycle.
The proposed framework offers statistical organizations a practical tool for integrating AI/ML into production processes while maintaining established quality standards.