Proposal Description

The Total Machine Learning Error (TMLE) framework, introduced by Puts, Salgado and Daas (2025), provides a conceptual basis for understanding and assessing the quality of machine learning applications in official statistics. The framework starts with the idea that ML-based statistical output cannot be reduced to predictive accuracy alone. Instead, quality depends on the entire process through which units are selected, data are collected or observed, features are constructed, models are chosen, trained and evaluated, and predictions are ultimately used in statistical production.
This broader perspective is especially relevant to official statistics, where machine learning is increasingly used in primary production. In such settings, an ML model is not merely a technical instrument, but part of a measurement process. Errors can therefore arise at several stages, such as coverage problems, selection mechanisms and measurement errors in input data. A high-performing model on a test set may still lead to biased or unreliable statistical outputs if these other error sources are ignored.
In practice, however, the TMLE framework can still appear abstract. This session therefore focuses on how TMLE can be translated into concrete quality considerations for statistical institutes. The aim is not to present TMLE as a theoretical model. Instead, it is to show how it can help practitioners identify, structure and assess quality risks when machine learning is introduced into statistical production.
The approach will be illustrated using several use cases in which machine learning is applied to the primary statistical process. These examples show how TMLE can distinguish between technical model performance and statistical quality. In addition, we include a use case in which the model is used in a more predictive fashion. This highlights how the interpretation of quality risks changes when the model's purpose shifts from measurement to prediction.
By the end of the session, participants will have a clearer understanding of how the TMLE framework can be used to move from abstract methodological principles to practical quality assessment in real machine learning applications.