Abstract

The quality of statistical data depends fundamentally on effective fieldwork supervision. However, methodological and logistical issues that arise during data collection are often recorded through informal channels, such as messaging applications. These communications constitute unstructured text data that is difficult to process manually, this results in delays in supervisory intervention, exacerbate non-sampling errors, and threaten the integrity of the final survey results. Therefore, there is an urgent need for real-time tools to support supervisory decision-making.
To enable real-time quality monitoring, this study proposes an innovative methodology based on the analytical power of Artificial Intelligence ) AI(models, It utilizes leveraging Large Language Models )LLMs (and Natural Language Processing )NLP(using tools like Gemini to systematically process unstructured field communication texts via platforms WhatsApp, transforming them into actionable data within the context of the International Migration Survey in Egypt.
The immediate goal of this processing is to instantly and automatically convert these texts into accurately classified structured data, key classification categories include sample challenges, methodological ambiguity in the questionnaire, and routine logistical problems, this precise classification provides the foundation necessary for transitioning from identifying problems to implementing solutions.
Based on the above, this paper will discuss a system that generates real-time quality reports identifying "hotspots" of problems in the statistical field, with application to international migration surveys, this output serves as the primary tool for ensuring the efficient allocation of supervisory support and resources to maximize effectiveness, critical mechanisms are activated, including the issuance of immediate corrective memos to address methodological measurement errors.