66th ISI World Statistics Congress

66th ISI World Statistics Congress

Generalising Earth Observation AI/ML pipelines for European statistics

Conference

66th ISI World Statistics Congress

Format: WSC 2027: IPS Abstract

Keywords: artificial-intelligence, crop-type, earth-observation, land-cover, machine-learning, quality

Abstract

Earth Observation (EO) data combined with AI/ML offers opportunities to produce statistics more efficiently and at greater detail. However, many EO-based models are developed for specific countries, datasets or periods, limiting their reuse by other statistical institutes.

We investigated how existing AI/ML pipelines can be transferred across countries (space) and time periods (time), and what is needed to enable this. The ambition is a “build once, run everywhere” approach, allowing statistical institutes to reuse proven solutions, reduce duplicated development efforts and improve efficiency.

Land cover
IGN developed several land-cover models using different configurations. These models were evaluated using different quality assessment methods, including photo interpretation and comparisons with external maps.
Austria, Denmark, Italy and Poland successfully installed and tested the models on selected NUTS 2 areas. Based on these experiences, a generic processing pipeline was developed, deployed on the Copernicus Data Space Ecosystem (CDSE) cloud infrastructure and applied across the participating countries.
This demonstrated the feasibility of cross-country application at national scale. Cloud-based processing is particularly important given the volume of EO data and computing resources required. Further work will explore improvements, including retraining models.

Crop type
The crop-type use case followed a similar approach. GUS initially developed a crop-type pipeline using Sentinel-1 and Sentinel-2 data. Austria, Ireland, the Netherlands, Poland and Portugal tested it on selected NUTS 2 areas, using manually coded samples for training and validation.
Based on these tests, the pipeline was substantially redesigned using state-of-the-art techniques, including a foundation model. The new generic pipeline was deployed on the CDSE and applied across the participating countries.
The results are impressive and demonstrate the potential of a reusable, large-scale approach. The redesigned pipeline identified significantly more crop types while maintaining high accuracy across countries.

Key findings
Both use cases demonstrate that EO-based AI/ML pipelines can be transferred across countries, but successful generalisation requires more than applying an existing model to new data. The project also highlights the value of cloud platforms such as the CDSE in making EO-based statistical production scalable and accessible to multiple statistical institutes.

For crop-type classification, the Total Machine Learning Error (TMLE) framework was used to assess the soundness of the approach, providing insights for improving both the pipeline and the assessment methodology. Its application will be extended to the land-cover use case.

Overall, the project provides practical evidence that reusable EO-based AI/ML pipelines are feasible and can reduce duplicated development efforts across European statistical institutes.