Abstract

Automated classification of text into statistical classification systems is an important task for reducing manual effort in official statistics. Many such classification systems, including NACE, which classifies econimic activities, are inherently hierarchical, raising the question of whether this structure can be exploited to improve classification performance.

In this work, we investigate and compare different approaches to hierarchy-aware text models using NACE code prediction as a case study. We compare a conventional flat text classifier with three approaches that explicitly incorporate the hierarchical structure of the target classification: a stacked model with separate classifiers for each hierarchy level, a single model with multiple outputs corresponding to the different levels, and a model using a hierarchy-aware loss function. The models are evaluated using conventional evaluation metrics such as accuracy and top-k accuracy as well as hierarchy-aware metrics.

The results show that incorporating the hierarchy does not automatically lead to improved classification performance. Of the three approaches, only one model outperforms the flat classifier. These findings highlight the trade-off between the potential benefits of hierarchy-aware classification and the additional complexity, computational resources, and maintenance required to implement such approaches. For official statistics, where classification systems evolve and models need to be maintained in production, these practical considerations are particularly relevant. This talk provides an empirical comparison of hierarchical classification strategies and a discussion of the challenges involved in applying them to real-world statistical classification systems.