Abstract

Coding free text to standard classifications is a core task in official statistics. Assigning NACE codes to enterprises is a typical case: the classification has hundreds of classes with subtle semantic differences, the input texts are often short, noisy or incomplete, and the class distribution is strongly imbalanced.
This paper aims to explore the practical choice between lightweight models that are fast and cheap, and large language models (LLMs) that promise higher quality at a much higher computational cost. A key constraint is that business data must remain strictly on-premise, requiring solutions that run on secure national infrastructure with often limited hardware.
To address this challenge, we investigate a fundamental question: how large model do we actually need for NACE classification? By comparing approaches of increasing complexity under identical conditions, we assess them not only by classification quality but also by resource consumption.
This work is carried out within the international project “One-stop-shop For Artificial Intelligence and Machine Learning for Official Statistics (AIML4OS)”. Results of the full implementations will be shared at the conference presentation.
We compare four groups of approaches:
1. fastText, a fast classifier trained from scratch, which serves as a low-cost reference point.
2. Sentence embeddings with a simple classifier, where texts are converted into dense vectors by a pretrained embedding model and a lightweight classifier (e.g. logistic regression) is trained on top of them. The embeddings are used as features for classification, not for retrieval.
3. Open LLMs used with prompting only (zero- and few-shot, without retrieval of external knowledge), where the model assigns a NACE code on the basis of instructions and examples given in the prompt. We use Polish Large Language Model (Bielik), trained with a focus on the Polish language, and Qwen, a multilingual model, which allows us to compare a language-specialised model with a general-purpose one.
4. Parameter-efficient fine-tuning (LoRA) of the better-performing LLM, where low-rank adapters specialise the pretrained model for NACE classification without the cost of full fine-tuning.
The study will show whether a lightweight classifier is sufficient or whether the additional cost of an LLM is justified, whether a model specialised in Polish has an advantage over a multilingual one, and how much fine-tuning improves results compared with prompting alone. Particular attention will be paid to rare classes, which are the most difficult in statistical practice.
The results are intended to provide practical guidance on selecting and adapting a model for NACE classification, depending on the required quality and the acceptable processing time.