Abstract

Breiman (2001) framed the divide between data modeling and algorithmic modeling as two cultures: one privileging interpretable parameters and proper likelihoods through rigid functional forms, the other privileging flexibility to capture nonlinear structure at the cost of interpretability. We propose a different answer to this divide: a semi-parametric framework that couples a self-attention encoder, which learns structure directly from covariates, with a modular parametric likelihood component that preserves full statistical inference. An attention-based variable selection mechanism gives direct interpretability of which covariates drive predictions, and the likelihood can be substituted to match the outcome at hand.
We illustrate the idea through two instantiations: a forecasting setup under non-stationarity, high dimensionality, structural zeros, and heavy tails; and a cross-sectional regression setup for over-dispersed, skewed count data. In both, attention locates the signal while the likelihood layer keeps the model anchored to statistical inference and interpretability.