Abstract

Variable selection is often unstable when applied to datasets featuring numerous, strongly correlated, or organized in complex structures predictors. This is, for instance, particularly relevant in high-dimensional regression, functional data analysis, multi-block modelling, and multi-omics settings. Interpreting a single selected model can be misleading, even when its predictive performance is satisfactory. This communication presents recent developments around a correlated resampling method named SelectBoost and its extensions. The central idea of SelectBoost is to perturb the predictor space while keeping the main correlation patterns of the data, then, for each model fit, to record the variables that keep reappearing in the selected set. The algorithm was initially motivated by multi-omics problems for which stable links between variable may be favored. The scope of the models that may benefit from the algorithm initially included linear and generalised linear models but was recently extended to other settings such as beta regression, quantile regression, multi-block and functional models. With functional models, SelectBoost can be used at different levels: coefficients, curve segments or derived components. We will focus on three points: the generation of the perturbations, the selection frequencies and their use to enhance the performance of variable selection methods as well as many benchmarks. We will also provide some theoretical insights on the methods.