Statistical Learning with Sparsity: the Lasso and the Elastic Net
Abstract
The Lasso is a widely used method for feature selection in machine
learning and statistics. It uses an L1 regularizer to induce
sparsity in a linear model. The Elastic Net is similar to the Lasso,
and uses a regularizer that is a mix between L2 and L1 penalties.
It also produces sparse solutions, but is designed to select groups of
collinear variables together, rather than in competition with each
other. We describe these methodologies, and the popular R "glmnet"
package that implements them. We also mention some newer Python
packages, and demonstrate the techniques using several applications.