Strong Rules for Discarding Predictors in Lasso-Type Problems

Summary We consider rules for discarding predictors in lasso regression and related problems, for computational efficiency. El Ghaoui and his colleagues have proposed ‘SAFE’ rules, based on univariate inner products between each predictor and the outcome, which guarantee that a coefficient will be 0 in the solution vector. This provides a reduction in the number of variables that need to be entered into the optimization. We propose strong rules that are very simple and yet screen out far more predictors than the SAFE rules. This great practical improvement comes at a price: the strong rules are not foolproof and can mistakenly discard active predictors, i.e. predictors that have non-zero coefficients in the…

• NS
  National Science Foundation

  Awards: DMS-9971405, DMS-0906801
• NI
  National Institutes of Health

  Award: N01-HV-28183