Sure Independence Screening for Ultrahigh Dimensional Feature Space

Summary Variable selection plays an important role in high dimensional statistical modelling which nowadays appears in many areas and is key to various scientific discoveries. For problems of large scale or dimensionality p, accuracy of estimation and computational cost are two top concerns. Recently, Candes and Tao have proposed the Dantzig selector using L1-regularization and showed that it achieves the ideal risk up to a logarithmic factor log(p). Their innovative procedure and remarkable result are challenged when the dimensionality is ultrahigh as the factor log(p) can be large and their uniform uncertainty principle can fail. Motivated by these concerns, we introduce the concept of sure screening and…

• NS
  National Science Foundation

  Awards: DMS-0714554, DMS-0354223, DMS-0704337, DMS-0806030
• NI
  National Institutes of Health

  Award: R01-GM072611