### Abstract

The classical multivariate linear regression problem assumes p variables X_{1}, X_{2}, . . ., X_{p} and a response vector y, each with n observations, and a linear relationship between the two: y = Xß + z, where z ∼ N(O,σ^{2}). We point out that when p > n, there is a breakdown point for standard model selection schemes, such that model selection only works well below a certain critical complexity level depending on n/p. We apply this notion to some standard model selection algorithms (Forward Stepwise, LASSO, LARS) in the case where p ≫ n. We and that 1) the breakdown point is well-de ned for random X -models and low noise, 2) increasing noise shifts the breakdown point to lower levels of sparsity, and reduces the model recovery ability of the algorithm in a systematic way, and 3) below breakdown, the size of coefcient errors follows the theoretical error distribution for the classical linear model.

Original language | English (US) |
---|---|

Title of host publication | International Joint Conference on Neural Networks 2006, IJCNN '06 |

Pages | 1916-1921 |

Number of pages | 6 |

State | Published - Dec 1 2006 |

Externally published | Yes |

Event | International Joint Conference on Neural Networks 2006, IJCNN '06 - Vancouver, BC, Canada Duration: Jul 16 2006 → Jul 21 2006 |

### Publication series

Name | IEEE International Conference on Neural Networks - Conference Proceedings |
---|---|

ISSN (Print) | 1098-7576 |

### Other

Other | International Joint Conference on Neural Networks 2006, IJCNN '06 |
---|---|

Country | Canada |

City | Vancouver, BC |

Period | 7/16/06 → 7/21/06 |

### ASJC Scopus subject areas

- Software

## Fingerprint Dive into the research topics of 'Breakdown point of model selection when the number of variables exceeds the number of observations'. Together they form a unique fingerprint.

## Cite this

*International Joint Conference on Neural Networks 2006, IJCNN '06*(pp. 1916-1921). [1716344] (IEEE International Conference on Neural Networks - Conference Proceedings).