Sparse Linear Regression with Constraints: A Flexible Entropy-Based Framework

Amber Srivastava, Alisina Bayati, Srinivasa M. Salapaka

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This work presents a new approach to solve the sparse linear regression problem, i.e., to determine a k - sparse vector w ∈ ℝd that minimizes the cost || y-Aw||22. In contrast to the existing methods, our proposed approach splits this k - sparse vector into two parts - (a) a column stochastic binary matrix V, and (b) a vector x ∈ ℝk. Here, the binary matrix V encodes the location of the k non-zero entries in w. Equivalently, it encodes the subset of k columns in the matrix A that map w to y. We demonstrate that this enables modeling several non-trivial application specific structural constraints on w as constraints on V. The vector x comprises of the actual non-zero values in w. We use Maximum Entropy Principle (MEP) to solve the resulting optimization problem. In particular, we ascribe a probability distribution to the set of all feasible binary matrices V, and iteratively determine this distribution and the vector x such that the associated Shannon entropy gets minimized, and the regression cost attains a pre-specified value. The resulting algorithm employs homotopy from the convex entropy function to the non-convex cost function to avoid poor local minimum. We demonstrate the efficacy and flexibility of our proposed approach in incorporating a variety of practical constraints, that are otherwise difficult to model using the existing benchmark methods.

Original languageEnglish (US)
Title of host publication2024 European Control Conference, ECC 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2105-2110
Number of pages6
ISBN (Electronic)9783907144107
DOIs
StatePublished - 2024
Externally publishedYes
Event2024 European Control Conference, ECC 2024 - Stockholm, Sweden
Duration: Jun 25 2024Jun 28 2024

Publication series

Name2024 European Control Conference, ECC 2024

Conference

Conference2024 European Control Conference, ECC 2024
Country/TerritorySweden
CityStockholm
Period6/25/246/28/24

ASJC Scopus subject areas

  • Control and Optimization
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'Sparse Linear Regression with Constraints: A Flexible Entropy-Based Framework'. Together they form a unique fingerprint.

Cite this