High-dimensional joint sparsity random effects model for multi-task learning

Krishnakumar Balasubramanian, Kai Yu, Tong Zhang

Research output: Contribution to conferencePaperpeer-review


Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the looseness of the relaxation. To remedy this problem, we view jointly sparse multi-task learning as a specialized random effects model, and derive a convex relaxation approach that involves two steps. The first step learns the covariance matrix of the coefficients using a convex formulation which we refer to as sparse covariance coding; the second step solves a ridge regression problem with a sparse quadratic regularizer based on the covariance matrix obtained in the first step. It is shown that this approach produces an asymptotically optimal quadratic regularizer in the multitask learning setting when the number of tasks approaches infinity. Experimental results demonstrate that the convex formulation obtained via the proposed model significantly outperforms group Lasso (and related multi-stage formulations).

Original languageEnglish (US)
Number of pages10
StatePublished - 2013
Externally publishedYes
Event29th Conference on Uncertainty in Artificial Intelligence, UAI 2013 - Bellevue, WA, United States
Duration: Jul 11 2013Jul 15 2013


Other29th Conference on Uncertainty in Artificial Intelligence, UAI 2013
Country/TerritoryUnited States
CityBellevue, WA

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'High-dimensional joint sparsity random effects model for multi-task learning'. Together they form a unique fingerprint.

Cite this