Block regularized lasso for multivariate multi-response linear regression

Weiguang Wang, Yingbin Liang, Eric P. Xing

Research output: Contribution to journalConference Articlepeer-review

10 Scopus citations

Abstract

The multivariate multi-response (MVMR) linear regression problem is investigated, in which design matrices are Gaussian with covariance matrices Σ(1:K) = Σ(1); : : : ;Σ(K) ) for K linear regressions. The support union of K p-dimensional regression vectors (collected as columns of matrix B∗) are recovered using l1=l2-regularized Lasso. Sufficient and necessary conditions to guarantee successful recovery of the support union are characterized via a threshold. More specifically, it is shown that under certain conditions on the distributions of design matrices, if n > cp1ψ(B∗;Σ(1:K)) log(p - s) where cp1 is a constant, and s is the size of the support set, then l1=l2-regularized Lasso correctly recovers the support union; and if n < cp2ψ(B∗;Σ(1:K)) log(p - s) where cp2 is a constant, then l1=l2-regularized Lasso fails to recover the support union. In particular, ψ(B∗;Σ(1:K)) captures the impact of the sparsity of K regression vectors and the statistical properties of the design matrices on the threshold for support recovery. Numerical results are provided to demonstrate the advantages of joint support union recovery using multi-task Lasso over individual support recovery using single-task Lasso.

Original languageEnglish (US)
Pages (from-to)608-617
Number of pages10
JournalJournal of Machine Learning Research
Volume31
StatePublished - 2013
Event16th International Conference on Artificial Intelligence and Statistics, AISTATS 2013 - Scottsdale, United States
Duration: Apr 29 2013May 1 2013

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Control and Systems Engineering
  • Statistics and Probability

Fingerprint

Dive into the research topics of 'Block regularized lasso for multivariate multi-response linear regression'. Together they form a unique fingerprint.

Cite this