Abstract
Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms that we expect a measure of potential correlation to satisfy; (ii) we show that the rate of information bottleneck, i.e., the hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we provide a novel estimator to estimate the hypercontractivity coefficient from samples; and (iv) we provide numerical experiments demonstrating that this proposed estimator discovers potential correlations among various indicators of WHO datasets, is robust in discovering gene interactions from gene expression time series data, and is statistically more powerful than the estimators for other correlation measures in binary hypothesis testing of canonical examples of potential correlations.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 4578-4588 |
| Number of pages | 11 |
| Journal | Advances in Neural Information Processing Systems |
| Volume | 2017-December |
| State | Published - Jan 1 2017 |
| Event | 31st Annual Conference on Neural Information Processing Systems, NIPS 2017 - Long Beach, United States Duration: Dec 4 2017 → Dec 9 2017 |
Fingerprint
ASJC Scopus subject areas
- Computer Networks and Communications
- Information Systems
- Signal Processing
Cite this
Discovering potential correlations via hypercontractivity. / Kim, Hyeji; Gao, Weihao; Kannan, Sreeram; Oh, Sewoong; Viswanath, Pramod.
In: Advances in Neural Information Processing Systems, Vol. 2017-December, 01.01.2017, p. 4578-4588.Research output: Contribution to journal › Conference article
}
TY - JOUR
T1 - Discovering potential correlations via hypercontractivity
AU - Kim, Hyeji
AU - Gao, Weihao
AU - Kannan, Sreeram
AU - Oh, Sewoong
AU - Viswanath, Pramod
PY - 2017/1/1
Y1 - 2017/1/1
N2 - Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms that we expect a measure of potential correlation to satisfy; (ii) we show that the rate of information bottleneck, i.e., the hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we provide a novel estimator to estimate the hypercontractivity coefficient from samples; and (iv) we provide numerical experiments demonstrating that this proposed estimator discovers potential correlations among various indicators of WHO datasets, is robust in discovering gene interactions from gene expression time series data, and is statistically more powerful than the estimators for other correlation measures in binary hypothesis testing of canonical examples of potential correlations.
AB - Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms that we expect a measure of potential correlation to satisfy; (ii) we show that the rate of information bottleneck, i.e., the hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we provide a novel estimator to estimate the hypercontractivity coefficient from samples; and (iv) we provide numerical experiments demonstrating that this proposed estimator discovers potential correlations among various indicators of WHO datasets, is robust in discovering gene interactions from gene expression time series data, and is statistically more powerful than the estimators for other correlation measures in binary hypothesis testing of canonical examples of potential correlations.
UR - http://www.scopus.com/inward/record.url?scp=85047003843&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85047003843&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85047003843
VL - 2017-December
SP - 4578
EP - 4588
JO - Advances in Neural Information Processing Systems
JF - Advances in Neural Information Processing Systems
SN - 1049-5258
ER -