Information for inference

Ge Xu, Biao Chen

Research output: Chapter in Book/Entry/PoemConference contribution

4 Scopus citations

Abstract

Wyner defined the notion of common information of two discrete random variables as the minimum of I(W; X, Y) where W induces conditional independence between X and Y. Its generalization to multiple dependent random variables revealed a surprising monotone property in the number of variables. Motivated by this monotonicity property, this paper explores the application of Wyner's common information to inference problems and its connection with other performance metrics. A central question is that under what conditions Wyner's common information captures the entire information contained in the observations about the inference object under a simple Bayesian model. For infinitely exchangeable random variables, it is shown using the de Finetti-Hewitt-Savage theorem that the common information is asymptotically equal to the information of the inference object. For finite exchangeable random variables, such conclusion is no longer true even for infinitely extendable sequences. However, for some special cases, including both the binary and the Gaussian cases, concrete connection between common information and inference performance metrics can be established even for finite samples.

Original languageEnglish (US)
Title of host publication2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011
Pages1516-1520
Number of pages5
DOIs
StatePublished - 2011
Event2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 - Monticello, IL, United States
Duration: Sep 28 2011Sep 30 2011

Publication series

Name2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

Other

Other2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011
Country/TerritoryUnited States
CityMonticello, IL
Period9/28/119/30/11

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Control and Systems Engineering

Fingerprint

Dive into the research topics of 'Information for inference'. Together they form a unique fingerprint.

Cite this