### Abstract

Wyner defined the notion of common information of two discrete random variables as the minimum of I(W; X, Y) where W induces conditional independence between X and Y. Its generalization to multiple dependent random variables revealed a surprising monotone property in the number of variables. Motivated by this monotonicity property, this paper explores the application of Wyner's common information to inference problems and its connection with other performance metrics. A central question is that under what conditions Wyner's common information captures the entire information contained in the observations about the inference object under a simple Bayesian model. For infinitely exchangeable random variables, it is shown using the de Finetti-Hewitt-Savage theorem that the common information is asymptotically equal to the information of the inference object. For finite exchangeable random variables, such conclusion is no longer true even for infinitely extendable sequences. However, for some special cases, including both the binary and the Gaussian cases, concrete connection between common information and inference performance metrics can be established even for finite samples.

Original language | English (US) |
---|---|

Title of host publication | 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 |

Pages | 1516-1520 |

Number of pages | 5 |

DOIs | |

State | Published - 2011 |

Event | 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 - Monticello, IL, United States Duration: Sep 28 2011 → Sep 30 2011 |

### Other

Other | 2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011 |
---|---|

Country | United States |

City | Monticello, IL |

Period | 9/28/11 → 9/30/11 |

### Fingerprint

### ASJC Scopus subject areas

- Computer Networks and Communications
- Control and Systems Engineering

### Cite this

*2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011*(pp. 1516-1520). [6120347] https://doi.org/10.1109/Allerton.2011.6120347