A lossy source coding interpretation of Wyner's common information

Ge Xu, Wei Liu, Biao Chen

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Wyner's common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray-Wyner network, it is established that Wyner's common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner's common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray-Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner's common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray-Wyner lossy source coding problem and for identifying conditions under which Wyner's common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained.

Original languageEnglish (US)
Article number7349201
Pages (from-to)754-768
Number of pages15
JournalIEEE Transactions on Information Theory
Volume62
Issue number2
DOIs
StatePublished - Feb 1 2016

Fingerprint

coding
interpretation
Random variables
Byproducts
Decoding
equality
Values

Keywords

  • Common information
  • Gray-Wyner network
  • Rate distortion function

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Cite this

A lossy source coding interpretation of Wyner's common information. / Xu, Ge; Liu, Wei; Chen, Biao.

In: IEEE Transactions on Information Theory, Vol. 62, No. 2, 7349201, 01.02.2016, p. 754-768.

Research output: Contribution to journalArticle

@article{cbf1b5daa88f4ac899158dc05b1c96a6,
title = "A lossy source coding interpretation of Wyner's common information",
abstract = "Wyner's common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray-Wyner network, it is established that Wyner's common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner's common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray-Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner's common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray-Wyner lossy source coding problem and for identifying conditions under which Wyner's common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained.",
keywords = "Common information, Gray-Wyner network, Rate distortion function",
author = "Ge Xu and Wei Liu and Biao Chen",
year = "2016",
month = "2",
day = "1",
doi = "10.1109/TIT.2015.2506560",
language = "English (US)",
volume = "62",
pages = "754--768",
journal = "IEEE Transactions on Information Theory",
issn = "0018-9448",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - A lossy source coding interpretation of Wyner's common information

AU - Xu, Ge

AU - Liu, Wei

AU - Chen, Biao

PY - 2016/2/1

Y1 - 2016/2/1

N2 - Wyner's common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray-Wyner network, it is established that Wyner's common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner's common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray-Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner's common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray-Wyner lossy source coding problem and for identifying conditions under which Wyner's common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained.

AB - Wyner's common information was originally defined for a pair of dependent discrete random variables. Its significance is largely reflected in, and also confined to, several existing interpretations in various source coding problems. This paper attempts to expand its practical significance by providing a new operational interpretation. In the context of the Gray-Wyner network, it is established that Wyner's common information has a new lossy source coding interpretation. Specifically, it is established that, under suitable conditions, Wyner's common information equals to the smallest common message rate when the total rate is arbitrarily close to the rate distortion function with joint decoding for the Gray-Wyner network. A surprising observation is that such equality holds independent of the values of distortion constraints as long as the distortions are within some distortion region. The new lossy source coding interpretation provides the first meaningful justification for defining Wyner's common information for continuous random variables and the result can also be extended to that of multiple variables. Examples are given for characterizing the rate distortion region for the Gray-Wyner lossy source coding problem and for identifying conditions under which Wyner's common information equals that of the smallest common rate. As a by-product, the explicit expression for the common information between a pair of Gaussian random variables is obtained.

KW - Common information

KW - Gray-Wyner network

KW - Rate distortion function

UR - http://www.scopus.com/inward/record.url?scp=84959354328&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84959354328&partnerID=8YFLogxK

U2 - 10.1109/TIT.2015.2506560

DO - 10.1109/TIT.2015.2506560

M3 - Article

VL - 62

SP - 754

EP - 768

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

SN - 0018-9448

IS - 2

M1 - 7349201

ER -