Decentralized data reduction with quantization constraints

Ge Xu, Shengyu Zhu, Biao Chen

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impact of quantization on decentralized data reduction, which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.

Original languageEnglish (US)
Article number6728713
Pages (from-to)1775-1784
Number of pages10
JournalIEEE Transactions on Signal Processing
Volume62
Issue number7
DOIs
StatePublished - Apr 1 2014

Fingerprint

Data reduction
Statistics
Communication
Sensors

Keywords

  • Decentralized inference
  • quantization
  • sufficiency principle
  • sufficient statistic

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Signal Processing

Cite this

Decentralized data reduction with quantization constraints. / Xu, Ge; Zhu, Shengyu; Chen, Biao.

In: IEEE Transactions on Signal Processing, Vol. 62, No. 7, 6728713, 01.04.2014, p. 1775-1784.

Research output: Contribution to journalArticle

@article{a803faa23fc74ca995b894f6a4ee8de0,
title = "Decentralized data reduction with quantization constraints",
abstract = "A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impact of quantization on decentralized data reduction, which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.",
keywords = "Decentralized inference, quantization, sufficiency principle, sufficient statistic",
author = "Ge Xu and Shengyu Zhu and Biao Chen",
year = "2014",
month = "4",
day = "1",
doi = "10.1109/TSP.2014.2303432",
language = "English (US)",
volume = "62",
pages = "1775--1784",
journal = "IEEE Transactions on Signal Processing",
issn = "1053-587X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "7",

}

TY - JOUR

T1 - Decentralized data reduction with quantization constraints

AU - Xu, Ge

AU - Zhu, Shengyu

AU - Chen, Biao

PY - 2014/4/1

Y1 - 2014/4/1

N2 - A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impact of quantization on decentralized data reduction, which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.

AB - A guiding principle for data reduction in statistical inference is the sufficiency principle. This paper extends the classical sufficiency principle to decentralized inference, i.e., data reduction needs to be achieved in a decentralized manner. We examine the notions of local and global sufficient statistics and the relationship between the two for decentralized inference under different observation models. We then consider the impact of quantization on decentralized data reduction, which is often needed when communications among sensors are subject to finite capacity constraints. The central question we intend to ask is: if each node in a decentralized inference system has to summarize its data using a finite number of bits, is it still optimal to implement data reduction using global sufficient statistics prior to quantization? We show that the answer is negative using a simple example and proceed to identify conditions under which sufficiency based data reduction followed by quantization is indeed optimal. They include the well known case when the data at decentralized nodes are conditionally independent as well as a class of problems with conditionally dependent observations that admit conditional independence structure through the introduction of an appropriately chosen hidden variable.

KW - Decentralized inference

KW - quantization

KW - sufficiency principle

KW - sufficient statistic

UR - http://www.scopus.com/inward/record.url?scp=84897911777&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84897911777&partnerID=8YFLogxK

U2 - 10.1109/TSP.2014.2303432

DO - 10.1109/TSP.2014.2303432

M3 - Article

AN - SCOPUS:84897911777

VL - 62

SP - 1775

EP - 1784

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

SN - 1053-587X

IS - 7

M1 - 6728713

ER -