On Guo and Nixon's criterion for feature subset selection: Assumptions, implications, and alternative options

Kiran S. Balagani; Vir V. Phoha; S. S. Iyengar; N. Balakrishnan

doi:10.1109/TSMCA.2009.2036935

On Guo and Nixon's criterion for feature subset selection: Assumptions, implications, and alternative options

Kiran S. Balagani, Vir V. Phoha, S. S. Iyengar, N. Balakrishnan

Research output: Contribution to journal › Article › peer-review

3 Scopus citations

Abstract

Guo and Nixon proposed a feature selection method based on maximizing I(x;Y), the multidimensional mutual information between feature vector x and class variable Y. Because computing I(x;Y) can be difficult in practice, Guo and Nixon proposed an approximation of I(x;Y) as the criterion for feature selection. We show that Guo and Nixon's criterion originates from approximating the joint probability distributions in I(x;Y) by second-order product distributions. We remark on the limitations of the approximation and discuss computationally attractive alternatives to compute I(x;Y).

Original language	English (US)
Article number	5395688
Pages (from-to)	651-655
Number of pages	5
Journal	IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans
Volume	40
Issue number	3
DOIs	https://doi.org/10.1109/TSMCA.2009.2036935
State	Published - May 2010
Externally published	Yes

Keywords

Entropic spanning graphs
Feature selection
Mutual information
Parzen window

ASJC Scopus subject areas

Software
Control and Systems Engineering
Human-Computer Interaction
Computer Science Applications
Electrical and Electronic Engineering

Access to Document

10.1109/TSMCA.2009.2036935

Cite this

@article{43110b03fba04f4eaa93a94da0e9d8ee,

title = "On Guo and Nixon's criterion for feature subset selection: Assumptions, implications, and alternative options",

abstract = "Guo and Nixon proposed a feature selection method based on maximizing I(x;Y), the multidimensional mutual information between feature vector x and class variable Y. Because computing I(x;Y) can be difficult in practice, Guo and Nixon proposed an approximation of I(x;Y) as the criterion for feature selection. We show that Guo and Nixon's criterion originates from approximating the joint probability distributions in I(x;Y) by second-order product distributions. We remark on the limitations of the approximation and discuss computationally attractive alternatives to compute I(x;Y).",

keywords = "Entropic spanning graphs, Feature selection, Mutual information, Parzen window",

author = "Balagani, {Kiran S.} and Phoha, {Vir V.} and Iyengar, {S. S.} and N. Balakrishnan",

note = "Funding Information: Manuscript received February 11, 2009; revised June 13, 2009. First published January 22, 2010; current version published April 14, 2010. This work was supported in part by the Louisiana Board of Regents under PKSFI Grant LEQSF(2007-12)-ENH-PKSFI-PRS-03. This paper was recommended by Associate Editor J. Wu. K. S. Balagani and V. V. Phoha are with Louisiana Tech University, Ruston, LA 71270 USA (e-mail: [email protected]; [email protected]). S. S. Iyengar is with Louisiana State University, Baton Rouge, LA 70809 USA (e-mail: [email protected]). N. Balakrishnan is with the Indian Institute of Science, Bangalore 560012, India (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCA.2009.2036935",

year = "2010",

month = may,

doi = "10.1109/TSMCA.2009.2036935",

language = "English (US)",

volume = "40",

pages = "651--655",

journal = "IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans",

issn = "1083-4427",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "3",

}

TY - JOUR

T1 - On Guo and Nixon's criterion for feature subset selection

T2 - Assumptions, implications, and alternative options

AU - Balagani, Kiran S.

AU - Phoha, Vir V.

AU - Iyengar, S. S.

AU - Balakrishnan, N.

N1 - Funding Information: Manuscript received February 11, 2009; revised June 13, 2009. First published January 22, 2010; current version published April 14, 2010. This work was supported in part by the Louisiana Board of Regents under PKSFI Grant LEQSF(2007-12)-ENH-PKSFI-PRS-03. This paper was recommended by Associate Editor J. Wu. K. S. Balagani and V. V. Phoha are with Louisiana Tech University, Ruston, LA 71270 USA (e-mail: [email protected]; [email protected]). S. S. Iyengar is with Louisiana State University, Baton Rouge, LA 70809 USA (e-mail: [email protected]). N. Balakrishnan is with the Indian Institute of Science, Bangalore 560012, India (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TSMCA.2009.2036935

PY - 2010/5

Y1 - 2010/5

N2 - Guo and Nixon proposed a feature selection method based on maximizing I(x;Y), the multidimensional mutual information between feature vector x and class variable Y. Because computing I(x;Y) can be difficult in practice, Guo and Nixon proposed an approximation of I(x;Y) as the criterion for feature selection. We show that Guo and Nixon's criterion originates from approximating the joint probability distributions in I(x;Y) by second-order product distributions. We remark on the limitations of the approximation and discuss computationally attractive alternatives to compute I(x;Y).

AB - Guo and Nixon proposed a feature selection method based on maximizing I(x;Y), the multidimensional mutual information between feature vector x and class variable Y. Because computing I(x;Y) can be difficult in practice, Guo and Nixon proposed an approximation of I(x;Y) as the criterion for feature selection. We show that Guo and Nixon's criterion originates from approximating the joint probability distributions in I(x;Y) by second-order product distributions. We remark on the limitations of the approximation and discuss computationally attractive alternatives to compute I(x;Y).

KW - Entropic spanning graphs

KW - Feature selection

KW - Mutual information

KW - Parzen window

UR - http://www.scopus.com/inward/record.url?scp=77951203722&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77951203722&partnerID=8YFLogxK

U2 - 10.1109/TSMCA.2009.2036935

DO - 10.1109/TSMCA.2009.2036935

M3 - Article

AN - SCOPUS:77951203722

SN - 1083-4427

VL - 40

SP - 651

EP - 655

JO - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

JF - IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

IS - 3

M1 - 5395688

ER -

On Guo and Nixon's criterion for feature subset selection: Assumptions, implications, and alternative options

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this