Applying Bayesian Modeling and Receiver Operating Characteristic Methodologies for Test Utility Analysis

Qiu Wang, Matthew A. Diemer, Kimberly S. Maier

Research output: Contribution to journalArticlepeer-review


This study integrated Bayesian hierarchical modeling and receiver operating characteristic analysis (BROCA) to evaluate how interest strength (IS) and interest differentiation (ID) predicted low-socioeconomic status (SES) youth's interest-major congruence (IMC). Using large-scale Kuder Career Search online-assessment data, this study fit three models, the one-level BROCA, the two-level BROCA, and the ordinal Probit BROCA, to examine the moderating effects of gender and race/ethnicity. Both IS and ID displayed race/ethnicity differences in predicting low-SES females' IMC. Gender difference was found only on IS in predicting low-SES youth's IMC. Results suggested that low-SES White males and low-SES minority females may need help the most to develop stronger career interests and to differentiate their interests. This study illustrated that BROCA can be a powerful tool for test evaluation and utility analysis in the field because of its capacity of analyzing continuous, nominal, and ordinal data; its graphical nature of result presentation; multiple statistical test options; and its little requirement of Level 2 sample sizes.

Original languageEnglish (US)
Pages (from-to)275-292
Number of pages18
JournalEducational and Psychological Measurement
Issue number2
StatePublished - Apr 2013


  • Bayesian ROC analysis
  • classification analysis
  • utility analysis

ASJC Scopus subject areas

  • Education
  • Developmental and Educational Psychology
  • Applied Psychology
  • Applied Mathematics


Dive into the research topics of 'Applying Bayesian Modeling and Receiver Operating Characteristic Methodologies for Test Utility Analysis'. Together they form a unique fingerprint.

Cite this