Effective Learning During COVID-19: Multilevel Covariates Matching and Propensity Score Matching

Siying Guo, Jianxuan Liu, Qiu Wang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


In large-scale observational data with a hierarchical structure, both clusters and interventions often have more than two levels. Popular methods in the binary treatment literature do not naturally extend to the hierarchical multilevel treatment case. For example, most K-12 and universities have moved to an unprecedented hybrid learning module during the COVID-19 pandemic where learning modes include hybrid and fully remote learning, while students were clustered within a class and school region. It is challenging to evaluate the effectiveness of the learning outcomes of the multilevel treatments in a hierarchically data structured. In this paper, we study a covariates matching method and develop a generalized propensity score matching method to reduce the bias of estimation in the intervention effect. We also propose simple algorithms to assess the covariates balance for each approach. We examine the finite sample performance of the methods via simulation studies and apply the proposed methods to analyze the effectiveness of learning modes during the COVID-19 pandemic.

Original languageEnglish (US)
Pages (from-to)967-982
Number of pages16
JournalAnnals of Data Science
Issue number5
StatePublished - Oct 2022


  • COVID-19
  • Generalized propensity score
  • Matching
  • Multilevel hybrid learning
  • Potential outcome

ASJC Scopus subject areas

  • Business, Management and Accounting (miscellaneous)
  • Computer Science Applications
  • Statistics, Probability and Uncertainty
  • Artificial Intelligence


Dive into the research topics of 'Effective Learning During COVID-19: Multilevel Covariates Matching and Propensity Score Matching'. Together they form a unique fingerprint.

Cite this