Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance

Eric J. Barnett, Daniel G. Onete, Asif Salekin, Stephen V. Faraone

Research output: Contribution to journalArticlepeer-review

Abstract

Many studies have been conducted with the goal of correctly predicting diagnostic status of a disorder using the combination of genomic data and machine learning. It is often hard to judge which components of a study led to better results and whether better reported results represent a true improvement or an uncorrected bias inflating performance. We extracted information about the methods used and other differentiating features in genomic machine learning models. We used these features in linear regressions predicting model performance. We tested for univariate and multivariate associations as well as interactions between features. Of the models reviewed, 46% used feature selection methods that can lead to data leakage. Across our models, the number of hyperparameter optimizations reported, data leakage due to feature selection, model type, and modeling an autoimmune disorder were significantly associated with an increase in reported model performance. We found a significant, negative interaction between data leakage and training size. Our results suggest that methods susceptible to data leakage are prevalent among genomic machine learning research, resulting in inflated reported performance. Best practice guidelines that promote the avoidance and recognition of data leakage may help the field avoid biased results.

Original languageEnglish (US)
Pages (from-to)169-177
Number of pages9
JournalIEEE/ACM Transactions on Computational Biology and Bioinformatics
Volume21
Issue number1
DOIs
StatePublished - Jan 1 2024

Keywords

  • Machine learning
  • artificial intelligence
  • biology and genetics
  • computer applications
  • computing methodologies
  • learning
  • life and medical sciences

ASJC Scopus subject areas

  • Applied Mathematics
  • Genetics
  • Biotechnology

Fingerprint

Dive into the research topics of 'Genomic Machine Learning Meta-regression: Insights on Associations of Study Features With Reported Model Performance'. Together they form a unique fingerprint.

Cite this