Rare event-prediction with a hybrid algorithm under power-law assumption

Mina Jung, Jae C Oh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present an algorithm for predicting both common and rare events. Statistics show that occurrences of rare events are usually associated with common events. Therefore, we argue that predicting common events correctly is an important step toward correctly predicting rare events. The new algorithm assumes that frequencies of events exhibit a power-law distribution. The algorithm consists of components for detecting rare event types and common event types, while minimizing computational overhead. For experiments, we attempt to predict various fault types that can occur in distributed systems. The simulation study driven by the system failure data collected at the Pacific Northwest National Laboratory (PNNL) shows that fault-mitigation based on the new prediction mechanism provides 15% better system availability than the existing prediction methods. Furthermore, it allows only 10% of all possible system loss caused by rare faults in the simulation data.

Original languageEnglish (US)
Title of host publicationTrends in Applied Knowledge-Based Systems and Data Science - 29th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2016, Proceedings
PublisherSpringer Verlag
Pages43-55
Number of pages13
Volume9799
ISBN (Print)9783319420066
DOIs
StatePublished - 2016
Event29th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2016 - Morioka, Japan
Duration: Aug 2 2016Aug 4 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9799
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other29th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2016
CountryJapan
CityMorioka
Period8/2/168/4/16

Keywords

  • Bayesian inference
  • Event prediction
  • Fault mitigation
  • Logistic regression
  • Online learning
  • Power-law distribution
  • Rare event

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Fingerprint Dive into the research topics of 'Rare event-prediction with a hybrid algorithm under power-law assumption'. Together they form a unique fingerprint.

  • Cite this

    Jung, M., & Oh, J. C. (2016). Rare event-prediction with a hybrid algorithm under power-law assumption. In Trends in Applied Knowledge-Based Systems and Data Science - 29th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2016, Proceedings (Vol. 9799, pp. 43-55). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9799). Springer Verlag. https://doi.org/10.1007/978-3-319-42007-3_5