Hierarchical dynamic power management using model-free reinforcement learning

Yanzhi Wang, Maryam Triki, Xue Lin, Ahmed C. Ammari, Massoud Pedram

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Model-free reinforcement learning (RL) has become a promising technique for designing a robust dynamic power management (DPM) framework that can cope with variations and uncertainties that emanate from hardware and application characteristics. Moreover, the potentially significant benefit of performing application-level scheduling as part of the system-level power management should be harnessed. This paper presents an architecture for hierarchical DPM in an embedded system composed of a processor chip and connected I/O devices (which are called system components.) The goal is to facilitate saving in the system component power consumption, which tends to dominate the total power consumption. The proposed (online) adaptive DPM technique consists of two layers: an RL-based component-level local power manager (LPM) and a system-level global power manager (GPM). The LPM performs component power and latency optimization. It employs temporal difference learning on semi-Markov decision process (SMDP) for model-free RL, and it is specifically optimized for an environment in which multiple (heterogeneous) types of applications can run in the embedded system. The GPM interacts with the CPU scheduler to perform effective application-level scheduling, thereby, enabling the LPM to do even more component power optimizations. In this hierarchical DPM framework, power and latency tradeoffs of each type of application can be precisely controlled based on a user-defined parameter. Experiments show that the amount of average power saving is up to 31.1% compared to existing approaches.

Original languageEnglish (US)
Title of host publicationProceedings - International Symposium on Quality Electronic Design, ISQED
Pages170-177
Number of pages8
DOIs
StatePublished - 2013
Externally publishedYes
Event14th International Symposium on Quality Electronic Design, ISQED 2013 - Santa Clara, CA, United States
Duration: Mar 4 2013Mar 6 2013

Other

Other14th International Symposium on Quality Electronic Design, ISQED 2013
CountryUnited States
CitySanta Clara, CA
Period3/4/133/6/13

    Fingerprint

Keywords

  • Bayesian classification
  • Dynamic power management
  • reinforcement learning

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Safety, Risk, Reliability and Quality

Cite this

Wang, Y., Triki, M., Lin, X., Ammari, A. C., & Pedram, M. (2013). Hierarchical dynamic power management using model-free reinforcement learning. In Proceedings - International Symposium on Quality Electronic Design, ISQED (pp. 170-177). [6523606] https://doi.org/10.1109/ISQED.2013.6523606