Hierarchical dynamic power management using model-free reinforcement learning

Yanzhi Wang, Maryam Triki, Xue Lin, Ahmed C. Ammari, Massoud Pedram

Research output: Chapter in Book/Entry/PoemConference contribution

6 Scopus citations


Model-free reinforcement learning (RL) has become a promising technique for designing a robust dynamic power management (DPM) framework that can cope with variations and uncertainties that emanate from hardware and application characteristics. Moreover, the potentially significant benefit of performing application-level scheduling as part of the system-level power management should be harnessed. This paper presents an architecture for hierarchical DPM in an embedded system composed of a processor chip and connected I/O devices (which are called system components.) The goal is to facilitate saving in the system component power consumption, which tends to dominate the total power consumption. The proposed (online) adaptive DPM technique consists of two layers: an RL-based component-level local power manager (LPM) and a system-level global power manager (GPM). The LPM performs component power and latency optimization. It employs temporal difference learning on semi-Markov decision process (SMDP) for model-free RL, and it is specifically optimized for an environment in which multiple (heterogeneous) types of applications can run in the embedded system. The GPM interacts with the CPU scheduler to perform effective application-level scheduling, thereby, enabling the LPM to do even more component power optimizations. In this hierarchical DPM framework, power and latency tradeoffs of each type of application can be precisely controlled based on a user-defined parameter. Experiments show that the amount of average power saving is up to 31.1% compared to existing approaches.

Original languageEnglish (US)
Title of host publicationProceedings of the 14th International Symposium on Quality Electronic Design, ISQED 2013
Number of pages8
StatePublished - 2013
Externally publishedYes
Event14th International Symposium on Quality Electronic Design, ISQED 2013 - Santa Clara, CA, United States
Duration: Mar 4 2013Mar 6 2013

Publication series

NameProceedings - International Symposium on Quality Electronic Design, ISQED
ISSN (Print)1948-3287
ISSN (Electronic)1948-3295


Other14th International Symposium on Quality Electronic Design, ISQED 2013
Country/TerritoryUnited States
CitySanta Clara, CA


  • Bayesian classification
  • Dynamic power management
  • reinforcement learning

ASJC Scopus subject areas

  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Safety, Risk, Reliability and Quality


Dive into the research topics of 'Hierarchical dynamic power management using model-free reinforcement learning'. Together they form a unique fingerprint.

Cite this