Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation

Qinyun Zhu, Jae C Oh

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As autonomous robots are becoming a reality, we discover new challenges in coordination among these robots. We present a unique new problem that each robot makes decisions in achieving tasks that require multiple robots with different capabilities. Fair resource allocation is essential to ensure that all resource requesters acquire adequate robot resources and accomplish their tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters requiring heterogeneous robots with different capabilities to accomplish tasks. In particular, this work focuses on systems of single-tasking robots with multi-robot tasks (STR-MRT). In STR-MRT, the capability of a robot is the resource for accomplishing a specific task. In this problem, 1) each robot can perform only one task at a time, 2) tasks are divisible, and 3) accomplishing each task requires resources from one or more robots. We model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot strategically selects one resource requester. Then a consensus-based algorithm conducts formation of a robotic team for each task. We leverage the Dominant Resource Fairness (DRF) and Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the common Q-learning.

Original languageEnglish (US)
Title of host publicationProceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018
EditorsM. Arif Wani, Moamar Sayed-Mouchaweh, Edwin Lughofer, Joao Gama, Mehmed Kantardzic
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages460-466
Number of pages7
ISBN (Electronic)9781538668047
DOIs
StatePublished - Jan 15 2019
Event17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018 - Orlando, United States
Duration: Dec 17 2018Dec 20 2018

Publication series

NameProceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018

Conference

Conference17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018
CountryUnited States
CityOrlando
Period12/17/1812/20/18

Fingerprint

Reinforcement learning
Resource allocation
Robotics
Robots
Fairness
Robot
Resources

Keywords

  • Multi-robot system
  • Reinforcement learning
  • Resource allocation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Safety, Risk, Reliability and Quality
  • Signal Processing
  • Decision Sciences (miscellaneous)

Cite this

Zhu, Q., & Oh, J. C. (2019). Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation. In M. A. Wani, M. Sayed-Mouchaweh, E. Lughofer, J. Gama, & M. Kantardzic (Eds.), Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018 (pp. 460-466). [8614100] (Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICMLA.2018.00075

Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation. / Zhu, Qinyun; Oh, Jae C.

Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018. ed. / M. Arif Wani; Moamar Sayed-Mouchaweh; Edwin Lughofer; Joao Gama; Mehmed Kantardzic. Institute of Electrical and Electronics Engineers Inc., 2019. p. 460-466 8614100 (Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, Q & Oh, JC 2019, Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation. in MA Wani, M Sayed-Mouchaweh, E Lughofer, J Gama & M Kantardzic (eds), Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018., 8614100, Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, Institute of Electrical and Electronics Engineers Inc., pp. 460-466, 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018, Orlando, United States, 12/17/18. https://doi.org/10.1109/ICMLA.2018.00075
Zhu Q, Oh JC. Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation. In Wani MA, Sayed-Mouchaweh M, Lughofer E, Gama J, Kantardzic M, editors, Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018. Institute of Electrical and Electronics Engineers Inc. 2019. p. 460-466. 8614100. (Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018). https://doi.org/10.1109/ICMLA.2018.00075
Zhu, Qinyun ; Oh, Jae C. / Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation. Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018. editor / M. Arif Wani ; Moamar Sayed-Mouchaweh ; Edwin Lughofer ; Joao Gama ; Mehmed Kantardzic. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 460-466 (Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018).
@inproceedings{89b81169031244fb9bb2b78dfcbbd137,
title = "Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation",
abstract = "As autonomous robots are becoming a reality, we discover new challenges in coordination among these robots. We present a unique new problem that each robot makes decisions in achieving tasks that require multiple robots with different capabilities. Fair resource allocation is essential to ensure that all resource requesters acquire adequate robot resources and accomplish their tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters requiring heterogeneous robots with different capabilities to accomplish tasks. In particular, this work focuses on systems of single-tasking robots with multi-robot tasks (STR-MRT). In STR-MRT, the capability of a robot is the resource for accomplishing a specific task. In this problem, 1) each robot can perform only one task at a time, 2) tasks are divisible, and 3) accomplishing each task requires resources from one or more robots. We model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot strategically selects one resource requester. Then a consensus-based algorithm conducts formation of a robotic team for each task. We leverage the Dominant Resource Fairness (DRF) and Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the common Q-learning.",
keywords = "Multi-robot system, Reinforcement learning, Resource allocation",
author = "Qinyun Zhu and Oh, {Jae C}",
year = "2019",
month = "1",
day = "15",
doi = "10.1109/ICMLA.2018.00075",
language = "English (US)",
series = "Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "460--466",
editor = "Wani, {M. Arif} and Moamar Sayed-Mouchaweh and Edwin Lughofer and Joao Gama and Mehmed Kantardzic",
booktitle = "Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018",

}

TY - GEN

T1 - Deep Reinforcement Learning for Fairness in Distributed Robotic Multi-type Resource Allocation

AU - Zhu, Qinyun

AU - Oh, Jae C

PY - 2019/1/15

Y1 - 2019/1/15

N2 - As autonomous robots are becoming a reality, we discover new challenges in coordination among these robots. We present a unique new problem that each robot makes decisions in achieving tasks that require multiple robots with different capabilities. Fair resource allocation is essential to ensure that all resource requesters acquire adequate robot resources and accomplish their tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters requiring heterogeneous robots with different capabilities to accomplish tasks. In particular, this work focuses on systems of single-tasking robots with multi-robot tasks (STR-MRT). In STR-MRT, the capability of a robot is the resource for accomplishing a specific task. In this problem, 1) each robot can perform only one task at a time, 2) tasks are divisible, and 3) accomplishing each task requires resources from one or more robots. We model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot strategically selects one resource requester. Then a consensus-based algorithm conducts formation of a robotic team for each task. We leverage the Dominant Resource Fairness (DRF) and Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the common Q-learning.

AB - As autonomous robots are becoming a reality, we discover new challenges in coordination among these robots. We present a unique new problem that each robot makes decisions in achieving tasks that require multiple robots with different capabilities. Fair resource allocation is essential to ensure that all resource requesters acquire adequate robot resources and accomplish their tasks. We propose solutions to the fairness problem in multi-type resource allocation for multi-robot systems that have multiple resource requesters requiring heterogeneous robots with different capabilities to accomplish tasks. In particular, this work focuses on systems of single-tasking robots with multi-robot tasks (STR-MRT). In STR-MRT, the capability of a robot is the resource for accomplishing a specific task. In this problem, 1) each robot can perform only one task at a time, 2) tasks are divisible, and 3) accomplishing each task requires resources from one or more robots. We model the decentralized resource allocation in STR-MRT as a coordination game between the robots. Each robot strategically selects one resource requester. Then a consensus-based algorithm conducts formation of a robotic team for each task. We leverage the Dominant Resource Fairness (DRF) and Deep Q-learning Network (DQN) to support requester selection. The results suggest that the DQN outperforms the common Q-learning.

KW - Multi-robot system

KW - Reinforcement learning

KW - Resource allocation

UR - http://www.scopus.com/inward/record.url?scp=85062229788&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062229788&partnerID=8YFLogxK

U2 - 10.1109/ICMLA.2018.00075

DO - 10.1109/ICMLA.2018.00075

M3 - Conference contribution

T3 - Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018

SP - 460

EP - 466

BT - Proceedings - 17th IEEE International Conference on Machine Learning and Applications, ICMLA 2018

A2 - Wani, M. Arif

A2 - Sayed-Mouchaweh, Moamar

A2 - Lughofer, Edwin

A2 - Gama, Joao

A2 - Kantardzic, Mehmed

PB - Institute of Electrical and Electronics Engineers Inc.

ER -