Multi-Agent Cooperative Games Using Belief Map Assisted Training

Chen Luo, Qinwei Huang, Alex B. Wu, Simon Khan, Hai Li, Qinru Qiu

Research output: Chapter in Book/Entry/PoemConference contribution


In a multi-agent system, agents share their local observations to gain global situational awareness for decision making and collaboration using a message passing system. When to send a message, how to encode a message, and how to leverage the received messages directly affect the effectiveness of the collaboration among agents. When training a multi-agent cooperative game using reinforcement learning (RL), the message passing system needs to be optimized together with the agent policies. This consequently increases the model's complexity and poses significant challenges to the convergence and performance of learning. To address this issue, we propose the Belief-map Assisted Multi-agent System (BAMS), which leverages a neuro-symbolic belief map to enhance training. The belief map decodes the agent's hidden state to provide a symbolic representation of the agent's understanding of the environment and other agents' status. The simplicity of symbolic representation allows the gathering and comparison of the ground truth information with the belief, which provides an additional channel of feedback for the learning. Compared to the sporadic and delayed feedback coming from the reward in RL, the feedback from the belief map is more consistent and reliable. Agents using BAMS can learn a more effective message passing network to better understand each other, resulting in better performance in the game. We evaluate BAMS's performance in a cooperative predator and prey game with varying levels of map complexity and compare it to previous multi-agent message passing models. The simulation results showed that BAMS reduced training epochs by 66%, and agents who apply the BAMS model completed the game with 34.62% fewer steps on average.

Original languageEnglish (US)
Title of host publicationECAI 2023 - 26th European Conference on Artificial Intelligence, including 12th Conference on Prestigious Applications of Intelligent Systems, PAIS 2023 - Proceedings
EditorsKobi Gal, Kobi Gal, Ann Nowe, Grzegorz J. Nalepa, Roy Fairstein, Roxana Radulescu
PublisherIOS Press BV
Number of pages8
ISBN (Electronic)9781643684369
StatePublished - Sep 28 2023
Event26th European Conference on Artificial Intelligence, ECAI 2023 - Krakow, Poland
Duration: Sep 30 2023Oct 4 2023

Publication series

NameFrontiers in Artificial Intelligence and Applications
ISSN (Print)0922-6389
ISSN (Electronic)1879-8314


Conference26th European Conference on Artificial Intelligence, ECAI 2023

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Multi-Agent Cooperative Games Using Belief Map Assisted Training'. Together they form a unique fingerprint.

Cite this