Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning

Yilan Li, Hongjia Li, Zhe Li, Haowen Fang, Amit K. Sanyal, Yanzhi Wang, Qinru Qiu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728131979
DOIs
StatePublished - Aug 2019
Event25th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019 - Hangzhou, China
Duration: Aug 18 2019Aug 21 2019

Publication series

NameProceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019

Conference

Conference25th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019
CountryChina
CityHangzhou
Period8/18/198/21/19

    Fingerprint

Keywords

  • Actor-critic algorithm
  • Continuous trajectory tracking
  • Deep reinforcement learning
  • Unmanned aerial vehicles

ASJC Scopus subject areas

  • Artificial Intelligence
  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Safety, Risk, Reliability and Quality
  • Instrumentation

Cite this

Li, Y., Li, H., Li, Z., Fang, H., Sanyal, A. K., Wang, Y., & Qiu, Q. (2019). Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. In Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019 [8864571] (Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RTCSA.2019.8864571