Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning

Yilan Li, Hongjia Li, Zhe Li, Haowen Fang, Amit K. Sanyal, Yanzhi Wang, Qinru Qiu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

Original languageEnglish (US)
Title of host publicationProceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728131979
DOIs
StatePublished - Aug 2019
Event25th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019 - Hangzhou, China
Duration: Aug 18 2019Aug 21 2019

Publication series

NameProceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019

Conference

Conference25th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019
CountryChina
CityHangzhou
Period8/18/198/21/19

Fingerprint

pilotless aircraft
Reinforcement learning
Unmanned aerial vehicles (UAV)
reinforcement
learning
Trajectories
trajectories
fixed wings
trajectory control
position errors
field-programmable gate arrays
control theory
Fixed wings
torque
hardware
Degrees of freedom (mechanics)
Real time control
degrees of freedom
adjusting
Control theory

Keywords

  • Actor-critic algorithm
  • Continuous trajectory tracking
  • Deep reinforcement learning
  • Unmanned aerial vehicles

ASJC Scopus subject areas

  • Artificial Intelligence
  • Hardware and Architecture
  • Electrical and Electronic Engineering
  • Safety, Risk, Reliability and Quality
  • Instrumentation

Cite this

Li, Y., Li, H., Li, Z., Fang, H., Sanyal, A. K., Wang, Y., & Qiu, Q. (2019). Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. In Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019 [8864571] (Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RTCSA.2019.8864571

Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. / Li, Yilan; Li, Hongjia; Li, Zhe; Fang, Haowen; Sanyal, Amit K.; Wang, Yanzhi; Qiu, Qinru.

Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8864571 (Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Li, Y, Li, H, Li, Z, Fang, H, Sanyal, AK, Wang, Y & Qiu, Q 2019, Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. in Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019., 8864571, Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019, Institute of Electrical and Electronics Engineers Inc., 25th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019, Hangzhou, China, 8/18/19. https://doi.org/10.1109/RTCSA.2019.8864571
Li Y, Li H, Li Z, Fang H, Sanyal AK, Wang Y et al. Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. In Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8864571. (Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019). https://doi.org/10.1109/RTCSA.2019.8864571
Li, Yilan ; Li, Hongjia ; Li, Zhe ; Fang, Haowen ; Sanyal, Amit K. ; Wang, Yanzhi ; Qiu, Qinru. / Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning. Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019).
@inproceedings{4033d0cacebf41608351d68d7f10857a,
title = "Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning",
abstract = "Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14{\%} less position error, 21.77{\%} less system power consumption and 9.23{\%} faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.",
keywords = "Actor-critic algorithm, Continuous trajectory tracking, Deep reinforcement learning, Unmanned aerial vehicles",
author = "Yilan Li and Hongjia Li and Zhe Li and Haowen Fang and Sanyal, {Amit K.} and Yanzhi Wang and Qinru Qiu",
year = "2019",
month = "8",
doi = "10.1109/RTCSA.2019.8864571",
language = "English (US)",
series = "Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019",

}

TY - GEN

T1 - Fast and accurate trajectory tracking for unmanned aerial vehicles based on deep reinforcement learning

AU - Li, Yilan

AU - Li, Hongjia

AU - Li, Zhe

AU - Fang, Haowen

AU - Sanyal, Amit K.

AU - Wang, Yanzhi

AU - Qiu, Qinru

PY - 2019/8

Y1 - 2019/8

N2 - Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

AB - Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

KW - Actor-critic algorithm

KW - Continuous trajectory tracking

KW - Deep reinforcement learning

KW - Unmanned aerial vehicles

UR - http://www.scopus.com/inward/record.url?scp=85074213008&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074213008&partnerID=8YFLogxK

U2 - 10.1109/RTCSA.2019.8864571

DO - 10.1109/RTCSA.2019.8864571

M3 - Conference contribution

AN - SCOPUS:85074213008

T3 - Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019

BT - Proceedings - 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications, RTCSA 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -