Learning-Based UAV Trajectory Optimization with Collision Avoidance and Connectivity Constraints

Unmanned aerial vehicles (UAVs) are expected to be an integral part of wireless networks, and determining collision-free trajectories for multiple UAVs while satisfying requirements of connectivity with ground base stations (GBSs) is a challenging task. In this paper, we consider non-cooperative multi-UAV scenarios, in which multiple UAVs need to fly from initial locations to destinations, while satisfying collision avoidance, wireless connectivity, and kinematic constraints. We aim to find trajectories for the UAVs with the goal to minimize their mission completion time. We first formulate the multi-UAV trajectory optimization problem as a sequential decision making problem. We, then, propose a decentralized deep reinforcement learning approach to solve the problem. More specifically, a value network is developed to obtain values given the agent's joint state (including the agent's information, the nearby agents' observable information, and the locations of the nearby GBSs). A signal-to-interference-plus-noise ratio (SINR)-prediction neural network is also designed, using accumulated SINR measurements obtained when interacting with the cellular network, to map the GBSs' locations into the SINR levels in order to predict the UAV's SINR. Numerical results show that with the value network and SINR-prediction network, real-time navigation for multi-UAVs can be efficiently performed in various environments with high success rate.