Deep Learning-Based Obstacle Detection and Classification with Portable Uncalibrated Patterned Light

Autonomous navigation and obstacle avoidance systems are critically relevant and important for visually impaired people, assisted driving applications, and autonomous robots. Even though there has been significant amount of work on obstacle detection and avoidance using LiDAR and camera data, there has not been much effort focusing on providing a lightweight, cost conscious, energy efficient, reliable, and portable solution for the visually impaired. We propose a new method for autonomous obstacle detection and classification, which incorporates a different and novel type of sensor, namely, patterned light field, with camera. The proposed device is small in size, easily carried, as well as low cost. The grid, projected by the patterned light source, is apparent and differentiable as the sensing system is hand carried in natural indoor and outdoor environments over and toward different types of obstacles. Our proposed approach exploits these patterns, without calibration, by employing deep learning techniques, including a convolutional neural network-based classification on individual frames. We further refine our approach by smoothing the frame-based classifications over multiple frames using long short-term memory units. The proposed method provides very promising results with overall detection and classification accuracies of 98.37% for the binary case as well as 95.97% and 92.62% for two different multi-class scenarios. These results represent the average number of sequences correctly detected and classified and were obtained on a sequence-based analysis of over 120 sequences from four different users.