TY - JOUR
T1 - Deep Learning-Based Obstacle Detection and Classification with Portable Uncalibrated Patterned Light
AU - Cornacchia, Maria
AU - Kakillioglu, Burak
AU - Zheng, Yu
AU - Velipasalar, Senem
N1 - Funding Information:
Manuscript received June 1, 2018; revised July 20, 2018; accepted July 29, 2018. Date of publication August 17, 2018; date of current version September 25, 2018. This work was supported by the National Science Foundation under Grants 1302559 and 1739748. The associate editor coordinating the review of this paper and approving it for publication was Dr. Edward Sazonov. (Corresponding author: Maria Cornacchia.) The authors are with the Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY 13244 USA (e-mail: mlscalzo@syr.edu; bkakilli@syr.edu; yzheng04@syr.edu; svelipas@syr.edu). Digital Object Identifier 10.1109/JSEN.2018.2865306
Publisher Copyright:
© 2001-2012 IEEE.
PY - 2018/10/15
Y1 - 2018/10/15
N2 - Autonomous navigation and obstacle avoidance systems are critically relevant and important for visually impaired people, assisted driving applications, and autonomous robots. Even though there has been significant amount of work on obstacle detection and avoidance using LiDAR and camera data, there has not been much effort focusing on providing a lightweight, cost conscious, energy efficient, reliable, and portable solution for the visually impaired. We propose a new method for autonomous obstacle detection and classification, which incorporates a different and novel type of sensor, namely, patterned light field, with camera. The proposed device is small in size, easily carried, as well as low cost. The grid, projected by the patterned light source, is apparent and differentiable as the sensing system is hand carried in natural indoor and outdoor environments over and toward different types of obstacles. Our proposed approach exploits these patterns, without calibration, by employing deep learning techniques, including a convolutional neural network-based classification on individual frames. We further refine our approach by smoothing the frame-based classifications over multiple frames using long short-term memory units. The proposed method provides very promising results with overall detection and classification accuracies of 98.37% for the binary case as well as 95.97% and 92.62% for two different multi-class scenarios. These results represent the average number of sequences correctly detected and classified and were obtained on a sequence-based analysis of over 120 sequences from four different users.
AB - Autonomous navigation and obstacle avoidance systems are critically relevant and important for visually impaired people, assisted driving applications, and autonomous robots. Even though there has been significant amount of work on obstacle detection and avoidance using LiDAR and camera data, there has not been much effort focusing on providing a lightweight, cost conscious, energy efficient, reliable, and portable solution for the visually impaired. We propose a new method for autonomous obstacle detection and classification, which incorporates a different and novel type of sensor, namely, patterned light field, with camera. The proposed device is small in size, easily carried, as well as low cost. The grid, projected by the patterned light source, is apparent and differentiable as the sensing system is hand carried in natural indoor and outdoor environments over and toward different types of obstacles. Our proposed approach exploits these patterns, without calibration, by employing deep learning techniques, including a convolutional neural network-based classification on individual frames. We further refine our approach by smoothing the frame-based classifications over multiple frames using long short-term memory units. The proposed method provides very promising results with overall detection and classification accuracies of 98.37% for the binary case as well as 95.97% and 92.62% for two different multi-class scenarios. These results represent the average number of sequences correctly detected and classified and were obtained on a sequence-based analysis of over 120 sequences from four different users.
KW - Structured light
KW - activity classification
KW - convolutional neural network
KW - long-short term memory
KW - obstacle classification
KW - obstacle detection
UR - http://www.scopus.com/inward/record.url?scp=85051761855&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85051761855&partnerID=8YFLogxK
U2 - 10.1109/JSEN.2018.2865306
DO - 10.1109/JSEN.2018.2865306
M3 - Article
AN - SCOPUS:85051761855
SN - 1530-437X
VL - 18
SP - 8416
EP - 8425
JO - IEEE Sensors Journal
JF - IEEE Sensors Journal
IS - 20
M1 - 8438986
ER -