Deep Learning Based Obstacle Detection and Classification with Portable Uncalibrated Patterned Light

Maria Cornacchia, Burak Kakillioglu, Yu Zheng, Senem Velipasalar

Research output: Contribution to journalArticle

1 Scopus citations


Autonomous navigation and obstacle avoidance systems are critically relevant and important for visually impaired people, assisted driving applications, autonomous robots, and so on. Even though there has been significant amount of work on obstacle detection and avoidance using LiDAR and camera data, there has not been much effort focusing on providing a lightweight, cost-conscious, energy-efficient, reliable and portable solution for the visually impaired. We propose a new method for autonomous obstacle detection and classification, which incorporates a different and novel type of sensor, namely patterned light field, with camera. The proposed device is small in size, easily carried, as well as low cost. The grid, projected by the patterned light source, is apparent and differentiable as the sensing system is hand-carried in natural indoor and outdoor environments over and towards different types of obstacles. Our proposed approach exploits these patterns, without calibration, by employing deep learning techniques, including a Convolutional Neural Network (CNN)-based classification on individual frames. We further refine our approach by smoothing the frame-based classifications over multiple frames using Long Short Term Memory (LSTM) units. The proposed method provides very promising results with overall detection and classification accuracies of 98.37% for the binary case, as well as 95.97% and 92.62% for two different multi-class scenarios. These results represent the average number of sequences correctly detected and classified and were obtained on a sequence based analysis of over 120 sequences from four different users.

Original languageEnglish (US)
JournalIEEE Sensors Journal
StateAccepted/In press - Aug 16 2018



  • activity classification
  • convolutional neural network
  • long-short term memory
  • obstacle classification
  • obstacle detection
  • structured light

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering

Cite this