Deep Learning-Based Obstacle Detection and Classification with Portable Uncalibrated Patterned Light

Maria Cornacchia, Burak Kakillioglu, Yu Zheng, Senem Velipasalar

Research output: Contribution to journalArticlepeer-review

26 Scopus citations


Autonomous navigation and obstacle avoidance systems are critically relevant and important for visually impaired people, assisted driving applications, and autonomous robots. Even though there has been significant amount of work on obstacle detection and avoidance using LiDAR and camera data, there has not been much effort focusing on providing a lightweight, cost conscious, energy efficient, reliable, and portable solution for the visually impaired. We propose a new method for autonomous obstacle detection and classification, which incorporates a different and novel type of sensor, namely, patterned light field, with camera. The proposed device is small in size, easily carried, as well as low cost. The grid, projected by the patterned light source, is apparent and differentiable as the sensing system is hand carried in natural indoor and outdoor environments over and toward different types of obstacles. Our proposed approach exploits these patterns, without calibration, by employing deep learning techniques, including a convolutional neural network-based classification on individual frames. We further refine our approach by smoothing the frame-based classifications over multiple frames using long short-term memory units. The proposed method provides very promising results with overall detection and classification accuracies of 98.37% for the binary case as well as 95.97% and 92.62% for two different multi-class scenarios. These results represent the average number of sequences correctly detected and classified and were obtained on a sequence-based analysis of over 120 sequences from four different users.

Original languageEnglish (US)
Article number8438986
Pages (from-to)8416-8425
Number of pages10
JournalIEEE Sensors Journal
Issue number20
StatePublished - Oct 15 2018


  • Structured light
  • activity classification
  • convolutional neural network
  • long-short term memory
  • obstacle classification
  • obstacle detection

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering


Dive into the research topics of 'Deep Learning-Based Obstacle Detection and Classification with Portable Uncalibrated Patterned Light'. Together they form a unique fingerprint.

Cite this