Towards video streaming analysis and sharing for multi-device interaction with lightweight DNNs

Yakun Huang, Hongru Zhao, Xiuquan Qiao, Jian Tang, Ling Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multi-device interaction has attracted a growing interest in both mobile communication industry and mobile computing research community as mobile devices enabled social media and social networking continue to blossom. However, due to the stringent low latency requirements and the complexity and intensity of computation, implementing efficient multi-device interaction for real-time video streaming analysis and sharing is still in its infancy. Unlike previous approaches that rely on high network bandwidth and high availability of cloud center with GPUs to support intensive computations for multi-device interaction and for improving the service experience, we propose MIRSA, a novel edge centric multi-device interaction framework with a lightweight end-to-end DNN for on-device visual odometry (VO) streaming analysis by leveraging edge computing optimizations with three main contributions. First, we design MIRSA to migrate computations from the cloud to the device side, reducing the high overhead for large transmission of video streaming while alleviating the server load of the cloud. Second, we design a lightweight VO network by utilizing temporal shift module to support on-device pose estimation. Third, we provide on-device resource-aware scheduling algorithm to optimize the task allocation. Extensive experiments show MIRSA provides real-time high quality pose estimation as an interactive service and outperforms baseline methods.

Original languageEnglish (US)
Title of host publicationINFOCOM 2021 - IEEE Conference on Computer Communications
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9780738112817
DOIs
StatePublished - May 10 2021
Externally publishedYes
Event40th IEEE Conference on Computer Communications, INFOCOM 2021 - Vancouver, Canada
Duration: May 10 2021May 13 2021

Publication series

NameProceedings - IEEE INFOCOM
Volume2021-May
ISSN (Print)0743-166X

Conference

Conference40th IEEE Conference on Computer Communications, INFOCOM 2021
Country/TerritoryCanada
CityVancouver
Period5/10/215/13/21

ASJC Scopus subject areas

  • Computer Science(all)
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Towards video streaming analysis and sharing for multi-device interaction with lightweight DNNs'. Together they form a unique fingerprint.

Cite this