Heterogeneous Sensor Fusion with out of Sync Data

This paper tackles sensor fusion when data from different sensors are out of sync - observations may be sampled asynchronously and there may be no timing information at the fusion center for observations from some sensors. This often happens when sensors are heterogeneous in nature hence may employ different and even time varying sampling cycles. Furthermore, observations may need to go through wireless channels when tagging each observation with a timestamp may be too expensive. Treating the timing information in out of sync data as a hidden variable, this paper employs the expectation-maximization (EM) algorithm to recover the posterior distribution of the timing information. Data fusion utilizing the out of sync data can subsequently be carried out. Using a simple two-sensor off-line state estimation experiment where one sensor's data is available but is out of sync at the other sensor, we demonstrate that the performance of the EM based approach improves on the fusion performance when the out of sync data is discarded. Extensions to high dimensional observations to systems involving multiple out of sync data streams, and to online state tracking with out of sync data are discussed.