Citation

Alldieck T, Bahnsen CH, Moeslund TB. Sensors (Basel) 2016; 16(11): s16111947.

Affiliation

Visual Analysis of People Lab, Aalborg University, 9000 Aalborg, Denmark. thiemoall@gmail.com.

Copyright

(Copyright © 2016, MDPI: Multidisciplinary Digital Publishing Institute)

DOI

10.3390/s16111947

PMID

27869730

Abstract

In order to enable a robust 24-h monitoring of traffic under changing environmental conditions, it is beneficial to observe the traffic scene using several sensors, preferably from different modalities. To fully benefit from multi-modal sensor output, however, one must fuse the data. This paper introduces a new approach for fusing color RGB and thermal video streams by using not only the information from the videos themselves, but also the available contextual information of a scene. The contextual information is used to judge the quality of a particular modality and guides the fusion of two parallel segmentation pipelines of the RGB and thermal video streams. The potential of the proposed context-aware fusion is demonstrated by extensive tests of quantitative and qualitative characteristics on existing and novel video datasets and benchmarked against competing approaches to multi-modal fusion.

Language: en