Improving Situational Awareness of the As-Is Building Conditions through Multi-Modal Sensing and Analytics Using Thermal Camera-Equipped Smartphones

The demand for improving energy efficiency of existing buildings is rising, hence many construction firms are shifting their efforts from constructing new buildings to rehabilitating the existing ones. Currently, visual inspections are conducted for building envelopes or mechanical systems, and auditors try to collect energy-related contextual information to examine if their performance is maintained as expected by the design. However, occupants still continue to face challenges in identifying and being aware of 'where' and 'what' energy problems are emerging or likely to emerge in their buildings for taking timely corrective actions. This paper proposes a new ubiquitous context-aware application for building energy diagnostics using thermal camera-equipped smartphones. The proposed mobile platform consists of: (1) thermography-based visual sensing and analytics for characterizing building energy problems; and (2) infrastructure-free (i.e., available in GPS or WLAN-denied environments) remote 3D localization of the problems by leveraging multi-modal visual and motion sensory data from smartphones. Experimental results on an instructional facility promise that the proposed method offers significant potentials for recording the location and severity of building energy problems. As an efficient practice in energy informatics, such near real-time feedback will improve occupants' situational awareness of the as-is building conditions and facilitate associated sustainable retrofits.