Image-based localization and content authoring in structure-from-motion point cloud models for real-time field reporting applications

Recent research efforts to improve information management on construction sites have focused on devising mobile augmented reality methods to provide on-demand access to project information and facilitate field reporting. To track position and orientation of the users, prior research focused on radio frequency based location tracking methods, fiduciary markers, or parallel visual tracking and mapping techniques. These methods have potential in providing fast information access to field personnel, yet their application heavily depends on preinstalled infrastructure, may suffer from accumulated drift errors, and in the case of visual methods do not typically scale well. Furthermore, these methods do not provide content authoring capability in three dimensions, which is necessary for reciprocal application of augmented reality for field reporting and asynchronous personnel-to-personnel communications. To overcome these limitations, this paper presents a scalable markerless and infrastructure-independent mobile augmented reality system for commodity smartphones and tablets, with client-server architecture and cloud computing basis, to facilitate both accessing to project information and conducting field reporting in three dimensions. The method relies on structure-from-motion point clouds generated from site images and/or videos to enable image-based localization and single-image based content authoring in three dimensions. By using mobile client devices with the proposed method, onsite personnel can associate field reports with two-dimensional (2D) patches from mobile images and store that information directly in the underlying three-dimensional (3D) point cloud model. In addition, the client-server architecture of the proposed system allows an onsite access to most updated project information and field reports as they are being documented on the server. The presented architecture also handles multiple onsite information retrieval and field reporting requests simultaneously. The overall system is validated on several indoor/outdoor work areas of three real-world construction projects with varying degrees of occlusions and lighting conditions. The experimental results show the potential of the proposed method in prompt field reporting in three dimensions, minimizing the time field personnel spent on accessing project information, and communicating with others to resolve an issue/conflict on the jobsite within 1-3 min.