In an effort to document work-in-progress, many construction companies take hundreds of images on their project sites on a daily basis. These images together with 4D BIM can serve as a great resource for analyzing progress deviations. To facilitate image-vs.-BIM comparison, several methods have been introduced that tie in all images together in 3D using standard Structure-from-Motion (SfM) procedures. The resulting point clouds are then superimposed with the 4D BIM, resulting in back-projection of BIM on all images that were successfully registered through the SfM procedure. However, often site images exhibit wide baselines and thus are not successfully registered with BIM. To address current limitations, this paper presents a method together with experimental results that leverages BIM as a priori to initiate the SfM procedure. It is shown that by interactively guiding BIM into one or a few images that have significant overlap with the rest, the proposed BIM-assisted SfM procedure results in more complete point clouds and also generate more accurate BIM overlays on site images.