Interferometric synthetic aperture microscopy (ISAM) reconstructs the scattering potential of a sample with spatially invariant resolution, based on the incident beam profile, the beam scan pattern, the physical model of light sample interaction, and subsequent light collection by the system. In practice, aberrations may influence the beam profile, particularly at higher NA, when ISAM is expected to provide maximum benefit over optical coherence microscopy. Thus it is of interest to determine the effects of aberrations on ISAM reconstructions. In this paper we present the forward model incorporating the effects of aberrations, which forms the basis for aberration correction in ISAM. Simulations and experimental results show that when operating far from focus, modest amounts of spherical aberration can introduce artifacts to the point-spread function, even at relatively low NA ~ 0.1-0.2. Further work will investigate computational methods to correct the effects of aberrations, i.e. to perform virtual adaptive optics.