Prior image-based medical image reconstruction using a style-based generative adversarial network

Varun A. Kelkar, Mark A. Anastasio

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Computed medical imaging systems require a computational reconstruction procedure for image formation. In order to recover a useful estimate of the object to-be-imaged when the recorded measurements are incomplete, prior knowledge about the nature of object must be utilized. In order to improve the conditioning of an ill-posed imaging inverse problem, deep learning approaches are being actively investigated for better representing object priors and constraints. This work proposes to use a style-based generative adversarial network (StyleGAN) to constrain an image reconstruction problem in the case where additional information in the form of a prior image of the sought-after object is available. An optimization problem is formulated in the intermediate latent-space of a StyleGAN, that is disentangled with respect to meaningful image attributes or "styles", such as the contrast used in magnetic resonance imaging (MRI). Discrepancy between the sought-after and prior images is measured in the disentangled latent-space, and is used to regularize the inverse problem in the form of constraints on specific styles of the disentangled latent-space. A stylized numerical study inspired by MR imaging is designed, where the sought-after and the prior image are structurally similar, but belong to different contrast mechanisms. The presented numerical studies demonstrate the superiority of the proposed approach as compared to classical approaches in the form of traditional metrics.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2022
Subtitle of host publicationPhysics of Medical Imaging
EditorsWei Zhao, Lifeng Yu
ISBN (Electronic)9781510649378
StatePublished - 2022
EventMedical Imaging 2022: Physics of Medical Imaging - Virtual, Online
Duration: Mar 21 2022Mar 27 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2022: Physics of Medical Imaging
CityVirtual, Online


  • Inverse problems
  • compressive sensing
  • generative adversarial network
  • reference-based MRI

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Prior image-based medical image reconstruction using a style-based generative adversarial network'. Together they form a unique fingerprint.

Cite this