Evaluating the Capacity of Deep Generative Models to Reproduce Measurable High-order Spatial Arrangements in Diagnostic Images

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Given the recent interest in the role of deep generative models (DGM) in medical imaging pipelines, it is imperative to evaluate the capacity of such models to generate medically accurate images. Popular methods of evaluation of natural images generated using generative adversarial networks (GANs), a type of DGM, are often applied to medical data. Such methods are insufficient to evaluate anatomical realism, representations of which include high-order spatial information. To our knowledge, no test exists for the faithful replication of spatial statistics beyond the second-order. In this work, purposefully designed stochastic object models (SOMs) are proposed to encode predetermined rules governing the prevalence of features within single images, thus encoding known high-order spatial information within each realization. These SOMs are independent of the network architecture being tested and can also be applied to any new architecture that may be proposed. Two popular GANs are trained on these SOM datasets and the generated images are tested for the encoded statistics. It is observed that although ensemble statistics might be well replicated, this is not necessarily true for realization i.e., per-image statistics. Thus, GAN-generated images might not be ready for clinical use. With the proposed SOMs, the rate of image errors and the rate of feature malformation can be quantified for any architecture, while providing one measure of GAN utility in a diagnostic scenario.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2022
Subtitle of host publicationImage Processing
EditorsOlivier Colliot, Ivana Isgum, Bennett A. Landman, Murray H. Loew
PublisherSPIE
ISBN (Electronic)9781510649392
DOIs
StatePublished - 2022
Externally publishedYes
EventMedical Imaging 2022: Image Processing - Virtual, Online
Duration: Mar 21 2021Mar 27 2021

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume12032
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2022: Image Processing
CityVirtual, Online
Period3/21/213/27/21

Keywords

  • deep generative models
  • generative adversarial network (GAN) evaluation
  • high-order statistics
  • stochastic object models

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Fingerprint

Dive into the research topics of 'Evaluating the Capacity of Deep Generative Models to Reproduce Measurable High-order Spatial Arrangements in Diagnostic Images'. Together they form a unique fingerprint.

Cite this