TY - GEN
T1 - Evaluating the Capacity of Deep Generative Models to Reproduce Measurable High-order Spatial Arrangements in Diagnostic Images
AU - Deshpande, Rucha
AU - Anastasio, Mark A.
AU - Brooks, Frank J.
N1 - This work was supported in part by NIH awards EB020604, EB023045, NS102213, EB028652 and the Imaging Sciences Pathway (5T32EB01485505). This work utilized resources supported by the National Science Foundations Major Research Instrumentation program, grant 1725729, as well as the University of Illinois at Urbana-Champaign.6
PY - 2022
Y1 - 2022
N2 - Given the recent interest in the role of deep generative models (DGM) in medical imaging pipelines, it is imperative to evaluate the capacity of such models to generate medically accurate images. Popular methods of evaluation of natural images generated using generative adversarial networks (GANs), a type of DGM, are often applied to medical data. Such methods are insufficient to evaluate anatomical realism, representations of which include high-order spatial information. To our knowledge, no test exists for the faithful replication of spatial statistics beyond the second-order. In this work, purposefully designed stochastic object models (SOMs) are proposed to encode predetermined rules governing the prevalence of features within single images, thus encoding known high-order spatial information within each realization. These SOMs are independent of the network architecture being tested and can also be applied to any new architecture that may be proposed. Two popular GANs are trained on these SOM datasets and the generated images are tested for the encoded statistics. It is observed that although ensemble statistics might be well replicated, this is not necessarily true for realization i.e., per-image statistics. Thus, GAN-generated images might not be ready for clinical use. With the proposed SOMs, the rate of image errors and the rate of feature malformation can be quantified for any architecture, while providing one measure of GAN utility in a diagnostic scenario.
AB - Given the recent interest in the role of deep generative models (DGM) in medical imaging pipelines, it is imperative to evaluate the capacity of such models to generate medically accurate images. Popular methods of evaluation of natural images generated using generative adversarial networks (GANs), a type of DGM, are often applied to medical data. Such methods are insufficient to evaluate anatomical realism, representations of which include high-order spatial information. To our knowledge, no test exists for the faithful replication of spatial statistics beyond the second-order. In this work, purposefully designed stochastic object models (SOMs) are proposed to encode predetermined rules governing the prevalence of features within single images, thus encoding known high-order spatial information within each realization. These SOMs are independent of the network architecture being tested and can also be applied to any new architecture that may be proposed. Two popular GANs are trained on these SOM datasets and the generated images are tested for the encoded statistics. It is observed that although ensemble statistics might be well replicated, this is not necessarily true for realization i.e., per-image statistics. Thus, GAN-generated images might not be ready for clinical use. With the proposed SOMs, the rate of image errors and the rate of feature malformation can be quantified for any architecture, while providing one measure of GAN utility in a diagnostic scenario.
KW - deep generative models
KW - generative adversarial network (GAN) evaluation
KW - high-order statistics
KW - stochastic object models
UR - http://www.scopus.com/inward/record.url?scp=85131920744&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85131920744&partnerID=8YFLogxK
U2 - 10.1117/12.2611807
DO - 10.1117/12.2611807
M3 - Conference contribution
AN - SCOPUS:85131920744
T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE
BT - Medical Imaging 2022
A2 - Colliot, Olivier
A2 - Isgum, Ivana
A2 - Landman, Bennett A.
A2 - Loew, Murray H.
PB - SPIE
T2 - Medical Imaging 2022: Image Processing
Y2 - 21 March 2021 through 27 March 2021
ER -