TY - JOUR
T1 - Deepfake Histologic Images for Enhancing Digital Pathology
AU - Falahkheirkhah, Kianoush
AU - Tiwari, Saumya
AU - Yeh, Kevin
AU - Gupta, Sounak
AU - Herrera-Hernandez, Loren
AU - McCarthy, Michael R.
AU - Jimenez, Rafael E.
AU - Cheville, John C.
AU - Bhargava, Rohit
N1 - Publisher Copyright:
© 2022 United States & Canadian Academy of Pathology
PY - 2023/1
Y1 - 2023/1
N2 - A pathologist's optical microscopic examination of thinly cut, stained tissue on glass slides prepared from a formalin-fixed paraffin-embedded tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of pathologists is dependent on their direct experience with common and rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task, and artificially synthesized histologic images can prove greatly beneficial. In this study, we present an approach to not only generate histologic images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves developing a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting the diagnostic ability of machine learning methods and their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data when training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images, and their analyses showed a similar level of interobserver agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the morphology of the complex microenvironment in such tissues can be reproduced. Finally, we present the ability for a user to generate deepfake histologic images using a simple markup of sematic labels.
AB - A pathologist's optical microscopic examination of thinly cut, stained tissue on glass slides prepared from a formalin-fixed paraffin-embedded tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of pathologists is dependent on their direct experience with common and rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task, and artificially synthesized histologic images can prove greatly beneficial. In this study, we present an approach to not only generate histologic images that reproduce the diagnostic morphologic features of common disease but also provide a user ability to generate new and rare morphologies. Our approach involves developing a generative adversarial network model that synthesizes pathology images constrained by class labels. We investigated the ability of this framework in synthesizing realistic prostate and colon tissue images and assessed the utility of these images in augmenting the diagnostic ability of machine learning methods and their usability by a panel of experienced anatomic pathologists. Synthetic data generated by our framework performed similar to real data when training a deep learning model for diagnosis. Pathologists were not able to distinguish between real and synthetic images, and their analyses showed a similar level of interobserver agreement for prostate cancer grading. We extended the approach to significantly more complex images from colon biopsies and showed that the morphology of the complex microenvironment in such tissues can be reproduced. Finally, we present the ability for a user to generate deepfake histologic images using a simple markup of sematic labels.
KW - computer vision
KW - deep learning
KW - deepfake pathology
KW - digital pathology synthetic data
UR - http://www.scopus.com/inward/record.url?scp=85147460992&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85147460992&partnerID=8YFLogxK
U2 - 10.1016/j.labinv.2022.100006
DO - 10.1016/j.labinv.2022.100006
M3 - Article
C2 - 36748189
AN - SCOPUS:85147460992
SN - 0023-6837
VL - 103
JO - Laboratory Investigation
JF - Laboratory Investigation
IS - 1
M1 - 100006
ER -