TY - JOUR
T1 - Real-time three-dimensional histology-like imaging by label-free nonlinear optical microscopy
AU - Sun, Yi
AU - You, Sixian
AU - Du, Xiaoxi
AU - Spaulding, Allison
AU - George Liu, Z.
AU - Chaney, Eric J.
AU - Spillman, Darold R.
AU - Marjanovic, Marina
AU - Tu, Haohua
AU - Boppart, Stephen A.
N1 - We also thank the study coordinators and Research Office at Carle Foundation Hospital. Additional information can be found at http://biophotonics.illinois.edu. Funding: This research was supported in part by grants from the National Institutes of Health (R01CA213149, R01EB023232) and the National Science Foundation (CBET 18-41539). Allison Spaulding received support from the NSF REU Program when conducting this study.
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi. org/10.21037/qims-20-381). YS, SY, EJC, DRS, MM, SAB report grants from the National Institutes of Health during the conduct of this study; SAB also reports grants from the National Science Foundation during the conduct of the study; HT reports grants from LiveBx, LLC outside the submitted work; in addition, HT has a patent Molecular Imaging Biomarkers issued, and a patent Clipping-Assisted Dual-Fluorophore Sensing pending; SY has a patent No. 10,445,880 issued. The other authors have no conflicts of interest to declare.
PY - 2020/11
Y1 - 2020/11
N2 - Background: The current gold-standard formalin-fixed and paraffin-embedded (FFPE) histology typically requires several days for tissue fixing, embedding, sectioning, and staining to provide depth-resolved tissue feature visualization. During these time- and labor- intense processes, the in vivo tissue dynamics and three-dimensional structures undergo inevitable loss and distortion. Methods: A simultaneous label-free autofluorescence multiharmonic (SLAM) microscope is used to conduct ex vivo and in vivo imaging of fresh human and rat tissues. Four nonlinear optical imaging modalities are integrated into this SLAM microscope, including second harmonic generation (SHG), two-photon fluorescence (2PF), third harmonic generation (THG), and three-photon fluorescence (3PF). By imaging fresh human and rat tissues without any tissue processing or staining, various biological tissue features are effectively visualized by one or multiple imaging modalities of the SLAM microscope. In particular, some of the most essential features in hematoxylin and eosin (H&E)-stained histology, such as collagen fibers and nuclei, are also present in the SLAM microscopy images with good contrast. Because nuclei are evident from negative contrast, the nuclei are segmented from the SLAM images using deep learning. Finally, a color-transforming algorithm is developed to convert the grey-scale images acquired by the SLAM microscope to the virtually H&E-stained histology-like images. The converted histology-like images are later compared with the FFPE histology at the same tissue site. In addition, the nuclear-to-cytoplasmic ratios (N/C ratios) of the cells in the SLAM image are quantified, which has diagnostic relevance for cancer. Results: Various histological correlations are identified with high similarities for the color-converted histology-like SLAM microscopy images. By applying the color transforming algorithm on real-time SLAM image sequences and 3D SLAM image stacks, we report, for the first time and to the best our knowledge, real-time 3D histology-like imaging. Furthermore, the quantified N/C ratio of the cells in the SLAM image are overlaid on the converted histology-like image as a new image contrast. Conclusions: We demonstrated real-time 3D histology-like imaging and its future potential using SLAM microscopy aided by color remapping and deep-learning-based feature segmentation.
AB - Background: The current gold-standard formalin-fixed and paraffin-embedded (FFPE) histology typically requires several days for tissue fixing, embedding, sectioning, and staining to provide depth-resolved tissue feature visualization. During these time- and labor- intense processes, the in vivo tissue dynamics and three-dimensional structures undergo inevitable loss and distortion. Methods: A simultaneous label-free autofluorescence multiharmonic (SLAM) microscope is used to conduct ex vivo and in vivo imaging of fresh human and rat tissues. Four nonlinear optical imaging modalities are integrated into this SLAM microscope, including second harmonic generation (SHG), two-photon fluorescence (2PF), third harmonic generation (THG), and three-photon fluorescence (3PF). By imaging fresh human and rat tissues without any tissue processing or staining, various biological tissue features are effectively visualized by one or multiple imaging modalities of the SLAM microscope. In particular, some of the most essential features in hematoxylin and eosin (H&E)-stained histology, such as collagen fibers and nuclei, are also present in the SLAM microscopy images with good contrast. Because nuclei are evident from negative contrast, the nuclei are segmented from the SLAM images using deep learning. Finally, a color-transforming algorithm is developed to convert the grey-scale images acquired by the SLAM microscope to the virtually H&E-stained histology-like images. The converted histology-like images are later compared with the FFPE histology at the same tissue site. In addition, the nuclear-to-cytoplasmic ratios (N/C ratios) of the cells in the SLAM image are quantified, which has diagnostic relevance for cancer. Results: Various histological correlations are identified with high similarities for the color-converted histology-like SLAM microscopy images. By applying the color transforming algorithm on real-time SLAM image sequences and 3D SLAM image stacks, we report, for the first time and to the best our knowledge, real-time 3D histology-like imaging. Furthermore, the quantified N/C ratio of the cells in the SLAM image are overlaid on the converted histology-like image as a new image contrast. Conclusions: We demonstrated real-time 3D histology-like imaging and its future potential using SLAM microscopy aided by color remapping and deep-learning-based feature segmentation.
KW - Cancer diagnostics
KW - Deep learning
KW - Digital pathology
KW - Digital staining
KW - Label-free
KW - Virtual histology
UR - http://www.scopus.com/inward/record.url?scp=85092125831&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85092125831&partnerID=8YFLogxK
U2 - 10.21037/QIMS-20-381
DO - 10.21037/QIMS-20-381
M3 - Article
AN - SCOPUS:85092125831
SN - 2223-4292
VL - 10
SP - 2177
EP - 2190
JO - Quantitative Imaging in Medicine and Surgery
JF - Quantitative Imaging in Medicine and Surgery
IS - 11
ER -