TY - JOUR
T1 - Deep learning-based protocols to enhance infrared imaging systems
AU - Falahkheirkhah, Kianoush
AU - Yeh, Kevin
AU - Mittal, Shachi
AU - Pfister, Luke
AU - Bhargava, Rohit
N1 - Funding Information:
This work was supported by the National Institutes of Health via awards R01CA197516 and R01EB009745 .
Publisher Copyright:
© 2021
PY - 2021/10/15
Y1 - 2021/10/15
N2 - Infrared (IR) spectroscopic imaging provides both morphologic and chemical detail; however, obtaining this extensive spectral-spatial information requires the ability to rapidly record high-quality data. Discrete frequency infrared (DFIR) imaging using a point scanning microscope strikes a balance between data quality and acquisition speed that, in principle, can further be aided by computational methods. Here, we report a deep learning-based framework to complement the process of data acquisition and information extraction. First, we introduce a convolutional neural network (CNN) to leverage both spatial and spectral information for segmenting data into informative sub-classes, which we call the IR-SEG network. We show that this framework increases accuracy by using approximately half of the features used in the typical pixel-wise classification of IR data. Second, we present a generative adversarial network (GAN)-based approach to reconstruct full data sets with low loss in the information from incomplete spatial and spectral data recording. Termed IR-REC, this approach is shown to speed up data acquisition by up to 20-fold for typical biomedical samples. In addition to enhancing the speed and quality of data, we also propose a method to utilize complementary morphologic detail to estimate the spatial details of a single band IR image beyond the diffraction limit. Finally, we discuss potential pitfalls and new opportunities that can be addressed by developing these methods further. Together, these deep learning techniques provide new capabilities for IR imaging to extract better quality information faster.
AB - Infrared (IR) spectroscopic imaging provides both morphologic and chemical detail; however, obtaining this extensive spectral-spatial information requires the ability to rapidly record high-quality data. Discrete frequency infrared (DFIR) imaging using a point scanning microscope strikes a balance between data quality and acquisition speed that, in principle, can further be aided by computational methods. Here, we report a deep learning-based framework to complement the process of data acquisition and information extraction. First, we introduce a convolutional neural network (CNN) to leverage both spatial and spectral information for segmenting data into informative sub-classes, which we call the IR-SEG network. We show that this framework increases accuracy by using approximately half of the features used in the typical pixel-wise classification of IR data. Second, we present a generative adversarial network (GAN)-based approach to reconstruct full data sets with low loss in the information from incomplete spatial and spectral data recording. Termed IR-REC, this approach is shown to speed up data acquisition by up to 20-fold for typical biomedical samples. In addition to enhancing the speed and quality of data, we also propose a method to utilize complementary morphologic detail to estimate the spatial details of a single band IR image beyond the diffraction limit. Finally, we discuss potential pitfalls and new opportunities that can be addressed by developing these methods further. Together, these deep learning techniques provide new capabilities for IR imaging to extract better quality information faster.
KW - Deep learning
KW - Discrete frequency infrared imaging
KW - Image reconstruction
KW - Infrared spectroscopic imaging
KW - Tissue segmentation
UR - http://www.scopus.com/inward/record.url?scp=85111547589&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85111547589&partnerID=8YFLogxK
U2 - 10.1016/j.chemolab.2021.104390
DO - 10.1016/j.chemolab.2021.104390
M3 - Article
AN - SCOPUS:85111547589
SN - 0169-7439
VL - 217
JO - Chemometrics and Intelligent Laboratory Systems
JF - Chemometrics and Intelligent Laboratory Systems
M1 - 104390
ER -