TY - JOUR
T1 - End-to-end online quality prediction for ultrasonic metal welding using sensor fusion and deep learning
AU - Wu, Yulun
AU - Meng, Yuquan
AU - Shao, Chenhui
N1 - Funding Information:
This research has been supported by the National Science Foundation, United States under Grant No. 1944345 .
Publisher Copyright:
© 2022 The Society of Manufacturing Engineers
PY - 2022/11
Y1 - 2022/11
N2 - In industrial-scale production applications of ultrasonic metal welding (UMW), there is a strong need for predicting joint quality quickly, reliably, and non-destructively. State-of-the-art quality assessment methods such as destructive tensile testing and binary quality classification cannot meet such requirements. This paper develops a novel end-to-end online quality prediction method for UMW based on sensor fusion and deep learning. This method first preprocesses 1-dimensional signals from multiple sensors including an acoustic emission sensor, a linear variable differential transformer, and a microphone, and transforms them to 2-dimensional images using wavelet transform. Then, these images are fed into ResNet20, which is a 20-layer convolutional neural network, to automatically generate feature maps and predict joint strength. The proposed method offers important advantages compared to state-of-the-art approaches, including automatic feature generation and good robustness to UMW tool conditions. The effectiveness of the developed method is demonstrated using real-world data generated from an UMW process with four different tool conditions. Additionally, we propose three feature fusion strategies (early fusion, middle fusion, and late fusion) and present a comparative case study to compare their performance. It is found that the late fusion strategy achieves the best prediction performance. Towards interpretability and explainability in deep learning, we perform a correlation analysis to reveal the connection between ResNet-generated features and features that are manually extracted based on UMW process physics. It is shown that many manual features are strongly correlated with ResNet features, proving that ResNet is able to resemble physical knowledge. The proposed method is readily applicable to industrial-scale UMW processes to enable accurate online quality prediction.
AB - In industrial-scale production applications of ultrasonic metal welding (UMW), there is a strong need for predicting joint quality quickly, reliably, and non-destructively. State-of-the-art quality assessment methods such as destructive tensile testing and binary quality classification cannot meet such requirements. This paper develops a novel end-to-end online quality prediction method for UMW based on sensor fusion and deep learning. This method first preprocesses 1-dimensional signals from multiple sensors including an acoustic emission sensor, a linear variable differential transformer, and a microphone, and transforms them to 2-dimensional images using wavelet transform. Then, these images are fed into ResNet20, which is a 20-layer convolutional neural network, to automatically generate feature maps and predict joint strength. The proposed method offers important advantages compared to state-of-the-art approaches, including automatic feature generation and good robustness to UMW tool conditions. The effectiveness of the developed method is demonstrated using real-world data generated from an UMW process with four different tool conditions. Additionally, we propose three feature fusion strategies (early fusion, middle fusion, and late fusion) and present a comparative case study to compare their performance. It is found that the late fusion strategy achieves the best prediction performance. Towards interpretability and explainability in deep learning, we perform a correlation analysis to reveal the connection between ResNet-generated features and features that are manually extracted based on UMW process physics. It is shown that many manual features are strongly correlated with ResNet features, proving that ResNet is able to resemble physical knowledge. The proposed method is readily applicable to industrial-scale UMW processes to enable accurate online quality prediction.
KW - Deep learning
KW - Interpretability
KW - Quality prediction
KW - ResNet
KW - Sensor fusion
KW - Ultrasonic metal welding
UR - http://www.scopus.com/inward/record.url?scp=85138809080&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85138809080&partnerID=8YFLogxK
U2 - 10.1016/j.jmapro.2022.09.011
DO - 10.1016/j.jmapro.2022.09.011
M3 - Article
AN - SCOPUS:85138809080
SN - 1526-6125
VL - 83
SP - 685
EP - 694
JO - Journal of Manufacturing Processes
JF - Journal of Manufacturing Processes
ER -