TY - GEN
T1 - Automated Visual Testing for Mobile Apps in an Industrial Setting
AU - Ran, Dezhi
AU - Li, Zongyang
AU - Liu, Chenxu
AU - Wang, Wenyu
AU - Meng, Weizhi
AU - Wu, Xionglin
AU - Jin, Hui
AU - Cui, Jing
AU - Tang, Xing
AU - Xie, Tao
N1 - Tao Xie’s work was partially supported by National Natural Science Foundation of China (Grant No. 62161146003), a grant from Alibaba, and XPLORER PRIZE.
PY - 2022
Y1 - 2022
N2 - User Interface (UI) testing has become a common practice for quality assurance of industrial mobile applications (in short as apps). While many automated tools have been developed, they often do not satisfy two major industrial requirements that make a tool desirable in industrial settings: high applicability across platforms (e.g., Android, iOS, AliOS, and Harmony OS) and high capability to handle apps with non-standard UI elements (whose internal structures cannot be acquired using platform APIs). Toward addressing these industrial requirements, automated visual testing emerges to take only device screenshots as input in order to support automated test generation. In this paper, we report our experiences of developing and deploying VTest, our industrial visual testing framework to assure high quality of Taobao, a highly popular industrial app with about one billion monthly active users. VTest includes carefully designed techniques and infrastructure support, outperforming Monkey (which has been popularly deployed in industry and shown to perform superiorly or similarly compared to state-of-the-art tools) with 87.6% more activity coverage. VTEST has been deployed both internally in Alibaba and externally in the Software Green Alliance to provide testing services for top smart-phone vendors and app vendors in China. We summarize five major lessons learned from developing and deploying VTEST.
AB - User Interface (UI) testing has become a common practice for quality assurance of industrial mobile applications (in short as apps). While many automated tools have been developed, they often do not satisfy two major industrial requirements that make a tool desirable in industrial settings: high applicability across platforms (e.g., Android, iOS, AliOS, and Harmony OS) and high capability to handle apps with non-standard UI elements (whose internal structures cannot be acquired using platform APIs). Toward addressing these industrial requirements, automated visual testing emerges to take only device screenshots as input in order to support automated test generation. In this paper, we report our experiences of developing and deploying VTest, our industrial visual testing framework to assure high quality of Taobao, a highly popular industrial app with about one billion monthly active users. VTest includes carefully designed techniques and infrastructure support, outperforming Monkey (which has been popularly deployed in industry and shown to perform superiorly or similarly compared to state-of-the-art tools) with 87.6% more activity coverage. VTEST has been deployed both internally in Alibaba and externally in the Software Green Alliance to provide testing services for top smart-phone vendors and app vendors in China. We summarize five major lessons learned from developing and deploying VTEST.
KW - UI testing
KW - mobile testing
KW - robotic testing
KW - visual testing
UR - http://www.scopus.com/inward/record.url?scp=85132821525&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85132821525&partnerID=8YFLogxK
U2 - 10.1109/ICSE-SEIP55303.2022.9793948
DO - 10.1109/ICSE-SEIP55303.2022.9793948
M3 - Conference contribution
AN - SCOPUS:85132821525
T3 - Proceedings - International Conference on Software Engineering
SP - 55
EP - 64
BT - Proceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering
PB - IEEE Computer Society
T2 - 44th ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2022
Y2 - 22 May 2022 through 27 May 2022
ER -