Automated Visual Testing for Mobile Apps in an Industrial Setting

Dezhi Ran, Zongyang Li, Chenxu Liu, Wenyu Wang, Weizhi Meng, Xionglin Wu, Hui Jin, Jing Cui, Xing Tang, Tao Xie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

User Interface (UI) testing has become a common practice for quality assurance of industrial mobile applications (in short as apps). While many automated tools have been developed, they often do not satisfy two major industrial requirements that make a tool desirable in industrial settings: high applicability across platforms (e.g., Android, iOS, AliOS, and Harmony OS) and high capability to handle apps with non-standard UI elements (whose internal structures cannot be acquired using platform APIs). Toward addressing these industrial requirements, automated visual testing emerges to take only device screenshots as input in order to support automated test generation. In this paper, we report our experiences of developing and deploying VTest, our industrial visual testing framework to assure high quality of Taobao, a highly popular industrial app with about one billion monthly active users. VTest includes carefully designed techniques and infrastructure support, outperforming Monkey (which has been popularly deployed in industry and shown to perform superiorly or similarly compared to state-of-the-art tools) with 87.6% more activity coverage. VTEST has been deployed both internally in Alibaba and externally in the Software Green Alliance to provide testing services for top smart-phone vendors and app vendors in China. We summarize five major lessons learned from developing and deploying VTEST.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 ACM/IEEE 44th International Conference on Software Engineering
Subtitle of host publicationSoftware Engineering in Practice, ICSE-SEIP 2022
PublisherIEEE Computer Society
Pages55-64
Number of pages10
ISBN (Electronic)9781665495905
DOIs
StatePublished - 2022
Event44th ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2022 - Pittsburgh, United States
Duration: May 22 2022May 27 2022

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Conference

Conference44th ACM/IEEE International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP 2022
Country/TerritoryUnited States
CityPittsburgh
Period5/22/225/27/22

Keywords

  • UI testing
  • mobile testing
  • robotic testing
  • visual testing

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'Automated Visual Testing for Mobile Apps in an Industrial Setting'. Together they form a unique fingerprint.

Cite this