Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines

Peng Peng, Limin Yang, Linhai Song, Gang Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Online scan engines such as VirusTotal are heavily used by researchers to label malicious URLs and files. Unfortunately, it is not well understood how the labels are generated and how reliable the scanning results are. In this paper, we focus on VirusTotal and its 68 third-party vendors to examine their labeling process on phishing URLs. We perform a series of measurements by setting up our own phishing websites (mimicking PayPal and IRS) and submitting the URLs for scanning. By analyzing the incoming network traffic and the dynamic label changes at VirusTotal, we reveal new insights into how VirusTotal works and the quality of their labels. Among other things, we show that vendors have trouble flagging all phishing sites, and even the best vendors missed 30% of our phishing sites. In addition, the scanning results are not immediately updated to VirusTotal after the scanning, and there are inconsistent results between VirusTotal scan and some vendors' own scanners. Our results reveal the need for developing more rigorous methodologies to assess and make use of the labels obtained from VirusTotal.

Original languageEnglish (US)
Title of host publicationIMC 2019 - Proceedings of the 2019 ACM Internet Measurement Conference
PublisherAssociation for Computing Machinery
Number of pages8
ISBN (Electronic)9781450369480
StatePublished - Oct 21 2019
Event19th ACM Internet Measurement Conference, IMC 2019 - Amsterdam, Netherlands
Duration: Oct 21 2019Oct 23 2019


Conference19th ACM Internet Measurement Conference, IMC 2019

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications


Dive into the research topics of 'Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines'. Together they form a unique fingerprint.

Cite this