New discoveries in science are often built upon previous knowledge. Ideally, such dependency information should be made explicit in a scientific knowledge graph. The Keystone Framework was proposed for tracking the validity dependency among papers. A keystone citation indicates that the validity of a given paper depends on a previously published paper it cites. In this paper, we propose and evaluate a strategy that repurposes rhetorical category classifiers for the novel application of extracting keystone citations that relate to research methods. Five binary rhetorical category classifiers were constructed to identify Background, Objective, Methods, Results, and Conclusions sentences in biomedical papers. The resulting classifiers were used to test the strategy against two datasets. The initial strategy assumed that only citations contained in Methods sentences were methods keystone citations, but our analysis revealed that citations contained in sentences classified as either Methods or Results had a high likelihood to be methods keystone citations. Future work will focus on fine tuning the rhetorical category classifiers, experimenting with multiclass classifiers, evaluating the revised strategy with more data, and constructing a larger gold standard citation context sentence dataset for model training.

Original languageEnglish (US)
Title of host publicationThe Web Conference 2021 - Companion of the World Wide Web Conference, WWW 2021
PublisherAssociation for Computing Machinery
Number of pages5
ISBN (Electronic)9781450383134
StatePublished - Apr 19 2021
Event30th World Wide Web Conference, WWW 2021 - Ljubljana, Slovenia
Duration: Apr 19 2021Apr 23 2021

Publication series

NameThe Web Conference 2021 - Companion of the World Wide Web Conference, WWW 2021


Conference30th World Wide Web Conference, WWW 2021


  • Knowledge dependency
  • argumentation
  • citation context classification
  • methods
  • validity

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software


Dive into the research topics of 'Finding Keystone Citations for Constructing Validity Chains among Research Papers'. Together they form a unique fingerprint.

Cite this