Validation and inference of schema-level workflow data-dependency annotations

Shawn Bowers, Timothy McPhillips, Bertram Ludäscher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An advantage of scientific workflow systems is their ability to collect runtime provenance information as an execution trace. Traces include the computation steps invoked as part of the workflow run along with the corresponding data consumed and produced by each workflow step. The information captured by a trace is used to infer “lineage” relationships among data items, which can help answer provenance queries to find workflow inputs that were involved in producing specific workflow outputs. Determining lineage relationships, however, requires an understanding of the dependency patterns that exist between each workflow step’s inputs and outputs, and this information is often under-specified or generally assumed by workflow systems. For instance, most approaches assume all outputs depend on all inputs, which can lead to lineage “false positives”. In prior work, we defined annotations for specifying detailed dependency relationships between inputs and outputs of computation steps. These annotations are used to define corresponding rules for inferring fine-grained data dependencies from a trace. In this paper, we extend our previous work by considering the impact of dependency annotations on workflow specifications. In particular, we provide a reasoning framework to ensure the set of dependency annotations on a workflow specification is consistent. The framework can also infer a complete set of annotations given a partially annotated workflow. Finally, we describe an implementation of the reasoning framework using answer-set programming.

Original languageEnglish (US)
Title of host publicationProvenance and Annotation of Data and Processes - 7th International Provenance and Annotation Workshop, IPAW 2018, Proceedings
EditorsKhalid Belhajjame, Ashish Gehani, Pinar Alper
PublisherSpringer-Verlag Berlin Heidelberg
Pages128-141
Number of pages14
ISBN (Print)9783319983783
DOIs
StatePublished - Jan 1 2018
Event7th International Provenance and Annotation Workshop, IPAW 2018 - London, United Kingdom
Duration: Jul 9 2018Jul 10 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11017 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other7th International Provenance and Annotation Workshop, IPAW 2018
CountryUnited Kingdom
CityLondon
Period7/9/187/10/18

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Validation and inference of schema-level workflow data-dependency annotations'. Together they form a unique fingerprint.

  • Cite this

    Bowers, S., McPhillips, T., & Ludäscher, B. (2018). Validation and inference of schema-level workflow data-dependency annotations. In K. Belhajjame, A. Gehani, & P. Alper (Eds.), Provenance and Annotation of Data and Processes - 7th International Provenance and Annotation Workshop, IPAW 2018, Proceedings (pp. 128-141). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11017 LNCS). Springer-Verlag Berlin Heidelberg. https://doi.org/10.1007/978-3-319-98379-0_10