A model for user-oriented data provenance in pipelined scientific workflows

Shawn Bowers, Timothy McPhillips, Bertram Ludäscher, Shirley Cohen, Susan B. Davidson

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Integrated provenance support promises to be a chief advantage of scientific workflow systems over script-based alternatives. While it is often recognized that information gathered during scientific workflow execution can be used automatically to increase fault tolerance (via checkpointing) and to optimize performance (by reusing intermediate data products in future runs), it is perhaps more significant that provenance information may also be used by scientists to reproduce results from earlier runs, to explain unexpected results, and to prepare results for publication. Current workflow systems offer little or no direct support for these "scientist-oriented" queries of provenance information. Indeed the use of advanced execution models in scientific workflows (e.g., process networks, which exhibit pipeline parallelism over streaming data) and failure to record certain fundamental events such as state resets of processes, can render existing provenance schemas useless for scientific applications of provenance. We develop a simple provenance model that is capable of supporting a wide range of scientific use cases even for complex models of computation such as process networks. Our approach reduces these use cases to database queries over event logs, and is capable of reconstructing complete data and invocation dependency graphs for a workflow run.

Original languageEnglish (US)
Title of host publicationProvenance and Annotation of Data - International Provenance and Annotation Workshop, IPAW 2006, Revised Selected Papers
Number of pages15
ISBN (Print)354046302X, 9783540463023
StatePublished - 2006
Externally publishedYes
EventInternational Provenance and Annotation Workshop, IPAW 2006 - Chicago, IL, United States
Duration: May 3 2006May 5 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4145 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


OtherInternational Provenance and Annotation Workshop, IPAW 2006
Country/TerritoryUnited States
CityChicago, IL

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'A model for user-oriented data provenance in pipelined scientific workflows'. Together they form a unique fingerprint.

Cite this