TY - JOUR
T1 - Parsl
T2 - 10th International Workshop on Science Gateways, IWSG 2018
AU - Babuji, Yadu
AU - Chard, Kyle
AU - Foster, Ian
AU - Katz, Daniel S.
AU - Wilde, Michael
AU - Woodard, Anna
AU - Wozniak, Justin
N1 - This work was supported in part by NSF award ACI-1550588 and DOE contract DE-AC02-06CH11357.
PY - 2019
Y1 - 2019
N2 - Computational and data-driven research practices have significantly changed over the past decade to encompass new analysis models such as interactive and online computing. Science gateways are simultaneously evolving to support this transforming landscape with the aim to enable transparent, scalable execution of a variety of analyses. Science gateways often rely on workflow management systems to represent and execute analyses efficiently and reliably. However, integrating workflow systems in science gateways can be challenging, especially as analyses become more interactive and dynamic, requiring sophisticated orchestration and management of applications and data, and customization for specific execution environments. Parsl (Parallel Scripting Library), a Python library for programming and executing data-oriented workflows in parallel, addresses these problems. Developers simply annotate a Python script with Parsl directives wrapping either Python functions or calls to external applications. Parsl manages the execution of the script on clusters, clouds, grids, and other resources; orchestrates required data movement; and manages the execution of Python functions and external applications in parallel. The Parsl library can be easily integrated into Python-based gateways, allowing for simple management and scaling of workflows.
AB - Computational and data-driven research practices have significantly changed over the past decade to encompass new analysis models such as interactive and online computing. Science gateways are simultaneously evolving to support this transforming landscape with the aim to enable transparent, scalable execution of a variety of analyses. Science gateways often rely on workflow management systems to represent and execute analyses efficiently and reliably. However, integrating workflow systems in science gateways can be challenging, especially as analyses become more interactive and dynamic, requiring sophisticated orchestration and management of applications and data, and customization for specific execution environments. Parsl (Parallel Scripting Library), a Python library for programming and executing data-oriented workflows in parallel, addresses these problems. Developers simply annotate a Python script with Parsl directives wrapping either Python functions or calls to external applications. Parsl manages the execution of the script on clusters, clouds, grids, and other resources; orchestrates required data movement; and manages the execution of Python functions and external applications in parallel. The Parsl library can be easily integrated into Python-based gateways, allowing for simple management and scaling of workflows.
KW - Parallel scripting
KW - Parsl
KW - Python
KW - Scientific Workflows
UR - http://www.scopus.com/inward/record.url?scp=85065527336&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85065527336&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85065527336
SN - 1613-0073
VL - 2357
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
Y2 - 13 June 2018 through 15 June 2018
ER -