TY - GEN
T1 - Scalable parallel programming in python with PArsL
AU - Babuji, Yadu
AU - Woodard, Anna
AU - Li, Zhuozhao
AU - Katz, Daniel S.
AU - Clifford, Ben
AU - Foster, Ian
AU - Wilde, Michael
AU - Chard, Kyle
N1 - Publisher Copyright:
© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.
PY - 2019/7/28
Y1 - 2019/7/28
N2 - Python is increasingly the lingua franca of scientific computing. It is used as a higher level language to wrap lower-level libraries and to compose scripts from various independent components. However, scaling and moving Python programs from laptops to supercomputers remains a challenge. Here we present Parsl, a parallel scripting library for Python. Parsl makes it straightforward for developers to implement parallelism in Python by annotating functions that can be executed asynchronously and in parallel, and to scale analyses from a laptop to thousands of nodes on a supercomputer or distributed system. We examine how Parsl is implemented, focusing on syntax and usage. We describe two scientific use cases in which Parsl’s intuitive and scalable parallelism is used.
AB - Python is increasingly the lingua franca of scientific computing. It is used as a higher level language to wrap lower-level libraries and to compose scripts from various independent components. However, scaling and moving Python programs from laptops to supercomputers remains a challenge. Here we present Parsl, a parallel scripting library for Python. Parsl makes it straightforward for developers to implement parallelism in Python by annotating functions that can be executed asynchronously and in parallel, and to scale analyses from a laptop to thousands of nodes on a supercomputer or distributed system. We examine how Parsl is implemented, focusing on syntax and usage. We describe two scientific use cases in which Parsl’s intuitive and scalable parallelism is used.
UR - http://www.scopus.com/inward/record.url?scp=85071010264&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85071010264&partnerID=8YFLogxK
U2 - 10.1145/3332186.3332231
DO - 10.1145/3332186.3332231
M3 - Conference contribution
AN - SCOPUS:85071010264
T3 - ACM International Conference Proceeding Series
BT - Proceedings of the Practice and Experience in Advanced Research Computing
PB - Association for Computing Machinery
T2 - 2019 Conference on Practice and Experience in Advanced Research Computing: Rise of the Machines (Learning), PEARC 2019
Y2 - 28 July 2019 through 1 August 2019
ER -