Pegasus (workflow management)
   HOME

TheInfoList



OR:

Pegasus is an open-source workflow management system. E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, and K. Wenger
"Future Generation Computer Systems"
''
Elsevier Elsevier () is a Dutch academic publishing company specializing in scientific, technical, and medical content. Its products include journals such as '' The Lancet'', ''Cell'', the ScienceDirect collection of electronic journals, '' Trends'', ...
''; 46, pp. 17-35 (2015)
E.A. Huerta, R. Haas, E. Fajardo, D.S. Katz, S. Anderson, P. Couvares ,J. Willis, T. Bouvet, J. Enos, W.T.C. Kramer, H.W. Leong, and D. Wheeler
"BOSS-LDG: A Novel Computational Framework That Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery"
'' 2017 IEEE 13th International Conference on e-Science (e-Science)''; pp. 335-344 (2017)
B. Riedel, B. Bauermeister, L. Bryant, J. Conrad, P. de Perio, R. W. Gardner ,L. Grandi, F. Lombardi, A. Rizzo, G. Sartorelli, M. Selvi, E. Shockley, J. Stephen, S. Thapa, and C. Tunnel
"Distributed Data and Job Management for the XENON1T Experiment"
''PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing'';9, pp. 1-8 (2018)
It provides the necessary abstractions for scientists to create scientific workflowsG. Amalarethinam, T. Lucia, A. Beena
“Scheduling Framework for Regular Scientific Workflows in Cloud”
'' International Journal of Applied Engineering Research''; 10, no. 82 (2015)
and allows for transparent execution of these workflows on a range of computing platforms including high performance computing clusters, clouds, and national cyberinfrastructure.The Scientific Workflow Integrity with Pegasus (SWIP)
by Center for Applied Cybersecurity Research; published 16 September 2016; retrieved 1 May 2020
In Pegasus, workflows are described abstractly as directed acyclic graphs (DAGs) using a provided API for Jupyter Notebooks,
Python Python may refer to: Snakes * Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia ** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia * Python (mythology), a mythical serpent Computing * Python (pro ...
, R, or
Java Java (; id, Jawa, ; jv, ꦗꦮ; su, ) is one of the Greater Sunda Islands in Indonesia. It is bordered by the Indian Ocean to the south and the Java Sea to the north. With a population of 151.6 million people, Java is the world's mos ...
.D. Weitzel, B. Bockelman, D. Brown, P. Couvares, F. Würthwein, and E.F. Hernandez
“Data Access for LIGO on the OSG”
''Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact - PEARC17''; 24, no. 1-6 (2017)
During execution, Pegasus translates the constructed abstract workflow into an executable workflowDuncan Brown and Ewa Deelman
"Looking for gravitational waves: A computing perspective"
at ''Science Node''; published June 8, 2011; retrieved April 30, 2020
which is executed and managed by HTCondor.$1M NSF award goes to IU-led data integrity project
by
Indiana University Indiana University (IU) is a system of public universities in the U.S. state of Indiana. Campuses Indiana University has two core campuses, five regional campuses, and two regional centers under the administration of IUPUI. *Indiana Universi ...
; published 16 September 2016; retrieved 1 May 2020
Brian Mattmiller
"High Throughput Computing helps LIGO confirm Einstein's last unproven theory"
at '' Morgridge Institute for Research''; published March 7, 2016; retrieved May 1, 2020
Pegasus is being used in a number of different disciplines including
astronomy Astronomy () is a natural science that studies celestial objects and phenomena. It uses mathematics, physics, and chemistry in order to explain their origin and evolution. Objects of interest include planets, moons, stars, nebulae, g ...
, gravitational-wave physics, bioinformatics, earthquake engineering, and
helioseismology Helioseismology, a term coined by Douglas Gough, is the study of the structure and dynamics of the Sun through its oscillations. These are principally caused by sound waves that are continuously driven and damped by convection near the Sun's sur ...
.Sanden Totten
"Caltech Wasn't the Only SoCal School Helping Discover Gravitational Waves"
at '' KPCC''; published 11 February 2016; retrieved May 1, 2020
Notably, the LIGO Scientific Collaboration has used it to directly detect a gravitational wave for the first time.D.A. Brown, P.R. Brady, A. Dietz, J. Cao, B. Johnson, J. McNabb,
“A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis. In: I.J Taylor, E. Deelman, D.B. Gannon, M. Shields (eds) Workflows for e-Science”
'' Springer, London''; 13, pp. 39-59 (2007)
D. Davis, T. Massinger, A. Lundgren, J.C. Driggers, A.L. Urban, and L. Nuttall,
“Improving the sensitivity of Advanced LIGO using noise subtraction”
''
Classical and Quantum Gravity ''Classical and Quantum Gravity'' is a peer-reviewed journal that covers all aspects of gravitational physics and the theory of spacetime. Its scope includes: *Classical general relativity *Applications of relativity *Experimental gravitation ...
''; 36, no. 5 (2019)


Area of applications

Application examples: E. Deelman, G. Singh, M. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, B. G. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz
“Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems”
'' Scientific Programming''; 13, pp. 19 (2005)
* Gravitational-Wave Physics * Earthquake Science * Bioinformatics * Workflows for Volcanic Mass Flows * Diffusion Image Processing and Analysis * Spallation Neutron Source (SNS)


History

The development of Pegasus started in 2001.


See also

*
Distributed computing A distributed system is a system whose components are located on different networked computers, which communicate and coordinate their actions by passing messages to one another from any system. Distributed computing is a field of computer sci ...
*
Workflow Management System A workflow management system (WfMS or WFMS) provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application. International standards There are several international standards ...


References

{{DEFAULTSORT:Spark Workflow applications Free software