r55 - 13 Feb 2013 - 17:18:31 - SergeyPanitkinYou are here: TWiki >  PanDA Web > WebHome

Next Generation Workload Management and Analysis System for Big Data


Welcome to Big PanDA

These pages contain information related to ASCR funded project "Next Generation Workload Management and Analysis System for Big Data".

An important foundation underlying the impressive success of ATLAS Experiment data processing and analysis is the Production and Distributed Analysis (PanDA) workload management system that was designed specifically for the ATLAS at the Large Hadron Collider (LHC) and proved to be highly successful in meeting all the distributed computing needs of the experiment. However, the core design of PanDA is not experiment specific. The PanDA workload management system is capable of meeting the needs of other data intensive scientific applications. PanDA has been used to run CHARMM protein folding code on the Open Science Grid (OSG). Several big data experiments such as Alpha-Magnetic Spectrometer (AMS, an astro-particle experiment on International Space Station) and the Compact Muon Solenoid (CMS, a LHC experiment) are considering the adoption of PanDA. We proposed generalizing PanDA as meta-application, providing location transparency of processing and data access, for High Energy Physics, other data-intensive sciences, and a wider exascale community.

Project work packages

  • WP1 (Factorizing the core): Factorizing the core components of PanDA to enable adoption by a wide range of exascale scientific communities (UTA, K.De)
  • WP2 (Extending the scope): Evolving PanDA to support extreme scale computing clouds and Leadership Computing Facilities (BNL, S.Panitkin)
  • WP3 (Leveraging intelligent networks): Integrating network services and real-time data access to the PanDA workflow (BNL, D.Yu)
  • WP4 (Usability and monitoring): Real time monitoring and visualization package for PanDA (BNL, T.Wenaus)

Meetings

  • Kick-off Meeting at BNL, September 24, 2012
  • Face to Face Meeting at BNL, November 30, 2012
  • Face to Face Meeting at BNL, January 28, 2013

Related Information and Activities (May require authentication)

PanDA Publications and Presentations

  1. PanDA Overview. K. De, talk at ORNL GEANT4 Meeting, Oak Ridge, TN, February 11, 2013. https://www.usatlas.bnl.gov/twiki/pub/PanDA/WebHome/panda-ornl-feb13-final.pdf
  2. Interfacing PanDA to OLCF. S. Panitkin, talk at ORNL GEANT4 Meeting, Oak Ridge, TN, February 11, 2013. https://www.usatlas.bnl.gov/twiki/pub/PanDA/WebHome/PanDA_on_SC_v1.pdf
  3. Networking and Workload Management. K. De, talk at LHCONE Point-to-Point Service Workshop - CERN, Geneva, Switzerland, December 13, 2012. http://indico.cern.ch/getFile.py/access?contribId=5&sessionId=1&resId=1&materialId=slides&confId=215393
  4. Recent Improvements in the ATLAS PanDA Pilot. P. Nilsson et al 2012 J. Phys.: Conf. Ser. 396 032080
  5. PanDA: Next Generation Workload Management and Analysis System for Big Data. Flyer for the International Conference for High Performance Computing, Networking, Storage and Analysis 2012, Salt Lake, UT , November 10-16, 2012. http://cdsweb.cern.ch/record/1492076
  6. Next Generation Workload Management and Analysis System for Big Data. Poster presented at International Conference for High Performance Computing, Networking, Storage and Analysis 2012, Salt Lake, UT , November 10-16, 2012. http://sc12.supercomputing.org/schedule/event_detail.php?evid=post286
  7. Common Analysis Framework for ATLAS and CMS. Feasibility Study. ATLAS Distributed Computing Technical Interchange Meeting. Annecy, France, Apr 18-20,2012. https://indico.cern.ch/getFile.py/access?contribId=12&sessionId=5&resId=1&materialId=slides&confId=176443
  8. The ATLAS PanDA Monitoring System and its Evolution. 2011 J. Phys.: Conf. Ser. 331 072058, A.Klimentov, P.Nevski, M.Potekhin and T.Wenaus
  9. PanDA Production and Analysis backend. 17th International Conference on Computing in High Energy and Nuclear Physics (CHEP09), March 2009. Journal of Physics vol. 219, 210. K.De, A.Klimentov, T.Maeno, P.Nilsson and T.Wenaus
  10. PanDA : Distributed Production and Distributed Analysis System for ATLAS. T.Maeno et al. Journal for Physics vol. 119 part 6, 2008

PanDA in Press

General information

  • Lead Institution: Brookhaven National Laboratory
  • Lead PI: Alexei Klimentov
  • Principal Investigators:
    • Brookhaven National Laboratory:
    • Argonne National Laboratory:
    • The University of Texas at Arlington:


Major updates:
-- TWikiAdminGroup - 30 Mar 2017

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments


pdf panda-ornl-feb13-final.pdf (552.7K) | SergeyPanitkin, 13 Feb 2013 - 17:12 |
pdf PanDA_on_SC_v1.pdf (729.0K) | SergeyPanitkin, 13 Feb 2013 - 17:13 |
 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback