r37 - 11 Jan 2013 - 12:38:27 - MaximPotekhinYou are here: TWiki >  AtlasSoftware Web > PandaExtensions

Workload Management Extensions of Panda


Introduction

This page is mostly of historical interest, as the Open Science Grid no longer supports this activity. The Applications Area of OSG used to organize and manage a small number of science-driven projects in extending the existing grid middleware to support higher level services and offer them to the OSG community. One of these projects was the workload management, specifically in developing a 'just-in-time' workload management system and a tool set based on the 'pilot job' approach to job submission and management.

For some background see this excerpt of an OSG proposal in this area. For more information on the OSG extensions program at BNL see OSGAtBNL.

A major part of the project was generalizing the US ATLAS Panda system to become a VO-neutral workload management system supported on OSG for general use. This involved generalized mechanisms of data movement which should be easy to deploy and maintain for VOs with limited resources.

Program outline

These are principal activities:

  • Generalization of existing Panda to an project-neutral just-in-time workload manager
    • Remove ATLAS specificity and make it a generic, modular system usable by any VO via standard interfaces and VO-specific customization via plugins (such as for data movement)
    • Usable by, and supported for, any OSG VO
    • Supporting VO-defined back end job submission tools and data management tools
  • Selective middleware technology studies and functionality/performance evaluations
    • Select technologies for integration with generic Panda
  • Inter-experiment collaboration
    • Identifying and integrating high level components from other experiments which are/could be common tools

Project participants

  • Torre Wenaus, BNL (ATLAS)
  • Maxim Potekhin, BNL (OSG)
  • Miron Livny et al, U Wisconsin Madison (Condor)

Current activities

Data handling

  • Integration with Globus Online. This is related to the needs of non-Atlas users of Panda who would like to have a platform-neutral tool for reliable data movements

noSQL Solution for Panda Monitoring Upgrade

  • Mnemosyne -- a Cassandra-based Web Service for archived Panda Monitoring Data

Pilot and Job Submission for Virtual Organizations outside of Atlas

Former Projects

Older information (archives)


Major updates:
-- MaximPotekhin - 11 Jan 2013
-- MaximPotekhin - 23 Sep 2011
-- MaximPotekhin - 05 May 2010
-- MaximPotekhin - 18 Mar 2010
-- MaximPotekhin - 23 Nov 2009
-- MaximPotekhin - 06 Nov 2009
-- MaximPotekhin - 26 May 2009
-- MaximPotekhin - 14 Oct 2008
-- TorreWenaus - 19 Sep 2006

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments

 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback