PPDG is developing and integrating Grid-enabled tools for data intensive distributed computing into production quality end-to-end distributed computing solutions for HENP experiments. PPDG is a collaboration of physicists with computer scientists working at the forefront of Grid technologies.

The principal PPDG deliverable for ATLAS in the first year (through June '02) is the delivery of a production distributed data service deployed to users. The purpose of this project is to develop the distributed data manager which will be deployed to meet this deliverable. The production distributed data service is to exist between CERN, the ATLAS Tier 1 facility at BNL, and a number of US ATLAS grid testbed sites (some or all of ANL, LBNL, Boston University, Indiana University, U Michigan, U Oklahoma, and UT Arlington). The objective is a multi-point U.S. Grid (in addition to the CERN link) providing distributed data services as early as possible.

The deployed service will be a vertically integrated suite of tools extending from a number of grid toolkit components (listed below) at the foundation, through a metadata cataloging and distributed data infrastructure that is partly an ATLAS-specific infrastructure layer and partly a generic testbed for exploring distributed data management technologies and approaches, to primarily experiment-specific interfaces to ATLAS users and software.

Project leaders

Activities

• Magda - Distributed data manager prototype
  Magda is the distributed data manager to be deployed to meet the principal year 1 PPDG deliverable of a production distributed data management system serving the US ATLAS Tier 1 center at BNL, CERN, and many US ATLAS grid testbed sites.
• U.S. ATLAS Grid Testbed
• Data history
  A full specification or 'signature' of a data set that is complete enough to (at least in principle) regenerate it is required both to unambiguously discriminate the data set from others and to allow 'virtual data' mechanisms to regenerate it. We are working on the 'data histories' which will implement this.
• Distributed data services testbed using test beam data
• U.S. ATLAS grid computing work breakdown and work plan

Common components

• Globus GSIftp/Gridftp file transfer
• Globus remote execution tools
• Globus replica catalog
• Globus replica management
• GDMP file mirroring package
• SRM storage resource management (to come)
• Condor/DAGMAN distributed processing management (to come)

Group pages of PPDG-ATLAS participants

• ANL: PPDG Applications Working Group
• BNL: PPDG at BNL
• LBNL: Scientific Data Management Research Group

Work plan

The ATLAS PPDG work plan for the next 3 years (commencing about July 1, 2001) is shown here:

• Summary plan from the PPDG/SCIDAC proposal 3/2001
• Full plan of PPDG and related work
  • Year 1
  • Year 2
  • Year 3

As much of the ATLAS PPDG work as possible will be done in collaboration with the computer science participants in PPDG. Collaborative PPDG work between computer scientists and the experiments is in these 7 areas. See the proposal for details on these areas and their work packages.

CS-1: Job description language

Definition of job processing requirements and policies, file placement and replication in distributed systems

CS-2: Job scheduling and management

Job processing, data placement, resource discovery and optimization over the grid

CS-3: Monitoring and status reporting

CS-4: Storage resource management

CS-5: Reliable replica management services

CS-6: File transfer services

CS-7: Collect and document experiment practices

ATLAS involvement in these areas through the three years of PPDG is principally in the following work packages (P1 is in CS-1, etc.).

Year 1

• P3-1 Monitoring and status reporting for initial production deployment
• P4-1 HRM extensions and integration for local storage system
• P5-2 Distributed file and replica catalogs between a few sites
• P6-1 Reliable file transfer

Year 2

• P1-3 Deployment of job and production computing control
• P2-2 Remote job submission and management of production computing
• P3-3 Fully integrated monitoring and availability of information
• P6-2 Enhanced data transfer and replication services

Year 3

• P1-4 Extensions to support object collections, event level access etc.
• P2-4 Distributed data management and enhanced resource discovery and optimization
• P3-3 Fully integrated monitoring and availability of information to job control and management
• P4-3 Enhanced resource discovery and scheduling
• P5-3 Enhanced replication services including cache management

Documents and reports

• ATLAS PPDG quarterly reports
  • Jan-Mar 2002 (draft)
  • Oct-Dec 2001 PDF, Word
  • Jul-Sep 2001
  • Apr-Jun 2001
  • Jan-Mar 2001
• 2001 PPDG SciDAC proposal
  • See the PPDG web site for the final version of the full proposal
  • ATLAS section of proposal -- program and deliverables
    • Version 3 (Mar 5): PDF, Word
    • Version 1 (Feb 28): PDF, Word
    • Preliminary text version (2/2001)