r4 - 04 Mar 2008 - 17:32:49 - RobertGardnerYou are here: TWiki >  Admins Web > DistributedFacilityWorkshop2008

DistributedFacilityWorkshop2008

Introduction

A meeting of the US ATLAS Facility to assess readiness and confront remaining obstacles for distributed analysis involving all Tiers and resources in the distributed facility, with principal focus on Tier1, Tier2 and Tier3 resources in the US ATLAS Faclity. The meeting is co-located with the OSG All Hands Consortium meeting at the University of North Carolina.

Background

Introduction, required capabilities - Michael Ernst

  • Lots more CPU required to simulate events due to showering. This implies changes in the admixture Tier2 job mix of analysis and simulation. See tables resulting from meetings last week.
  • US share of simulation budget - 20%.
  • Computing in the US - there will be a new role for a Production & Analysis Operation coordinator.
  • Data location and sites - Tier 1/2 defined within the model - but we have to develop this for the Tier3.
  • While there was a task force for the Tier3, there as yet no report, and the model is still evolving.
  • Disk resources at the Tier1 - consider data requirements for the next year.
  • We are on target for CPU, short on disk by 20%. We expect a budget shortfall in 2011; it will become a challenge to replace aging equipment and maintaing salaries.

Planning points

  • SRM v2.2 functionality now required for all ATLAS sites - implications for storage upgrades at Tier2, and bestman-xrootd development.
  • Supporting user analysis jobs through functional testing and standard templates.
  • Reprocessing FDR-1 - by March 17? Where?
  • FDR2 - Simulation already started. Digitization to start April 1. 90% RDO at BNL by April 20.
    May 19: data on SFOs. Plan:
    • 7h at 10^32 (2.5 pb-1), with fakes
    • 10h at 10^33 data (10.8 pb-1), probably no fakes buth with 75ns pileup.
  • Job-data-site brokering in Panda is in development. Will need to be tested when available, and before then to support analysis we'll need to replicate all FDR data everywhere.
  • Incorporating TAGs into the FDR exercise - lack of automated tools to upload them presently.
  • Data placement within the US cloud - opting out of the centrally controlled model for transfers within the BNL cloud. New tools to be developed to manage T1->T2 subscriptions.
  • Data deletion centrally is too ridgid down to the Tier2 level. Development of new tools for US-cloud management, using Panda where appropriate.
  • Definition of partitions (and tokens) for Tier2 storage needs to be defined.
  • A schedule is needed and approach for LFC migration.
  • ESD-like analysis on AOD datasets, possible now with the merger that happened between Rels 12/13.
  • T/P separation has led to separate optimizations for stroage and memory access - now AOD reading is near optimal.
  • Need to evaluate the benefits of D1PDs on tier 2s.
  • Tier3 issues
    • Can we support replication of D1PD to tier 3? Implication of many (5-10 TB) transfers monthly to tier3.
    • A technical plan for Tier3 facilities within the next three months
      • Deploy/test/collect experiences of PROOF at Tier3 facilities.
    • Tier3 and data access
    • Software must be simple to install and troubleshoot
  • PROOF
    • Determine role of PROOF at Tier2
    • See ProofXrootd/WebHome twiki web
    • Multi-Tier3 proof - associating Tier3 proof farms to process large datasets.
    • Need to bring data to the cluster

-- RobertGardner - 18 Jan 2008

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments

 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback