r2 - 26 Mar 2010 - 20:39:57 - RichardMountYou are here: TWiki >  AtlasSoftware Web > Minutes26Mar2010

Minutes26Mar2010 RAC Minutes, March 26, 2010 DRAFT

Members (*=present, #=apologies)

*Richard Mount, Kevin Black (Physics Forum Chair), *Jim Cochran (Analysis Support Manager), Alexei Klimentov (ATLAS ADC), #Ian Hinchliffe (Physics Advisor), #Rik Yoshida (Tier3 Coordinator), *Michael Ernst (U.S. Facilities Manager), Rob Gardner (Integration Coordinator), *Kaushik De (U.S. Operations Manager), Armen Vartapetian (U.S. Operations Deputy)

Ex-Officio: *Torre Wenaus, *Stephane Willocq, *Mike Tuts, *Howard Gordon

Correction/approval of minutes of previous meetings

The minutes of the last full RAC and the core team meeting were approved.

Summary of core team discussions and decisions since last month's full RAC meeting (Richard)

See the minutes of the core team meeting for details.

Regional Production - progress in clarifying conditions for validity (Richard)

Most of the progress is captured in the US Regional Production: Summary of ATLAS Policy and US Practice. The CREM took up the issue of Regional Production at its meeting the day before. The trigger was a Canadian regional production that had successfully mopped up Tier-2 cycles, but had written its output to an unsuspecting TRIUMF. The CREM discussion was wide ranging. There was no disagreement on the desirability of regionally organized production using unpledged or unused resources and delivering validated datasets. There was intense discussion of the detailed mechanisms that should be involved, and general acceptance that disk space would be a key resource. The CREM agreed to hold further discussions at the BNL ADC Workshop next week, and to try to present a convergent summary at the Software and Computing week in April.

Stephane asked whether the clarified regional production situation was now known to US ATLAS physicists. Richard noted that it would be part of the (now) regular "News from the RAC" topic at the Monday Physics Forum, but also agreed to send mail to usatlas-l.

Jim commented that for most physicists, getting a regional production performed in accordance with all ATLAS approval processes would still be a daunting task. He proposed identifying a single point-of-contact in the US who could help physicists with this task. This proposal was welcomed and it was agreed to hold discussions next week to try to identify the person.

Available disk usage monitoring tools - summary (Kaushik)

Kaushik outlined the capabilities of the new monitoring tool released by the DQ2 team (http://voatlas89.cern.ch/run_search.py). The new tool was a vast improvement on the previous situation. It was still not possible to break down the dataset-access statistics according to analysis group. Under heavy usage (by Armen) the new tool seemed to crash frequently, but this was expected to improve. US operations had an outstanding request to the DQ2 team to make it possible to record an owner for replicas. Derived (dESD etc.) datasets on UserDisk did already have owners.

Richard repeated his view that ATLAS would need a mechanism to make analysis groups become stakeholders in datasets (and their replicas) involving some "disk space currency" allocated by collaboration management. It was pointed out that an allocation mechanism already exists for UserDisk.

Stephane triggered a discussion of how deletions should be handled and how potentially affected users could be informed:

  • Kaushik:
    • Derived datasets (dESD ...) mostly have owners and 2 weeks notice is given before deletion. Up to now this has been performed using central ATLAS mechanisms
    • Other datasets do not have replica owners (but there is a request to the DQ2 team to implement this concept, and also expiration dates for replicas)
    • Jobs submitted via Panda to the Tier 2 cluster should be unaffected by deletions (as long as some replicas remain)
  • Richard: There should be single place to go to find out what is scheduled for deletion and details of past deletions
  • Kaushik: All the data about past deletions is in the database, but there is still no Web tool to extract the information in a useful form. This should be a request to the DQ2 team.
  • Stephane: DQ2 or AMI?
  • Kaushik: Probably DQ2

Revisions to official ATLAS CPU and Disk requirements for 2011 and 2012 (Michael)

The discussion referenced ATLAS Computing Resource Needs for Data-taking in 2010-2012. Some key changes with respect to earlier plans were:
  • 60% of simulation to be at the Tier 1 and 40% at the Tier 2s (previously 40:60
  • Almost no raw data at the Tier 1s
  • AODs - two copies spread over Tier 1s (previously one copy per Tier 1)
  • ESDs - two copies spread over Tier 1s (previously one copy at BNL)
  • dESDs - no copy at Tier 1s (previously 2 copies spread over all Tier 1s)

This new approach seemed to cripple analysis on the facilities best able to perform it. Michael strongly argued (with no detectable dissent) that a disk buffer of about 2 PB at the Tier 1 (over and above the new lower requested pledge) would be critical for successful physics analysis.

There was enthusiasm from the meeting to present this issue, (as an issue, not a definite planned solution), at next week's Program Manager's Review.



Action Items

  1. 3/26/2010: Richard, Further publicize "US Regional Production: Summary of ATLAS Policy and US Practice".
  2. 3/26/2010: All but especially core team, Find time during the ADC Workshop next week to identify a point-of-contact for Valid US Regional Production
  3. 3/26/2010: Kaushik, Make formal request to the DQ2 team for a tool to display a record of what has been deleted when.
  4. 2/12/2010: All (but especially Kevin/Jim/Ian), Propose the parts of the activity list above that are your responsibility and identify any activities that do not appear to be an appropriate task for any of the current RAC members. (Now declared complete)
  5. 2/12/2010: Richard Non-urgent simulation tasks - consult with physics groups on the best way to establish two-way communication with the RAC to meet this and other needs (Ongoing – see discussion of regional production)

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback