r2 - 07 Apr 2009 - 13:56:31 - KaushikDeYou are here: TWiki >  Admins Web > MinutesDataManageApr7



Minutes of the US ATLAS Data Management meeting, Apr 7, 2009
  • Previous meetings and background : IntegrationProgram
  • Coordinates: Tuesdays, Noon Central
    • (309) 946-5300, Access code: 735188; Dial *6 to mute/un-mute.


  • Meeting attendees: Shawn, Patrick, Saul, John, Charles, Armen, Rob, Wensheng, Bob, Michael, Pedro
  • Apologies: Wei, Pedro (first half)
  • Guests: None

Topics for this week

  • Reprocessing status - Kaushik
    • We had daily meetings past 5 days (except weekend)
    • Armen created special web page with important links http://www-hep.uta.edu/~vartap/monitoring/monitoring_links.htm
    • 55k jobs in 4.5 days (30k success, 25k failures)
    • Many service interruptions (LFC twice, Panda server twice, BNL queues...)
      • Michael - BNL queue problem fixed, GRAM space filled up, will implement more aggressive cleaning in future
      • Will revisit GRAM issue for all sites during facilities meeting
    • Job definition slow the first ~36 hours
    • HPSS throughput 10-15k files daily? (Need ATLAS accessible monitoring)
      • Michael - web page with HPSS snapshots will be available soon
      • Not enough requests, not all tape cartridges being used, increase nprestage to 4000
  • BNL staging/dcache - Pedro/Michael
    • Data carousel (using priority) coming soon
    • Checking a possible corruption problem
  • Tier 2 space cleanup - Bob/Shawn, Patrick
    • AGLT2 - working with Hiro, cleaning up files in wrong partition etc, v12 deleted, have some breathing room
    • AGLT2 - not too many ghosts and orphans, also cleaned up
    • We need to make scripts public, for now Charles' repsository, which will migrate to ATLAS subversion
    • SWT2 - looking at problem files (on LFC but not in DQ2 for example), about 15% files like this, still studying
    • SWT2 - concentrating on MCDISK and DATADISK
  • DQ2 adler32 plugin - Hiro
    • On vacation - pick up next week
  • User space cleanup at BNL - Armen
    • User data cleanup is on hold till space is found to keep some datasets needed by user
  • Local storage at BNL - Armen
    • Studying LOCALGROUPDISK issue
    • Report in future meeting
  • SCRATCHDISK deployment
    • Tied in with GROUPDISK and LOCALGROUPDISK discussions
    • BNL - done, AGLT2 - 9TB, MWT2 - 10 TB, NET2 - 10 TB, all by next week, SLACT2, SWT2 - later
    • New storage estimates from Kaushik needed
    • Pedro - need descriptions from Armen of all space tokens
  • Hot issues
    • Rob - Kaushik needs to act on Eric's ESD request
    • Saul - pilot scratch space filling up, AGLT2 cleans automtically after 5 days, SWT2 after 100 hours, NET2 will implement something similar
  • AOB

-- KaushikDe - 07 Apr 2009

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback