r5 - 14 Apr 2009 - 16:25:14 - ShawnMckeeYou are here: TWiki >  Admins Web > MinutesDataManageApr14



Minutes of the US ATLAS Data Management meeting, Apr 14, 2009
  • Previous meetings and background : IntegrationProgram
  • Coordinates: Tuesdays, Noon Central
    • (309) 946-5300, Access code: 735188; Dial *6 to mute/un-mute.


  • Meeting attendees: Charles, Shawn, Patrick, Rob, Saul, Bob, Hiro, Wensheng, Pedro, Armen, Jim S., Torre, Jose, Alexei, Wei
  • Apologies:John
  • Guests: None

Topics for this week

  • Reprocessing status - Kaushik
    • Going at average rate of 6k jobs finished per day
    • Still 30k jobs left to do
  • Storage cleanup - All
    • All Tier 2's low on space, Tier 1 cleanup also needed
    • Stephane started central cleanup of aborted tasks at SWT2 and SLACT2
      • Seeing high rate of LFC dying at SWT2 since central deletion started (half dozen time in 3-4 days)
      • Seen ~4TB cleaned up, no LFC crashes, but problem with Bestman crashes (may be unrelated - probably firewall related)
      • Many LFC entries deleted, but files not deleted, but file deletion is a delayed process by 1-2 days (will provide statistics next week of how many files left behind on storage after 1 week)
    • Problem noticed by Patrick - central deletion leaves metadata in LFC
    • Will try other 3 Tier 2's, and then BNL next
    • After that, will delete 'obsolete' datasets too
    • Need plan to cleanup old deleted datasets http://panda.cern.ch:25880/server/pandamon/query/?mode=listAbortedDatasetsTXT
      • Charles and Hiro will work on a script
    • ESD deletion - check with users, reduce number of copies (Armen)
  • DQ2 adler32 plugin - Hiro
    • Passive mode in both site services at BNL
    • Flagged problem with some files due to network card
    • Look into making plugin active
    • NET2, AGLT2, SLACT2 are also interested in trying this
    • Wei is following up with Simone about FTS checksum validation
  • SCRATCHDISK deployment
    • BNL - done, AGLT2 - 9TB (45TB total, done, see http://head02.aglt2.org/dcache_pool.html or http://gate02.grid.umich.edu/aglt2se_tot.shtml; Need ToA entry added), MWT2 - 10 TB (created space already, create token in next few days), NET2 - 10 TB (by next week), SLACT2, SWT2 - later, after cleanup
    • What should the protection settings be for this space? (Who can write?) - consensus is usatlas1.
    • AGLT2 has setup space-tokens setup for the new SCRATCHDISK area for '/atas/Role=production', '/atlas/usatlas/Role=production', 'usatlas3','usatlas4'. See:
http://head02.aglt2.org/dcache_space_reservation.html. Not sure it this is the best way to proceed. Further details at: https://hep.pa.msu.edu/twiki/bin/view/AGLT2/HowToSetUpATLASSpaceToken
  • Hot issues
  • AOB
    • Major failure at BNL last week, some corrupted files from that, Hiro cleaning up
    • AGLT2 seeing files coming from TRIUMF, FZK. LYON, INFN, NDGF... to MCDISK (follow up with Alexei, Hiro)

-- KaushikDe - 14 Apr 2009

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback