r4 - 28 Apr 2009 - 14:10:51 - NurcanOzturkYou are here: TWiki >  Admins Web > FacilityWGAPMinutesApr28



Meeting of the Facilities working group on analysis queue performance, April 29, 2009



  • Meeting attendees: Rob, Saul, Nurcan, Rik, Mark
  • Apologies: Patrick, Horst

Schedule & Coordination

  • Updates, progress on sample generation production
  • MegaJam - collecting some info on the analysis datasets
  • Which email lists?

Analysis Queue testing reports (Nurcan)

  • AnalysisQueueJobTests
  • last week
    • SUSY nearly done
    • Now working on DPD making jobs. Tested on a full simulation sample (mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579_tid028664) and a private sample subscribed by hand by Akira (user.RichardHawkings.0108173.topmix_Egamma.AOD.v2).
    • Will start DPD making jobs this week. Next is TAG selection jobs.
    • See also notes at facility meeting, MintuesApr22?
  • this week
    • See site certification table for the status of stress testing of queues with different job types, AnalysisSiteCertification
    • Will still need to run SUSYValidation job at AGLT2 (91% success rate from last run, failures with "error code 256" were being investigated)
    • Stress testing with DPD making job is almost done, used a container dataset (mc08.105200.T1_McAtNlo_Jimmy.recon.AOD.e357_s462_r579/). MWT2 needs to be tested (dcache related problems last time, site will run a test), AGLT2 jobs were caught by failures with "error code 256" in the second run (on the container dataset).
    • Rik defined a TAG selection job, will use it in my testing
    • Data reprocessing job was to define by Mark Slater from Ganga/HammerCloud team , will check the status.
    • HammerCloud is now running in the US cloud, we need to discuss how often we like to run it and at what scale. I asked Mark Slater to add SUSYValidation and DPD making jobs into HammerCloud? , currrenly running a simple muon analysis (calculating invariant mass of dimuons).

Site readiness (Rob)

Additional ANALY queue jobs (Rik)

  • last time
    • Need a viable TAG job
    • Will get several people to start submitting jobs at MWT2
    • Difficult to figure out which datasets are at the Tier 2's - especially if you're not sure what you're looking for. Browser is very slow - and wildcards as well.
    • mc08 datasets should be available at Tier 2s.
    • Most tests at MWT2
    • Submitted ~ 2K jobs;
    • Find two types of periods - when jobs fail ~ 1%; other times - something is going at the site and large numbers of jobs fail.
    • ESD jobs accessing FDR - need to subscribe a small sample to Tier 2s. Rik will send this to Rob.
    • NET2 & SLAC - ~500 jobs.
    • Slowness of dataset browser or dq2_ls to answer questions about completeness of a container at a site (few hours). Can't wildcard search with container.
    • Will look for a TAG job, ARA job
  • this meeting
    • TAG selection job defined.
    • BNL, SLAC, NET2 - no problems
    • Two job errors at MWT2. Poolfilecatalog.xml failures. Follow-up with Marco.
    • Will start making it systematic, clean up and make as a regular test job.
    • Akria communicated w/ Kaushik to subscribe ESDs to Tier 2s - not sure if they're available yet.

Job metrics


  • Still problems in the Panda monitor - still issues with Oracle migration? Wrong port numbers. Also very slow. Will also be adding a hammer-cloud attribute. Can we have programmatic access to the database and/or web?
  • Decide when to meet again after US ATLAS discussion forum is setup.

-- RobertGardner - 22 Apr 2009

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback