AnalysisSiteCertification

Background

Part of the FacilityWGAP working group on analysis queue performance in the the IntegrationProgram. This page organizes analysis queue certification activities for each site in the facility.

• We need to certify sites to handle each type of analysis job, in advance of the STEP09 stress testing (June 2-12).
• Job preparations, see AnalysisQueueJobTests
• Site preparations and ANALY queue profiles: AnalyQueueCertificationPrep
• Metrics will be taken, see: AnalyQueueMetrics

Schedule and tasks

General analysis stress testing

• See notes from Akira at FacilityWGAPMinutesMar10
• Data generation/simulation job to be defined by Akira
• List of possible participants to be compiled by Rik
• A table of site configuration to be produced by Rob
  • See AnalyQueueCertificationPrep
• A table of well-defined job types and instructions for users (Nurcan, Rik)
  • See below

Other possible tasks

• Initial stress testing via UTA submit scripts (Nurcan)
• Specification of metrics for comparison to Hammer Cloud results, http://atlas-ganga-storage.cern.ch/test_188/
• Creation of Panda monitor views for analysis testing services

Schedule:

• March 15: Specify and test a well-defined set of job archetypes representing likely user analysis workflows hitting the analysis queues.
• March 31: Specify set of queue metrics and test measurements made, see AnalyQueueMetrics
• April 15: Initial job type testing to all queues
• May 1: Metrics monitoring in place
• May 1: all job type definitions finalized
• May 1: initial stress tests to all sites completed
• May 15: final set of job types tested
• May 15: dataset placement certification
• May 25-June 6: analysis stress test weeks begin

Certifications by job type

Notation:		completed		work is in progress		defer to next phase		table to be updated

Job type	Athena release	input	T1	AGLT2	MWT2	NET2	SWT2	WT2
SusyValidation	14.5.0	AOD
D3PD making with TopPhysTools	14.5.0	AOD
TAG selection	14.5.0	TAG
DB access job	14.5.1.4	AOD
Data reprocessing	14.5.0.5	DPD
ANLASC1	14.5.1	AOD
ANLASC2	14.2.23	ESD
ANLASC2 Jet sampling	14.5.1	ESD
More jobs types to come ...
HammerCloud Jobs

Panda monitoring of stress tests

• http://panda.cern.ch:25890/server/pandamon/query?dash=analysis&processingType=stresstest&reload=yes

Hammer Cloud monitoring

• Stress testing with the Ganga Robot - http://gangarobot.cern.ch/st/
• http://panda.cern.ch:25890/server/pandamon/query?dash=analysis&processing=hammercloud&reload=yes

References

• GangaRobot, HammerCloud stress testing

-- RobertGardner - 14 Apr 2009