Tier Two Policy and Accounting Planning

Questions and Issues

• NotesFromHarvardWorkshop
• What information is provided by the RAC
• The RAC is supposed to resolve specific responsibilities and usage of Tier2 Centers
• What specific requirements come from analysis?
• What are the available resources and what is the expected ramp up profile?
• What is future of LCG accounting system, effort required?
• What accounting systems / information are available, by source and system.
  • from Panda (site, user identified by DN); cpu usage only in SI2K? -s; and walltime; additional information from Condor-G
  • from Monalisa (site, usatlas total)
  • from LCG
  • from OSG (in pipeline)
  • from local accounting sources (eg. BNL from Condor)
  • from future OSG-EGEE initiatives
• Do we distinguish between overall job consumption (pilot view) from actual ATLAS job payload.
• What is the future of the physics working groups roles as established at the ATLAS level,and how will these roles be implemented at the tier2-system level and within Panda?
• How are individual users consumption collected US ATLAS-wide?
• How can the current Privilege infrastructure developed in OSG be fully exploited by the production and analysis systems. Example: using extended proxy attributes to distinguish "pilot-types" such as production vs analysis, or by physics working group.

Tasks and Deliverables

• Reviewed 8/18/06, CB, RWG.

1. Provide current and projected-planned inventory of resources (CPU, storage, bandwidth).
   • This was discussed in the facilities group.
   • Static and dyanmic versions of this should be available.
   • See:
     • NortheastTier2ActionItems#Facilities
     • MidwestTier2ActionItems#Facilities
     • SouthwestTier2ActionItems#Facilities
2. Develop policy description for resource allocation by site (scheduling mechanisms, storage).
   • Attempt to write down a PolicyTemplate
3. Provide a single location where site-level policies are referenced.
   • SiteLevelPolicies
4. Develop a coherent strategy for the set of accounting systems avaliable, or to be developed, and identify shortcomings if any in the existing or near term systems.
   • Use internal Panda accounting information
   • Encourage development of OSG accounting systems and use what they provide.
   • Use local job scheduler accounting systems where possible to augment the above.
5. Develop a RAC-friendly accounting portal which includes references to all relevant accounting and reporting/display systems.
   • Done, see: AccountingInformation
6. Translate RAC policy decisions and priorities into site-level implementation plans at the VO and role-based account levels.
7. Define a multi-site quota/shares system.
   • Should come naturally for workloads run under Panda.
   • Question is whether this sufficient. For the moment rely on accounting and reporting systems so that usage is transparent.
8. Deploy/implement quota policies as specified by the RAC using whatever systems available.
   • Job schedulers must be configured to ensure managed production quotas are met.
   • Group/user level quotas will use Panda internal mechanisms.
   • Non-Panda production and analysis will be governed by the job scheduler priority settings.
9. Verify that policies are being implement by Tier2 sites.
   • Relies on accounting and reporting systems from the three sources identified above.

Milestones

• Reviewed 8/18/06, CB, RWG.

1. Report on current and projected resource inventory delivered to RAC on periodic basis (6 months?) as determined by RAC.
   • Incomplete
2. Delivery of policy description for BNL.
   • First need a common policy template
3. Delivery of policy description for NET2.
   • First need a common policy template
4. Delivery of policy description for MWT2.
   • First need a common policy template
5. Delivery of policy description for SWT2.
   • First need a common policy template
6. Delivery of policy description page for all sites.
   • Done, SiteLevelPolicies
7. Delivery of list of viable accounting systems available or planned.
   • Internal Panda, Monalisa, Gratia, local job scheduler information
8. Delivery of statement of minimal feature set required to carry out RAC policies.
   • Not done - presumably CPU resouce consumption in wall time, number of jobs processed, storage consumed.
9. Delivery of inventory of accounting systems capabilities and feature sets.
10. Decision on accounting system services for facilties.
    • Nothing imminent to decide.
11. Delivery of accounting system specifications for installation/configuration of chosen accounting systems at each Tier2 facility.
12. Delivery of RAC friendly accounting portal.
    • Started a twiki to gather information, AccountingInformation
13. Delivery of policy implementation instructions for Tier2 facilties on a periodic basis, according to RAC processes.
    • No progress
14. Delivery of tools to RAC to provide auditing capability.
    • Will be done via AccountingInformation.
15. Report on available toolsets for a multi-site quota system. (Perhaps use the RAC-friendly accounting portal.)
    • Covered by the above.

Schedule

• See notes above, 8/18/06, CB, RWG.

1. MS1: May 31
2. MS2-6: May 31
3. MS7: June 15
4. MS8: June 15
5. MS9: July 1
6. MS10: Aug 1
7. MS11: Aug 15
8. MS12: Aug 20
9. MS13: Periodic
10. MS14: Oct 1
11. MS15: Dec 1

Resources

• Resource Allocation Committee
• OSG site policy page summary: http://vors.grid.iu.edu/site_policies.html

-- RobertGardner - 09 May 2006