ReplicaCatalog Local Replica Catalog (LRC) for US ATLAS DDM

LRC development work program as of Oct 2006

For operations and site support tasks see Alexei's action list from the BNL workshop (URL?)

All work described here should be documented here (documentation section below)

LRC DB in MySQL

• Establish BNL DDM ops testbed for LRC and other development/testing (Wensheng et al)
• Grid-based authentication (Wensheng)
• Schema review and update
  • Move metadata from LFN association to PFN association
    • archive bit, last touch time
    • size, checksum
  • Partitioning support, for scalability
    • Supplementary metadata supplied on writes, or inferred from LFNs, or eventually derived from GUID, to drive partitioning
• Scheme for avoiding transactions by using optimal ordering of SQL operations
• Support for multiple instances in one MySQL DB. DB name 'localreplicas-instancename' instead of 'localreplicas'

httpd service

• Eliminate POOL dependency
• Bulk operations: return PFNs, metadata for list of LFNs, GUIDs
• Support for multiple instances. Specify instance name in URL.

Client tools

• Command line tools based on http interface
  • For list of LFNs or GUIDs, return PFNs and metadata
  • Site specification is any sitename in the ToA hierarchy -- map to appropriate LRC instance
• Use this tool (as alternative to LFC lookup, not replacement) in dq2_* tools
  • Avoids LCG UI dependency

CERN deployment

• LRC instance, one for each LFC to be mirrored, on dedicated CERN machines
  • Multiple instances per MySQL server, with different DBs. DB name localreplicas-instancename
• httpd service serving all instances on a given server

LFC mirroring

• LFC extraction and LRC loading script based on direct access to LFC back end DB
  • Rod has script to do extraction from LFC back end
  • Script usable either standalone or invoked during LFC consistency scans
    • Mirror only consistency-checked data?
  • Script needs to support full mirroring and incremental mirroring based on last update time

LRC performance demonstration, LFC/LRC evaluation

• Does centralized (CERN) LRC deployment work
• Relative performance, robustness of LRC and LFC for real use cases

LRC development program status and documentation