r1 - 09 Oct 2006 - 12:07:51 - TorreWenausYou are here: TWiki >  AtlasSoftware Web > ReplicaCatalog

ReplicaCatalog Local Replica Catalog (LRC) for US ATLAS DDM


LRC development work program as of Oct 2006

For operations and site support tasks see Alexei's action list from the BNL workshop (URL?)

All work described here should be documented here (documentation section below)

LRC DB in MySQL

  • Establish BNL DDM ops testbed for LRC and other development/testing (Wensheng et al)
  • Grid-based authentication (Wensheng)
  • Schema review and update
    • Move metadata from LFN association to PFN association
      • archive bit, last touch time
      • size, checksum
    • Partitioning support, for scalability
      • Supplementary metadata supplied on writes, or inferred from LFNs, or eventually derived from GUID, to drive partitioning
  • Scheme for avoiding transactions by using optimal ordering of SQL operations
  • Support for multiple instances in one MySQL DB. DB name 'localreplicas-instancename' instead of 'localreplicas'

httpd service

  • Eliminate POOL dependency
  • Bulk operations: return PFNs, metadata for list of LFNs, GUIDs
  • Support for multiple instances. Specify instance name in URL.

Client tools

  • Command line tools based on http interface
    • For list of LFNs or GUIDs, return PFNs and metadata
    • Site specification is any sitename in the ToA hierarchy -- map to appropriate LRC instance
  • Use this tool (as alternative to LFC lookup, not replacement) in dq2_* tools
    • Avoids LCG UI dependency

CERN deployment

  • LRC instance, one for each LFC to be mirrored, on dedicated CERN machines
    • Multiple instances per MySQL server, with different DBs. DB name localreplicas-instancename
  • httpd service serving all instances on a given server

LFC mirroring

  • LFC extraction and LRC loading script based on direct access to LFC back end DB
    • Rod has script to do extraction from LFC back end
    • Script usable either standalone or invoked during LFC consistency scans
      • Mirror only consistency-checked data?
    • Script needs to support full mirroring and incremental mirroring based on last update time

LRC performance demonstration, LFC/LRC evaluation

  • Does centralized (CERN) LRC deployment work
  • Relative performance, robustness of LRC and LFC for real use cases

LRC development program status and documentation


Major updates:
-- TorreWenaus - 09 Oct 2006

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments

 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback