r2 - 14 Jan 2009 - 13:28:33 - CharlesWaldmanYou are here: TWiki >  Admins Web > StorageValidation



Task in the IntegrationProgram to validate storage systems and consistency at US ATLAS sites.

Consistency checking

Atlas-wide tools

US-developed tools

  • "Complete Consistency Check" - ccc.py - more thorough checking of all datasets
  • Compares DQ2 catalogs, LFC entries, PNFS namespace, and contents of dCache pools.
  • Does not make any changes to filesystem or databases, but prints a list of "ghosts" and "orphans".
  • Additional scripts exist for dealing with ghost/orphan files.
  • Link for ccc.py: http://repo.mwt2.org/viewvc/admin-scripts/lfc/ccc.py?view=markup
  • Sample output: http://www.mwt2.org/sys/ccc
  • To be done:
    • Automated running via cron or similar
    • HTML output
    • version for non-dCache sites:
      • ccc.py reads output of pnfsDump, so all that is needed is a way to dump the storage namespace to a text file.

Site cleanup:

US-developed tools

  • Updated version of cleanse.py which uses LFC instead of LRC, renamed dq2site-cleanse.py
  • Has been modified to also clean up panda mover datasets which are not registered as DQ2 replicas.
  • dq2site-cleanse.py at: http://repo.mwt2.org/viewvc/admin-scripts/lfc/dq2site-cleanse.py?view=markup
  • Some stability problems due to segfaults in LFC module - varies by site.
  • To be done:
    • diagnose/fix LFC stability problems
    • pure-Python version that does not use _lfc.so (access LFC via SQL).

-- RobertGardner - 14 Jan 2009 -- CharlesWaldman - 14 Jan 2009

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback