r56 - 29 Jul 2014 - 13:04:44 - ShuweiYeYou are here: TWiki >  AtlasSoftware Web > DatasetsOnXRD

Datasets at BNL dCache or T3 XROOTD server


This page summarizes the datasets that are currently being used on the BNL Tier-3 xrootd servers. The datasets are stored at BNL dCache, or at one of the two xrootd servers. The clist files for the datasets are kept accordingly in these two directories:

  1. dCache (xrootd door): /usatlas/u/xrdadmin/dcdcap_dataset/; free space on LOCALGROUPDISK: about 180 TB
  2. T3 xrootd Server 1 (xrd01): /usatlas/u/xrdadmin/xrd_copied_dataset/; free space: 27 TB; Recommended Proof server: bnlt3a10
  3. T3 xrootd Server 2 (bnlt3lrd1): /usatlas/u/xrdadmin/xrdT3_copied_dataset/; free space: 49 TB; Recommended Proof server: bnlt3s01

Software details about using PROOF and xrootd can be found on Proof Farms at BNL .

Generation of clist files for private datasets on BNL dCache

A tool pnfs_ls.py is provided to help generate clist files for given datasets on BNL dCache

% /afs/usatlas/scripts/pnfs_ls.py -h
     pnfs_ls.py [options] dsetListFile
     pnfs_ls.py [options] dsetNamePattern
     pnfs_ls.py [options] dsetNamePattern1,dsetNamePattern2[,more name patterns]

  This script generate file pnfs-path, xrootd-path, or dcache-path 
on BNL dcache for given datasets

  -h, --help            show this help message and exit
  -v                    Verbose
  -V, --version         print my version
  -i, --incomplete      Use incomplete sites if complete not available
                        write pfn list into a file instead of printing to the
                        write pfn list into a directory with a file per
  -x, --useXrootd       using dcache xrootd access, default is NFS-4.1
  -D, --useDCache       using dcache: access, default is NFS-4.1
                        specify a BNL site, overriding the one choosen by the

Note: dCache has been mounted as NFS on machine acas0785, and will be mounted as "NFS" on other acas machines. NFS and dcache access excludes each other. So on acas0785, you can NOT use dcache: to access, instead you need access via NFS or dCache xrootd door root://dcdcap02.usatlas.bnl.gov.

Datasets on bnl xrootd storage (out of date)

NTUP type/stream tag Comment N(datasets) Size(MB) Status
SMWZ mc11_7TeV p1035 WW, ZPrime 56 672,349 bnlT3

Major updates:
-- ShuweiYe - 31 Mar 2014
-- HongMa - 28 Apr 2011

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback