r20 - 05 Jan 2007 - 20:22:39 - HongMaYou are here: TWiki >  AtlasSoftware Web > EventDataAccessAtBNL

Access to Event Data Produced by ATLAS Data Challenges

Introduction

Many of the datasets produced by the ATLAS data challenges are available at USATLAS Tier-1. The majority of the data are stored in the dCache file system. Please make sure your account is set up for accessing data in dCache.

The DC3 datasets can be accessed through Panda and DQ2, while the older DC2 and Rome datasets are located in some well defined directories.

CSC production

The current data challenge production is DC3 for Computing System Commissioning. Some useful links at CERN:

The status of datasets available at BNL can be best viewed through Panda physics dataset monitoring website. The database behind this page provide information about all datasets registered in DQ2. So you can find datasets available at CERN or other Tier-1 or Tier-2 sites.

The procedure for locating the data that are catalogued in DQ2 is as follows.

  • Locate the dataset you are interested in. Pay also attention to what release was used to produce the dataset. For example, this is a dataset of single muon produced by production version 11004103, csc11.007200.singlepart_mu2.recon.ESD.v11004103
  • Follow the instruction under Access on the above page for accessing the data. One can use the dq2_ls command to find the physical files at BNL:
dq2_ls -f -p csc11.007200.singlepart_mu2.recon.ESD.v11004103
and
dq2_poolFCjobO   -j csc11.005141.JimmyZmumu.recon.AOD.v11004204
to generate jobOpt for athena job. For instructions on how to use DQ2 end-user commands, see UsingDQ2 .

There is also a more detailed page by Tadashi, AthenaUsingDQ2 for using with Athena.

As usual, it is mandatory that the US part of the data challenge production is made available at BNL. The current plan is to subscribe for BNL all AOD datasets of non-US CSC data produced by recent releases in LCG and NG.

For other datasets (digit, ESD and CBNT) that already exist in DQ2 but not at BNL and you want to access them at BNL, we suggest you submit subscription request to dq2. For example,

/afs/usatlas.bnl.gov/Grid/Don-Quijote/DQ2_0_2_12_client_slc3/dq2 registerDatasetSubscription csc11.007001.singlepart_e_Et15.recon.ESD.v11004103 BNLTAPE

You need to have valid grid certificate to submit the request. The data files will be automatically replicated at BNL, and will be shown on the dq2 monitoring page. If you do not have grid certificate, see ObtainingGridCertificate for how to get one. If you can not submit the request, please send your request to Hong Ma. If the dataset replication does not succeed after a few days, you can submit a savannah ticket to the ATLAS DDM Operation.

Rome datasets

The Rome data sample is explained in this wiki page at CERN: RomeOverviewWiki. All AOD/CBNT files have been copied to BNL on an nfs disk:
/usatlas/magdacache001/common/Rome-AOD/datafiles/rome/recov10.
The complete chain (evgen/simul/digit/reco) of datasets produced in US can be found under
/pnfs/usatlas.bnl.gov/data/prod/rome/datafiles/rome/, and they are also registered in DQ2 monitoring page.

For the datasets produced outside of US, all the reco (ESD/AOD/CBNT) have been replicated, while some other types (evgen/simul/digit) are also available in the directory under this:
/pnfs/usatlas.bnl.gov/data/replica/rome/datafiles/rome

Some of the rome datasets are also registered in dq2. See rome.

If you need to replicate additional Rome data from other sites, and the datasets are already defined in dq2, you can submit subscription request as for csc data above. If the datasets are not registered in dq2, it is better to register them, then submit a subscription for replication.

To register a new dataset that has files on a site, prepare a PoolFileCatalog? .xml file for all the files, then use dq2_put command to register. For example:

dq2_put -o -p PoolFileCatalog.xml rome.004860.digit.pyt_WH120_munubb

Then submit subscription request:

/afs/usatlas.bnl.gov/Grid/Don-Quijote/DQ2_0_2_12_client_slc3/dq2 registerDatasetSubscription rome.004860.digit.pyt_WH120_munubb BNLPANDA


DC2 datasets

Description of DC2 datasets can be found in here.

The US production is stored in this directory: /pnfs/usatlas.bnl.gov/data/atlasdat/dc2/datafiles/dc2/ .

Most of the digit datasets are replicated to US. They can be found under this directory:

/pnfs/usatlas.bnl.gov/data/replica/dc2/datafiles/dc2

-- HongMa - 24 Feb 2006

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments

 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback