ATLAS Software Releases and How To Use Them

ATLAS software releases at BNL

ATLAS software releases are compiled on a Scientific Linux platform. Two compiler options are available:

• tag=dbg: debug, unoptimized (but unavailable for most releases)
• tag=opt: optimized (-O2)

There are stable and nightly ATLAS software releases. Stable releases were installed from mirror of ATLAS kit distribution with Pacman tool. However, some old releases under /afs/usatlas/software/dist were not installed from kit, and you can find more documentations about them and setup at "old AFS releases".

Tools and services for ATLAS releases

Code repository

U.S. ATLAS Tier I Center maintains the read-only mirror of ATLAS SVN repository at SVNROOT=file:///usatlas/projects/svn. Follow this link for details.

LXR

The LXR cross-referencing tool allows to jump easily to the declaration of any global identifier.

Tag Collector

Tag Collector is a database of packages versions with a powerful web interface

Work with ATLAS releases downloaded from kits

1. The ATLAS releases 12 and lower downloaded from kits are available for SLC3 only and are installed in /usatlas/OSG/atlas_app/atlas_rel (and symbolic link /opt/usatlas/kit_rel/OSG_rel).
2. ATLAS releases 13.0.0 and higher are downloaded from kits and located in /afs/usatlas/software/downloads/releases (and a shorter symbolic link /opt/usatlas/kit_rel).
3. ATLAS nightly releases are available in /afs/usatlas/software/downloads/nightlies (bugfix and dev branches) also with shorter sym-links /opt/usatlas/kit_rel/bugfix and /opt/usatlas/kit_rel/dev.

They are self-sufficient and include all necessary setup scripts. See for details:

• Installing Atlas Software (on ATLAS Wiki)
• Beginning users should look at the Computing Workbook to get a basic idea on how to setup their account.

There are three different setups for the kit releases (>11.2.0 only): one using AtlasSetup, another using the requirements coming with the kit, the other using user's private requirements. Please refer to here for setup instruction of old releases (≤11.2.0).

Recipe-1 of setup for kit releases

You can use tool AtlasSetup. The setup is very simple:

• Example-1 for setting up AtlasOffline 15.9.0: source /opt/usatlas/kit_rel/AtlasSetup/aSetup.[c]sh 15.9.0.
• Example-2 for setting up AtlasProduction 15.9.0.3: source /opt/usatlas/kit_rel/AtlasSetup/aSetup.[c]sh 15.9.0.3.
• Example-3 for setting up with GroupArea: source /opt/usatlas/kit_rel/AtlasSetup/aSetup.[c]sh --grouparea=${ATLAS_GROUP_AREA} 15.9.0.3.

The default setting has already included your current directory into CMTPATH. If you do not like to add your current directory into CMTPATH, you can specify an option "--testarea=None" something like:

• source /opt/usatlas/kit_rel/AtlasSetup/aSetup.[c]sh --testarea=None 15.9.0

You can run source /opt/usatlas/kit_rel/AtlasSetup/aSetup.[c]sh --help for more options. For more details on using AtlasSetup, you can visit its wiki at CERN.

To use releases in OSG area, you only need replace "aSetup" with "osgSetup".

Recipe-2 of setup for kit releases

Instead of creating a private requirements file, you can use the requirements file coming with the kit as the following setup:

1. export CMTROOT=/afs/usatlas/cernsw/contrib/CMT/v1r20p20090520 for zsh/bash/ksh;
   setenv CMTROOT /afs/usatlas/cernsw/contrib/CMT/v1r20p20090520 for tcsh
2. source /path_to_kit/cmtsite/setup.sh [or .csh] -tag=32,release_number[,projectName]
3. export CMTPATH=${working_directory}:${CMTPATH} for zsh/bash/ksh;
   setenv CMTPATH ${working_directory}:${CMTPATH} for tcsh
   • You can check out packages and compile them under ${working_directory}.
4. If you need add GroupArea, you can do:
   export CMTPATH=${ATLAS_GROUP_AREA}:${CMTPATH} for zsh/bash/ksh;
   setenv CMTPATH ${ATLAS_GROUP_AREA}:${CMTPATH} for tcsh
5. source ${working_directory}/${your_package}/cmt/setup.sh [or .csh]

The first command performs CMT, ATLAS release management tool, setup, where projectName is optional and it default project is AtlasOffline. The last command sets full run-time environment. The middle one is needed if you like to add your own package on top of the release. For release 14.5.0, path_to_kit is /afs/usatlas/software/downloads/releases/14.5.0 or /opt/usatlas/kit_rel/14.5.0.

If no additional package is required, the above setup can be simpler. That is, you can skip the middle step and combine other two commands:

1. source /path_to_kit/cmtsite/setup.sh [or .csh] -tag=32,release_number,setup[,projectName]

Please note:

• The projectName must be specified as AtlasProduction for production patched releases,
  and as AtlasPoint1 for 13.0.25 patched releases (designed for M4 data) such as 13.0.25.4.

• If you are running pathena, you also need define an env variable like
  export PATHENA_GRID_SETUP_SH=/afs/usatlas/lcg/current/etc/profile.d/grid_env.sh
  In addition, run the following command to have pathena and/or prun available:
  source /opt/usatlas/bin/panda_setup.[c]sh
  For more details, go to https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda

Recipe-3 of setup for kit releases

As stated in the Computing Workbook, you can use your own requirements file to setup the environment:

• Prepare your requirements files in $HOME/cmthome directory with the following basic content:

       #---------------------------------------------------------------------
       set CMTSITE STANDALONE
       macro PROJ_BASE_RELEASE    "latest"
  
       macro PROJ_SUBDIR    "$(PROJ_BASE_RELEASE)" \
                bugfix      "bugfix/$(PROJ_BASE_RELEASE)" \
                dev         "dev/$(PROJ_BASE_RELEASE)"
  
       set SITEROOT /opt/usatlas/kit_rel/${PROJ_SUBDIR}
       macro ATLAS_DIST_AREA ${SITEROOT}
       macro ATLAS_TEST_AREA ${HOME}/testarea/${PROJ_BASE_RELEASE}
  
       # macro ATLAS_GROUP_AREA "/afs/usatlas/project/analysis/EventViewGroupArea/Numbered/13.0.40.1
       # apply_tag groupArea
  
       apply_tag oneTest 
       apply_tag 32
       # apply_tag setupCMT
       apply_tag setup
       use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)
       # setup_script "$(SITEROOT)/setup"
       # set CMTCONFIG i686-slc4-gcc34-opt
       set SVNROOT svn+ssh://svn.cern.ch/reps/atlasoff
  
       #---------------------------------------------------------------------
       

• Please note that you must NOT apply tag "setupCMT" anymore.

• If you are running pathena, you also need a line like set PATHENA_GRID_SETUP_SH /afs/usatlas.bnl.gov/osg/client/@sys/current/setup.sh
  In addition, run the following command to have pathena and/or prun available:
  source /opt/usatlas/bin/panda_setup.[c]sh
  For more details, go to https://twiki.cern.ch/twiki/bin/view/Atlas/DAonPanda
• Go to ~/cmthome/ directory and run "cmt config" (please ignore warning message like "/opt/usatlas/kit_rel/not_numbered not found") to create files such as setup.[c]sh under ~/cmthome/:

       cd $HOME/cmthome
       source /afs/usatlas/cernsw/contrib/CMT/v1r20p20090520/mgr/setup.[c]sh
       cmt config
       

Once you set up ~/cmthome/ as described in the above two steps, normally you need NOT do it again. However, you can later add additional CMT macros and patterns in the requirements file. Then for each release or a new terminal you execute the following (12.0.6, 14.5.0 and bugfix nightly rel_1 for examples):

1. cd ~/testarea/12.0.6
   or cd ~/testarea/14.5.0
   or cd ~/testarea/rel_1 for bugfix nightly rel_1.
2. source ~/cmthome/setup.[c]sh -tag=12.0.6,gcc323,slc4
   or source ~/cmthome/setup.[c]sh -tag=14.5.0
   or source ~/cmthome/setup.[c]sh -tag=bugfix,rel_1 for bugfix nightly rel_1.
3. Now you should be able to see an alias athena to athena.py .
4. If you need additional packages, then use "cmt co" to check them out, and/or compile them.

The tag "gcc323" and "slc4" are necessary for kit releases <13.0.0.

Notes on ATLAS releases downloaded from kits

• Distribution software is under development and Kits are not created successfully for all nightlies
• Nightlies installation status is reflected at BNL nightlies global page. This "global" page shows the list of nightly jobs running at BNL. The jobs responsible for downloads of nightly releases have titles "bugfix mirror" and "dev mirror"
• The available releases including nightlies at BNL can be found at here.
• ATLAS software releases used in the data production are downloaded in /usatlas/OSG/atlas_app/atlas_rel (that is on NFS) by OSG group
• For more information about releases please refer to ATLAS Offline Release Page

Use SVN at BNL

All old releases at BNL have been updated to allow to use svn now.

Use SVN repository at CERN

If your account username at CERN is same as that at BNL, then you can simply define the env variable SVNROOT to svn+ssh://svn.cern.ch/reps/atlasoff. Otherwise, you need explicitly specify your CERN account name in SVNROOT in the format of svn+ssh://YourCERNName@svn.cern.ch/reps/atlasoff. There is another option to omit your CERN account name in SVNROOT, i.e., just to add the following lines in $HOME/.ssh/config:

Host svn.cern.ch
   GSSAPIAuthentication yes
   GSSAPIDelegateCredentials yes
   # GSSAPITrustDNS yes
   Protocol 2
   ForwardX11 no
   User YourCERNName

Use SVN repository mirror at BNL

Just define the env variable SVNROOT to file:///usatlas/projects/svn, or use command bnl_cmt instead cmt. Please be aware that the SVN repository at BNL is just a mirror of ATLAS SVN repository, and this mirror is available for checkouts only. The import and commit operations should be performed with CERN ATLAS SVN repository. Please visit here for details.

Overriding DBRelease and poolcond

Overriding DBRelease

A default (symbolically linked by current under DBRelease/) version is provided in the kit distribution. Sometimes, esp. for AtlasProduction patches, other later version is required, which can found Atlas Database Release. You can override the default version of DBRelease with other versions already installed under DBRelease/ in that release by setting the following env variable:

   export DBRELEASE_OVERRIDE=${DBRELEASE_VERSION}

before setting up runtime environment.

There are more DBRelease versions installed at /afs/usatlas/software/downloads/DB/DBRelease, to use them, you need set both env variables DBRELEASE_OVERRIDE and export ATLAS_DB_AREA=/afs/usatlas/software/downloads/DB before runtime environment setup.

For setup using AtlasSetup (recipe-1), you need specify option --dbarea="/afs/usatlas/software/downloads/DB" and --dbrelease="${DBRELEASE_VERSION}" to override the default DBRelease, e.g.

   source /opt/usatlas/kit_rel/AtlasSetup/aSetup.sh 16.0.1 --dbarea="/afs/usatlas/software/downloads/DB" --dbrelease="12.7.1"

In case you are interested in overriding geomDB or sqlite200, part of DBRelease, you can simply create a sym-link to them under your run directory.

In case of pathena jobs, you can specify an option --dbRelease such as --dbRelease=ddo.000001.Atlas.Ideal.DBRelease.v070401:DBRelease-7.4.1.tar.gz. You can dq2-ls command to look up the available DBRelease datasets.

Overriding poolcond

The file catalogues (poolcond) of condition database coming with the kit may not be up-to-date enough. For event reprocessing, a more up-to-date poolcond might be necessary. We also provide a nightly replication in /usatlas/workarea/atlas/PFC/catalogue/poolcond with condition files residing in dcache. You can make your athena job to use this poolcond by creating a sym-link to this path under your working directory (or ${ATLAS_TEST_AREA}) by:

   ln -s /usatlas/workarea/atlas/PFC/catalogue/poolcond ${working_directory}/InstallArea/share/poolcond

Or you can add path /usatlas/workarea/atlas/PFC/catalogue directly into ${DATAPATH} before running your jobs.

Using poolcond in HOTDISK

Since release 15.4.0, IOVDbSvc provides a new way to use poolcond replicated in HOTDISK to override the default poolcond in the kit. To use HOTDISK, you can just need define an env varilable in the following:

• export ATLAS_POOLCOND_PATH=/usatlas/workarea/atlas/PFC_HOTDISK for zsh/bash/ksh;
  setenv ATLAS_POOLCOND_PATH /usatlas/workarea/atlas/PFC_HOTDISK for tcsh

In case of pathena jobs, pilot job will take care of defining this env variable for you.

CernVM? -FS at BNL

• CVMFS tests on aplay3 machine

Relevant links

• ATLAS Offline Release Page
• Beginniners should consult the Computing Workbook and the Physics Analysis Workbook for an introduction to ATLAS computing, physics topics, analysis recipes, etc. These two workbooks are actively maintained.
  Older guides/tutorials are available at,
  • Physics Analysis Tutorial with ATLAS software How To recipes
  • ASK tool tutorial with event generation, simulation, digitization and reconstruction recipes
• Additional information for ATLAS software users (complementary to ATLAS Workbooks). This page contains many useful recipes and workarounds.
• List of stable releases at U.S. ATLAS Tier I Center
• Status of nightlies downloads at BNL
• Status of nightlies at CERN
• Nightly Control System (NICOS) web page
• CMT documentation

Questions and comments

• Please send your concerns to Shuwei Ye and Alex Undrus