Instructions for Participants

Version 4.0: 11 Janvier, 2006
Ketevi A. Assamagan

What should I do before coming to the tutorial?

It is assumed that the user is familiar with how to use ATHENA. If that is not the case, he or she should consult The ATLAS WorkBook and glance at these tutorials before coming. The user may also want to take this tutorial before proceeding further.

Keep the Physics Analysis Tools page and the how-to for users as a reference.

If you are not familiar with the ATLAS software, refer to the documentation mentioned above before coming to the tutorial. We will use the ATLAS software release 11.0.3 for this tutorial.

In the following tutorial, the tcsh is assumed, the default on the US ATLAS linux farm. If you work in the bash shell, for example on the CERN lxplus, simply use setup.sh everywhere instead of setup.csh. If you are working at CERN, the input RDO file used in the Exercises 1-5 can be founded in castor: /castor/cern.ch/atlas/csc/valiprod/sampleA/mc11/004100.T1_McAtNLO_top/digit/v11000301/ 

Exercise 0: Make sure that CMT is setup correctly

The Agenda

You need a grid certificate to do Exercise 4 - Distributed Analysis. If you do not have a grid certificate, contact your system administrator and manage to get one before attempting Exercise 4: follow the instructions here.

Physics Analysis Tools - Overview

Exercise 1: ESD/AOD/TAG Production
Exercise 2: User Analysis Algorithm
Exercise 3: Tag Based Event Selection and Athena-Aware NTuple
Exercise 4: User Analysis in Distributed Analysis Environment
Exercise 5: Interactive analysis and Event Display
Exercise 6: An example on AOD
Exercise 7: TrackParticle and Navigation
Exercise 8: Monte Carlo Truth
Exercise 9: An example on ESD - Adding another Algorithm
Exercise 10: Overlap checking, Redundancy Removal, Event View
Some AOD Analysis Tools

Exercise 0: Make sure that CMT is setup correctly

If you have already taken this tutorial, some of the Exercise 0 may not be necessary for you: skip the un-necessary steps.

0.1 Login to a US ATLAS Linux Machine at BNL or to lxplus at CERN

ssh atlasgw00.bnl.gov
xterm -i
At CERN, you would login to lxplus as follows: 
ssh lxplus.cern.ch
If you are working on the US ATLAS machines, make sure that this line is in your .login file in your home directory. If not, you should add it, log out and login in again (do not do this if you are working at CERN): 
source /afs/usatlas.bnl.gov/i386_redhat72/opt/dcache/dcache_root_client_config.csh

0.2 Create a directory called tutorial

cd
mkdir tutorial
cd tutorial

0.3 Check that my requirements file is set for 11.0.3

Here is Jane Doe's requirements file. If you do not have a file called "requirements", create one add the lines below to it.

If you are working on the US ATLAS machines, your requirements file should look like this: 

###################################################################
set     CMTSITE         BNL
set     SITEROOT        /afs/usatlas.bnl.gov
macro   ATLAS_DIST_AREA ${SITEROOT}/software/dist
macro   ATLAS_TEST_AREA "" \
        11.0.3          "${HOME}/tutorial"

use     AtlasLogin      AtlasLogin-*    $(ATLAS_DIST_AREA)
###################################################################
Then do this: 
source /afs/usatlas.bnl.gov/cernsw/contrib/CMT/v1r18p20050501/mgr/setup.csh
cmt config
If you are working at CERN, your requirements file should look like this: 
###################################################################
set     CMTSITE         CERN
set     SITEROOT        /afs/cern.ch
macro   ATLAS_DIST_AREA ${SITEROOT}/atlas/software/dist
macro   ATLAS_TEST_AREA "" \
        11.0.3          "${HOME}/tutorial"

use     AtlasLogin      AtlasLogin-*    $(ATLAS_DIST_AREA)
###################################################################
then do this - if you are working at CERN: 
/afs/cern.ch/sw/contrib/CMT/v1r18p20050501/mgr/setup.sh
cmt config

0.4 Create the working directory

mkdir 11.0.3

0.5 Setup CMT

Jane Doe's working directory for this tutorial is "${HOME}/tutorial/11.0.3" where
${HOME}=/usatlas/u/janedoe. So, Jane Doe will setup CMT by doing this

source setup.csh -tag=11.0.3,opt
Then, Jane Doe checks that the CMT path is correctly defined by doing this:
echo $CMTPATH
Something like this is printed to the screen
/usatlas/u/ketevi/tutorial/11.0.3:
/afs/usatlas.bnl.gov/software/dist/11.0.3:
/afs/usatlas.bnl.gov/offline/external/Gaudi/0.14.6.14-pool201:
/afs/usatlas.bnl.gov/offline/external/LCGCMT/LCGCMT_33
The path to Jane Doe's working directory, to the ATLAS software distribution, to external libraries are all defined.

0.6 Change Directory to your working area for this tutorial

Jane Doe does the following:

cd 11.0.3

0.7 Reconstruction/RecExample/RecExCommon

So Jane Doe checks out this package by doing this.

cmt co -r RecExCommon-00-04-38 Reconstruction/RecExample/RecExCommon

0.8 Compile, and build this package

cd Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/cmt
cmt config
source setup.csh
cmt broadcast gmake

0.9 Test it

Get the following files for the release and run ATHENA: 

cd ../run
get_files PDGTABLE.MeV
get_files HelloWorldOptions.py
athena.py -b HelloWorldOptions.py
Stuff like this should be printed to the screen
HelloWorld           INFO execute()
HelloWorld           INFO An INFO message
HelloWorld           WARNING A WARNING message
HelloWorld           ERROR An ERROR message
HelloWorld           FATAL A FATAL error message
AthenaEventLoopMgr   INFO   ===>>>  end of event 9    <<<===
HistorySvc           INFO Service finalised successfully
ChronoStatSvc.f...   INFO  Service finalized succesfully
ToolSvc              INFO Removing all tools created by ToolSvc
ApplicationMgr       INFO Application Manager Finalized successfully
ApplicationMgr       INFO Application Manager Terminated successfully

If you successfuly reach this point, you are ready for the tutorial, otherwise, please seek help before continuing.

0.10 The next time you login

The next you login to the USATLAS Linux machines, you set up CMT by just doing the following

In ${HOME}/tutorial directory or where you have your "requirements" file

source setup.csh -tag=11.0.3,opt
Then
cd 11.0.3/Reconstruction/RecExample/RecExCommon/.../cmt
source setup.csh
cd ../run
Take a look at Information for New Users for further details on how to get started with the ATLAS software.

Back to the top

Exercise 1: ESD/AOD/TAG Production

Since the release 10.4.1, the ESD, AOD and TAG are produced by default from the standard reconstruction.

1.1 Set up for the ESD/AOD/TAG Production

You should already be in the "run" directory of RecExCommon, if not, change directory to the "run" directory of RecExCommon. Get the reconstruction top options file from the release: 

get_files myTopOptions.py
Add the following lines to your myTopOptions.py file just above this line:
# DetFlags modifications are best set here (uncomment RecExCommon_flags first)
to produce the ESD/AOD/TAG. 
############################################################################
#The Detector Description
DetDescrVersion="ATLAS-DC3-02"
 
#Number of Events to process
EvtMax=5

#Input Raw Data
PoolRDOInput=["/usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root"]

#Suppress the production of NTuple Histograms
doCBNT = False
doHist = False

# AOD Flags are set here - comment out and set appropriately
from ParticleEventAthenaPool.AODFlags import AODFlags
AODFlags.Streaming = False # switch off streaming
#############################################################################
Note that if you are working at CERN, your input Raw Data (RDO) is on castor. This means that in the above and wherever relevant in the Exercises 1-5, PoolRDOInput should be set to: 
PoolRDOInput=[
"castor:/castor/cern.ch/atlas/csc/valiprod/sampleA/mc11/004100.T1_McAtNLO_top/digit/v11000301/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root"
]
Now produce the ESD, AOD and TAG:
athena.py -b myTopOptions.py
 You can can change the name and set the location of the ESD/AOD/TAG files by adding these lines to your myTopOptions.py file: 
PoolESDOutput = "ESD output file path and name"
PoolAODOutput = "AOD output file path and name"
CollOutput = "TAG output file path and name - no extension: .root will be appended"

1.2 Configuring your ESD/AOD production jobs

(a) Create a new file called optRecExToESD.AOD.TAG.py for the options to produce the ESD/AOD/TAG and add these lines to the file and save it.

############################################################################
#The Detector Description
DetDescrVersion="ATLAS-DC3-02"

#Number of Events to process
EvtMax=5
 
#Input Raw Data
PoolRDOInput=["/usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root"]

#Suppress the production of NTuple Histograms
doCBNT = False
doHist = False

# AOD Flags are set here - comment out and set appropriately
from ParticleEventAthenaPool.AODFlags import AODFlags
AODFlags.Streaming = False # switch off streaming
#############################################################################
(b) Get the reconstruction top options file: RecExCommon_topOptions.py:
get_files RecExCommon_topOptions.py
 (c) Rename the file myTopOption.py to ESD.AOD.TAG_topOptions.py and get a new copy of myTopOptions.py from the release: 
mv myTopOptions.py ESD.AOD.TAG_topOptions.py
get_files myTopOptions.py
(c) You can also produce the ESD/AOD/TAG in the following way - but do not do it in this tutorial. This is just show the many different ways to produce the ESD/AOD/TAG.
athena.py -b optRecExToESD.AOD.TAG.py RecExCommon_topOptions.py
 or 
athena.py -b optRecExToESD.AOD.TAG.py myTopOptions.py
 (d) It is also possible to specify the options at the command line: 
athena.py -b -c 'EvtMax=100; PoolESDInput=["ESD.pool.root"]; PoolAODOutput="AOD.pool.root"; ...' RecExCommon_topOptions.py
Where do I find the list of the reconstruction flags?. Look also here for further details

1.3 Produce your own AOD from a given ESD

Create a new file called optESDtoAOD.py for the options to produce the AOD from the ESD and add the following lines to the file and save it. (Since we are producing the AOD from the ESD, we need to have the ESD file available. Use the ESD that you produced in Exercise 1.1 as the input for the AOD production. The default output name for the AOD is AOD.pool.root). 

#############################################################################
AllAlgs = False
EvtMax=5
DetDescrVersion="ATLAS-DC3-02" 
PoolESDInput = ["ESD.pool.root"] # change file name and path appropriately
PoolAODOutput = "AOD.FromESD.pool.root" # change file name an dpath
doWriteESD = False
doWriteTAG = False 
readESD = True
doCBNT = False
doHist = False

# AOD Flags are set here - comment out and set appropriately
from ParticleEventAthenaPool.AODFlags import AODFlags
AODFlags.Streaming = False # switch off streaming
#############################################################################
Produce the AOD: 
athena.py -b optESDtoAOD.py myTopOptions.py
 To specify a different file name for your AOD, add this line:
PoolAODOutput = "file path and name"

Check that the following files are available in your "run" directory after the production of the ESD/AOD/TAG: 
-rw-r--r--    1 ketevi   usatlas   1194711 Dec 26 12:13 AOD.FromESD.pool.root
-rw-r--r--    1 ketevi   usatlas   1201618 Dec 26 12:00 AOD.pool.root
-rw-r--r--    1 ketevi   usatlas   5693334 Dec 26 12:00 ESD.pool.root
-rw-r--r--    1 ketevi   usatlas     74516 Dec 26 11:57 MyColl.root               <--- TAG
-rw-r--r--    1 ketevi   usatlas       657 Dec 26 12:13 PoolFileCatalog.xml

AOD Production Flags

The AOD production flags allows one to customize the production of AOD. For example, if the input raw data is byte stream, no Monte Carlo (MC) truth information is available: on may want to turn of the production of the MC truth AOD. For the list of the AOD production flags and their default status, see this page To use these flags, for example to turn off the production of the MC truth AOD, you will include this fragment in your job options: 
# AOD Flags
from ParticleEventAthenaPool.AODFlags import AODFlags
AODFlags.SpclMC = False

1.4 Running on many input files

For the ESD/AOD/TAG production in Exercise 1.1, we only used one raw data file which has only 100 events. For more statistics on the same dataset, you can specify more input files. Suppose you want 200 events in the ESD: since each of these ttbar files have 100 events, the PoolRDOInput (see Exercise 1.1 above) could be changed to the following:

PoolRDOInput = [
"/usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root",
"/usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00002.pool.root"
]
The same is true for the production of the AOD, if you run over many different input ESD files.

1.5 Fast Simulation AOD

Fast simulation AOD can be obtained from generated events. Suppose you use Pythia to generate ttbar events and you save the output of the generator as POOL file:

get_files produceGenEvents_jobOptions.py
Edit this file to see what is in it: the generation of inclusive ttbar events in Pythia. In 11.0.3, comments out the following lines: 
Pythia.PythiaCommand += ["pypars mstp 51 7"]
Pythia.PythiaCommand += ["pypars mstp 128 1"]
Pythia.PythiaCommand += ["pyinit output EventListing.out","pyinit dumpr 1 3"];
Pythia.PythiaCommand += ["pystat 3"]
Pythia.PythiaCommand += ["pyinit pylistf 1"]
Pythia.PythiaCommand += ["pyinit pylisti 12"]
Run this job options:
athena.py -b produceGenEvents_jobOptions.py
It will create an output file, McEvent.root. The file contains the output of generator. Note that the simulation steps are as follow:

  1. Event generation: you run Pythia or some other generator and you save the output in POOL files as we just did above - look here for details.
  2. Detector simulation: you simulate the detector response by using 1. as input. The output of this process is also saved as POOL files - for details, look here.
  3. Digitization: simulation of the electronic output or the raw data. 2. is used as input and the output can either be byte stream files or POOL files - look here for details.
  4. Reconstruction and ESD production: 3. is used as input. One could also use simulated byte stream files, or combined test beam byte stream files - look here for details. The generated files, the simulated files, the digitized files and ESD all contain the generated event record. This means you can use any of these files as input for the fast simulation AOD. Alternatively, you can produce your own generated events, using the job options produceGenEvents_jobOptions.py for example, then in a subsequent steps, use these generated events as input to your fast simulation AOD production. For further details on fast simulation in ATLAS, see this page.
Get the fast simulation AOD production options from the release: 
get_files FastSimToAOD_topOptions.py
Edit this file and replace this line 
    INPUT = [ "rfio:/castor/cern.ch/atlas/project/dc2/preprod/evgen804/dc2.002896.pyt_h130_llll.evgen804/data/dc2.002896.pyt_h130_llll.evgen804._0001.pool.root" ]
with this one 
    INPUT = [ "McEvent.root" ]
Also, after the following line: 
AODFlags.MissingEtTruth = True
add this line so that the output of the generation in the McEvent.root is used as input to the fast simulation: 
 AODFlags.McEventKey="GEN_EVENT"
Now produce your fast simulation AOD
athena.py -b FastSimToAOD_topOptions.py
Note that you can change number of events, the input file name and the AOD output file name, directly at the command line: look in FastSimToAOD_topOptions.py for further details. But try this:
athena.py -b -c 'INPUT=["McEvent.root"]; OUTPUT="fastAOD.root"; EVTMAX=100' FastSimToAOD_topOptions.py
You can also use the output of the Geant 4 simulation - i.e., the generator output plus the Geant interactions - as the input to your fast simulation: this is called "TruthEvent" as opposed to "GEN_EVENT". "TruthEvent" is saved by default in the simulated files, the digitized (or Raw Data files) and in the ESD, any of which can therefore be used as the input to the fast simulation. Before you try this, edit FastSimToAOD_topOptions.py and comment out this line: 
 
AODFlags.McEventKey="GEN_EVENT"
so that "TruthEvent" from the ESD will be taken by default. Now run the fast simulation on the ESD: 
 
athena.py -b -c 'INPUT=["ESD.pool.root"]; OUTPUT="fastAOD.root"; EVTMAX=100' FastSimToAOD_topOptions.py
Note however that ONLY the generator part of "TruthEvent" is used for Atlfast. Furthermore, since the relase 11.1.0, the fast and full simulated AOD are saved together in the same output file.

1.6 Large Scale Production of ESD/AOD

For large scale production, one should consider creating shell scripts to submit several jobs on the LSF batch system, on Condor, or on the grid - or customize job transformations used in the central ATLAS production. Some example scripts are available in the AnalysisExample. A job transformation for ESD, combined NTuple and AOD production can be in rome.reco.trf

1.7 Comments of ESD/AOD Production

Since the release 10.4.1, the default reconstruction produces the ESD, AOD and Tags by default: this means that you do not need to explicitly switch ON these production flags (doWriteAOD=True for example). Rather, you explicitly switch them OFF when you do not want ESD/AOD/Tags produced. To see how to configure your ESD, AOD and Tag production in the release 10.4.1 or higher and how to customize contents of your ESD/AOD, see this page.

1.8 Merging AOD files

It is sometimes necessary to merge small AOD files into bigger, more manageable chunks. The Utilities exist to do this. Look in MergingAOD and follow the instructions there.

Back to the top

Exercise 2: The Analysis Algorithm

2.1 Create your own Analysis Algorithm

We provide a package, UserAnalysis, to help you get started with the development of your analysis codes. This package consists of a simple Gaudi Algorithm, the AnalysisSkeleton. The user may check out this package and start implementing his/her analysis code: look here for further details. This package has a "run" directory where you can also run the reconstruction, the ESD and AOD and tag production as you've just done above in the RecExCommon package.

cd $HOME/tutorial/11.0.3
cmt co -r UserAnalysis-00-05-11 PhysicsAnalysis/AnalysisCommon/UserAnalysis
Just for the purpose of the tutorial, we will also check out 3 additional packages as follows (you will not need to do this in the release 11.0.4 or higher): 
cmt co Reconstruction/CBNT_Utils
cmt co Reconstruction/CBNT_Athena
cmt co PhysicsAnalysis/AnalysisCommon/AnalysisTools
Now change directory to the "cmt" directory of the UserAnalysis package: 
cd PhysicsAnalysis/AnalysisCommon/UserAnalysis/UserAnalysis-00-05-11/cmt
then compile and build everything: 
cmt config
source setup.csh
cmt broadcast gmake
After a successful build, change directory to the "run" directory of the UserAnalysis package, get the files AnalysisSkeleton_jobOptions.py and PDGTABLE.MeV: 
cd ../run
get_files AnalysisSkeleton_jobOptions.py
get_files PDGTABLE.MeV
Edit the AnalysisSkeleton_jobOptions.py to specify the input AOD file: use the AOD that you produced in Exercise 1 in the RecExCommon package: you will replace these lines (substitute the appropriate user name for "ketevi" or supply the correct path to the AOD that you produced in Exercise 1).: 
EventSelector.InputCollections = [ 
  "AOD.pool.root"
                                 ]
with these lines, to point to the location of the AOD that you produced in Exercise 1; for example: 
EventSelector.InputCollections = [ 
  "/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root"
                                 ]
Now run the AnalysisSkeleton_jobOptions.py: 
athena.py -b AnalysisSkeleton_jobOptions.py
It should successfully, producing a ROOT output file called AnalysisSkeleton.aan.root. This contains an Athena-Aware NTuple (AAN) and some histograms. Let's browse in the content of the file in ROOT: 
root AnalysisSkeleton.aan.root
At the ROOT prompt type this: b=TBrowser()
The ROOT Browser will appear. Double click on "ROOT Files", then double click on AnalysisSkeleton.aan.root
The histograms are under the directory Electron and the NTuple is under the directory CollectionTree (the histograms and NTuple contains the same information).

2.2 Histograms and NTuples in your Analysis Algorithm

Now, let's look at the AnalysisSkeleton algorithm to understand the following: 

(1) How to produce your own customized, small AAN from the ESD or the AOD
(2) How to produce histograms
(3) How to save histograms and NTuples
It is recommended that you produce a small AAN as an output and finish your analysis in ROOT: with histograms, everytime you need to change, for example, the number of bins, you will have to recompile and re-run your analysis codes. For further details on histograms and NTuple in Athena, look here. We will come back the AAN after the exercises on the Event Tags and on the Event View.

Back to the top

Exercise 3: Tag based Event Selection and Athena-Aware NTuple

The tags are produced by default together with the ESD and AOD: 
-rw-r--r--    1 ketevi   usatlas   5693374 Dec 26 15:02 ESD.pool.root
-rw-r--r--    1 ketevi   usatlas    252779 Dec 26 15:02 fastAOD.root
-rw-r--r--    1 ketevi   usatlas    653218 Dec 26 15:02 McEvent.root
-rw-r--r--    1 ketevi   usatlas     74516 Dec 26 15:02 MyColl.root           <---- Event Tags
-rw-r--r--    1 ketevi   usatlas    353574 Dec 26 15:02 AOD_FastSim.pool.root
-rw-r--r--    1 ketevi   usatlas   1194711 Dec 26 15:02 AOD.FromESD.pool.root
-rw-r--r--    1 ketevi   usatlas   1201147 Dec 26 15:02 AOD.pool.root
-rw-r--r--    1 ketevi   usatlas     19858 Dec 26 14:08 AnalysisSkeleton.aan.root
For further details and feedback on the Event Tag, look here. The definition of the tag content for initial running and steady state operation is still evolving: Look the activities of the Event Tag working group. We will use the tags that you produced in Exercise 1. We will select events where the loose leading electron Pt > 20 GeV and the number of loose electrons < 3.

3.1 The PoolFileCatalog.xml

Generate the PoolFileCatalog.xml as follows: 

At BNL do this (replace the path to your ESD and AOD as appropriate):

cp ~ketevi/tutorial/11.0.3/data/PoolFileCatalog.xml .
pool_insertFileToCatalog /usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root
pool_insertFileToCatalog /direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/ESD.pool.root
pool_insertFileToCatalog /direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root

At CERN do this (replace the path your ESD and AOD as appropriate):

cp ~ketevi/scratch0/Tutorial/11.0.3/data/PoolFileCatalog.xml .
pool_insertFileToCatalog castor:/castor/cern.ch/atlas/csc/valiprod/sampleA/mc11/004100.T1_McAtNLO_top/digit/v11000301/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00030.pool.root
pool_insertFileToCatalog /direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root
pool_insertFileToCatalog /direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root
You may look in your PoolFileCatalog.xml to check that the raw data, the ESD and AOD are defined.

3.2 Analysis on Tag Selected Events

Edit AnalysisSkeleton_jobOptions.py and replace these lines (substitute the appropriate user name for "ketevi" or supply the correct path to the tags that you produced in Exercise 1). 

EventSelector.InputCollections = [ 
  "/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root"
                                 ]
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.aan.root' OPT='UPDATE'"]
with these lines 
EventSelector.InputCollections = [ 
  "/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/MyColl"
                                 ]
EventSelector.CollectionType="ExplicitROOT"
EventSelector.Query="(NElectron < 3) && (abs(ElectronPt1) > 20000)"
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.tag.root' OPT='UPDATE'"]
Run the AnalysisSkeleton_jobOptions.py. Now open both AnalysisSkeleton.aan.root and AnalysisSkeleton.tag.root in ROOT and check that the above selection has been applied.

3.3 Collection of Events of Interest

In this exercise, we will collect the events of interest --- by doing queries on the tags --- into our own ESD and AOD, and then do the analysis only of those events. For further information on how to collect events of interst, follow this link. Let's copy the relevant job options for the RecExCommon package: 

cp $HOME/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/myTopOptions.py .
Edit myTopOptions.py, switch off the TAG production flag (we've already produced the tags in Exercise 1. We just want to use the tags for selection and event collections), comment out the input RDO file and add the tag based options that we used above in Exercise 3.2. Further, switch off all the algorithms, switch of the MC Truth, switch off all the AOD builders and switch off all the detectors. Your myTopOptions.py should look like this (substitute the appropriate user name for "ketevi" or supply the correct path to the tags that you produced in Exercise 1): 
#switch off all Algorithms and also the MC Truth
AllAlgs = False
doTruth = False

# new AOD to contain ONLY the selected events of interest
PoolAODOutput = "selAOD.pool.root"
doWriteTAG = False

# Read in the tags and make selections
readColl = True
CollInput = [
"/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/MyColl"
            ]
CollInputQuery = "(NElectron < 3) && (abs(ElectronPt1) > 20000)"

# Switch off histograms and NTuples
doCBNT = False
doHist = False

#Switch off all the detectors
include ("RecExCommon/RecExCommon_flags.py")
DetFlags.ID_setOff()
DetFlags.Calo_setOff()
DetFlags.Muon_setOff()

# Switch off all the AOD buildeers
from ParticleEventAthenaPool.AODFlags import AODFlags
AODFlags.allSetOff()

# needed for the AOD Truth - will not be needed in 11.0.4
theApp.Dlls += ["TruthParticleAlgs"]

# The main Recontruction options
include ("RecExCommon/RecExCommon_topOptions.py")

#To collect Event at the ESD level - integrated into RecExCommon as a flag from 11.2.0
#EventSelector = Service( "EventSelector" )
#EventSelector.RefName="StreamESD"
Run this job options as follows: 
athena.py -b myTopOptions.py
It should produce an AOD file called selAOD.pool.root: this contains only the events that pass the selection on the tags. Now you can just analysize these events. Compare the sizes of the original, un-cut AOD and this new one: 
-rw-r--r--    1 ketevi   usatlas   1201147 Dec 26 12:33 AOD.pool.root
-rw-r--r--    1 ketevi   usatlas    806894 Dec 27 10:05 selAOD.pool.root
In RecExCommon, It will possible to collect interesting events at the ESD, or even at the Raw Data level as well from 11.2.0. For more information on how to build tag based queries, or about the tag infrastructure in general, look here.

3.4 The Athena-Aware NTuple (AAN)

The Athena-Aware NTuple can be used as input to other Athena jobs, for example to pre-select events or to navigate back to the full event at AOD, ESD, RDO level, in the same way as the tags; in addition, the AAN is your analysis data in ROOT. Edit AnalysisSkeleton_jobOptions.py and replace these lines (substitute the appropriate user name for "ketevi" or supply the correct path to the tags that you produced in Exercise 1). 

EventSelector.InputCollections = [ 
  "/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/MyColl"
                                 ]
EventSelector.Query="(NElectron < 3) && (abs(ElectronPt1) > 20000)"
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.tag.root' OPT='UPDATE'"]
AANTupleStream.OutputName = 'AnalysisSkeleton.aan.root'
with thes ones: 
EventSelector.InputCollections = [ 
  "AnalysisSkeleton.aan"
                                 ]
EventSelector.Query="EventNumber != 2 && ElecRMatch < 0.05"
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.sel.root' OPT='UPDATE'"]
AANTupleStream.OutputName = 'AnalysisSkeleton.sel.root'
and run it as follows: 
athena.py -b AnalysisSkeleton_jobOptions.py
It should produce a new AAN file which contains the results of the analysis on the selected events. And We've used the user's own AAN for this selection: 
EventSelector.Query="EventNumber != 2 && ElecRMatch < 0.05"
Note that we are now making the selection using any variable (attribute) that the user has put into his/her own AAN, for example ElecRMatch - the central, common tag database does not contain any attribute called ElecRMatch. Thus, the AAN serves two purposes: not only is it the user's analysis data object in ROOT but it is also the user's own customized tags in addition to the central, common tags. What you've just done with the AAN is NOT possible with the ordinary CBNT.

Back to the top

Exercise 4: User Analysis in Distributed Analysis Environment

We refer the user to the following tutorial and/or documentation:
  1. DIAL examples and tutorial, look here.
  2. Tutorial on GANGA.
  3. Distributed Analysis on Panda - how to submit user analysis jobs on OSG system.
Today, we will concentrate on item 3 above: Distributed Analysis on Panda. Let's go to the following page for details, taking note of these modifications:
  1. In the "Getting Started", you do not need to check out the UserAnalysis package - we already had it checked out. Just generate a grid-proxy as explained.
  2. For example.1, we need an input dataset. To see the list of registered datasets on Panda and their status, look on the Panda Monitoring Page. We will use this dataset as input: mc11.004201.ZeeJimmy.recon.AOD.v11000202_8179.
  3. To run in the Distributed Environment on Panda as follows --- see the following pages for accessing Panda data and for the dataset naming convention: AccessPandaData, pathena, naming convention
  4. (Each of you MUST use a different dataset output name - for example replace "ketevi" by your user name)!
  5. Edit AnalysisSkeleton_jobOptions.py and comment out the pre-selection on the tags for now, the lines affected should be as follows: 
    EventSelector.InputCollections = [
#"/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/MyColl"
#"/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool"
#"AnalysisSkeleton.aan"
                                 ]
#EventSelector.CollectionType="ExplicitROOT"
#EventSelector.Query="(NElectron < 3) && (abs(ElectronPt1) > 20000)"
#EventSelector.Query="EventNumber != 2 && ElecRMatch < 0.05"
THistSvc.Output = ["AANT DATAFILE='AnalysisSkeleton.aan.root' OPT='UPDATE'"]
    Submit the analysis job on Panda (PLease do NOT do this in THIS tutorial: I have already done it for you. But try this at your leisure on datasets on interest to you with large statistics and give us feedback): 
    pathena AnalysisSkeleton_jobOptions.py --inDS mc11.004201.ZeeJimmy.recon.AOD.v11000202_8179  --outDS tutorial.001103.reco.cbntAAN.pool.v1
Now browse the Panda monitoring page to follow the status of this job (the job id is given to you after the successful pathena command). Now go back to instructions on this page and copy the output to a location of choice and finish the analysis in ROOT - do this at your leisure.

Exercise

Define analysis job that makes selection on the tags and collects the events of interest into a new AOD. Submit this job in distributed environment on Panda on a large statistics. Take the output dataset as input to your AOD analysis and produce AAN, also in the distributed environment if the statistics is still large after the tag pre-selection. The final output is your AAN dataset containing the AAN files, to be merged with final plots in ROOT. (note that the tags need to be produced first for all the events before trying this exercise. If the tags are not already available you can produce them in the distributed environment by using the AOD dataset as input: - this is fast). If you attempt this exercise, please let me know.

Back to the top

Exercise 5: The interactive Analysis in Athena

There are several aspects to the interactive analysis:

5.1 Interactive Athena

How access data, create, plot and fit histograms interactively without writing any piece of code or run snippets of analysis code interactively. Let's get this job options: Interactive_topO.py:
get_files Interactive_topO.py

Edit Interactive_topO.py and replace this line 

EventSelector.InputCollections = [ "AOD.pool.root" ]
with this one (make sure the path to the AOD that you produced in Exercise 1 set correctly). The AOD needs to have at least 20 events to proceed with this exercise. 
EventSelector.InputCollections = [ 
"/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root" 
]
In the release 11.0.X, you will need to check out and build these tags: PyKernel-00-00-22 and PyAnalysisCore-00-00-12 - to do so, you would proceed as follows: 

cd $HOME/tutorial/11.0.3
cmt co -r PyKernel-00-00-22 Control/PyKernel
cmt co -r PyAnalysisCore-00-00-12 PhysicsAnalysis/PyAnalysis/PyAnalysisCore
cd -
cd ../cmt

edit the requirements file add these lines:


use PyKernel                    PyKernel-00-*                   Control
use PyAnalysisCore		PyAnalysisCore-00-*		PhysicsAnalysis/PyAnalysis



just after this line:
use StoreGate                   StoreGate-02-*                  Control
save the requirements file, then:

cmt config
source setup.csh
cmt broadcast gmake
cd ../run
After this, proceed with item 4) on this page: Interactive Analysis and AOD Browsing. Noe the interactive AOD browser does not work in 11.0.X: it will be fixed in 11.X.0. To seek to some event, use this: 
theApp.seekEvent(n)
where "n" is the event number, counting from zero. To terminate the interactive Athena session, press "Ctrl d".

For the interactive analysis, add this line at the bottom of your job options before starting the Interactive Athena: 
include ("PyAnalysisCore/InitPyAnalysisCore.py")
See this page for further details. Another aspect of the intereactive analysis is how to pass selected events to the display and visualization tools: we will look at this in Exercises 5.2-5.4.

5.2 Event Display and Visualization - JiveXML

In this exercise, we will use the Event Display and visualization tools of Atlantis. Dedicated Athena Algorithms convert event data to a special XML format, JiveXML, readable by Atlantis. There exists a reconstruction flag, "doJiveXML" --- which is false by default --- to produce the JiveXML files (one for each event) if set to true in your RecExCommon job options: for example, take the ESD/AOD/TAG production options of Exercise 1, i.e., ESD.AOD.TAG_topOptions.py and add this line to it: 

doJiveXML = True
For Details, look at the JiveXML page. Now if you run this job options, it should reproduce the ESD, AOD, TAG and 5 JiveXML files (one for each event) - but do not run it in this live tutorial (try it later at your own leisure).

For this exercise, create a job options file, called it RDO.ESD.AODtoJiveXML_topOptions.py and add the following lines to it. Use the AOD that you produced in Exercise 1 as input. We will back navigate to the ESD and RDO to collect all the event data needed for the display. At the time of this tutorial, the display of AOD objects is at the development and testing stage. Eventually, going to back to the RDO will not be necessary: 
######################################################################
#Detector Description
DetDescrVersion="ATLAS-DC3-02"
include ( "RecExCommon/AllDet_detDescr.py" )

theApp.EvtMax=5
doTruth = True

include ( "RecExCommon/RecExCommon_flags.py" )

#Event selector tools
include ( "AthenaPoolCnvSvc/ReadAthenaPool_jobOptions.py" )
EventSelector = Service( "EventSelector" )
EventSelector.InputCollections = [
"/direct/usatlas+u/ketevi/tutorial/11.0.3/Reconstruction/RecExample/RecExCommon/RecExCommon-00-04-38/run/AOD.pool.root"
]
EventSelector.BackNavigation = True

#Data converters
include ("RecExCommon/RecoOutputPoolCnv_jobOptions.py")
include("TrigT1EventAthenaPool/TrigT1EventAthenaPool_joboptions.py")
include( "ParticleEventAthenaPool/AOD_PoolCnv_jobOptions.py")

include( "PartPropSvc/PartPropSvc.py" )

#Inner Detector - will not necessary when the RIO on Tracks are in the ESD
include( "InDetRecExample/InDetRec_jobOptions.py" )

#Calo DLLs
theApp.Dlls += [ "CaloTools" ]
theApp.DLLs += ["TileRecAlgs"]
theApp.DLLs += ["LArClusterRec"]
theApp.DLLs += ["CaloRec"]
theApp.DLLs += ["TileRec"]

#Muons - will not be necessary when the RIO on Tracks are in the ESD
include( "MuonByteStreamCnvTest/MuonRdoDigit_jobOptions.py" )
include( "MuonByteStreamCnvTest/jobOptions_MuonRDOToDigit.py" )
include( "CscClusterization/G4CscClusterOptions.py" )
include( "MuonRdoToPrepData/MuonRdoToMuonPrepData_jobOptions.py" )
include( "MooAlgs/Moore_RecEx_jobOptions_G4.py" )
include( "MuonboyAth/MuonboyG4dry_jobOptions.py" )
include( "MuidAlgs/MuidAlgsSA_jobOptions.py" )
include( "MuidAlgs/MuidAlgsCB_jobOptions.py" )
include( "STACO/Staco_jobOptions.py" )
include( "MuTag/MuTag_jobOptions.py" )

#Jive XML tools to produce the XML files for Atlantis
include ( "RecExCommon/JiveXML_config.py")
ToolSvc = Service( "ToolSvc" )
ToolSvc.EventData2XML.FileNamePrefix = "mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001"

MessageSvc = Service( "MessageSvc" )
MessageSvc.OutputLevel = FATAL
######################################################################
If you run this job options file, it should produce 5 JiveXML files, one for each of the 5 events. But please do not run the job options during this live tutorial: I have already done this and produced the XML files, just copy them from my area: 
cp /direct/usatlas+u/ketevi/tutorial/11.0.3/JiveXML/*.xml .
In the above job options, you notice that we go back to the RDO, and re-run some Inner Detector and Muons reconstruction algorithms since some of the objects needed are not in the ESD (yet) and must be re-created from the RDO. In some cases, the requested objects are not found and the back navigation attempts to go beyond the RDO: you can see this by setting the Message Output Level to INFO - but we will not worry about that here.

Eventually, the user should be able to use the event tags or the AAN, as described in Exercise 3 above, to select/identify some events of interest and produce the JIveXML files for only those events and visualize in greater details.

5.3 Event Display and Visualization - Atlantis

To start the Atlantis event display, simply type the command 

atlantis
Then follow in the instructions described on this page: scroll down to the section on Using Atlantis and select the JiveXML file produced in Exercise 5.2 (for the first event).

It is recommended that you install Atlantis on your computer and move the events (JiveXML files) there (your computer) for faster graphical operations.

After playing with this for a while, exit the Atlantis session before starting the next exercise: go to "file", then "exit".

5.4 Interactive Athena and Atlantis Display

  1. Look in the ATLAS CVS for a script called InteractiveServer.py and copy it your "run" directory. Note that from the release 11.2.0, you can get files directly from the release (you do not have to look for it and copy it) using the "get_files" command as follows: 
    get_files InteractiveServer.py
  2. Read the instructions in the InteractiveServer.py
  3. Create a job options file call it InteractiveAthenaToAtlantis_topOptions.py and add these lines to it: 
    ############################################################################
#The Detector Description
DetDescrVersion="ATLAS-DC3-02"

#Number of Events to process
EvtMax=5

#Input Raw Data
#Set the path and the file name as appropriate
PoolRDOInput = [
"/usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root"
]

#Suppress the production of NTuple Histograms
doCBNT = False
doHist = False

#We do not want to reproduce the ESD/AOD/TAG in this exercise - otherwise comment these lines out
doWriteESD = False
doWriteAOD = False
doWriteTAG = False

#Message Output Level
OutputLevel = FATAL

#Produce the JiveXML out for the Atlantis display
doJiveXML = True

#Main job Options
include ("RecExCommon/RecExCommon_topOptions.py")

###################### For interactive analysis - comment it out otherwise
include ("PyAnalysisCore/InitPyAnalysisCore.py")
#############################################################################
Now start the interactive session as we did in Exercise 5.1: 
athena -i InteractiveAthenaToAtlantis_topOptions.py -l FATAL
At the interactive Athena prompt, execute these commands: 
theApp.initialize()
execfile ("InteractiveServer.py")
After this, 3 pieces of information will be output to the screen, for example 
         server name: 'acas0002.usatlas.bnl.gov'
         port number: '13260'
         key: 'EELK#/^Qk`)N5/m'
You will need to copy this information to the Atlantis dialog box - note that the key is case sensitive. Open a new working session, set up CMT as in Exercise 0.10, and launch Atlantis, for example: 
ssh acas00XY (or rterm -i)
cd tutorial
source setup.csh -tag=11.0.3,opt
cd 11.0.3/PhysicsAnalysis/AnalysisCommon/UserAnalysis/../cmt
source setup.csh
cd ../run
atlantis
Now, we want to read the event from the Interactive Athena, NOT from some JiveXML files that must have been produced in a prior step as in Exercise 5.3. Once the Atlantis GUI appears: 
Go to "File" and select Interactive Athena.
A dialog box will appear: fill in the Remote Server Name, the Port and the Key as printed to you on the interactive athena window. In my case (which will be different from you case), the information is this: 
         server name: 'acas0002.usatlas.bnl.gov'
         port number: '13260'
         key: 'EELK#/^Qk`)N5/m'
Now remember that the interactive Athena has not produced any event yet: so far we only run "theApp.initialize()". Let's execute Athena for one event: in the Interactive Athena dialog box of Atlantis, you will see a small box at the bottom saying "athena command". In that box, type the following athena command and click OK: 
theApp.nextEvent()
to execute Athena for one event, the next event, which in this particular case would be the first event: now watch the interactive Athena window and see that Athena is reconstructing the event. Wait for this operation to complete successfully. Then, in the Interactive Athena dialog box of Atlantis, 
click on "Get Event"
to read the event into Atlantis: check the Atlantis display and see the event summary is updated accordingly. You may examine this event, then execute Athena for next event, etc. To terminate the the communication between the Interactive Athena session and Atlantis, press "Cancel" in the dialog box, then press "Ctrl c" in the Interactive Athena session. This will terminate the communication and take you back to the Interactive Athena prompt. Then, to exit Interactive Athena, press "Ctrl d". Also, exit the Atlantis display before proceeding to the next Exercise.

You see that running Atlantis on your own machine, you could communicate and pass event data --- JiveXML output --- to Atlantis in an interactive way: no need to produce the JiveXML files in the step, then copy them to your machine in the second step before displaying. There may be firewall issues in some cases: for example, the US ATLAS acas machines are not in the BNL DNS, thus not accessible directly from the outside (in this exercise, both the interactive Athena and the Atlantis sessions were on different machines, all of which are inside the firewall so there was no communication problems). For the US ATLAS case, a short term solution would be to run the Interactive Athena on the aftpexp0x.bnl.gov machines which are within the BNL DNS, and accessible from the outside.

5.5 v-Atlas

The 3-D visualization program, HepVis/v-atlas, designed mainly to debug the ATLAS detector geometry can also display hit and energy deposition information. Unlike Atlantis, v-atlas is highly coupled to Athena, can access directly StoreGate objects, change the geometry and re-run algorithms. Atlantis and v-atlas thus complement eah other quite well. To get going with v-atlas, do this (later at your own leisure): 

v-atlas DetectorDescription fileName
You may use the RDO file of Exercise 1 for which the detector description is ATLAS-DC3-02. Thus, the command line would look like this: 
v-atlas /usatlas/magdacache001/ketevi/ttbar/digit/4100/mc11.004100.T1_McAtNLO_top.digit.RDO.v11000301._00001.pool.root
Play in the graphics window and exit v-atlas before proceeding to the next exercise. Note that in 11.0.3, you may observe a crash in v-atlas - the authors have been looking into this for a fix for 11.2.0.

5.6 Persint

There is also the Persectively Interacting package, known as Persint. It has been developed for ATLAS, for 3-dimensional representation of objects, with access to a variety of applications, in an interactive environment. Consult the Persint page for further information.

Back to the top

An Example on AOD: Z-> ee

At the end of this exercise, you should have a plot of the Z -> ee invariant mass. The data file data you are using is Z -> ee, simulated with the ATLAS-DC3-01 initial layout. You will retrieve a container AOD electrons from the persistency storage. You will loop over the electrons in the container, and for each electron, you will plot the isEM flag. You will calculate the ee pair invariant mass and put it in a histogram. So, you are adding 2 items to the AAN of Exercise 2.

Some Clarifications

i. A container of AOD objects: by container, we mean an STL vector, or rather, a DataVector of AOD objects. The DataVector is just an STL vector which, by default, in the C++ language, "owns" the stuff that it contains. Each AOD object has an associated container. For example, the ParticleJet has a corresponding ParticleJetContainer.h class.

ii. Retrieve the container of AOD object: the AOD data file that you are using, for example the one that you will use in this exercise, was created for you by somebody - althought you now know how to create your own ESD and AOD. When you want to us the AOD objects in your analysis code, on event by event basis, you ask for a pointer to the container of the AOD objects. For example, in the execute() method of the default AnalysisSkeleton.cxx, we are asking for the container of the Electron AOD:
  const ElectronContainer* elecTES = 0; 
  sc=m_storeGate->retrieve( elecTES, m_electronContainerName); 
  if( sc.isFailure()  ||  !elecTES ) { 
     mLog << MSG::WARNING 
          << "No AOD electron container found in TDS"
          << endreq; 
     return StatusCode::SUCCESS;
  }  
  mLog << MSG::DEBUG << "ElectronContainer successfully retrieved" << endreq;
iii. Container Names: you need to know the "name" or the "key" of the container/object that you want to access. What we mean by name or key here is the std::string code that was used to create, store and identify thw that AOD container. You need to know that so you can ask for that container if you need it. In the above snippet of code, the data member m_electronContainerName defines the name of the ElectronContainer that you want to retrieve. If you look in the constructor of AnalysisSkeleton.cxx, you will see that m_electronContainerName is initialized to "ElectronCollection". That is the name used when the AOD electron container was produced for you. For the list of containers/objects in the ESD and AOD, see the following pages: 
 
ESD Content and Keys
AOD Content and Keys
iv. Access to the Kinematics: Some of the AOD objects are 4-Momentum objects, meaning that they should be able to answer all your questions about their kinematics. For example, to ask the Electron object for its transverse momentum and pseudo rapidity: 
 
  for (; elecItr != elecItrE; ++elecItr) {
    if( (*elecItr)->hasTrack() &&
        (*elecItr)->pt()> m_etElecCut ) { 
        double electronPt  =  (*elecItr)->pt(); 
        double electronEta = (*elecItr)->eta();
        ...
The complete list is the following:
  	double px() 
  	double py()
  	double pz()
  	double m()
  	double p()
  	double eta()
  	double phi()
  	double e() 
  	double et()
  	double pt()
  	double iPt() 
  	double cosTh()
  	double sinTh()
  	double cotTh()
  	HepLorentzVector hlv()
For further details, look at the 4Momentum implementation

v. Histogramming: Follow this link to Histograms and Ntuples in ATHENA and the THist Service.

vi. Loop over the Objects in the Container: after you successfully retrieve the container and have a pointer to it in the transient data store (TDS), you can get the iterators over the container since the container is a DataVector. Look at the example in the execute() method of AnalysisSkeleton.cxx: 
  const ElectronContainer* elecTES=0; 
  sc=m_storeGate->retrieve( elecTES, m_electronContainerName);
  if( sc.isFailure()  ||  !elecTES ) {
     mLog << MSG::WARNING 
          << "No AOD electron container found in TDS"
          << endreq; 
     return StatusCode::SUCCESS;
  }  
  mLog << MSG::DEBUG << "ElectronContainer successfully retrieved" << endreq;

  /// iterators over the container 
  ElectronContainer::const_iterator elecItr  = elecTES->begin();
  ElectronContainer::const_iterator elecItrE = elecTES->end();
  for (; elecItr != elecItrE; ++elecItr) {
    if( (*elecItr)->hasTrack() &&
        (*elecItr)->pt()> m_etElecCut ) { 
        m_h_elecpt->Fill( (*elecItr)->pt(), 1.); 
	m_h_eleceta->Fill( (*elecItr)->eta(), 1.); 
      ...
vii. CompositeParticle: Its design is still evolving. The composite object, for example, the Z boson as the composite of electron-positron, is itself a ParticleBase or an IParticle. But it is also a 4Momentum object. Further it needs to provide navigation to its constituents somehow. The current implementation of the CompositeParticle Class is most likely to evolve while maintaining somehow the main ingredients of IParticle, 4Momentum and constituent navigation.

viii. Now you have all that you need to do this exercise: proceed as described below. When you finish the exercise, your ROOT output file, AnalysisSkeleton.aan.root, should have 2 additional items in the AAN, namely, the isEM flag and the ee invariant mass. Make a 20 GeV cut on the electron Pt before calculation the ee mass.

Write your Analysis Code

1. Include the follwing line in AnalysisSkeleton.h
#include "ParticleEvent/ElectronContainer.h"
2. Add a method called zee_on_aod() where you will implement this exercise: 
StatusCode zee_on_aod();
2.1 Define the 2 AAN items mentioned above: put the key word "aod".
2.2 Add the piece of code to retrieve the AOD electron container from the TDS
2.3 Add the piece of code to iterate over the AOD container
2.4 For each electron, the isEM flag and fill the relevant AAN item.
2.5 Add the piece of code to calculate the ee invariant mass if each electron has Pt greater than 20 GeV within a pseurapidity of +/-2.5 and the fill the relevant AAN item. Make sure that the cuts adjustable in your job options
2.6 Compile and build your analysis code

Run your Analysis Code

You need more statistics, so proceed as follows: (if you are working at CERN, copy these files from ~ketevi/scratch0/Tutorial/11.0.3/data)
1. Change directory to your run directory
2. cp ~ketevi/tutorial/11.0.3/data/dc3.004201.zee.tag.py .
3. cp ~ketevi/tutorial/11.0.3/data/dc3.004201.zee.aod.py .
4. cp ~ketevi/tutorial/11.0.3/data/dc3.004201.zee.esd.py .
5. Look inside dc3.004201.zee.aod.py to see the list of the input AOD files
6. Edit AnalysisSkeleton_jobOptions.py and replace these lines 

EventSelector.InputCollections = [
   "AOD.pool.root"
]
by these ones
include ("dc3.004201.zee.aod.py")
Also, switch ON the back navigation to the ESD in AnalysisSkeleton_jobOptions.py: 
EventSelector.BackNavigation = True
5. Now, run your analysis code by doing this 
athena.py -b AnalysisSkeleton_jobOptions.py
 6. Open AnalysisSkeleton.aan.root in ROOT, examine the ee invariant mass distribution and make sure that it is OK: a narrow peak at the Z-mass.

Here is the solution to this exercise:

  std::vector * m_aan_isEM;
  std::vector * m_aan_mee;
  ...
  addBranch("ElecIsEM",    m_aan_isEM);
  addBranch("ElecMee",     m_aan_mee);
  ...
  m_aan_isEM->clear();
  m_aan_mee->clear();
  ...
  ElectronContainer::const_iterator elecItr  = elecTES->begin();
  ElectronContainer::const_iterator elecItrE = elecTES->end();
  for (; elecItr != elecItrE; ++elecItr) {
    if ( (*elecItr)->author() != ElectronParameters::egamma || (*elecItr)->pt() < m_etElecCut ) continue;
    int bitPosition = 1;
    int isEM = (*elecItr)->isEM();
    if ( isEM == 0 ) m_aan_isEM->push_back( (isEM-1.0) ); /// put isEM=0 at -1 
    /// fill the isEM histogram as a bit pattern of the isEM flag 
    if (isEM > 0) {
       for (int i=0; i<16; ++i) {
         if (isEM & bitPosition) m_aan_isEM->push_back(i);
         bitPosition *= 2;
       }
    }
  }

  AnalysisUtils::Combination < const ElectronContainer > comb(elecTES,2);
  ElectronVect ePair;
  while (comb.goodOnes(this, ePair, selectElectron)) {
    /// Create the Z -> ee as a composite object
    CompositeParticle zee;
    zee.add(ePair[0],ePair[1]);
    /// fill the Z -> ee invariant mass
    m_aan_mee->push_back(zee.m());
  }
  ...
  /// electron selection
  bool selectElectron(AnalysisSkeleton * self, const ElectronVect &ll) {
    bool test1 = ll[0]->charge() == -(ll[1]->charge());
    bool test2 = (ll[0]->pt() > self->m_etElecCut) &&
                  (ll[1]->pt() > self->m_etElecCut);
    bool test3 = (fabs(ll[0]->eta()) < self->m_etaElecCut ) &&
                  (fabs(ll[1]->eta()) < self->m_etaElecCut );
    bool test4 = ( (ll[0]->isEM()%16)==0 && (ll[1]->isEM()%16) == 0);
    bool test5 = ( ll[0]->author() == ElectronParameters::egamma && ll[1]->author() == ElectronParameters::egamma);
    return (test1 && test2 && test3 && test4 && test5);
  }
You may take a look at my version of AnalysisSkeleton.h/.cxx. It is in 
~ketevi/tutorial/11.0.3/PhysicsAnalysis/AnalysisCommon/UserAnalsyis/UserAnalysis-00-05-03 at BNL
~ketevi/scratch0/Tutorial/11.0.3/PhysicsAnalysis/AnalysisCommon/UserAnalsyis/UserAnalysis-00-05-03 at CERN

Back to the top

Exercise 7: TrackParticle and Navigation

At the end of this exercice, you should know how to access and do analysis with track parameters, how to do back navigation from AOD to ESD, how to use constituent navigation, how to do SymLink and how to manipulate composite objects.

Some Clarifications

i. The TrackParticle: It is designed to handle tracking and vertexing output. It is the interface between the user analysis domain and track and vertex reconstruction. Since the track object itself is not in the AOD. The TrackParticle collections are in the ESD and in the AOD. The TrackParticle has link to the original track whose collections are only in the ESD. The TrackParticle class can be found in TrackParticle. Note that the TrackParticle is also a 4Momentum object like the AOD Electron or Muon.

ii. Constituent Navigation: It is the ability to access information at any point in a relational tree. A jet, for example, may consists of calorimeter clusters as constituents, and each cluster having calorimeter cells as constiutents. Accessing information at any node in the relation tree behind the jet itself is the constituent navigation. The navigation tools are implemented in Navigation.

iii. Back Navigation: It is the ability to access an object which is not in the AOD but is in the ESD. An AOD, created from an ESD, "knows" the ESD from which it came. As a concrete example, the electron object is in the AOD but the e/gamma object from which the electron was created is not in the AOD, rather it is in the ESD. A user doing analysis on AOD may want to access the e/gamma object of a particular electron: this would be done through back navigation provided the ESD POOL file are also available and accessible. To enable back nagivation, you just have to add the following line to your job options, for example to AnalysisSkeleton_jobOptions.py: 
# enable Back Navigation in your job options
EventSelector.BackNavigation = True
iv. SymLink: Suppose you want to retrieve the AOD ElectronContainer, as a ParticleBaseContainer, for some reason. After all, the Electon class derives from the ParticleBase which is the common implementation AOD particles: maybe you want to do something with electrons and muons, perhaps, just 4Momentum manipulation, you not care whether the particle is an Electron or Muon, just a particle of type ParticleBase or IParticle. You want then want to retrieve the MuonContainer as ParticlebaseContainer. The ability to record a container of type Electron for example, and later retrieve it as a container of type ParticleBase for example is known as SymLink.

Exercice 7.1: Access to TrackParticle in the AOD

7.1.1 In AnalysisSkeleton.h/.cxx, add another method: StatusCode trackParticleAndNavigation()

7.1.2 Make a call to trackParticleAndNavigation() in the execute() method as you did for the zee_on_aod() method above

7.1.3 Add an item to the AAN to plot the momentum of the track

7.1.4 In the method trackParticleAndNavigation(), retrieve the ElectronContainer from the TDS

7.1.5 Iterate over the ElectonContainer, for each Electron access its TrackParticle and fill the relevant ANN item with the track momentum: 
  Retrieve the ElectronContainer elecTDS
  ElectronContainer::const_iterator elecItr  = elecTDS->begin();
  ElectronContainer::const_iterator elecItrE = elecTDS->end();
  for (; elecItr != elecItrE; ++elecItr) {
      if ( (*elecItr)->author() != ElectronParameters::egamma || (*elecItr)->pt() < m_etElecCut ) continue;

      /// access to the Electron's TrackParticle
      const Rec::TrackParticle *trackParticle = (*elecItr)->track();
      if (trackParticle != 0) {
         double trackP = trackParticle->p();
         fill the AAN item
      } 
    }
7.1.6 If you are just working with TrackParticles, not electron nor muon, etc, in particular, you can just retrieve the collection of the TrackParticles of interest: 
  Retrieve the Inner Detector TrackParticleContainer trackTDS
  Rec::TrackParticleContainer::const_iterator trackItr  = trackTDS->begin();
  Rec::TrackParticleContainer::const_iterator trackItrE = trackTDS->end();
  for (; trackItr != trackItrE; ++trackItr) {
      /// access to the Inner detector TrackParticle
      const Rec::TrackParticle *trackParticle = (*trackItr);
      double trackP = trackParticle->p();
      double charge = trackParticle->charge();
      ... 
    }

Exercice 7.2: Back Navigation to ESD

In the same iteration loop, access the EMShower object of the Electron and the Et leakage into the hadronic calorimeter

7.2.1 Add an item to the AAN for the Et leackage into the hadronic calorimeter
7.2.2 In the same loop above for the TrackParticle, ask the Electron for its egamma object by back navigating to the ESD since the egamma object is not in the AOD

7.2.3 From the egamma, get the EMShower object and ask for the hadronic Et leakage

7.2.4 When you finish doing all this, the trackParticleAndNavigation() method should look, a sort of, like this: 

  Retrieve the ElectronContainer elecTDS
  ElectronContainer::const_iterator elecItr  = elecTDS->begin();
  ElectronContainer::const_iterator elecItrE = elecTDS->end();
  for (; elecItr != elecItrE; ++elecItr) {
      if ( (*elecItr)->author() != ElectronParameters::egamma || (*elecItr)->pt() < m_etElecCut ) continue;

      /// access to the Electron's TrackParticle
      const Rec::TrackParticle *trackParticle = (*itE)->track();
      if (trackParticle != 0) { 
         double trackP = trackParticle->p();
         fill the histogram
      } 
      /// back navigation to ESD to access Electron's egamma object
      const egamma *eg = (*elecItr)->eg();
      if (eg) { 
          const EMShower *shower = eg->get_EMShower();
          if(shower) { 
            double et_had = (shower)->ethad();
	    fill the relevant AAN item
          } 
       } 
     
    }
7.2.5 This line should be included your AnalysisSkeleton.h: 
#include "egammaEvent/EMShower.h">
#include "egammaEvent/egamma.h">
7.2.6 Compile the code and resolve any compilation problem: proceed as follow

7.2.6.1 Change directory to your "cmt" directory

7.2.6.2 edit the "requirements" file
7.2.6.3 add the following line to the requirements file: 
use egammaEvent         egammaEvent-01-*   Reconstruction
just below this line 
use ParticleEvent ParticleEvent-00-* PhysicsAnalysis/AnalysisCommon
7.2.6.4 gmake
We need the above line for the egamma object. Make sure that the back navigation is ON in your job options:
EventSelector.BackNavigation = True
7.2.7 Run your code and examine the AAN.

Exercice 7.3: Consitutent Navigation

In the same iteration loop, access the EM Cluster of the Electron and navigate to the cells. Since the release 11.0.X, the clusters are in the AOD. But the cells are not in the AOD. Thus, the constituent navigation to the cells, will trigger back navigation to the ESD. Add an item to the AAN for the summed cell Et. The snippet of code might look like this: 
#include "Navigation/Navigable.h"
    ...
    const Calocluster * cluster = (*electIr)->cluster();
    if (cluster) {
       NavigationToken cellToken;
       (cluster)->fillToken(cellToken,double(1.));
       NavigationToken::const_iterator c =  cellToken.begin();
       NavigationToken::const_iterator cend = cellToken.end();
       mLog << MSG::DEBUG << "# cells found " << cellToken.size() <<  endreq;
       double etSum = 0;
       for(; c != cend; ++c) {
         const CaloCell* thisCell = *c;
         double weight = cellToken.getParameter(thisCell);
         double et    = weight * thisCell->et();
         etSum += et;
       }
       m_aan_sumEtCell->push_back(etSum);
       ...
And your requirements file should include this line: 
use Navigation    	Navigation-00-*   	Control

Exercice 7.4: SymLink

7.4.1 Add another method to AnalysisSkeleton, StatusCode symLinkExample(): call this method from your "execute()" method. In the zmm_on_aod() method, you retrieved the container of particle muons as MuonContainer. Now, in symLinkExample(), you will retrieve the same container as ParticleBaseContainer. Sometimes, you may want to do this because you want to use common tools which to do not operate on Muon, but rather on ParticleBase, or you are interested in Muon, Electron, etc, as simply 4Momentum objects.

7.4.2 Add another AAN item for the transverse momentum of the electrons from the ParticleBaseContainer. Iterate over this container and fill the AAN item. Later on when you run your code, make sure that this histogram is identical to the one created in the electron() method for the electron transverse momentum in Exercise 2: 
  const ParticleBaseContainer* particleBaseTES=0;
  sc=m_storeGate->retrieve( particleBaseTES, m_electronContainerName);
  if( sc.isFailure()  ||  !particleBaseTES ) {
     mLog << MSG::WARNING
          << "No AOD electron container found in TDS as symLink"
          << endreq;
     return StatusCode::SUCCESS;
  }
  mLog << MSG::DEBUG << "ParticleBaseContainer successfully retrieved as symLink" << endreq;

  /// iterators over the ParticleBase container
  ParticleBaseContainer::const_iterator partItr  = particleBaseTES->begin();
  ParticleBaseContainer::const_iterator partItrE = particleBaseTES->end();

  for (; partItr != partItrE; ++partItr) {
     m_aan_pt_symLink->push_back( (*particleItr)->pt() );
  }
  ...
The symLink feature should not be abused. It is recommended to use symLink only if you absolutely have to. Do not do none sensical stuff, for example trying to retrieve an ElectronContainer as symLink to a ParticleJetContainer.

Back to the top

Exercice 8: Monte Carlo Truth

The following Monte Carlo (MC) Truth information is available in the ESD and AOD:
  1. The full MC event record, the generator truth and Geant4 interaction truth. This is known as "TruthEvent"(McEventColloction.h) and it is available in the ESD but it too big for the AOD. In the AOD, there is a slimmed version of the "TruthEvent" and that is known as "GEN_AOD". The slimming aims at keeping th especial, interesting particles with their parents and children. However, to be able to use the same tools on all particles, the particles of "GEN_AOD" are converted, on fly, on the user's request, into the TruthParticle and put into the TruthParticleContainer.
  2. The collection of Track Truth, TrackParticleTruth is also in the ESD/AOD.
  3. The Muon Entry Record: these are particles escaping the calorimeter and entering the muon spectrometer.
Take a look at the content list of the ESD and the AOD to identify all these various MC Truth collections and their data access keys: 
 
ESD Content and Keys
AOD Content and Keys
In this exercise, you will retrieve the Electron Collection and the Truth Particle Collection and you will calculate and plot the efficiency of the reconstructed Electrons as a funtion of eta: for a given reconstructed electron, find the closest, in deltaR, truth electron. Plot the deltaR of the match, the pt ratio of reconstructed to truth and the efficiency: 
  /// get the MC truth particle AOD container from StoreGate
  const TruthParticleContainer*  mcpartTES = 0;
  sc=m_storeGate->retrieve( mcpartTES, m_truthParticleContainerName);
  if( sc.isFailure()  ||  !mcpartTES ) {
     mLog << MSG::WARNING
          << "No AOD MC truth particle container found in TDS"
          << endreq; 
     return StatusCode::SUCCESS;
  }
  mLog << MSG::DEBUG << "MC Truth Container Successfully Retrieved" << endreq;
You already know how to retrieve the Electron Collection. Take a look at this solution which is already in the method electron() of your AnalysisSkeleton.cxx - please look there for details: 
  /// iterators over the container 
  ElectronContainer::const_iterator elecItr  = elecTES->begin();
  ElectronContainer::const_iterator elecItrE = elecTES->end();
 
  for (; elecItr != elecItrE; ++elecItr) {
    if( (*elecItr)->author() == (*elecItr)->parameter(ElectronParameters::egamma) && (*elecItr)->pt() > m_etElecCut ) {
       m_aan_eta->push_back((*elecItr)->eta());
       m_aan_pt->push_back((*elecItr)->pt());

       int index = -1;
       double deltaRMatch;
       /// find a match to this electron in the MC truth container
       /// the index and deltaR are returned
       const TruthParticleContainer * truthContainer = mcpartTES;
       bool findAMatch = m_analysisTools->matchR((*elecItr), truthContainer, 
		       index, deltaRMatch, (*elecItr)->pdgId());
       if (findAMatch) {
          deltaRMatch = (deltaRMatch > m_maxDeltaR) ? m_maxDeltaR : deltaRMatch;
          m_aan_elecrmatch->push_back( deltaRMatch );
          if ( deltaRMatch < m_deltaRMatchCut) {
             const TruthParticle*  electronMCMatch = (*mcpartTES)[index]; 
             double res = (*elecItr)->pt() / electronMCMatch->pt();
             m_ann_elecetres->push_back(res);
          }

	  m_aan_mcelecpt->push_back( (*mcpartTES)[index]->pt() );
	  m_aan_mceleceta->push_back( (*mcpartTES)[index]->eta() );
       }
    }
    
  }
Compile and run your code.

Back to the top

Exercise 9: An example on ESD - Adding another Algorithm

We will repeat the Z to ee reconstruction but one on the ESD. You will retrieve the egamma object container from the ESD and the Zee reconstruction on the egamma objects.

9.0 Create a different algorithm and call it ZeeOnESD. This means that your UserAnalysis package has 2 algorithms: 
AnalysisSkeleton - which we've been using so far
ZeeOnESD - the new one that you are adding
For this new algorithm, you should have ZeeOnESD.h, ZeeOnESD.cxx, ZeeOnESD_jobOptions.py, and you should add the appropriate lines to the file UserAnalysis_entries.cxx in ../src/components/ directory. Do the following in your new algorithm:

9.1 Create an Athena-Aware NTuple (AAN) and add 4 items to it: the electron pt, eta, E/P and the ee pair invariant mass, as you did in exercises 2 and 6 for the AOD.

9.2 Add a method called zee_on_esd() and make a call to it in the execute() method of ZeeOnESD.cxx as you did for zee_on_aod() in AnalysisSkeleton.cxx.

9.3 In zee_on_esd(), retrieve the egamma container and write the analysis code. The name of the egamma container is "egammaCollection". For solf-electrons, it is "softeCollection". Look here for details.

9.4 In the requirements file, you should have this line if it is not already in there: 
use egammaEvent         egammaEvent-01-*        	Reconstruction
9.5 In your ZeeOnESD_jobOptions.py, declare the ZeeOnESD algorithm and set its properties: follow the example of the AnalysisSkeleton_jobOptions.py.
9.6 In your ZeeOnESD_jobOptions.py, specify the ESD input data (dc3.004201.zee.esd.py, see Exercise 6), and also make sure the back navigation is OFF. Compile and run your ZeeOnESD_jobOptions.py and compare the results to the analysis on the AOD: AnalysisSkeleton_jobOptions.py of Exercise 6 above.

You may take a look at my version of ZeeOnESD.h/.cxx. It is is 
~ketevi/tutorial/11.0.3/PhysicsAnalysis/AnalysisCommon/UserAnalsyis/UserAnalysis-00-05-03 at BNL
~ketevi/scratch0/Tutorial/11.0.3/PhysicsAnalysis/AnalysisCommon/UserAnalsyis/UserAnalysis-00-05-03 at CERN

Back to the top

Exercise 10: Overlap Checking, Redundancy Removal, Event View

Follow this link to Event View Basics. You will also find this Event View tutorial useful.

Back to the top

Some AOD Analysis Tools

Look here for some the available tools and how to use them.

Back to the top

Last modified January 11, 2006, Ketevi A. Assamagan