Using the BNL PROOF Test Bed
Introduction
What is PROOF?
The Parallel ROOT Facility, PROOF, is an extension of ROOT allowing transparent analysis of large sets of ROOT files in parallel on clusters of computers or multi-core machines. There is an
official PROOF website and a
PROOF forum in Root Talk, as well as an
ATLAS hypernews forum.
What can I use PROOF for?
PROOF can be used to analyze any ROOT ntuple, including tertiary DPDs and AODs (using
AthenaROOTAccess). It's
fast, allowing you to analyze the events in a
TTree in parallel over many CPUs.
What is the BNL PROOF Test Bed?
The BNL PROOF Test Bed currently has 10 nodes (each with 4 GB of RAM) for a total of
40 1.8-GHz Opteron cores. You can process files on NFS at BNL or ask to have them transferred to the xrootd server (for faster processing). For
more details about the farm, see
Sergey's talk at the June 2008 BNL Jamboree. To
monitor the status of the farm, see the
Ganglia monitor.
Getting Started with PROOF at BNL
Right now the test bed has restricted access. If you would like to try this, contact Sergey at
panitkin@bnl.gov to add your BNL account name to the list. If you would like to try to run PROOF locally instead of at BNL, follow the
xproofd setup instructions on the ROOT website (NOTE: in order to run
xrootd, you need to setup ROOT).
Check out the Getting Started package
First setup CVSROOT; e.g.,
setenv CVSROOT /afs/usatlas.bnl.gov/software/cvs for tcsh at BNL
Check out the PROOF package:
cvs co -d PROOF groups/PROOF
Notice that under PROOF, there are 2 directories:
ARA-PROOF_BNL and
ProofNtupleAnalysis. The directory
ARA-PROOF_BNL contains examples for running PROOF with
AthenaROOTAccess (both in python and CINT), and the directory
ProofNtupleAnalysis contains an example for running PROOF over a standard ROOT ntuple.
Run a Very Simple Example
In this example, you will open a PROOF session, add a file to a
TChain, and
Draw() a simple quantity.
cd PROOF/ProofNtupleAnalysis
source SetupROOTforPROOF.csh
root
TProof::Open("acas0420");
TChain *ch = new TChain("FullRec0");
ch->Add("/usatlas/workarea/yesw2000/root/Data/HPTV/user.TARRADEFabien.trig1_misal1_csc11.005145.PythiaZmumu.Athena_12.0.6.GroupArea_12.0.6.6.Jamboree_II-HightPtView-00-00-30.AAN.AANT3._00010.root");
ch->SetProof(kTRUE,kTRUE);
TStopwatch t;
t.Start();
ch->Draw("Jet_C4_p_T","Jet_C4_N>0");
t.Stop();
t.Print();
The output should look something like this:
Reset Your PROOF Session
Sometimes your PROOF session will get corrupted and TProof::Open() will hang. Reset your session to restore functionality:
TProof::Reset("acas0420");
Find Available Data
You can use
dq2_ls to see the datasets on the
xrootd server:
dq2_ls -fp -s BNLXRDHDD1 "my_dataset" > file_list.txt
You need to use the physical filename (
-p option). If you would like a dataset to be copied over to the
xrootd server, contact Sergey at
panitkin@bnl.gov. Alternatively, you can access data on NFS at BNL by providing the full path name.
Examples
NOTE: This section is under construction
Examples that show you how to use PROOF:
Additional Useful Links
Major updates:
--
TWikiAdminGroup - 07 Oct 2008
About This Site
Please note that this site is a content mirror of the BNL USATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your BNL USATLAS account.
Attachments
Picture_11.png (100.8K) |
KyleCranmer, 13 Jul 2007 - 14:55 | EventProcessingRate?
Xrootd_farm_status_Aug_2007.pdf (120.4K) | Main.serp, 01 Aug 2007 - 18:42 |
AllHPTV_Ntuples.png (37.8K) |
KyleCranmer, 16 Jul 2007 - 14:46 |
xrdmon_presentation.pdf (389.8K) | Main.serp, 01 Aug 2007 - 18:44 | Xrootd monotoring talk from Edgar and Ofer
ProofTest.h (125.7K) |
KyleCranmer, 16 Jul 2007 - 16:19 | An Example TSelector
AllHPTVFiles.txt (683.6K) |
KyleCranmer, 13 Jul 2007 - 17:43 |
PROOF_Analysis_Aug1.pdf (1280.0K) |
KyleCranmer, 01 Aug 2007 - 16:33 | KyleProofAnalysis? _Aug1
ProofTest.C (3.6K) |
KyleCranmer, 16 Jul 2007 - 16:19 |
ListII.txt (16.8K) |
KyleCranmer, 13 Jul 2007 - 17:32 | List of HPTV datasets available via PROOF
BasicProofCommands.C (1074.1K) |
KyleCranmer, 18 Jul 2007 - 11:10 | Basic commands to setup proof, add many files to a dataset, and make a simple plot, run a TSelector
Picture_9.png (274.7K) |
KyleCranmer, 13 Jul 2007 - 14:50 | ExampleSession1?
Xrootd_farm_plans.pdf (114.3K) |
KyleCranmer, 13 Jul 2007 - 16:57 |
ExampleSession.png (100.1K) |
StephanieMajewski, 10 Jul 2008 - 16:01 | PROOF Simple Example Session Screenshot