Root demo 2 for DIAL 0.92
David Adams
July 2004

Uses MasterScheduler to do distributed processing on a local farm.
Edit rootlogon.C if you do not want to use LSF queue dial which is
appropriate for interactive processing at BNL.

Any of the applications described on the application page at
  http://www.usatlas.bnl.gov/~dladams/dial/releases/0.92/apps.html
may be used. The default is dial_cbnt.

Tested on the BNL ACF (ATLAS Computing Facility). Should work on any
RedHat 7.3 machine where DIAL is installed.

For ATLAS dataset catalogs see
  http://www.usatlas.bnl.gov/~dladams/dataset/atlas_catalog.

------------------------------------------------------------------------

Here are instructions for running the demo:

1. Define the location where you want DIAL to write its tasks and jobs:
     > DIAL_PRIVATE_DIR=$HOME/dial
   The default is $HOME/dial if you do not define this variable.

2. Set up the DIAL environment. From a bash shell, run
     > . $DIALSETDIR/setup.sh [VERSION]
   DIALSETDIR is the location of the DIAL setup script.
   At BNL, set
     > DIALSETDIR=/afs/usatlas.bnl.gov/users/dladams/apps/dial
   At CERN, set
     > DIALSETDIR=/afs/cern.ch/user/d/dadams/apps/dial
   If you want a particular version specify it on the command line.

3. Create an empty working directory and cd there.

4. To install the files for this demo: 
     > $DIAL_ROOT_DIR/demos/setup demo2

5. Start ROOT (should be version 3.10.02) and and use the submit macro
   to submit a job:
     > root.exe
     root> submit();
   This will use dial_cbnt to process the default virtual dataset.
   The applications aodhisto or atlasreco may also be used.
   To choose another application or dataset, find the dataset name in the
   DSC and provide it and the application names as arguments:
     root> submit("my_application_name", "my_dataset_name");
   The ID of the submitted job will be in the variable jid. To display:
     root> cout << jid.to_string() << endl;

6. Display the new job status with
     root> cout << msch.job(jid) << endl << endl;
   and, when it is done or an intermediate result is available, examine
   the result with:
     root> cout << msch.job(jid).result() << endl << endl;
   If the application is dial_cbnt, the result will include a filled
   HBOOK logical file most likely in a LocalFileCatalog or
   AfsFileCatalog. The ID of that logical file gives the location of the
   output HBOOK file. (Note: Unlike root demo 1, there is need to update
   the job. This is done automatically by the scheduler in a separate
   thread.)

7. The macro open_hbook may be used to extract the filled HBOOK file
   from the result, convert it to ROOT format and open the ROOT file:
     root> open_hbook();
   The job status and a list of the histograms will be displayed.
   The histograms may be manipulated in the usual manner, e.g.
     root> h5.Draw();
   to draw a histogram of event Id's

------------------------------------------------------------------------

Reconfiguring
-------------

1. The task may be modified by replacing task.xml or editing the
   relevant files and deleting the existing task.xml. For dial_cbnt, the
   relevant files are init.kumac and cbnt.f. If absent, task.xml, is
   automatically created the next time submit.C is run.

2. To modify the scheduler configuration, edit rootlogon.C.

------------------------------------------------------------------------

Cleaning up
-----------

1. Job output (actually the run directories) may be found at $DIAL_JOBS.
   By default the subjob outputs (in $DIAL_JOBS/LocalScheduler) are
   deleted when you exit root. If space is tight and you do now want any
   results from earlier jobs, you may safely delete all job directories
   any time DIAL is not running:
     > rm -rf $DIAL_JOBS; mkdir $DIAL_JOBS

2. If DIAL is interrupted during job submission, there is a chance that
   files used to track job ID's may be invalidated. The symptom is that
     > ls $DIAL_UIDS/Job
   shows files with the suffix ".sav". Note it is normal to see such files
   while DIAL is running. If this occurs, you can recover by renaming the
   file to drop the .sav extension.