Command line demo 1 for DIAL 0.92
David Adams
August 2004
Questions or problems to dladams@bnl.gov.

This demo submits and monitors a DIAL job from the command line
using the command dial_submit.

The scheduler is defined in dial_submit.cnf. The processing may be
distributed using fork, LSF, Condor or a couple flavors of lsrun.
The application, task and dataset are defined in local XML files.

Any of the applications described on the application page at
  http://www.usatlas.bnl.gov/~dladams/dial/releases/0.92/apps.html
may be used. The instructions below assume the application dial_cbnt.

Tested on the BNL ACF (ATLAS Computing Facility). Should work on any
RedHat 7.3 machine where DIAL is installed.

For ATLAS dataset catalogs see
  http://www.usatlas.bnl.gov/~dladams/dataset/atlas_catalog.

-----------------------------------------------------------------------

Instructions for running the demo. Note that C-shells are not supported
in this release.

1. Define the location where DIAL is to write its tasks and jobs, e.g.:
     > DIAL_PRIVATE_DIR=$HOME/dial_private
   Choose a directory that exists and is accessible from all
   processing nodes. This must be defined before step 2.

2. Set up the DIAL environment. From a bash shell, run
     > . $DIAL_SETUP_DIR/setup.sh [VERSION]
   The variable DIAL_SETUP_DIR is defined when DIAL is installed. To
   use the BNL version installed on AFS, set
     > DIAL_SETUP_DIR=/afs/usatlas.bnl.gov/users/dladams/apps/dial
   If you want a particular version specify it on the command line.

3. Create an empty working directory and cd there.

4. Run
     > $DIAL_COM_DIR/demos/setup demo1
   to install the files for this demo.

5. Edit the histogram definitions in init.kumac and event processing
   subroutine in cbnt.f. Or accept the defaults.

6. Build the task description from init.kumac and cbnt.f.
     > make_cbnt_task
   The task description task.xml should appear.

7. Extract a dataset for processing, e.g.
     > dataset_extract -n test_2001_100
   The dataset description dataset.xml will appear.
   Catalogs of ATLAS datasets can be found at the above URL. Select any
   name from the DSC with content cbnt.

8. Edit dial_submit.cnf to select scheduling options. The initial
   setting should be fine. The job limit max_subjobs may be increased
   to something like the number of available CPU's.
   At BNL, I suggest:
     lsf dial
     max_subjobs = 100
     max_start_jobs = 30
   You may need to ask for permission to use the dial queue. All ATLAS
   members should have access to at_cas and can use it in place of dial.
   The default (fork) is fine for small datasets.

9. Process the dataset:
     > dial_submit

If successful, a description of the generated result will appear
on the screen. It will include the name of the filled HBOOK file
which may be viewed in the normal manner. To avoid case problems,
link to a lowercase name in the local directory:
  > ln -s <somedir>/dataset<XXX>.hbk res.hbk
  > paw
  PAW> hi/file 1 res.hbk
  PAW> hi/plot 2
  PAW> exit
In the first line, substitute the name (ID) from the result description.

The above instructions assume the application dial_cbnt. For other
applications, change the input XML files accordingly:
  Application in app.xml
  Task in task.xml
  Dataset in dataset.xml