r1 - 20 Mar 2013 - 19:03:02 - DavidLesnyYou are here: TWiki >  Admins Web > ALRBwrap

ALRBwrap

Altlas Local Root Base Command Wrapper

The attached bash script ALRBwrap can be used to execute an arbitrary command within the context of an environment of the Atlas Local Root Base.

The ALRB is a package contained within the CVMFS repository atlas.cern.ch and should be available on all Atlas nodes. The ALRB provides many of the tools needed for an Atlas job. These include the DQ2Client (dq2-ls, dq2-list-dataset-site, etc) and GLite (lfc-ls, lcg-cp, etc).

All packages in ARLB require python 2.6. When ALRB is setup, an environment using python 2.6 is created. Unfortunately other components in the Atlas software stack are incompatible with python 2.6 and require python 2.4.

If a command is "softlink" to the ALRBwrap script, when the command is invoked, the ALRB environment will be created. The command is then executed within that context as an "exec". When the command completes, the ALRB is "deleted" and the original context of the process is returned.

In this way, a process can start in a python 2.4 environment (the default on SL5), execute a command requiring python 2.6, but never leave the original python 2.4 environment.

How to use

Create a directory to hold the ALRBwrap script and the command to be executed in the ALRB context. Download the script ALRBwrap into this location. A second script, ALRBlink.sh is also provided. This script will create many of the "softlink" commands you might want to execute.

[ddl@uct2-int ~]$ cd ALRB
[ddl@uct2-int ALRB]$ ls-l
total 12
-rw-r--r-- 1 ddl mwt2 4313 Mar 20 17:26 ALRBlink.sh
-rwxr-xr-x 1 ddl mwt2 3275 Mar 20 17:26 ALRBwrap

Enable some ALRB commands

Create any "softlink" commands you would like to use in the ARLB environment

[ddl@uct2-int ALRB]$ ln -s ALRBwrap dq2-ls
[ddl@uct2-int ALRB]$ ln -s ALRBwrap dq2-version
[ddl@uct2-int ALRB]$ ln -s ALRBwrap dq2-python

[ddl@uct2-int ALRB]$ ls -l
total 12
-rw-r--r-- 1 ddl mwt2 4313 Mar 20 17:26 ALRBlink.sh
-rwxr-xr-x 1 ddl mwt2 3275 Mar 20 17:26 ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:30 dq2-ls -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:30 dq2-python -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:30 dq2-version -> ALRBwrap

Put them in the $PATH

To use these commands, the directory they reside in must be added to your $PATH

[ddl@uct2-int ~]$ PATH=$HOME/ALRB:$PATH
[ddl@uct2-int ~]$ echo $PATH

Execute them

You can now excute those commands. The current process context will not be changed.

[ddl@uct2-int ~]$ python -V
Python 2.4.3
[ddl@uct2-int ~]$ dq2-ls
Usage: dq2-ls [ -h/--help | options ] <PATTERN>
[ddl@uct2-int ~]$ dq2-version
2.3.0
[ddl@uct2-int ~]$ dq2-python -V
Python 2.6.5
[ddl@uct2-int ~]$ python -V
Python 2.4.3
[ddl@uct2-int ~]$ 

In the above example, the initial python version available is 2.4.

The "dq2-ls" command is then run (and gives the standard usage statement).

The next command "dq2-version" (an internal command in ARLBwrap) displays the current version of DQ2Clients? . It shows that version 2.3.0 of DQ2Clients? is available (the value of $ATLAS_LOCAL_DQ2CLIENT_VERSION).

The command "dq2-python" (also an internal ALRBwap command), is the exeuction of "python -V" and show that python 2.6 is being used.

However afterward, "python -V" shows that we are still in a python 2.4 environment.

No environment variables modified within the ALRBwrap script exit that world to the current.

Create a full command set

The provided script ALRBlink.sh crfeates many of the commands one might want to execute with ALRB. These include all the DQ2Client? and GLite commands.

Download this script in the same ARLB directory, edit it to make certain you want all those commands, then execute it

[ddl@uct2-int ALRB]$ ls -l
total 12
-rw-r--r-- 1 ddl mwt2 4313 Mar 20 17:26 ALRBlink.sh
-rwxr-xr-x 1 ddl mwt2 3275 Mar 20 17:26 ALRBwrap
[ddl@uct2-int ALRB]$ sh ALRBlink.sh 
[ddl@uct2-int ALRB]$ ls -l
total 12
-rw-r--r-- 1 ddl mwt2 4313 Mar 20 17:26 ALRBlink.sh
-rwxr-xr-x 1 ddl mwt2 3275 Mar 20 17:26 ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-addfs -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-addpool -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-drain -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-getspacemd -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-getspacetokens -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-listspaces -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-modifyfs -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-modifypool -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-ping -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 dpm-qryconf -> ALRBwrap
.
.
.
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-modifygrpmap -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-modifyusrmap -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-ping -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-rename -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-rm -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-rmgrpmap -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-rmusrmap -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-setacl -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 lfc-setcomment -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfcat -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfchmod -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfcp -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfdf -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfdir -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfmkdir -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfrename -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfrm -> ALRBwrap
lrwxrwxrwx 1 ddl mwt2    8 Mar 20 17:46 rfstat -> ALRBwrap

Potential problems

There are several potentail problems with this solution

Inifinite Loops

One of the potential problems is ending up creating an infinite loop.

The script attempts to prevent this by removing the directory the executing command resides in from the current $PATH

PathCMD=$(dirname $0)
PathCMD=$(cd "$PathCMD" && pwd)
PATH=${PATH/#$PathCMD:/}
PATH=${PATH/%:$PathCMD/}
PATH=${PATH//:$PathCMD:/}

The above could cause problems if the executed command relies on another which is also a "softlink" command in this $PATH. If this is the case, these lines could be removed.

To stop "all" possible infinite loops, the script checks for a set variable, $ALRBpid and exits if set.

if [[ -n $ALRBpid ]]; then
   echo "%ALRBwrap infinite loop detected - exiting"
   exit 1
else
   export ALRBpid=$$
fi

Slow Hz rate

Since every command residing in the ALRB world must "source" the ARLB environment, executing a command in this fashion is "High Cost". The Hz rate will be very slow and will not be very useful in an situation where many dq2, lfc or lcg commands need to be executed.

But this might be a good solution when only a few invocations are needed.

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Attachments


alrbwrap ALRBwrap (3.2K) | DavidLesny, 20 Mar 2013 - 18:09 |
sh ALRBlink.sh (4.2K) | DavidLesny, 20 Mar 2013 - 18:10 |
 
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback