DQ2 Clients in OSG

Introduction

DQ2 Clients are programs that allow to interact with the ATLAS Distributed Data Management system, DQ2. DQ2 Clients include both client programs to interact with site services and central catalog (dataset subscriptions) and also the enduser tools to move datasets (dq2-get/dq2-ls/...).

DQ2 Clients can use different Grid middleware. Here is documented only the installation with WlcgClient, for all other configurations and uses refer to the DQ2 documentation. DQ2-Client Pacman installation is compatible with OSG's wlcg-client and wlcg-client is sufficient to access all ATLAS data: can check all catalogs and move files correctly from/to all OSG/LCG/NG storage elements.

NOTE: If you use dq2-put on Athena (.pool.root) files you will need to setup also an Athena release. DQ2 uses POOL utilities to extract the GUIDs (see DQ2 documentation).

IMPORTANT: The default python from SL5 is python 2.4 whereas DQ2-Client (production version as of 9/20) requires 2.5 or greater. To install and use python 2.6 see the Install and setup Python 2.6 section in the WlcgClient document.

DQ2-Client Pacman installation

To simplify maintenance we recommend to install DQ2-Client in a separate directory, e.g.:

mkdir /share/dq2-client
cd /share/dq2-client
pacman -trust-all-caches -allow tar-overwrite -get http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/project/ddm/releases/pacman/cache:DQ2Clients

To use the client source the setup file created by Pacman during the installation, e.g.:

source /share/dq2-client/setup.sh

Make sure to have the latest DQ2Client version to have all the features and bug fixes.

If you want the latest cutting edge client install the DQ2ClientsTesting instead of DQ2Clients. If you install DQ2ClientsTesting you may have to modify the code to remove the version check from dq2-get ( DQ2ClientsTesting/usr/lib/python2.3/site-packages/dq2/clientapi/cli/Get.py) and dq2-put ( DQ2ClientsTesting/usr/lib/python2.3/site-packages/dq2/clientapi/cli/Put.py).

For documentation about the DQ2-Client refer to the pages provided by the DQ2 team:

• how to install DQ2-Client using Pacman
• Check DQ2-Clients wiki page and user manual
• Since 2009 the most uptodate DQ2 instructions are in the DQ2 clients how-to

Update

Supposing that you installed DQ2-Client in a separate directory as described here you can update it by removing the old installation and installing it again. It will take few minutes, DQ2Clients is a small package:

cd /share/dq2-client/
rm -rf *
pacman -trust-all-caches -allow tar-overwrite -get http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/project/ddm/releases/pacman/cache:DQ2Clients

-- MarcoMambelli - 29 Oct 2008