BestMan

Introduction

This page organizes ATLAS-specific setups for various Bestman storage element setups; part of the US ATLAS IntegrationProgram.

More information about Bestman in general can be found on the OSG TWiki at https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/Bestman , and at LBNL at http://datagrid.lbl.gov/bestman/ and http://wt2.slac.stanford.edu/xrootdfs/bestman-gateway.html .

For Bestman2 installations, simply replace bestman with bestman2 and Bestman with Bestman2 in all places below.

Bestman Gateway

For Posix filesystems, this is the recommended installation from the VDT:

• https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/BestmanGateway

Here is one possible way to install it. This assumes an up-to-date grid-mapfile in /etc/grid-security/ and an http*.pem certificate -- which must be a copy of the host certificate -- owned by daemon in /etc/grid-security/http/ .

mkdir -p /opt/bestman
cd /opt/bestman
. /opt/pacman/latest/setup.sh 

time pacman -trust-all-caches -get http://software.grid.iu.edu/osg-1.2:Bestman

. setup.sh
vdt-post-install
vdt-ca-manage setupca --location local --url osg

export EDITOR=emacs
visudo               # add:
----
#Defaults requiretty # be sure that line is commented out if it already exists

Cmnd_Alias SRM_CMD = /bin/rm, /bin/mkdir, /bin/rmdir, /bin/mv, /bin/cp, /bin/ls
Runas_Alias SRM_USR = ALL, !root
daemon     ALL=(SRM_USR) NOPASSWD: SRM_CMD
----

$VDT_LOCATION/vdt/setup/configure_bestman --server y --user daemon --cert /etc/grid-security/http/httpcert.pem --key /etc/grid-security/http/httpkey.pem --http-port 8080 --https-port 8443 \
--globus-tcp-port-range 63001,65000 --enable-gateway --with-tokens-list "\
ATLASDATADISK[desc:ATLASDATADISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasdatadisk];\
ATLASMCDISK[desc:ATLASMCDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasmcdisk];\
ATLASUSERDISK[desc:ATLASUSERDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasuserdisk];\
ATLASPRODDISK[desc:ATLASPRODDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasproddisk];\
ATLASGROUPDISK[desc:ATLASGROUPDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasgroupdisk];\
ATLASHOTDISK[desc:ATLASHOTDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlashotdisk];\
ATLASLOCALGROUPDISK[desc:ATLASLOCALGROUPDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlaslocalgroupdisk];\
ATLASSCRATCHDISK[desc:ATLASSCRATCHDISK][175000][owner:atlas][retention:REPLICA][latency:ONLINE][usedBytesCommand:/opt/bestman_apps/spacetoken/token_space.sh atlasscratchdisk]\
" --with-transfer-servers "gsiftp://tier2-02.ochep.ou.edu;gsiftp://tier2-03.ochep.ou.edu;gsiftp://tier2-04.ochep.ou.edu" --with-allowed-paths=/storage/data

# Edit bestman/sbin/bestman.server and change this line:

MAX_JAVA_HEAP=2048

# Edit bestman/conf/server-config.wsdd and change this line:

<parameter name="containerThreads" value="512"/>


vdt-control --enable vdt-rotate-logs
vdt-control --list
vdt-control --on

cd /storage/data/
mkdir atlasdatadisk atlasmcdisk atlasuserdisk atlasproddisk atlasgroupdisk atlashotdisk atlaslocalgroupdisk atlasscratchdisk
chown atlas. atlas*
chmod g+w atlas*
cd -

You will also need to configure Bestman to user adler32 checksum instead of md5. Please look at BestmanAdler32Checksum for directions for that.

If you would like to have Bestman "intelligently" decide the best gridftp server depending on the storage path it is trying to access, you can install the BestmanGridftpPathPlugin plugin.

Then, to test it as a regular user:

srm-copy file://///home/hs/tmp/test1 srm://tier2-02.ochep.ou.edu:8443/srm/v2/server\?SFN=/storage/data/griddata/test/test1 -spacetoken ATLASDATADISK
srm-copy srm://tier2-02.ochep.ou.edu:8443/srm/v2/server\?SFN=/storage/data/griddata/test/test1 file://///home/hs/tmp/test2
srm-ls srm://tier2-02.ochep.ou.edu:8443/srm/v2/server\?SFN=/storage/data/griddata/test/test1 -fulldetailed
srm-ping srm://tier2-02.ochep.ou.edu:8443/srm/v2/server
srm-sp-tokens srm://tier2-02.ochep.ou.edu:8443/srm/v2/server
srm-get-space-tokens srm://tier2-02.ochep.ou.edu:8443/srm/v2/server?SFN=/
srm-get-space-tokens -space_desc=ATLASPRODDISK srm://tier2-02.ochep.ou.edu:8443/srm/v2/server?SFN=/
srm-get-space-metadata -space_tokens=ATLASPRODDISK,ATLASUSERDISK srm://tier2-02.ochep.ou.edu:8443/srm/v2/server?SFN=/

Bestman-Xrootd

For an Xrootd backend storage system, this is the recommended installation from the VDT:

• https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/BestmanGatewayXrootd
• More information for ATLAS can be found on the BestmanXrootd page

Space tokens for ATLAS

• From Wei: In the way we use Bestman-gateway (pathToToken=false), bestman-g doesn't know the mapping between path and token. The original SRM specification doesn't assign a space token to a directory path. WLCG static space token does that. dCache follows the WLCG specification. Xrootd follows the original SRM specification. So dCache maps a space token to a directory path. Files under that path belong to that space token. In Xrootd, each file has an extend attribute called oss.cgroup, which is used to map to space token. It is a per file attribute regardless of the location. Since ToA? associates a path to a token anyway, choosing to follow dCache, Xrootd or ignoring space token (GPFS, Ibrix, Lustre) will all work.

• Setup at SLAC:

staticTokenList=ATLASDATADISK[desc:ATLASDATADISK][151200];ATLASMCDISK[51200]
[desc:ATLASMCDISK];ATLASPRODDISK[20480][desc:ATLASPRODDISK]

(and userdisk, groupdisk)

Support

• Main support list is osg-storage@opensciencegrid.org
• Use usatlas-ddm-l@lists.bnl.gov for ATLAS communications.
• Alex Sim - Bestman developer
• Wei Yang - Bestman-xrootd developer

References

• Original installation notes from Wei:
  • http://datagrid.lbl.gov/bestman/docs/bestman-guide.html
  • http://wt2.slac.stanford.edu/xrootdfs/bestman-gateway.html

-- RobertGardner - 14 Jan 2009