UpdateOSGFull

OSG 1.2.25

• Current OSG production verison is 1.2.25 (1.2.21 plus a couple of security patches)
• See announcement from OSG Operations, August 11, 2011.
• LFC update, including python-interfaces to support latest DQ2 clients

Introduction

This version of OSG worker-node client includes updated LFC bindings for both 32 and 64 bit versions of python 2.4, 2.5, and 2.6. Python 2.6 is required by the latest dq2 client.

Requirements for the new configuration including the latest production DQ2Clients:

1. RHEL 5 based OS (SL5 or Centos5)
2. OSG WN Client 1.2.25 (see below for installation - 1.2.21 or greater)
3. ATLAS-wn 1.3rolling (including ATLAS-OSG-compat 0.6 package - see below for installation)
4. python 2.6 (see below for installation)

IMPORTANT! This version of OSG worker-node client (specifically the LFC binding included into it) will not work on RHEL4 or SL4. It will work only on RHEL 5 based OS (SL5 or Centos5)

This document lists 2 possible update procedures:

• suggested one: simple but requires downtime
• a more complex one that allows to avoid downtime

Then it lists detailed instructions for each of the steps involved, with possible alternatives and advantages/disadvantages.

For a simpler document with only the instructions for site administrators (only suggested update procedure) see UpdateOSG.

Suggested update procedure:

To simplify the update procedure we recommend a downtime during which all the updates will be made. An alternative update procedure mentioned below allows to update without interrupt production but it is more complex. In the list below each step has a prefix: Xin are the steps that are usually performed by Xin (remote sw. installation via grid jobs), Admin are the steps that should be performed by a site administrator:

1. Admin will coordinate with Xin the schedule for a downtime when no jobs are running on the cluster and both (Admin and Xin) can do the update (the gatekeeper should be up to allow Xin to run installation jobs)
2. Admin: Update to the new OSG wn-client (v 1.2.21)
3. Admin: Install Python 2.6 using the python26 yum package from EPEL
4. Xin: Update to a version of ATLAS-wn that contains the new ATLAS-OSG-compat package and the latest production DQ2Clients (ATLAS-wn)

You can find descriptions and instructions for each step below in this document.

Update procedure not requiring downtime:

This is an alternative to the suggested update procedure described in the previous section. To avoid downtimes while updating a package installed via Pacman the recommandation is to reference (in the programs, path, ...) a symbolic link instead of the actual installation directory install the new version in a new directory then change a link from the old to the new version, e.g. for the program mysoftware, installed in /opt/mysoftware-old and with /opt/mysoftware linked to /opt/mysoftware-old:

mkdir /opt/mysoftware-new
cd /opt/mysoftware-new
pacman -get CACHE:mysoftware
# continue the installation and control that it completed successfully (e.g. setup files are there)
ln -sf /opt/mysoftware-new /opt/mysoftware

This is an update order that takes into account requirements, allows a site to remain up and do not require simultaneous updates of multiple packages. Xin are the steps that are usually performed by Xin (remote sw. installation via grid jobs), Admin are the steps that should be performed by a site administrator.

1. Xin: Update to a version of ATLAS-wn-old that contains the new ATLAS-OSG-compat package but still the currently deployed DQ2Clients 0.1.35 (ATLAS-wn)
2. Admin: Update to the new OSG wn-client (1.2.23)
3. Admin/Xin: Install Python 2.6 (this could be done any time before this point)
4. Xin: Update to a version of ATLAS-wn that contains the new ATLAS-OSG-compat package and the latest production DQ2Clients (ATLAS-wn)

Install OSG WN Client

You can follow the instructions in the OSG Worker Node Client Install guide. E.g.:

[root@wn /opt/wn-client]$ pacman -get http://software.grid.iu.edu/osg-1.2:wn-client

Remember to setup the CA certificates following the usual OSG instructions.

Install the ATLAS workernode (ATLAS-wn) package

This package is necessary because:

• It provides the DQ2Clients (latest release in the version ATLAS-wn , DQ2Clients 0.1.35 in ATLAS-wn-old)
• It sets environment variables
• It fixes a compatibility issue with the ATLAS releases

Generic instructions to install ATLAS-wn are here.

In step 1 above you need to install the ATLAS-wn-old (the old version, it points to DQ2Clients 0.1.35), i.e.

pacman -get http://www.mwt2.org/caches/:ATLAS-wn-old

In step 4 above you will need to install the ATLAS-wn version (the current production version (rolling), it points to the production DQ2Clients 0.1.36 as of 10/24 but 0.1.37 will come out soon), i.e.

pacman -get http://www.mwt2.org/caches/:ATLAS-wn

IMPORTANT! Before issuing the Pacman command above you have to have a OSG wn-client already installed and the environment variable OSG_GRID must contain the root installation directory (e.g. export OSG_GRID=/opt/wn-client, if don't have it in your environment).

IMPORTANT 2! DQ2Clients? 0.1.36 (latest release included in ATLAS-wn as of 10/24) and following do require python 2.5 or greater. The system python in RHEL 5 based systems is 2.4.

Install python 2.6

The default python from SL5 is python 2.4 whereas dq2 requires 2.5 or greater. We suggest python 2.6 (and not 2.5 or 2.7) because it is available from EPEL. Here we present 2 alternatives:

• Install python 2.6 from EPEL (preferred)
• Install python 2.6 using Pacman

Install python 2.6 from EPEL

You need to be root. Add the EPEL repository. Note that this will alter your system yum repositories!

rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Install python26:

yum install python26

Change the DQ2Clients package to use python26. Installing python26 from EPEL will not make it your default Python. DQ2Clients uses the default python in your environment (/usr/bin/env python). In order to have DQ2Clients using it you have at least these 2 options:

• add python26 to the PATH using pythonloader (will be hiding the default system one) (preferred)
• add python26 to the PATH as python (will be hiding the default system one)
• change the shebang (hashbang) line in all DQ2Clients executable scripts

Add pythonloader as python in the path

The directory atlasosgcompat/bin part of both the ATLAS-wn and the wlcg-client packages is added to the PATH when the setup file for ATLAS-wn or wlcg-client is sourced. Link atlasosgcompat/bin/python to pythonloader, so that it will become the default python in the users PATH once they source the ATLAS-wn or wlcg-client setup:

ln -s $VDT_LOCATION/atlasosgcompat/bin/pythonloader $VDT_LOCATION/atlasosgcompat/bin/python

This script checks your current python, the one on top of the PATH. If you have already python>= 2.5 does nothing and uses that one to run your python script. If not, it checks in possible places where python 2.5 or greater could be (all the python or python26 executables in the path) and uses the first one it encounters (instead of the one on top of the PATH) to run your python script. You don't choose exactly which version to run, id depends on the PATH content and order. This solution is the least invasive. It is not changing external packages and it is not replacing your custom python that may have been added to the PATH (e.g. by Athena setup).

Add python26 as python in the path

The directory atlasosgcompat/bin part of the ATLAS-wn package is added to the PATH when the setup file for ATLAS-wn is sourced. Adding a link named python will hide/replace the system python, e.g.:

cd /osg/app/atlas_app/atlaswn/atlasosgcompat/bin
ln -s /usr/bin/python26 python

This solution is not changing packages different from ATLAS-OSG-compat but it will change the default python once the setup file is sourced and DQ2Clients my end up using a different python if the PATH is changed afterwards (this is not a problem as long as it is any Python 2.5/6 32bit or 64bit).

Change the shebang line in DQ2Clients scripts

All dq2 scripts start with /usr/bin/env python and the default system python in the path is python 2.4 (RHEL5 based OS). To change them to use python26 do the equivalent of:

cd /osg/app/atlas_app/atlaswn/DQ2Clients/opt/dq2/bin/
for i in * ; do sed -i "1s|#!/usr/bin/env python|#!/usr/bin/python26|" $i; done

This solution is changing an external package (DQ2Clients not developed by us) and needs to be reapplied each time that package is updated/installed, but it will not change the default python and DQ2Clients my always use /usr/bin/python26.

Install python 2.6 with Pacman

Choose the correct package:

• Python32bit26 if you have a 32 bit OS (can work also on 64 bit OS if you need it for a test)
• Python64bit26 if you have a 64 bit OS and prefer 64 bit python (recommended for 64 bit OS)

Go in the same directory where you installed ATLAS-wn and install it, e.g.:

pacman -get http://www.mwt2.org/caches/:Python64bit26

This will add python 2.6 as default python once the setup file is sourced, so it is equivalent to the solution adding python26 to the PATH: it is not changing DQ2Clients package but it will change the default python once the setup file is sourced and DQ2Clients my end up using a different python if the PATH is changed afterwards (this is not a problem as long as it is any Python 2.5/6 32bit or 64bit).

-- MarcoMambelli - 23 Aug 2011