InstallLFConOSG

Site installation inventory

Site	Host	Full LFC installation	LFC with MySQL installed separately	MySQL version used	On OSG CE?	LRC Migrated?
AGLT2
NET2
MWT2	uct2-grid5, tier2-05			5.0.67
SWT2-OU	tier2-02.ochep.ou.edu			5.0.67
SWT2-UTA
WT2	atl-lfc.slac.stanford.edu			4.1.22

The LFC Administrators Guide is essential:

https://twiki.cern.ch/twiki/bin/view/LCG/LfcAdminGuide

Installation: full LFC + MySQL

This installation works with or without a pre-installed OSG CE installation on this host. If you already have a working CE -- or any other VDT installation which contains an updated CA directory -- then please enter its $GLOBUS_LOCATION as $CE_GLOBUS_LOCATION below, and it will link the CA directories to $GLOBUS_LOCATION/TRUSTED_CA. Otherwise, enter NONE, and the CA directory will be downloaded and the cron update scripts installed.

You will be asked for the CE_GLOBUS_LOCATION, the unix LFCUSER and LFCGROUP, and the mysql MYSQL_ROOT_PWD, which will be set for you during the installation.

NOTE: MYSQL_LFCUSER and MYSQL_LFCUSER_PWD are no longer being asked for, they are filled in automatically as lfc and a random password, which will be logged in lcg/etc/NSCONFIG, if you care to know it. That shouldn't really be necessary, though.

It is assumed that the unix user LFCUSER:LFCGROUP already exists; the mysql user lfc will be created for you.

You will also need the following additions in /etc/grid-security/, if you don't already have them:

mkdir /etc/grid-security/$LFCUSER
cd /etc/grid-security/
cp -p hostcert.pem $LFCUSER/lfccert.pem
cp -p hostkey.pem $LFCUSER/lfckey.pem
chmod 644 $LFCUSER/lfccert.pem
chmod 400 $LFCUSER/lfckey.pem
chown -R $LFCUSER:$LFCGROUP $LFCUSER

You can install the voms certs via tarball, or other methods.

wget http://www-hep.nhn.ou.edu/pacman-cache/packages/vomsdir.tar.gz
tar -zxvf vomsdir.tar.gz

Then setup pacman, and issue the following commands.

mkdir /opt/lfc/
cd /opt/lfc/
pacman -trust-all-caches -get OU:LFC-MySQL
ln -s /opt/lfc/lcg /opt/lcg
. setup.sh
vdt-control --on

You will need to make the above soft link because the lfcdaemon has the path /opt/lcg/etc/lcgdm-mapfile hard wired.

Then follow the directions below at the Post-install check-list .

Installation: LFC w/ MySQL installed separately

The LFC Administrators Guide is essential:

https://twiki.cern.ch/twiki/bin/view/LCG/LfcAdminGuide

Set up MySQL, if you do not already have a MySQL server that you want to use for this.

[root@tier2-04]# yum install mysql-server
[...] 
[root@tier2-04]# chkconfig --levels 2345 mysqld on
[root@tier2-04]# service mysql start

Get Pacman:

[root@tier2-04]# cd /opt
[root@tier2-04 opt]# wget http://physics.bu.edu/pacman/sample_cache/tarballs/pacman-latest.tar.gz
[...]

Get LFC from Alain's Pacman cache:

[root@tier2-04 opt]# pacman -get http://vdt.cs.wisc.edu/test-cache/roy:LFC
[...]  

LFC daemon wants to run as dedicated user lfcmgr.

[root@tier2-04]# useradd -c "LFC manager" -d /home/lfcmgr lfcmgr

Note the following values are hardcoded, at present:

NSCONFIGFILE=/opt/lcg/etc/NSCONFIG
LFCUSER=lfcmgr
LFCGROUP=lfcmgr

When I installed the VDT package at UC, I found that some directory permissions needed to be changed - many dirs under /opt/lfc are only readable by root, and the daemon wants to run as lfcmgr

[root@tier2-04 lcg]# ls -ld /opt/lcg/bin
drwx------ 2 root root 4096 Aug 26 19:17 bin

Correct all directory perms, if needed

[root@tier2-04 opt]# find . -type d  -exec chmod go+rx {} \;

(you could also probably chown these to lfcmgr)

Initialize the MySQL databases:

[root@tier2-04]# INSTALL_DIR=/opt/lcg
[root@tier2-04]# mysql -u root -p < $INSTALL_DIR/share/LFC/create_lfc_tables_mysql.sql
[root@tier2-04]# mysql -u root -p
use mysql
GRANT ALL PRIVILEGES ON cns_db.* TO 'lfc'@localhost IDENTIFIED BY 'db_password' WITH GRANT OPTION;
GRANT ALL PRIVILEGES ON cns_db.* TO 'lfc'@LFC_HOST IDENTIFIED BY 'db_password' WITH GRANT OPTION;

Create /opt/lcg/etc/NSCONFIG, containing

lfc/db_password@lfc_host

Now, to start it:

[root@tier2-04 init.d]# vdt-control --list
Service            | Type   | Desired State
-------------------+--------+--------------
fetch-crl          | cron   | enable
vdt-rotate-logs    | cron   | enable
vdt-update-certs   | cron   | enable
mysql5             | init   | do not enable

I copied init scripts to /etc/init.d

[root@tier2-04]# cp /opt/lcg/etc/init.d/lfc* /etc/init.d
[root@tier2-04]# chkconfig --add lfcdaemon
[root@tier2-04]# chkconfig --add lfc-dli
[root@tier2-04]# . /opt/setup.sh
[root@tier2-04]# /etc/init.d/lfcdaemon start
Starting lfcdaemon:                                        [  OK  ]
[root@tier2-04]# 
[root@tier2-04]# /etc/init.d/lfc-dli start
Starting lfc-dli:                                          [  OK  ]
[root@tier2-04]# 

In order to run the clients, the environment variable LFC_HOST (and possibly also LFC_PORT) must also be set.

You also need copies of the host cert/key copied (not linked) into /etc/grid-security/lfcmgr and renamed as follows, with the ownership of the account running the LFC daemon.

cgw@tier2-04~$ ls -l /etc/grid-security/lfcmgr
-rw-r--r-- 2 lfcmgr lfcmgr 1311 Aug 27 05:44 lfccert.pem
-r-------- 2 lfcmgr lfcmgr  887 Aug 27 05:44 lfckey.pem

NOTE: Also make sure the X509_CADIR and X509_CERT_DIR point to the correct location for your system. By default mine pointed to the $LFC_INSTALL/globus/TRUSTED_CA area (which didn't exist till I soft-linked it to the right place).

Post-install check-list

Testing with lfc-ls produces an error:

cgw@tier2-04~$ export LFC_HOST=tier2-04.uchicago.edu
cgw@tier2-04~$ lfc-ls /
Could not get virtual id for /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209: Can't open configuration file !
/: No user mapping

According to the LfcAdminGuide, if things are working correctly this command should produce no output.

Consulting the LFC logfile /var/log/lfc/log, I see

09/02 21:23:24 23553,0 Cns_vo_from_dn: NS023 - /opt/lcg/etc/lcgdm-mapfile is not accessible
09/02 21:23:24 23553,0 sendrep: Could not get virtual id for /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209: Can't open configuration file !

This error is happening because I used a plain grid proxy instead of a VOMS proxy.

I grabbed /etc/grid-security/gridmapfile from one of our other servers, and copied this to tier2-04 as /opt/lcg/etc/lcgdm-mapfile. Now, I get the expected "no reply" from the lfc-ls query, and see this message in the server log:

09/02 21:23:55 23553,0 getidmap: Creating a new Virtual gid for usatlas3
09/02 21:23:55 23553,0 Cns_srv_lstat: NS092 - lstat request by /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209 (101,101) from tier2-04.uchicago.edu
09/02 21:23:55 23553,0 Cns_srv_lstat: NS098 - lstat 0 /
09/02 21:23:55 23553,0 Cns_srv_lstat: returns 0
09/02 21:23:56 23553,0 Cns_srv_opendir: NS092 - opendir request by /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209 (101,101) from tier2-04.uchicago.edu
09/02 21:23:56 23553,0 Cns_srv_opendir: NS098 - opendir / 
09/02 21:23:56 23553,0 Cns_srv_opendir: returns 0
09/02 21:23:56 23553,0 Cns_srv_readdir: NS092 - readdir request by /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209 (101,101) from tier2-04.uchicago.edu
09/02 21:23:56 23553,0 Cns_srv_readdir: returns 0
09/02 21:23:56 23553,0 Cns_srv_readdir: NS092 - closedir request by /DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209 (101,101) from tier2-04.uchicago.edu
09/02 21:23:56 23553,0 Cns_srv_readdir: returns 0

This is clearly running in a highly verbose mode - in production conditions I don't think we'll want every access generating this much logging activity - in addition to generating more log files than we will know what to do with, it also must slow the server down. But for right now it's nice, because it's showing us that the LFC daemon is working. However, there are some questions about how authentication works - if we are going to support non-VOMS proxies, this mapfile needs to be managed/updated.

Next steps in the LfcAdminGuide:

[root@tier2-04]# lfc-mkdir /grid
[root@tier2-04]# lfc-mkdir /grid/atlas
cannot create /grid/atlas: Internal error
[root@tier2-04]# ls /grid/atlas
ls: /grid/atlas: No such file or directory
[root@tier2-04]# lfc-ls /grid
[root@tier2-04]# lfc-ls /
grid
[root@tier2-04]# lfc-ls /grid
[root@tier2-04]# lfc-mkdir /grid/atlas
[root@tier2-04]# lfc-ls /grid
atlas
[root@tier2-04]# lfc-ls /grid/atlas
[root@tier2-04]# lfc-entergrpmap --group atlas
[root@tier2-04]# lfc-chown root:atlas /grid/atlas
[root@tier2-04]# lfc-chmod 775 /grid/atlas
[root@tier2-04]# lfc-setacl -m d:u::7,d:g::7,d:o:5 /grid/atlas

The "Internal error" is a bit worrisome but on a subsequent attempt it did not recur.

To support VOMS proxies, one needs to have the correct certs in /etc/grid-security/vomsdir . You can install them via rpm or other methods.

[root@tier2-04]# rpm --install 'http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-UI/sl4/i386/RPMS.updates/lcg-vomscerts-5.1.0-1.noarch.rpm
[root@uct2-grid5 ~]# ls -l /etc/grid-security//vomsdir/
total 48
-rw-r--r--  1 root root 4612 May  7 21:57 cclcgvomsli01.in2p3.fr.3292
-rw-r--r--  1 root root 5932 May  7 21:57 lcg-voms.cern.ch.2008-04-07
-rw-r--r--  1 root root 4821 May  7 21:57 vo.racf.bnl.gov.15998
-rw-r--r--  1 root root 4832 May  7 21:57 vo.racf.bnl.gov.21772
-rw-r--r--  1 root root 5938 May  7 21:57 voms-test.cern.ch.2007-10-15
-rw-r--r--  1 root root 5912 May  7 21:57 voms.cern.ch.2007-10-15

See https://twiki.cern.ch/twiki/bin/view/LCG/VomsSetup

• Make sure port 5010 (lfc) is open, if you have any firewall in place. You may also want to add

lfc		5010/tcp			# LCG File Catalog

to /etc/services

Other validation tests

Remote users should be able to set LFC_HOST and issue lfc-ls commands, with a valid VOMS proxy.

cgw@iut2-mgt~/test$ lfc-ls /grid 
atlas
cgw@iut2-mgt~/test$ lfc-ls -l /grid
drwxrwxr-x   3 root     102                       0 Sep 08 12:36 atlas
cgw@iut2-mgt~/test$ lfc-ls -l /grid/atlas
drwxrwxr-x  88 102      103                       0 Sep 08 17:53 dq2

There are some test scripts in $INSTALL_DIR/lcg/share/LFC/test but they do not work out of the box:

cgw@uct2-grid5/opt/lcg/share/LFC/test$ ./python-interface-validation.py 
9838816b-beba-44e2-be31-1aecc1c1be85
Error when creating /grid/dteam/my_test_dir/my.test: Error 2 (No such file or directory)
[ERROR] There was an error while looking for /grid/dteam/my_test_dir/my.test: Error 2 (No such file or directory)
Listing replicas for GUID 9838816b-beba-44e2-be31-1aecc1c1be85
Found 0 replica(s)
result = -1
len(list) = 1 replicas found

Edit lines 12 and 13 of this file, changing /grid/dteam to /grid/atlas, and the script should run. This is a good post-intstallation test. (The directory name should be fixed in the next VDT release).

Command-line testing with lcg/lfc utilities

Here are some examples of creating registrations and deleting files using lcg and lfc utilities.

Note: if you do not have uuidgen, you probably need to install the e2fsprogs package.

1. Copy file to SE and register by lcg-cr

bash$ source /share/wn-client/setup.sh # to get lcg-cr
bash$ ls > /tmp/junk                  # create test file
bash$ lcg-cr -l /grid/atlas/test/cgw -g `uuidgen` -d srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001 file:///tmp/junk

LCG_GFAL_INFOSYS not set
lcg_cr: Invalid argument

This is due to the absence of BDII. There is a flag

-b/--nobdii

. But, using this flag results in the same error.

Trying with -v to get more info:

bash$ lcg-cr -b -l /grid/atlas/test/cgw -g `uuidgen` -d srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001 file:///tmp/junk

Using grid catalog type: lfc
LCG_GFAL_INFOSYS not set
Using grid catalog : (null)
Using LFN : /grid/atlas/test/cgw
Invalid BDII parameters
lcg_cr: Invalid argument

"Using grid catalog : (null)" is a clue, that we need to set LFC_HOST

bash$ export LFC_HOST=uct2-grid5.uchicago.edu
bash$ lcg-cr -v -b -l /grid/atlas/test/cgw -g `uuidgen` -d srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001 file:///tmp/junk

Using grid catalog type: lfc
liblfc.so: liblfc.so: cannot open shared object file: No such file or directory
Using grid catalog : (null)
Using LFN : /grid/atlas/test/cgw
liblfc.so: liblfc.so: cannot open shared object file: No such file or directory

Now, we need to source the setup file for LFC to get the needed liblfc.so (or perhaps this is the wrong client package?)

After setting up LFC, the new error is:

bash$ lcg-cr -v -b -l /grid/atlas/test/cgw -g `uuidgen` -d srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001 file:///tmp/junk

Using grid catalog type: lfc
Using grid catalog : uct2-grid5.uchicago.edu
Using LFN : /grid/atlas/test/cgw
Invalid request: When BDII checks are disabled, you must provide SURLs and endpoint type
lcg_cr: Invalid argument

The full command, which works, is this:

bash$ lcg-cr -T srmv2 -v -b -l /grid/atlas/test/cgw/lfctest.001  -g `uuidgen` -d 'srm://uct2-dc1.uchicago.edu:8443/srm/managerv2?SFN=/pnfs/uchicago.edu/test/cgw/lfctest.001' file:////tmp/junk

Additionally, this fails if you don't have the destination dirs in LFC: I had to to do lfc-mkdir /grid/atlas/test and lfc-mkdir /grid/atlas/test/cgw

2. Making sure if the file is in dcache.

bash$ ls -l /pnfs/uchicago.edu/test/cgw/lfctest.001
-rw-r--r--  1 usatlas1 usatlas 7578 Oct  2 10:48 /pnfs/uchicago.edu/test/cgw/lfctest.001

It's in dCache.

3. Making sure that file is in LFC.

bash$ lcg-lr lfn:/grid/atlas/test/cgw/lfctest.001
srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001

It's in LFC. Or, using LFC command,

bash$ lfc-ls /grid/atlas/test/cgw/lfctest.001
/grid/atlas/test/cgw/lfctest.001

works also.

4. Deleting file and catalog entry by LCG command

Again I run into the same BDII problem:

$ 
bash$ lcg-del srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001

LCG_GFAL_INFOSYS not set
lcg-del: Invalid argument

You can specify -b, but then you get:

Invalid request: When BDII checks are disabled, you must provide SURLs and endpoint type

The command which works is:

bash$ lcg-del -v -b -T srmv2 srm://uct2-dc1.uchicago.edu:8443/srm/managerv2?SFN=/pnfs/uchicago.edu/test/cgw/lfctest.001

This is a little bit unfortunate, because the SRM url that you need to specify to lcg-del is different from what is returned to you when you use lcg-lr (which returns

srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/test/cgw/lfctest.001

a simpler SRM url without port numbers, managerv2/SFN, etc)

In other words, if you use lcg-lr to retrieve PFNs (or SURLs or whatever the correct term is), you cannot then pass that URL back to lcg-del without modifying it to include additional information. It would be nice if the SRM utils had reasonable defaults, and one did not always have to specify the additional protocol info.

Perhaps things would be easiest in the long run if we just set up BDII as well as LFC?

5. Making sure it is not there.

bash$ ls -l /pnfs/uchicago.edu/test/cgw/lfctest.001
ls: /pnfs/uchicago.edu/test/cgw/lfctest.001: No such file or directory
bash$ lfc-ls /grid/atlas/test/cgw
/grid/atlas/test/cgw/lfctest.001: No such file or directory

.... it's gone.

MySQL performance tips

MySQL "out of the box" is not optimized for LFC. Here are some performance suggestions. These are just tips, not an authoritative guide to MySQL performance tuning.

• Make sure MySQL is using innodb storage engine. (Recent versions use this by default)
• Increase innodb_buffer_pool_size to 50%-80% of system RAM. For 32-bit OS, this cannot be higher than 2GB.
• Disable OS buffering by using O_DIRECT

I have the following in /etc/my.cnf

[mysqld]
innodb_buffer_pool_size = 2G
innodb_additional_mem_pool_size = 16M
# Secure writing for ACID compliance - 2 is faster but less safe
innodb_flush_log_at_trx_commit = 1
innodb_flush_method = O_DIRECT

• It may also help to reduce system swappiness:

[root@tier2-04]# echo 0 > /proc/sys/vm/swappiness

To make this setting stick, add the line

vm.swappiness = 0

to /etc/sysctl.conf

Operational issues for LRC->LFC migration

• DQ2 site services must be reconfigured to use LFC backend instead of LRC. Catalogue endpoint needs to be changed in TiersOfATLAS? . For testing purposes, it can be overridden in the dq2.cfg file - see: http://ddm-build.cern.ch/ddm/build/stable/doc/guides/dq2-siteservices/html/admin/maintainSection.html#catalogueEndpointsSection

• • Note: syntax for specifying LFC endpoint is

lfc://hostname.site.org:/grid/atlas

• Scripts for LFC integrity checking (replacemant for "check_se.py")

These are already written, see: https://twiki.cern.ch/twiki/bin/view/Atlas/DDMOperationsScripts

• Dataset cleanup. The above link has some cleanup scripts, also the dCache specific "poolcleaner.py" will need to be modified to work with LFC - see http://repo.mwt2.org/viewvc/admin-scripts/dcache/. This will only require a small change.

• MySQL version: Some sites do not want to install MySQL 5 for various reasons. Is it OK to use MySQL 4?

• Filename length - the initialization for the MySQL table Cns_file_replica has

 PRIMARY KEY  (`rowid`),
  KEY `fileid` (`fileid`),
  KEY `host` (`host`),
  KEY `sfn` (`sfn`(255))

The 255-character limit on sfn will probably be a problem.

Open questions

• Should LFC support plain grid proxies, or only VOMS proxies?
  • Answer: VOMS proxies only
  • Answer Part 2: You need to have a gridmap file
• If we support plain grid proxies, how to maintain the gridmap file?
• How to keep /etc/grid-security/vomsdir up-to-date? How often do these certs need to be updated?
• Should LFC be modified to use a GUMS server?
  • Answer: Probably not

Patrick McGuigan writes:

        I think that the lcgdm-mapfile is a small issue, but I don't believe  
        that the GUMS can provide a solution.  The file exists to map vanilla  
        proxies to a virtual group.  To be consistent, the virtual group would
	have to be one of the potential groups allowed via the voms server.  
        More information is available at: 

• https://twiki.cern.ch/twiki/bin/view/LCG/VirtualIds

Slow conversion

According to this paper, https://edms.cern.ch/cedar/plsql/doc.info?cookie=3327541&document_id=545811&ver sion=2 the LFC should be capable of about 100-200 inserts/sec. But the conversion script testLRCtoLFC_UC_1.py is doing 20 inserts/sec. (Script written by Hiro Ito @ BNL, available at http://repo.mwt2.org/viewvc/admin-scripts/lfc/)

From the LfcAdminGuide section on Troubleshooting:

Using transactions slows the performance by at least a factor of 2, 
depending on the number of operations performed per transaction. 
Changing into the working directory when performing a large number
of operations (e.g. inserts, deletes) in the same directory leads to 
significant performance improvement and is strongly recommended. 
Without changing directory, operation time increases linearly as a 
function of number of subdirectories in the path. 

The conversion script is not changing directories.

While the conversion script is running, load is very low. Why?

top - 02:01:06 up 14 min,  3 users,  load average: 0.19, 0.17, 0.07 
Tasks:  93 total,   1 running,  92 sleeping,   0 stopped,   0 zombie 
Cpu(s):  2.8% us,  0.8% sy,  0.0% ni, 96.5% id,  0.0% wa,  0.0% hi,  0.0% si 
Mem:   8316148k total,   641352k used,  7674796k free,    29856k buffers 
Swap:  2097136k total,        0k used,  2097136k free,   313344k cached 
 
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND         
 4182 mysql     16   0 1899m 172m 5652 S    9  2.1   0:59.53 mysqld          
 4636 lfcmgr    16   0  233m 8396 3584 S    4  0.1   0:22.48 lfcdaemon 
 4690 cgw       15   0 74956  46m 4464 S    1  0.6   0:07.43 python   

Running mytop shows that the majority of the MySQL transactions are "select" (66%)

           Command      Total  Pct  |  Last  Pct 
           -------      -----  ---  |  ----  --- 
            select     466272  66%  |     1   5% 
             begin      63915   9%  |     1   5% 
            update      63706   9%  |     1   5% 
            commit      56488   8%  |     1   5% 
            insert      40082   5%  |     1   5% 
          rollback       7427   1%  |     1   5% 

Since we are only updating the LFC, it sems odd that select outweighs update by this margin.

Improved conversion script

An improved script: LRCtoLFC.py which sorts files by directory, uses lfc_chdir, and uses multiple (20) threads was able to achieve 360 insertions/sec while the load on MySQL was 11,000 queries/sec. Thanks to Jean-Philippe Baud for helpful suggestions.

Note that in order to use this script you will have to set a few environment variables: LRC_HOST and LFC_HOST. The script also contains a hardcoded password for the LRC database, this may need to be changed for your site.

LRC_USER="dq2user"      ## may need to change these
LRC_PASSWORD="dqpwd"    ## for your site

LFC Launch Check-List

Open firewall ports

Make sure port 5010 (lfc) is open, if you have any firewall in place. You may also want to add

lfc		5010/tcp			# LCG File Catalog

to /etc/services

Set site offline in panda

curl -sS 'http://pandamon.usatlas.bnl.gov:25880/server/pandamon/query?tpmes=setoffline&queue=SITENAME'

Stop DQ2

/etc/init.d/dq2 stop

Update WN-Client

LFC works only with version >= 1.10.1k (or 1.10.1.mwt2), use vdt-version to check. source $OSG_GRID/setup.sh; python -c 'import lfc' should return no error. For install/update instructions see: https://www.usatlas.bnl.gov/twiki/bin/view/Admins/AtlasWorkerNode

Update DQ2 config

in dq2.cfg

catalog_SITENAME=lfc://lfc-host.domain:/grid/atlas

Remove your LRC information shown in [dq2-info] section.

Update TiersOfATLAS.py

Email usatlas-ddm-l@lists.bnl.gov and request site LRC entries be changed to LFC

Update scheddb

Request Paul Nillson or Torre Wenaus to change copyprefix, copytool, (if different copytoolin), lfchost, lfcpath, lfcprodpath and remove dq2url and check these spacetoken related se, sepath, seprodpath, setokens.

Mop-up run of LRCtoLFC.py

Run this script one more time, to pick up any entries which have gotten registered in LRC.

Make sure vomsdir is up to date

There are three methods for installing or updating the lcg-vomscerts. Choose the one that suits your site requirements.

Update via Yum: Create /etc/yum.repos.d/glite31-generic.repo with contents:

[glite31.generic]
name=gLite31.generic
baseurl=http://linuxsoft.cern.ch/EGEE/gLite/R3.1/glite-LFC_mysql/sl4/$basearch
enabled=1

Then, run yum install lcg-vomcerts.noarch or yum update lcg-vomscerts.noarch.

Update via tarball: This tarball is maintained by Horst Severini, hs@nhn.ou.edu.

wget http://www-hep.nhn.ou.edu/pacman-cache/packages/vomsdir.tar.gz
tar -zxvf vomsdir.tar.gz

Update via rpm: Browse to http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-UI/sl4/i386/RPMS.updates/ and search for the lcg-vomscerts rpm. Copy the url for the most up-to-date rpm, then use the rpm command to install it.

rpm --update http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-UI/sl4/i386/RPMS.updates/lcg-vomscerts-X.X.X-X.noarch.rpm

Make sure lcgdm-mapfile has needed entries

 cat /opt/lcg/etc/lcgdm-mapfile 
"/DC=org/DC=doegrids/OU=People/CN=Hironori Ito 564424" atlas
"/DC=org/DC=doegrids/OU=People/CN=Charles G Waldman 131209" atlas
"/DC=org/DC=doegrids/OU=People/CN=John Brunelle 583223" atlas
"/DC=org/DC=doegrids/OU=People/CN=Patrick McGuigan 416226" atlas
"/DC=org/DC=doegrids/OU=People/CN=wenjing wu 545240" atlas
"/DC=org/DC=doegrids/OU=People/CN=Neng Xu 462175" atlas
"/DC=org/DC=doegrids/OU=People/CN=Wei Yang 74203" atlas
"/DC=org/DC=doegrids/OU=People/CN=Shawn McKee 83467" atlas
"/DC=org/DC=doegrids/OU=People/CN=Nurcan Ozturk 18551" atlas

(These may not all be necessary. At the very least, Hiro's DN needs to be present.)

Change of production role

All proxies for production system should be using /atlas/Role=production instead of /atlas/usatlas/Role=production and/or simple grid-proxy.

If you run your DQ2 site service using your proxy, make sure that you have this role. Also, keep in mind that voms last only 12-96 hours depending on the voms server. Make sure to renew your voms proxy within that period. To request the role, send email to Alessandro.De.Salvo@cern.ch (with cc to jhover@bnl.gov)

For info on auto-updating VOMS proxy for DQ2, see http://twiki.mwt2.org/bin/view/Main/Dq2Proxies

Check directories and permissions

Create the required base directories and make sure that permissions are correct (generally 0x775). Create the required base paths in LFC and use lfc-setacl to set the correct permissions, e.g. for analysis:

$ lfc-mkdir -p /grid/atlas/users/pathena
$ lfc-setacl -m g:atlas/Role=production:rwx,m:rwx /grid/atlas/users/pathena
$ lfc-setacl -m g:atlas/usatlas/Role=production:rwx,m:rwx /grid/atlas/users/pathena

Bring site back online

curl -sS 'http://pandamon.usatlas.bnl.gov:25880/server/pandamon/query?tpmes=setonline&queue=SITENAME'

Restart DQ2

/etc/init.d/dq2 start

Disable LRC

This will vary depending on site.

Enable free space reporting via curl

See: XXX

References

• See FileCatalog and SubCommitteeLFC for LRC-LFC migration information in the US ATLAS Integration program.
• Offical guide: https://twiki.cern.ch/twiki/bin/view/LCG/LfcAdminGuide
• Charles' install log: http://twiki.mwt2.org/bin/view/Main/LfcInstallation
• MySQL InnoDB tuning: http://mysqldump.azundris.com/archives/78-Configuring-InnoDB-An-InnoDB-tutorial.html
• Troubleshooting/FAQ: https://twiki.cern.ch/twiki/bin/view/LCG/LfcTroubleshooting
• Data Management: https://twiki.cern.ch/twiki/bin/view/LCG/DataManagementDocumentation
• VOMS setup: https://twiki.cern.ch/twiki/bin/view/LCG/VomsSetup
• vomsdir RPM: http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-UI/sl4/i386/RPMS.updates/lcg-vomscerts-5.1.0-1.noarch.rpm
• DDM code repository: https://twiki.cern.ch/twiki/bin/view/Atlas/DDMOperationsScripts

-- CharlesWaldman - 22 Oct 2008
-- RobertGardner - 10 Sep 2008