SupportingGLOW

GLOW VO

• The Grid Laboratories of Wisconsin, a VO for the University of Wisconsin's Center for High Throughput Computing: http://chtc.cs.wisc.edu/
• Contact: Steve Barnet <barnet@cs.wisc.edu>, Brooklin Gore <BGore@Morgridgeinstitute.org>, Dan Bradley <dan@hep.wisc.edu>, chtc@cs.wisc.edu
• Submission system: GlideinWMS

Requirements

• Worker node OS should be RHEL 5 (or equivalent, such as SL5)
• Worker nodes require outbound internet access (nodes can be behind NAT)
• Worker node memory: 1GB / slot minimum, assume 1.5-2 GB
• Worker node scratch: assume 10G per job slot - though typically using less than 1GB
• The worker nodes require the OSG CA certs that are installed as part of the OSG Worker Node Client. Host certs on the worker nodes are not required.
• Site squid (optional): used for data in some jobs ~200 MB
• Job preemption is okay
• glexec is not required

Workflow and Security profile

Description:

• For every job slot, a pilot job process starts up.
• The pilot job sends outbound TCP traffic over HTTP to a host at UCSD ("factory") and via HTTP to a host at wisc.edu ("frontend").
• The pilot job spawns a condor startd, which spawns a condor starter.
• The startd sends outbound UDP traffic to a single port on the frontend. This is randomly chosen (per pilot) from a range of 200 ports currently. This can be changed to TCP if necessary.
• The startd and starter send outbound TCP traffic to two ports on one of the Condor submit machines at wisc.edu, to communicate with the condor_schedd and condor_shadow. The ports are chosen randomly (per glidein) from the host's ephemeral ports. The choice of ports can (and in some cases has) been reduced to a single static port, which is typically 9618.

Hosts and ports:

• The frontend is glidein.chtc.wisc.edu (128.104.59.136)
  • ports: 80, 9618-9820
• The factory is glidein-1.t2.ucsd.edu
  • port 8319
• The Condor submit machines (more may be added in future)
  • submit.chtc.wisc.edu
  • login01.hep.wisc.edu
  • poplar.hep.wisc.edu
  • spaldingsubmit.botany.wisc.edu
  • simon.stat.wisc.edu
  • chtcsubmit.ansci.wisc.edu
  • condor.morgridge.net

Setup

• GLOW in MyOSG
• Adding a VO to your site from the OSG documentation, https://twiki.grid.iu.edu/bin/view/ReleaseDocumentation/PreparingComputeElement#Required_User_Accounts
• The steps are generally:
  • Create local UNIX account for the VO (eg. "glow")
  • In GUMS, create a VO account mapping between the VO and the local UNIX account
  • There is no need to reconfigure the CE - the next time gums-host-cron runs the VO gets added to the osg-user-vo-map.txt file, as well as the osg-supported-vo-list.txt file.
  • You'll need this: vomss://glow-voms.cs.wisc.edu:8443/edg-voms-admin/GLOW

Status

GLOW VO status

• GLOW VOMS status

Notation:		completed		work is in progress		table to be updated

Site	OSG Resource	Status	Job Max
TIER1	gridgk01.racf.bnl.gov		500
AGLT2	gate02.grid.umich.edu		50
MWT2	osg-gk.mwt2.org		125
NET2_BU			15
UTA_SWT2			10
SWT2_OU_OSCER	grid1.oscer.ou.edu/jobmanager-lsf		64
WT2

Known Issues

-- RobertGardner - 02 Aug 2011