r7 - 25 Apr 2012 - 14:08:47 - MichaelErnstYou are here: TWiki >  Admins Web > FabricUpgradeP20


  • Place here anticipated hardware upgrade activities anticipated during this quarter (Jan 1 - Mar 31, 2012)
  • Update SiteCertificationP20 with progress, as appropriate.
  • At the end of the quarter we will update the US ATLAS Facility spreadsheet with newly deployed storage and compute resources (c.f. CapacitySummary).
  • Previously, FabricUpgradeP19


last phase:
  • Have issued a purchase order for R410/Westmere based farm nodes to upgrade the capacity and to replace obsolete (4 yr old) farm nodes
this phase:
  • Retired obsolete compute servers (~600 slots) and deployed ~4,500 job slots (X5660), total number of job slots is currently 12,600.
  • Have started planning on 100 GE-based interswitch links and 40 GE-based (storage) server connectivity


last phase:
  • Upgrade MD3000i (expired warranty) to MD3600i for VMWare systems
    • Allows 10Gb NIC usage for Virtual Machines
  • Upgrade to VMWare Enterprise Plus and vSphere v5 virtualization software
this phase:
  • Additional 192 job slots of blade server added
  • Will increment storage once grants close out to meet the 2.2PB storage requirement


last phase:
  • Re-arrangement of racks at IU
  • 720 TB useable deployed at UC
  • 540 TB useable deployed at IU
  • Two new head nodes at IU
this phase:
  • 39 R410 X5660 at UC
  • 24 R410 X5660 at IU
  • 16 HP X5650 at UIUC
  • 2x Dell R310 10G PerfSONAR boxes at UC, IU; 2x 1G PerfSONAR? boxes at UIUC, to be upgraded to 10G later.


last phase:
  • Brought one rack of Dell MD1200/r710 storage with 3TB drives, 432 TB useable into GPFS
this phase:
  • Two MD1200/r710 storage racks with 3TB drives
  • 19 C6100 X5650 (1824 job slots)
  • 4 R710s configured for: 1 NFS server, 1 Gatekeeper upgrade, 2 LSM/squid/misc.services nodes
  • 1 Dell R310 10G PerfSonar?
  • 2 8024F 10Gbps switches, 2 6248 1Gbpbs switches, misc additional networking gear


last phase:
  • OU: no updates
this phase:
  • OU: added 120 2TB DDN drives, to double Lustre storage to 350 TB


last phase:
  • Installed 24 MD1000 units with 2TB drives, ~550TB usable.
  • Installed 68 Dell R410 with dual Intel X5650, 48GB RAM. Hyperthreading turned on.
this phase:
  • no upgrades

-- RobertGardner - 28 Feb 2012

About This Site

Please note that this site is a content mirror of the BNL US ATLAS TWiki. To edit the content of this page, click the Edit this page button at the top of the page and log in with your US ATLAS computing account name and password.


Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback