ATLAS Support Center

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== ATLASHN NETWORKING SETUP ======

This procedure explains how to configure the networking for the head node.

=====What You Need=====
  * Hardware – server with two network cards
  * Services – running on headnode
  * Condor collector/negotiator
  * Xrootd redirector
  * Ganglia Metad server
  * Ganglia Gmond client
  * Apache Web server

=====Network Configuration=====
  * NIC #1 yellow network address, with MAC registered in IHW 
  * NIC #2 yellow network address, virtual interface in IHW

=====Steps done as root=====

**1) Stop xrootd, cmsd and condor**

<code>/sbin/service xrootd stop
/sbin/service cmsd stop
/sbin/service condor stop</code>

**2) Edit the "service interface" (In this case: eth1): /etc/sysconfig/network-scripts/ifcfg-eth1**

<code># Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet
DEVICE=eth1
HWADDR=00:26:B9:4E:B8:AD
ONBOOT=no
HOTPLUG=no
BOOTPROTO=static
NETMASK=255.255.255.128
IPADDR=146.139.33.75
GATEWAY=146.139.33.1</code>

**3) Edit the primary interface (In this case: eth5) /etc/sysconfig/network-scripts/ifcfg-eth5**

<code># Intel Corporation Unknown device 151c
DEVICE=eth5
BOOTPROTO=static
IPADDR=146.139.33.66
NETMASK=255.255.255.128
ONBOOT=yes
#ONBOOT=no
HWADDR=00:1b:21:d4:51:61
GATEWAY=146.139.33.1</code>

**4) Activate the service interface**

<code>/etc/sysconfig/network-scripts/ifup eth1</code>

**4b) Add the following to the routing table**

<code>146.139.33.0/25 dev eth5  proto kernel  scope link  src 146.139.33.66 
146.139.33.0/25 dev eth1  proto kernel  scope link  src 146.139.33.75 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
169.254.0.0/16 dev eth1  scope link 
default via 146.139.33.1 dev eth5</code> 

(Note the virtual interfaces come from the installation of the SL5 virtualization software)

**5) Remove routing reference for eth5**

<code>ip route delete 146.139.33.0/25 dev eth5  proto kernel  scope link  src 146.139.33.66 ; ip route delete default via 146.139.33.1 dev eth5;   ip route add default via 146.139.33.1 dev eth1</code>

(Note there are 3 ip commands on one line, which is important if you connected to the machine via ssh)

**6) Commands to test your handy work**

<code>ip route show
ip link show</code>

**7) Test how the routing looks**

<code>ip route get to 146.139.33.68</code>
146.139.33.68 dev eth1  src 146.139.33.75 
cache  mtu 1500 advmss 1460 hoplimit 64

<code>ip route get to 130.202.173.21</code>
130.202.173.28 via 146.139.33.1 dev eth1  src 146.139.33.75 
cache  mtu 1500 advmss 1460 hoplimit 64

<code>ip route get to 130.202.173.16</code>
130.202.173.16 via 146.139.33.1 dev eth1  src 146.139.33.75 
cache  mtu 1500 advmss 1460 hoplimit 64

**8) Check the contents of the working routing table**

<code>ip route show
146.139.33.0/25 dev eth1  proto kernel  scope link  src 146.139.33.75 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
169.254.0.0/16 dev eth1  scope link 
default via 146.139.33.1 dev eth1</code>

**9) Check to see if a nameserver is reachable**

<code>ping 146.139.254.3
dig atlas28.hep.anl.gov</code>

**10) Modifications to xrootd configuration file: /etc/xrootd/xrootd-clustered.cfg must have the following lines**
<code>#redirector name
set head = atlashn.hep.anl.gov</code>

**11) Create the /atlas subdirectory and set the ownership to "xrootd:xrootd"**
<code>mkdir /atlas
chown xrootd:xrootd /atlas</code>

**12) Set the hostname in the /proc file system to be atlashn.hep.anl.gov and check your work**

<code>echo "atlashn.hep.anl.gov" > /proc/sys/kernel/hostname
hostname
uname -a</code>

**13) Open the firewall to ports 1094 and 1213 and condor ports by adding these lines to /etc/sysconfig/iptables**

<code>-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1094 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1213 -j ACCEPT
-A RH-Firewall-1-INPUT  -s 146.139.33.66 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT  -s 146.139.33.75 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT  -s 130.202.173.21 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 9000:9999 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9000:9999 -j ACCEPT
-A RH-Firewall-1-INPUT -s  146.139.33.75 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
#</code>

Then restart iptables

<code>/sbin/service iptables restart</code>

**14) Restart xrootd/cmsd**

<code>
/sbin/service xrootd start
/sbin/service cmsd start</code>

**15) Modifications to condor file**

  * Move /etc/condor/condor_config.local (with reference to condor_config.cluster) in place
  * Move /etc/condor/condor_config.cluster in place, which should contain this line:<code>CONDOR_HOST=atlashn.hep.anl.gov</code>
  * Move /etc/condor/condor_config.head in place

**16) Create the file /etc/condor/condor_config.head.atlashn with the following two lines**

<code># set the network interface for the primary interface
NETWORK_INTERFACE = 146.139.33.75</code>

**17) Condor Security: Since we are using a share secret file (PASSWORD) make sure root owns the file**

<code>ls -l /var/lib/condor/condor_credential</code>

-rw------- 1 root condor 256 Jan  3 07:44 /var/lib/condor/condor_credential

**18) start condor**

<code>/sbin/service condor start</code>
ATLAS Support Center

User Tools

Site Tools

Page Tools
