====== ATLASHN NETWORKING SETUP ======
This procedure explains how to configure the networking for the head node.
=====What You Need=====
* Hardware – server with two network cards
* Services – running on headnode
* Condor collector/negotiator
* Xrootd redirector
* Ganglia Metad server
* Ganglia Gmond client
* Apache Web server
=====Network Configuration=====
* NIC #1 yellow network address, with MAC registered in IHW
* NIC #2 yellow network address, virtual interface in IHW
=====Steps done as root=====
**1) Stop xrootd, cmsd and condor**
/sbin/service xrootd stop
/sbin/service cmsd stop
/sbin/service condor stop
**2) Edit the "service interface" (In this case: eth1): /etc/sysconfig/network-scripts/ifcfg-eth1**
# Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet
DEVICE=eth1
HWADDR=00:26:B9:4E:B8:AD
ONBOOT=no
HOTPLUG=no
BOOTPROTO=static
NETMASK=255.255.255.128
IPADDR=146.139.33.75
GATEWAY=146.139.33.1
**3) Edit the primary interface (In this case: eth5) /etc/sysconfig/network-scripts/ifcfg-eth5**
# Intel Corporation Unknown device 151c
DEVICE=eth5
BOOTPROTO=static
IPADDR=146.139.33.66
NETMASK=255.255.255.128
ONBOOT=yes
#ONBOOT=no
HWADDR=00:1b:21:d4:51:61
GATEWAY=146.139.33.1
**4) Activate the service interface**
/etc/sysconfig/network-scripts/ifup eth1
**4b) Add the following to the routing table**
146.139.33.0/25 dev eth5 proto kernel scope link src 146.139.33.66
146.139.33.0/25 dev eth1 proto kernel scope link src 146.139.33.75
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
169.254.0.0/16 dev eth1 scope link
default via 146.139.33.1 dev eth5
(Note the virtual interfaces come from the installation of the SL5 virtualization software)
**5) Remove routing reference for eth5**
ip route delete 146.139.33.0/25 dev eth5 proto kernel scope link src 146.139.33.66 ; ip route delete default via 146.139.33.1 dev eth5; ip route add default via 146.139.33.1 dev eth1
(Note there are 3 ip commands on one line, which is important if you connected to the machine via ssh)
**6) Commands to test your handy work**
ip route show
ip link show
**7) Test how the routing looks**
ip route get to 146.139.33.68
146.139.33.68 dev eth1 src 146.139.33.75
cache mtu 1500 advmss 1460 hoplimit 64
ip route get to 130.202.173.21
130.202.173.28 via 146.139.33.1 dev eth1 src 146.139.33.75
cache mtu 1500 advmss 1460 hoplimit 64
ip route get to 130.202.173.16
130.202.173.16 via 146.139.33.1 dev eth1 src 146.139.33.75
cache mtu 1500 advmss 1460 hoplimit 64
**8) Check the contents of the working routing table**
ip route show
146.139.33.0/25 dev eth1 proto kernel scope link src 146.139.33.75
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1
169.254.0.0/16 dev eth1 scope link
default via 146.139.33.1 dev eth1
**9) Check to see if a nameserver is reachable**
ping 146.139.254.3
dig atlas28.hep.anl.gov
**10) Modifications to xrootd configuration file: /etc/xrootd/xrootd-clustered.cfg must have the following lines**
#redirector name
set head = atlashn.hep.anl.gov
**11) Create the /atlas subdirectory and set the ownership to "xrootd:xrootd"**
mkdir /atlas
chown xrootd:xrootd /atlas
**12) Set the hostname in the /proc file system to be atlashn.hep.anl.gov and check your work**
echo "atlashn.hep.anl.gov" > /proc/sys/kernel/hostname
hostname
uname -a
**13) Open the firewall to ports 1094 and 1213 and condor ports by adding these lines to /etc/sysconfig/iptables**
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1094 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1213 -j ACCEPT
-A RH-Firewall-1-INPUT -s 146.139.33.66 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT -s 146.139.33.75 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT -s 130.202.173.21 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 9000:9999 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 9000:9999 -j ACCEPT
-A RH-Firewall-1-INPUT -s 146.139.33.75 -m state --state ESTABLISHED,NEW -p tcp -m tcp -j ACCEPT
#
Then restart iptables
/sbin/service iptables restart
**14) Restart xrootd/cmsd**
/sbin/service xrootd start
/sbin/service cmsd start
**15) Modifications to condor file**
* Move /etc/condor/condor_config.local (with reference to condor_config.cluster) in place
* Move /etc/condor/condor_config.cluster in place, which should contain this line:CONDOR_HOST=atlashn.hep.anl.gov
* Move /etc/condor/condor_config.head in place
**16) Create the file /etc/condor/condor_config.head.atlashn with the following two lines**
# set the network interface for the primary interface
NETWORK_INTERFACE = 146.139.33.75
**17) Condor Security: Since we are using a share secret file (PASSWORD) make sure root owns the file**
ls -l /var/lib/condor/condor_credential
-rw------- 1 root condor 256 Jan 3 07:44 /var/lib/condor/condor_credential
**18) start condor**
/sbin/service condor start