Monday, June 1, 2009

Network Teaming/bonding on Linux

Network Interface Bonding

Interface Bonding is one of the simpler tasks which ensures high availability on the network. Bonding/Teaming is a way through which two or more physical NIC cards could work in tandem in order to ensure high availability from network perspective. It saves us from a network switch failure, network cable failure as well as NIC card failure. Ideal configuration for a high availability server should be:-

n/w switch A n/w switch B

NIC card 1 NIC card 2

SERVER

The only requirement is to have all of the participating slave NICs/Network switches in same subnet/network.


Doing it on Linux is even simpler!! Here are the steps to setup the bonding:-

1. Ensure that your version of OS is natively supporting interface bonding. By default, Redhat does provide bonding as a loadable module within the OS itself. If not, there are a lot of source rpms which could be build and loaded into the kernel.

locate bonding ## run updatedb if slocate database

the above mentioned command will list all the available bonding modules, alternatively you could also do modprobe and then rmmod.

2. Setup the network config files for all ensalved interfaces (i.e. which will form the bonded interface)

vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
HWADDR=xx:xx:xx:xx:xx:xx
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no


vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
HWADDR=yy:yy:yy:yy:yy:yy
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no

Boot protocol for NIC has been chosen to None as we are not assigning the IP address to slave NIC cards. HWADDR should NOT be edited. MASTER should be chosen to bond0 or any alias name (even apple0 will work ;)). As you could see above, the MASTER is same for both eth0, eth1. In case you want to ensalve three physical NICs, you need to keep the MASTER same for all three. SLAVE has been chosen as yes as eth0, eth1 will be slaves and will work depending upon the availability/config/requirement.

vi /etc/sysconfig/network-scripts/ifcfg-bond0 ##this file has to be created as its a logical interface
DEVICE=bond0
BOOTPROTO=none
USERCTL=no
IPADDR=.....
NETMASK=.....
GATEWAY=.....
NETWORK=.....
BROADCAST=.....
ONBOOT=yes

All .... entries need to be replaced with the ones which you would set for this logical/bonded interface.

3. vi /etc/modprobe.conf

Add the following lines in modprobe.conf file soon after the slave NICs configuration:-

alias bond0 bonding
options bonding mode=active-backup miimon=100

mode can be active-backup (1), balance-rr (0) etc. Here I am using it in active-backup mode where only one interface will be used for communication till the time a failure is detected on it. I could also use balance-rr which would load balance the packets based on Round Robin algo (sequential). miimon is the link detection interval which checks for the failure.



bonded Interface always have the MAC address of the first slave nic. It could, however, be changed via ifconfig command. Once the bonded logical NIC comes up, the ifconfig will report all slave/bonded NIC with same MAC address:-

# ip link show

2: eth0: mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:21:5a:d3:eb:b0 brd ff:ff:ff:ff:ff:ff
3: eth1: mtu 1500 qdisc pfifo_fast master bond0 qlen 1000
link/ether 00:21:5a:d3:eb:b0 brd ff:ff:ff:ff:ff:ff
4: bond0: mtu 1500 qdisc noqueue
link/ether 00:21:5a:d3:eb:b0 brd ff:ff:ff:ff:ff:ff

# ifconfig -a
bond0 Link encap:Ethernet HWaddr 00:21:5A:D3:EB:B0
....
....

eth0 Link encap:Ethernet HWaddr 00:21:5A:D3:EB:B0
....
....

eth1 Link encap:Ethernet HWaddr 00:21:5A:D3:EB:B0
.....
.....

Check either your ifconfig script or /proc filesystem for actual MAC address:-


# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.4 (January 28, 2008)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 500
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:21:5a:d3:eb:b0

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:21:5a:d3:eb:b1