[debian] NIC bonding

[dwhyte85]

Has anyone any practical hands-on experience of NIC bonding when a NIC has gone died, does it work and does it work well? Can you bond more than 2 and has anyone done it?

[teejay]

Yes it works well, yes it fails over if a nic dies, yes you can bond more than 2 nics.

[dwhyte85]

Thanks

[glennda]

I have done it on a debian based product (ubuntu) most probably similar but there are 6 options for the cards

 

0 (balance-rr) Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance.

 

1 (active-backup) Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. This mode provides fault tolerance. The primary option affects the behavior of this mode.

 

2 (balance-xor) XOR policy: Transmit based on [(source MAC address XOR'd with destination MAC address) modulo slave count]. This selects the same slave for each destination MAC address. This mode provides load balancing and fault tolerance.

 

3 (broadcast) Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.

 

4 (802.3ad) IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification.

 

Pre-requisites:

Ethtool support in the base drivers for retrieving the speed and duplex of each slave.

A switch that supports IEEE 802.3ad Dynamic link aggregation. Most switches will require some type of configuration to enable 802.3ad mode.

5 (balance-tlb) Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.

 

Prerequisite: Ethtool support in the base drivers for retrieving the speed of each slave.

6 (balance-alb) Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server.

 

taken from https://help.ubuntu.com/community/UbuntuBonding

 

but there is the same for debian Ethernet Bonding Configuration in Debian

[dhicks]

Can you bond more than 2 and has anyone done it?

 

Yes, you can bond up to four. Yes, it works. Well, just bear in mind that commuicating between two IP addresses will never manage above 1 port's speed (so 1 gibabit/sec between two servers is probably the most you'll get) - some further discussion is here:

 

http://www.edugeek.net/forums/thin-client-virtual-machines/39439-drbd-nic-bonding-load-balancing-openfiler.html

 

--

David Hicks

[teejay]

Forgot to say, the make of the switch can make a big difference in how well this works. It pains me to say it but Cisco switches really do make a big difference with this as you can do a lot with them to tune the bonded link, which you can't really do on things like HP switches. Just wish I could afford the Cisco gear...

[glennda]

You can do this with HP switches but only really on the higher end core switches - hopefully when I replace my core with an HP procurve 5406zl I can then setup an 802.3ad trunk using the switch aswell

[Guest Guest]

Personally I'd only ever go with 802.3ad unless there was a specific reason to go for another. As said the switch can make a big difference. Our old HP kit doesn't support it fully and caused us problems, the new kit however has been fine.

With 802.3ad you only assign 1 IP to the bonded interface - so no difference between 802.3ad and a single NIC in that respect.

Load balencing is only 1 NIC per client - so you aint ever going to get more throughput than a single NIC to any one client. This means out of hours backups arnt going to go quicker but backups during the day are possible without too much disruption to clients.

Failover is seemless.

[teejay]

You can do this with HP switches but only really on the higher end core switches - hopefully when I replace my core with an HP procurve 5406zl I can then setup an 802.3ad trunk using the switch aswell

 

We have some 5406zl's and yes it allows you to set up more trunk types, but it doesn't let you tweak how that trunk operates like you can with the Cisco's.