Bonded BT Infinity Project

[RobBaxter]

Hi All,

 

So, here at school we have been having a little brain storm over how best to "get with the times" when it comes to internet speed. For the most part with the introduction of many "cloud" services, such as Google Apps in education or maybe even Office Live (not that i use this), the internet connection you have at school is becoming one of your most vital assets.

 

After my boss/bursar bought a new School Information Database and wanted to have staff access this web portal from home I began the quite reasonable "thats not gonna work boss" issues. At the moment we have 1 single ADSL line which services the whole school, it's not really been an issue for us, but especially now that i'm implementing so many new things I had to look at alternate solutions.

 

WELL, bloody hell its expensive. Most companies want to charge in the region of £4500 + VAT for "bonded ADSL" solutions and don't even get me started on lease lines. But i think i found the solution. We have just ordered ourselfs a brand new BT Line with Infinity on it, the price is £60 (inc VAT) for 1 month (which includes your line rental and unlimited Bandwidth).

 

I should get 33.6mbit down / 5.8mbit up (with my Infinity Line) which is awsome to start with!! But i'm not stopping there!! Once the 1st line is installed and is tested of 3 months for stability / reliability purposes, i will then place an order for another 3 BT Infinity lines. Of course these are £60 each. So thats a £240/month Price tag or £2880 a year.

 

That figure for us is about 1/4 of the cost of a lease line per annum. AND what is the beauty of this imo is it's completely internally managed. If this goes the way I think it will, i should get 134mbit/s down and 23.2mbit/s up using 4 "Bonded" BT Infinity internet connections. I think this "for schools at least" this is the answer to getting the most "Bang for our very limited buck" so to speak in regards to "internet connectivity". I have 2 IT Suites and 40 students using the internet at the same time over 1 single ADSL connetion is laughable really. Google Earth would make you crindge and laugh at the same time.

 

Now bonding is not new, but what I want to test "for all of us" is if BT Infinity Bonding is really a answer once it's available in your area. Because if it really does work the way i think it should, it really is a cracking choice.

 

I will be documenting the whole process from begining to end along with testing metrics and stats. My plan here is to do the "leap of faith" on the behalf of our band of edugeekers. By the end of this "year" at school, I hope to be able to say one way or another with reasonably proofs and infos if this a suitable solution!!

[sidewinder]

Definitely an interesting idea, will be good to hear how it works in practice. Best of luck!

[sonofsanta]

I hate to rain on your parade, but I know ADSL bonding required specialist equipment at the BT end of things in the exchange as well as gear in our switching cab; I don't know either way, but I suspect aggregating 4 Infinity lines will be more complicated than just plugging them all in.

 

Is it consumer class Infinity you're looking at as well? i.e. rubbish contention ratios, no guaranteed speeds, no SLA? Obviously the saving is immense so if you're aware of the caveats of consumer connections then its your call, but if you're going to start relying on a connection it might end in tears...

 

(additional thought: with consumer contention ratios and 4 of your own lines, would you be contending with yourself and no better/only mildly better off?)

[tom_newton]

Depends how "bondy" you want to get. Without the ISP being complicit, you can at least loadbalance outgoing traffic and use different lines for different inbound traffic.

[localzuk]

Can I suggest you look at this thread? http://www.edugeek.net/forums/networks/89684-am-i-mad-school-broadband-cost-versus-home.html

[AngryTechnician]

Bonding is certainly complicated, but as Tom says, load balancing is just a case of having the right router on premises. If your goal is to provide decent speed to many users (rather than top-notch speed to a single user), it should work pretty well, though contention may be an issue during peaks times, as sonofsanta suggested.

 

Worth remembering that the linespeed available on BT Infinity is due to go up anyway in the next year or so.

[Domino]

What about bandwidth caps?

QOS?

fixed IPs?

SLAs for potential issues?

 

If this is the home inifinity package you'll likely fall foul of all these issues. Not to mention contention once more people move to it.

[AngryTechnician]

Good piont - make sure you have a business provision. We use Zen Internet who resell the FTTC product that BT call 'BT Infinity'. That has business level SLAs and usage allowances, along with a fixed IP. Many other companies can provide the same.

[FN-GM]

Form what i understand simply by putting the lines together wont give you 134mbs. You will still get 33mbs. But it will give you more bandwidth.

 

Its like a road. You have one lane and the speed limit is 50mph. The council turn up and at another lane (in the case another connection), the speed limit doesn't go upto 100mph but more traffic can go down the road at the same time.

[RobBaxter]

Hi all, nice to see activity here.

 

Your all right let me clear up a few "bits". First off, not i'm not really "bonding" them i'm load balancing across them. Again your right, its only a "total speed of" rather than a "i can download at" speed. I have the business unlimited packages (of which we unless doing really well, won't even get to their "fair use policy".

 

Again I agree, lease lines are far far superior, but if your like me, where getting a lease line to school is somewhat "out of the question" then this seems like a good alternate solution.

 

I am undecided as to which "router" to use for the "shall we say connection balancing". UBM 1500 ADSL Channel Bonding Router is one i'm looking at. If you have a link to "bonding / load balancing" routers then please post them . Failing a suitable appliance i can always setup a Linux box to do this with 6 or so network cards

[glennda]

Can you not load balance on a Smoothie using the advanced firewall?

 

Which would also sort your filtering requirements.

 

Tom will know more but i'm pretty sure a local school is using that here on adsl lines from multiple suppliers because they can only get 2mb on each

Edited February 10, 2012 by glennda

[tom_newton]

The xrio boxes are supposed to be OK - if a bit tough to configure in places. They're only up the road from us, have met a few of their folks.

 

Given the right presentation (static ethernet all round preferably) you can do simple load balancing with a Smoothwall UTM.

[RobBaxter]

Fantastic idea, I can also kill 2 birds with one stone here always wanted a smooth wall to replace my tiny watch guard pos!!! My bursar wont let me have one until the watch guard breaks or wont do the job. :D

 

Insolently, concidering the number of schools with smooth wall, makes this "case study" of sorts even better

[nicholab]

Andrew and Arnold are the best ISP for doing line bonding. They bond both ends of the connection so it can be a single link.

[jammmie999]

Form what i understand simply by putting the lines together wont give you 134mbs. You will still get 33mbs. But it will give you more bandwidth.

 

Its like a road. You have one lane and the speed limit is 50mph. The council turn up and at another lane (in the case another connection), the speed limit doesn't go upto 100mph but more traffic can go down the road at the same time.

 

Not strictly true I don't think. All traffic on the internet is traveling at the same speed,

e.g. when you download a file your request isn't sent to server any quicker than someone on a dial up connection instead you can fit more data through your pipe leading to a quicker download of that 100MB file. A dial up user is still getting the file at the same speed as you they are just not getting as much of it concurrently, makeing it take longer to download.

[jamesreedersmith]

Look at the drayteks for affordable load balancing multi wan routers.

[cpjitservices]

Simple solution is just to load balance it all, we have our 4 lines load balanced works a treat.

 

We have a pfsense server with 6 NIC's in - 4 From the single port modem/routers into the box then 2 NIC's to network.

 

It works very well and considering we get 10 Meg per line our speedtest usually shows us 33-40 Meg.

 

Pfsense is based on Smoothwall/Monowall so you'll be able to do this with other variations of the OS.

Edited February 13, 2012 by cpjitservices

[timbo343]

we have just replaced our cisco 2600xm router with a cisco 1941 to deal with our lines

[matt40k]

Pfsense is based on Smoothwall/Monowall so you'll be able to do this with other variations of the OS.

 

Smoothwall is fork from Redhat. MonoWall is based on FreeBSD. PFSense is based on FreeBSD but using OpenBSD PF rather then IPTables. Personally PF is WAAAY better as you've stated

[SchoolsBroadband]

We've been "doing" bonding for some time. It's important to understand the difference between bonding and balancing,

 

Balances sends packets / data down each different link giving you a different real world IP address each time you go out. So unless you use an application such as torrenting which makes thousands of separate connections to different IP endpoints then you'll never get the full bandwidth on a single download. e.g. you balance 2 x ADSL connections which both get a maximum of say 8Mbit then you won't get a throughput on single download of 16Mbit. You'll just get 8Mbit. If someone on a PC next to you then trys to download a file that is balanced onto the second link then they two should get 8Mbit. Add both together then it's 16Mbit but only on separate connections.

 

Balancing is ok, has the above issue and also others. They are the fact that the upload will also never be double the speed for the reason as above with the download. Also if you host anything on site or use a VPN then a remote user will never be able to get full use of the bandwidth on both lines. It'll only get the maximum speed from 1 line. Some websites / hosted applications also don't like balancing the connection as the real world IP address of your Internet connection keeps on changing. To get these sites to work you have to program your balancing device to permanently send all packets / data down the same link to either IP end point of via protocol used. e.g https sites can be broken by balancing sometimes.

 

Bonding is superior to balancing in every way. The speeds is truely bonding together. i.e you have 2 x 8mbit download lines then you will get 16Mbit download from 1 download stream. Your real world IP addresses are always the same so VPN'ing in and hosting web servers on site make use of the increased upload bandwidth. There are a couple of ways to do bonding.

 

MLPPP - This is done by having a device on site such as a Cisco and they create PPP links back to the ISP's LNS (this terminates the L2TP tunnels over BT's network). Packets are spread over each link and re-assembled on the LNS to give one throughput. The downside we've seen of MLPPP is that if you have for example 2 x ADSL lines, one syncs at 7Mbit and the other at 8Mbit then the max you'll get is 7Mbit out of each line. I'm not 100% sure on the technicalities as to why (perhaps one of my engineers would like to add to this thread, Luke?!!) but this is what we've seen in practise. It also means that if your ISP has an issue then the amount of resilience you have is lessened as if their core network has an issue then you can kiss your connection goodbye. Also ADSL lines can go up and down in speed which can also have quite a dramatic effect on performance. Some ISP's say you must only use MLPPP with fixed rate DSL. i.e. the now very old 512k, 1Mbit and 2Mbit options. This is generally a cheaper way than the next way to bond.

 

Xrio UBM - We actively sell this product and have our own Xrio UBM infrastructure. You must have a UBM bonding device on site and one "in the cloud". GRE tunnels are made from the extneral interface of all WAN connections on the onsite UBM back to a single WAN connection on the hosted UBM. Packets are distributed over all GRE links by use of Xrio's own algorithm. Using UBM's has the following advantages 1) The WAN links don't have to be the same speed. One could run at 5Mbit, the other at 7Mbit and you'd get 12Mbit of download, unlike with the previously mentioned MLPPP offering. 2) You can bond connections from multiple ISP's. This gives greater resilience. We for example sometimes offer 2 x BT wholesale based ADSL's and 2 x O2 / Be Wholesale ADSL's. If the BT ADSL's fail then the GRE links go down between both UBM devices. Therefore the O2 DSL's simply continue to function and the speed is reduced. As soon as the BT ADSL's come back on line the you are back to full speed.

 

There is some "bonding overhead" and we normally say about 10% of overall download bandwidth.

 

We've actually bonded 4 x FTTC 40Mbit services for a large school in Cheshire. The throughput they often get is about 105Mbit download and 32Mbit upload where the BRAS profiles are 38Mbit download and 10Mbit upload on all connections. This proved far more cost effective than the equivalent leased line. We also often bond dual ADSL's for primary schools when they are a long way from the telephone exchange.

 

Happy to discuss the merits or any more technicalities. I hope the above makes sense!

 

Thanks

 

Dave Tindall