bbc iplayer bandwidth limiting

[soapyfish]

Hi all,

 

I have been suffering terrible internet connectivity and after a log chat with my LEA and ISP it turns out that my bandwidth is being saturated with iplayer and streaming media requests. I am on a 10mb connection.

 

Unfortunatley/Fortunatley a significant number of teaching staff use IMDB iplayer and all number of other streaming media sites for legitimate use during lessons (especially music and media studies)

 

I have been using OpenBSD for some time and was thinking about using the inbuilt packet filter system "pf" to priortise and shape the traffic headed in and out of the internet connection.

 

the Plan is to use the ALTQ feature of "pf" to set all TCP traffic with a high priority and then to set HTTP IMAP POP etc with a higher priority using the CBQ method of queueing, and to also allow the HTTP que to "borrow upto 90% of the general TCP traffic queue when that que is under utilised.

 

If any of that makes sense to you and you have implimeneted such a system I would like to know of any pit falls that you may have encountered along the way !

 

I am still just planning the procedure but I can post updates is people are interested.

[powdarrmonkey]

I remember doing something similar with a NAT bodge, using Linux and iptables. I think I tagged packets from some nodes according to which outside line they should use, and then NATted according to the tags. Perhaps you could tag by destination, and prioritise accordingly?

 

I'd be interested to hear how you get on.

[soapyfish]

I have never used iptables, and I wondered about priotitising on destination but the iplayer site especially seems to be hosted from lots of different IP's and domains and so keep track of them and keeping the list upto date would constant task. I was not planning on using PF to do anything other than to priortise the traffic although I might see if its worth blocking excess broadcast traffic from arriving at the router too as that might add a small performance increase.

[powdarrmonkey]

Are you targetting the P2P traffic or the Flash site? The P2P stuff is obviously going to come from anywhere and everywhere, so that's trickier to control but easily blocked. I think they use a third-party content delivery network for the Flash stuff though, which as you say comes from lots of different places according to load. I guess if you proxied it you could do some kind of analysis on the pages for key iPlayer text, and throttle it, but this is at a higher level than just tagging and routing on source addresses.

 

I think the protocol used by the Flash players is RTMP, so maybe you could trace on that?

[soapyfish]

I am not looking to block anything at all I just plan to use the ALTQ process to prevent any one stealing all the bandwidth and so maintain a better quality of service

[Geoff]

That's one way... Here's another:

 

In Linux terms, set up a HTB with a queue for every PC. Set the base rate to whatever your backbone speed is (1/10th of the PC's line rate), the ceiling rate to the PC line rate, and give them a nice big bucket - say, 120 seconds times their line rate.

 

Then, people who are normal users - web surfing, downloading an occasional email attachment, etc - will go full bore, any time they want it. People who are streaming will go full speed for a couple minutes, and then decrease down to whatever bandwidth is available. If there's a lot of backbone free, it'll go fast. At lunchtime, they get best effort on whatever is available.

 

This is a very simplified example. You could additionally shape them so that their web and email will take priority over streaming when they're at the bottom of their token bucket, or other fine tuning...

[soapyfish]

Ok so I finally have a nearly working conf file I seem to have a syntax error on the last uncommented line but anyway I will update this when i have fixed that as well. any comments welcome etc...

 

 

 

 

# $OpenBSD: pf.conf,v 1.37 2008/05/09 06:04:08 reyk Exp $

#

# See pf.conf(5) for syntax and examples.

# Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1

# in /etc/sysctl.conf if packets are to be forwarded between interfaces.

 

### MACROS ###

#

#interfaces#

ext_if="re0"

int_if="re1"

#

#queues#

STN_TCP="tcp,ip"

STN_ENC="http,ftp,smtp,pop,imap,rtsp"

STN_EXS="https,ssh"

STN_OTH="udp"

#

### OPTIONS ###

#

#set drop-policy drop

set skip on { lo fxp0 }

#set loginterface ext_if

#

### SCRUB RULES ###

#

scrub all reassemble tcp

scrub out on ext_if random-id

#

### TRAFFIC SHAPING RULES ###

#

altq on re0 cbq bandwidth 10Mb queue { STN_TCP, STN_ENC, STN_EXS, STN_OTH }

#

# STN_TCP = Is all traffic that is NOT Any of the types listed below

# this will include iplayer,internet radio etc traffic.

# STN_ENC = Is all traffic that is HTTP,SMTP,POP,IMAP,RTSP, etc

# STN_EXS = Is all traffic that is HTTPS, SSL, other time dependant services

# STN_OTH = Is all traffic that is not TCP in any form

#

queue STN_TCP bandwidth 50% cbq (default red)

queue STN_ENC bandwidth 30% cbq (red borrow)

queue STN_EXS bandwidth 15% cbq (red borrow)

queue STN_OTH bandwidth 5% cbq (red borrow)

#

### FILTER RULES ###

#

allow all log

#