Core switch - Load balancing across default gateways

[Norphy]

This is probably a stupid question and I'm probably stupid for asking it but never mind.

 

So, I work for a MAT. We have three school clusters, we'll call them C, H and K. All three clusters have direct connections to one another with WAN links; C is directly connected to H and K, H is directly connected to C and K, K is directly connected to H and C.

 

Only C and H have connections to the Internet and right now, if a user in K wants to get out onto the internet, they go over the WAN link to H and use their internet connection. What I'd like to be able to do is to load balance K's internet traffic between H and C.

 

The core switches in C, H and K are connected to one another in VLAN 100. The Smoothwalls are also in VLAN 100. C's IP address in that VLAN is 10.0.0.8, H is on 10.0.0.5, K is 10.0.0.1. H's firewall is 10.0.0.150, C's is 10.0.0.152.

 

The default gateway for the core switches at H and K is 10.0.0.150, C's is 10.0.0.152. This is defined with a static route. To get the core at K to load balance traffic outside of its own networks between the two firewalls, is it just a matter of adding another static route with the address of 10.0.0.8 and the same metric?

 

Cheers

[AlanD]

Well...it depends on what kind of load balancing you are after....and no, you can’t simply add another gateway. You could allocate half the PCs with one gateway address and the other half to the other address...probably by creating static DHCP entries for each PC. That wouldn’t provide any fallback redundancy..You could put a loadbalancing proxy..like smoothwall..but it wouldn’t be cheap..but that would dynamically adjust sessions to balance by bandwidth...and provide auto fallover. Draytek routers will “load balance” and do fallback should one connection fail...but it wouldn’t dynamically share load automatically between two external routes.

[FN-GM]

Are C & H connected via VPN?

 

Do you have a darwing of this please? I am sturggling to picture it in my mind.

 

You can add another static route but this will cause major problems if one of the sites is down. You would use dynamic routing protocols for best results. If one site goes down the traffic is automatically stopped from going down that site and no longer goes into a black hole. Becuase not all your sites are directly conencted to a centeral point OSPF will be much more dificult to configure - you would need to create GRE tunnels. The best protocol will probably be EIGRP but this is a Cisco only protocol. But for both of these you will need C & H connected via VPN. What make are your core switches or routers please?

[geezersoft]

What firewalls have you got, what speed wans and what speed internet connetions?

[RobD]

We have a similar setup with HSRP, basically create a virtual IP that sits between the two IPS, if one goes down it will route to the second. If you're not Cisco then you could setup vrrp.

[Davit2005]

I could be wrong but I don't think you can get Load Balancing without some sort of hardware support. Resiliency should be achievable however.

[Norphy]

Thanks to all who have replied.

 

Please see attachment for a high level overview of the WAN here.

 

The lines between the three main clusters are all 1Gbps. They're not VPN and they don't go out over the internet.

 

All of the switches we have are HP. K has a 5400R, H and C both have a 8200ZL. We have OSPF handling routing between the sites.

 

Our firewalls are Smoothwalls. They're set up as transparent proxies, I don't want use explicit proxies as they're annoying. Both internet connections are 1Gbps. Which is nice.

 

The routers that have been provided by our ISPs for the internet connections are Juniper at H and Cisco at C but they're effectively black boxes to which I don't have any access to.

[AlanD]

Ok, I would do things differently.

 

Put both smoothwall boxes at Crawford set up as a redundant pair. Increase internet speed there, because one line at higher speed is always cheaper than two lines...and you should consider Virginzoom or FTTP which would give significantly higher speeds for less money...or keep what you have and add such a connection. Smoothwall will then hande all redundancy.

 

Add a leased line between Hatcham and Crawford...keeping the knights to Crawford...but take away knights to Crawford...unless you want redundancy ...

 

..or it both smoothwall boxes at Crawford..and do it the other way...

[mjk]

Rather than purchasing three 1 GB/s leased lines, have you thought about running your core servers in AWS, Google Compute Engine or Azure ? It's not all that expensive if you use a sensible scaling technology.

[Norphy]

Things are as they are. Suggesting that I move my firewalls around, move my services into the cloud or get rid of my leased lines are not helpful.

 

We want redundancy. That is why we want the ring and the two internet connections. Putting both of our internet connections in the same place reduces resiliency. Can we please get back to my original query rather than telling me to do things differently?

Edited April 9, 2018 by Norphy

[mjk]

Things are as they are. Suggesting that I move my firewalls around, move my services into the cloud or get rid of my leased lines are not helpful.

 

We want redundancy. That is why we want the ring and the two internet connections. Putting both of our internet connections in the same place reduces resiliency. Can we please get back to my original query rather than telling me to do things differently?

 

You are just working around a bad setup.

To get the best redundancy you should have redundant leased lines to the internet, not interconnecting sites. it's a waste of time and money.

[FN-GM]

We have a similar setup with HSRP, basically create a virtual IP that sits between the two IPS, if one goes down it will route to the second. If you're not Cisco then you could setup vrrp.

 

Thats not load balancing though :-) nor will that build redundant links. They are not the same as OSPF or EIGRP.

 

OP: As everything connects back to knights code. I would setup OSPF and make that switch in Area 0 as well as all other instances. Once is is all setup It will load balance between links and automatically failover if one your links drops out.

Edited April 9, 2018 by FN-GM

[RobD]

Thats not load balancing though :-) nor will that build redundant links. They are not the same as OSPF or EIGRP.

 

Very true and in fact we have issues with tromboning unless we change routes else where. Routing protocol is defiantly the best option really.

[Mustang]

OP: As everything connects back to knights code. I would setup OSPF and make that switch in Area 0 as well as all other instances. Once is is all setup It will load balance between links and automatically failover if one your links drops out.

 

Yes I also think that should work. When you enable OSPF on HP switches it defaults to load balancing 4 links (IP Load Sharing).

[Norphy]

OK, Thanks @FN-GM and @Mustang

 

As I say, OSPF is already in place for intra-site routing. The only static route that we have is 0.0.0.0/0 as the Smoothwalls don't support OSPF. So how do I get that route into OSPF?

[FN-GM]

In Cisco you can manually add it into OSPF with "network 0.0.0.0 0.0.0.0 area 0" I would test it though before going live.

 

Sorry I didn't see you already had OSPF

Edited April 10, 2018 by FN-GM

[Mustang]

OK, Thanks @FN-GM and @Mustang

 

As I say, OSPF is already in place for intra-site routing. The only static route that we have is 0.0.0.0/0 as the Smoothwalls don't support OSPF. So how do I get that route into OSPF?

 

where does the 0.0.0.0/0 route point to?

 

are you redistributing connected and static routes into ospf?

[Norphy]

where does the 0.0.0.0/0 route point to?

 

Depends on the site. At the moment, anything in the Knights and Hatcham cluster is being pointed at the Hatcham Smoothwall. Anything in the Crayford cluster is being pointed at the Crayford Smoothwall. What I'd like to do is for the Knights core to split its internet traffic between the two Smoothwalls but failing that, failover for all of the sites would be better than nothing.

 

are you redistributing connected and static routes into ospf?

 

I will be the first, second and third person to admit that networking at this level is not my primary subject. I don't entirely understand the question but putting the following command into the Knights core switch:

 

show ip route ospf

 

only shows subnets which belong to the other cores

[Mustang]

if you're allowed you could post the config of the switches and we can try and help?

[Norphy]

I don't see why not

 

; J9850A Configuration Editor; Created on release #KB.16.05.0004
; Ver #12:08.1d.fb.7f.bf.bb.ff.7c.59.fc.7b.ff.ff.fc.ff.ff.3f.ef:f6
hostname "HAK-CoreSW-SrvRm-001"
module A type j9986a
module B type j9990a
module E type j9993a
module F type j9993a
console idle-timeout 600
console idle-timeout serial-usb 600
trunk E2,F2 trk2 lacp
trunk E3,F3 trk3 lacp
trunk E4,F4 trk4 lacp
trunk E5,F5 trk5 lacp
trunk E6,F6 trk6 lacp
trunk E7,F7 trk7 lacp
trunk E8,F8 trk8 lacp
mvrp enable
logging 10.1.200.250
logging severity warning
radius-server host 10.10.0.17 key "timesync sntp
sntp unicast
sntp server priority 1 10.10.0.4
no telnet-server
time daylight-time-rule western-europe
no web-management
web-management ssl

ip access-list extended "WirelessNetworks"
    10 deny ip 172.24.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    20 deny ip 172.21.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    30 deny ip 172.22.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    40 deny ip 172.23.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    50 deny ip 172.25.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    60 deny ip 172.26.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    70 deny ip 172.27.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    80 deny ip 172.28.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    90 deny ip 172.29.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    100 deny ip 172.30.0.0 0.0.255.255 10.0.0.0 0.255.255.255
    110 permit ip 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255
  exit
ip route 0.0.0.0 0.0.0.0 192.168.1.150
ip router-id 10.0.0.1
ip routing
ip multicast-routing
interface A24
  name "KTG-UPLINK"
  exit
interface B20
  name "PROXY"
  exit
interface E1
  name "Hatcham-Uplink"
  speed-duplex 1000-full
  exit
interface F1
  name "Crayford-Uplink"
  speed-duplex 1000-full
  exit
interface Trk2
  mvrp enable
  exit
interface Trk3
  mvrp enable
  exit
interface Trk4
  mvrp enable
  exit
interface Trk5
  mvrp enable
  exit
interface Trk6
  mvrp enable
  exit
interface Trk7
  mvrp enable
  exit
interface Trk8
  mvrp enable
  exit
snmp-server community "nedi" operator
snmp-server location "GB;Bromley;Knights Academy_Launcelot Road, BR1 5EB;2;Server Room;Cab 4;5;6"
snmpv3 engineid "00:00:00:0b:00:00:f4:03:43:8f:87:00"
snmpv3 enable
snmpv3 group managerpriv user "HAAF-MGMT" sec-model ver3
snmpv3 user "HAAF-MGMT" auth sha "SECRET!!!!!" priv aes "ALSO A SECRET!!!"
aaa authentication login privilege-mode
aaa authentication web login peap-mschapv2 local
aaa authentication web enable peap-mschapv2 local
aaa authentication ssh login peap-mschapv2 local
aaa authentication ssh enable peap-mschapv2 local
oobm
  ip address dhcp-bootp
  exit
router ospf
  area backbone
  redistribute connected
  enable
  exit
router pim
  enable
  exit
vlan 1
  name "DEFAULT_VLAN"
  no untagged A1-A13,A19,A21,A23-A24,B1-B12,B16-B18,B20,E1,F1
  untagged A14-A18,A20,A22,B13-B15,B19,B21-B24,Trk2-Trk8
  no ip address
  ip igmp
  exit
vlan 16
  name "**CCTV**"
  untagged A3
  tagged A1,A13-A14,A16-A18,B13-B15,B19-B20
  ip address 10.16.0.254 255.255.240.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 18
  name "**IPTV**"
  untagged A5-A10,A12,B7-B8,B11-B12,B16-B18
  tagged A1,A13-A14,A16-A18,B13-B15,B19-B20
  ip address 10.18.0.254 255.255.240.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 19
  name "**Knights_Wireless**"
  tagged A1,A13-A14,A16-A18,B13-B15,B19-B20
  ip address 10.19.0.254 255.255.240.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 100
  name "**WAN**"
  untagged A24,E1
  tagged A1,A13-A14,A16-A18,B13-B15,B19-B20,F1
  ip address 10.0.0.1 255.255.255.0
  ip address 192.168.1.254 255.255.255.0
  ip ospf 10.0.0.1 area backbone
  exit
vlan 102
  name "**WAN-KTG**"
  tagged A1,A13-A14,A16-A18,B13-B15,B19-B20
  ip address 192.168.2.11 255.255.255.0
  exit
vlan 140
  name "Data-ServerRm"
  untagged B2
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.140.1 255.255.254.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 142
  name "VLAN142"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.142.1 255.255.254.0
  ip helper-address 10.1.201.10
  ip helper-address 10.1.201.11
  ip helper-address 10.1.200.21
  ip helper-address 10.1.200.22
  ip helper-address 10.1.200.23
  exit
vlan 144
  name "Data-NorthCab"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.144.1 255.255.254.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 148
  name "Data-MidCab"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.148.1 255.255.254.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 152
  name "Data-SouthCab123"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.152.1 255.255.254.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 156
  name "Data-SouthCab456"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.156.1 255.255.254.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 172
  name "**FederationServers**"
  untagged F1
  tagged A1,A13-A14,A16-A18,B12-B15,B19
  no ip address
  exit
vlan 200
  name "Servers"
  untagged A13,A19,A21
  tagged A1,A14,A16-A18,B13-B15,B19
  ip address 10.1.200.1 255.255.255.0
  ip igmp
  ip pim-dense
     ip-addr any
     exit
  exit
vlan 201
  name "New Domain Server Network"
  tagged A1,A14,A16-A18,B13-B15,B19
  ip address 10.1.201.1 255.255.255.0
  ip igmp
  ip pim-dense
     ip-addr any
     exit
  exit
vlan 202
  name "Federation Server"
  tagged A1,A13-A14,A16-A18,B12-B15,B19
  ip address 10.1.202.1 255.255.255.0
  exit
vlan 210
  name "ServerMgmt"
  untagged A23,B3,B5
  tagged A1,A8-A9,A13-A14,A16-A18,B4,B7-B8,B11,B13-B15,B19
  ip address 10.1.210.1 255.255.255.0
  ip igmp
  exit
vlan 212
  name "NetMgmt"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.212.1 255.255.255.0
  ip igmp
  exit
vlan 214
  name "WIFI-Mgmt"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.214.1 255.255.255.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 216
  name "BuildingMgmt"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.216.1 255.255.255.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 218
  name "CCTV"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.218.1 255.255.255.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 220
  name "Voice"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.220.1 255.255.255.0
  ip helper-address 10.10.0.4
  ip igmp
  voice
  exit
vlan 222
  name "WIFI-Guest"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip access-group "WirelessNetworks" in
  ip address 172.21.8.1 255.255.248.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 224
  name "WIFI-Trusted"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip address 10.1.224.1 255.255.240.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 240
  name "WIFI-BYOD"
  tagged A1,A13-A14,A16-A18,B13-B15,B19
  ip access-group "WirelessNetworks" in
  ip address 172.21.0.1 255.255.248.0
  ip helper-address 10.10.0.4
  ip igmp
  exit
vlan 998
  name "CSV Network"
  tagged A1-A24,B1-B24
  no ip address
  exit
vlan 999
  name "vMotion VLAN"
  tagged A1-A24,B1-B24
  no ip address
  exit
spanning-tree
spanning-tree E1 bpdu-filter
spanning-tree F1 bpdu-filter
spanning-tree Trk2 priority 4
spanning-tree Trk3 priority 4
spanning-tree Trk4 priority 4
spanning-tree Trk5 priority 4
spanning-tree Trk6 priority 4
spanning-tree Trk7 priority 4
spanning-tree Trk8 priority 4
spanning-tree priority 0
no tftp server
no autorun
no dhcp config-file-update
no dhcp image-file-update

 

Passwords have been trimmed out but that's the config otherwise.

[AlanD]

I don’t understand from any of discussions how you intend balancing the internet traffic. I can see how it might provide redundancy. I’m not sure about what the real world benefit of the the third additional leased line to complete a ring might bring...because in practice (unless you are paying for different physical external cabinets connections) you would be at the mercy of open reaches switches, and failure of one connection (unless it’s a local failure on one of your sites) would almost certainly mean a failure in the other....

 

The only way to balance traffic would be to allocate different gateways to different clients...and making this dynamic depending on demand would be impossible.

[mjk]

I don’t understand from any of discussions how you intend balancing the internet traffic. I can see how it might provide redundancy. I’m not sure about what the real world benefit of the the third additional leased line to complete a ring might bring...because in practice (unless you are paying for different physical external cabinets connections) you would be at the mercy of open reaches switches, and failure of one connection (unless it’s a local failure on one of your sites) would almost certainly mean a failure in the other....

 

I don't think he wants to hear why it's a bad idea. I tried that already. I suspect management told him to do it

[Norphy]

I don’t understand from any of discussions how you intend balancing the internet traffic.

 

This is what I'm trying to find out if I can do or not. If I can't with what I've got, fair enough.

 

I can see how it might provide redundancy. I’m not sure about what the real world benefit of the the third additional leased line to complete a ring might bring...because in practice (unless you are paying for different physical external cabinets connections) you would be at the mercy of open reaches switches, and failure of one connection (unless it’s a local failure on one of your sites) would almost certainly mean a failure in the other....

 

Yes, we thought of that. The P2P links are bought through one company (Charterhouse) but they get them from Virgin and Openreach so we do have some redundancy there. Having the loop also mitigates against a failure at one particular site. Knights is the "centre" of everything at the moment and if that goes pear shaped, the entire MAT loses its inter-site connectivity. The idea of bringing the loop into play is to mitigate against that and not making a single school the centre of things. Even the internet connections are through different providers; one is Virgin and the other is Openreach. The other benefit that it should bring is that it will reduce the amount of hops that traffic between Hatcham and Crayford and it will reduce the latency on that connection as well.

 

The only way to balance traffic would be to allocate different gateways to different clients...and making this dynamic depending on demand would be impossible.

 

I don't think that will work because none of our clients are in the same subnet as the Smoothwalls.

Edited April 12, 2018 by Norphy

[Norphy]

I don't think he wants to hear why it's a bad idea. I tried that already. I suspect management told him to do it

 

Can I be honest? You're right, I don't want to hear why you think it's a bad idea. I have the P2P links in already, albeit one of which is waiting to be configured. I have the internet connections. Telling me to do something in a different way when I already have the stuff in place is not even slightly helpful. You might consider it a waste of money and a bad idea. That's your opinion and that's fair enough. However I don't want to hear it.

 

Also bear in mind that my MAT has interconnected sites for other reasons than just sharing the internet connections. This will probably be reviewed in the future but right now, this is the model we've chosen and this is what we're using.

[mjk]

^ I appreciate it's hard being thrust into a situation where people have made poor decisions in the past which you now have to deal with. Sorry if I upset you, I just strive to configure the best possible configuration for all involved rather than polishing a t*rd.