New Switches Required

[davidstallwood]

Hi, we are a secondary school with approx 1500 students and 300 PCs. I am looking at upgrading our existing infrastructure as all our existing fibres are OM1 and will never run faster than 1Gig and we are experiencing network issues and some switches are daisy chained off others and are sharing only a 1 gig link. Ideally I want to replace all the fibres with OM3 so we can install edge switches with 10Gig fibre uplink SFPs. What would you recommend we install in the main server room as a core solution where the 8 fibre cables terminate along with approx 120 local clients over Cat5e? We have three Hyper-V Dell Servers & a Dell SAN also located in that main server room along with our internet feed and a Netgear NAS. We also have a Ruckus wi-fi network covering the whole school and approx 50 Hik-Vision CCTV cameras running on the same network. Any ideas would be most welcomed?

[chazzy2501]

Our site is also OM1 but it can do 10G just over very short runs, I bought 10G SPF+ and just tried them out, 2 of 8 links work at 10G! Other places I setup some trunks 2 and 4 gigabits. I'm using unify ubiquity.

[jb2201]

If you have the budget available I would suggest a pair of Aruba 5406zl chassis switches serving as the core with 1 x 24 port rj45 module and 3 x 8 port SFP+ modules.

 

For cabs I would recommend running enough OM3 fibre so that each switch can have two 10gig fibre links back to the cores. Also run two cable routes for the OM3 and give each switch one fibre from each route for resiliency.

 

Won’t be cheap but it will be built to last!! If you wanted a cheaper option you could drop down to one core switch or just have 1 x 10 gig link to each switch then put a resilient uplink in between the switches.

 

For edge switches in the main server room I would connect these with DAC cables rather than fibre to reduce costs.

[johnpd]

If you have the budget available I would suggest a pair of Aruba 5406zl chassis switches serving as the core with 1 x 24 port rj45 module and 3 x 8 port SFP+ modules.

 

For cabs I would recommend running enough OM3 fibre so that each switch can have two 10gig fibre links back to the cores. Also run two cable routes for the OM3 and give each switch one fibre from each route for resiliency.

 

Won’t be cheap but it will be built to last!! If you wanted a cheaper option you could drop down to one core switch or just have 1 x 10 gig link to each switch then put a resilient uplink in between the switches.

 

For edge switches in the main server room I would connect these with DAC cables rather than fibre to reduce costs.

 

Pretty much how each one of my sites are now, except I hadn’t thought of doing DAC for switch in server room... I’ll have a look at that ! I couldn’t afford two 5406zl but sooner or later you could add another I guess when money is available.

 

John

[Fazza]

We went for HP Switches as they come with a lifetime warranty. We have a couple of 10Gb links between 2 buildings and it works great.

[ColS]

I would certainly agree with the HP switch suggestion. We went HP 3 years ago and they have largely been rock solid. Two OM3 or 4 connections from each switch/stack back to your comms room with DAC cables between switches in the same location.

[sonofsanta]

I've got the HP Comware switches at my core and they've been flawless for... 6½ years now, I think? We put two A5800s in as a stacked pair, with redundant links to servers etc., then added a 5820 into the stack a couple of years back when we upgraded to 10GbE over the backbone. They were better value for money than the other HP stuff, and the edge switches (mostly 5130s) are brilliant too.

 

If it's a big upgrade project I'd definitely talk to a network specialist reseller--plenty on here, but SwitchShop have always been superb for me. And they send you a chocolate advent calendar if you're on the mailing list

 

If you're paying to have new cable run then definitely get double the capacity--the actual fibre cable doesn't cost that much, it's the work in hauling it through the ceiling, so while they're at it anyway you may as well get the job done right

Edited November 2, 2018 by sonofsanta

[LytchettNM]

Hi, we are a secondary school with approx 1500 students and 300 PCs. I am looking at upgrading our existing infrastructure as all our existing fibres are OM1 and will never run faster than 1Gig and we are experiencing network issues and some switches are daisy chained off others and are sharing only a 1 gig link. Ideally I want to replace all the fibres with OM3 so we can install edge switches with 10Gig fibre uplink SFPs. What would you recommend we install in the main server room as a core solution where the 8 fibre cables terminate along with approx 120 local clients over Cat5e? We have three Hyper-V Dell Servers & a Dell SAN also located in that main server room along with our internet feed and a Netgear NAS. We also have a Ruckus wi-fi network covering the whole school and approx 50 Hik-Vision CCTV cameras running on the same network. Any ideas would be most welcomed?

 

We are also a secondary with 1460 ish students + 200 ish staff but around 900+ computers and all the normal IP stuff printers, access points, cctv etc.

We replaced our server side around 4 years ago and replaced our edge switches the last summer.

As I’m sure you are aware every network / school is different so offering advise is limited.

Please feel free to correct me if I’m a little off track with any assumption I’ve made.

 

Assuming you have a flat network I would 100% recommend using VLANs and subnetting your school which will allow for access control and reduce broadcast traffic which will slow your network performance down, increasing the networks speed will not massively improve performance unless the IP space has an efficiently design.

 

My background was in networking before becoming NM so I would recommend 2 x layer 3 switches stacked for the core, and then each edge switch connected directly back to the core switches with 2 GBICs so that’s 4 fibres per edge switch using a LAG, with the first GBIC of each edge switch connecting the first core switch and the second GBICs connection to the other.

This is a very simple connection method but offers good resilience against a single device failure or human error like unplug the wrong cable.

The way you end up going will very much depend if your going to do the work yourself as we did or use a third party.

 

You need to have a plan before talking to suppliers as they will sell you the kit they get the best deals on, after all they need to make a profit to stay in business.

Hope this helps.

Darren

[AlanD]

 

Hang on a minute...before you simply throw money at something without understanding what problems you have. First, 10G works with OM1 fibre. In fact it works with some of our fibres which are pre- OM1 categorisation....even at 100m or more. BUT, we did re-terminate them with welded pigtails because the original mechanical joints were quite lossy. And it doesn't make much difference if you have a mismatch between 50u and 62.5u fibres, patch cables and modules. Maybe it would make a difference if you are working at the edge of their capabilities....but in our experience fibres work well beyond their normal quoted lengths and speeds. Take a look at the errors in the switch table. If you are not getting errors transmitting over the fibres you are intending to replace - its not going to fix any issues you refer to.

 

I can understand any external contractor/advisor suggesting you replace everything - because that gives you a clean start - but you may well still have the same issues. We once had problems with one building going "off line"....thought it was the fibre...thought it was the modules....thought it was the switch.... turned out to be a faulty network port in one of the PCs...

 

We have our "core" as a virtual stack with at least 2 switches in each location with a ring of fibre. All edge switches are fed by 2x link aggregated one from each of the pair of stack switches. This way - any switch can fail, and link can fail or a link fail in the ring and no one knows (including me, if I forget to look at the switch status). And if any switch fails, including a stack switch, only the PCs directly connected to that switch stop working. And although a few of the switches we bought new, most are second hand. And I think all the fibre modules are now the cheap £10 Chinese ones, because unlike the genuine £100+ modules, they don't fail. Didn't stop one troublesome PC causing a lot of network issues though.

 

And before you think 10G will make your network really quick...it rarely makes much difference, unless your servers are full of SSD storage. 2xLink aggregated lines between switches serves up a lot of data and doesn't make your PCs boot or work much faster, and almost certainly does nothing for wireless. But yes, if starting afresh it would give you headroom for the future. But, I'll say it again, it won't solve in "issues".

[ITGuyWestMidlands]

Hang on a minute...before you simply throw money at something without understanding what problems you have. First, 10G works with OM1 fibre. In fact it works with some of our fibres which are pre- OM1 categorisation....even at 100m or more. BUT, we did re-terminate them with welded pigtails because the original mechanical joints were quite lossy. And it doesn't make much difference if you have a mismatch between 50u and 62.5u fibres, patch cables and modules. Maybe it would make a difference if you are working at the edge of their capabilities....but in our experience fibres work well beyond their normal quoted lengths and speeds. Take a look at the errors in the switch table. If you are not getting errors transmitting over the fibres you are intending to replace - its not going to fix any issues you refer to.

 

I can understand any external contractor/advisor suggesting you replace everything - because that gives you a clean start - but you may well still have the same issues. We once had problems with one building going "off line"....thought it was the fibre...thought it was the modules....thought it was the switch.... turned out to be a faulty network port in one of the PCs...

 

We have our "core" as a virtual stack with at least 2 switches in each location with a ring of fibre. All edge switches are fed by 2x link aggregated one from each of the pair of stack switches. This way - any switch can fail, and link can fail or a link fail in the ring and no one knows (including me, if I forget to look at the switch status). And if any switch fails, including a stack switch, only the PCs directly connected to that switch stop working. And although a few of the switches we bought new, most are second hand. And I think all the fibre modules are now the cheap £10 Chinese ones, because unlike the genuine £100+ modules, they don't fail. Didn't stop one troublesome PC causing a lot of network issues though.

 

And before you think 10G will make your network really quick...it rarely makes much difference, unless your servers are full of SSD storage. 2xLink aggregated lines between switches serves up a lot of data and doesn't make your PCs boot or work much faster, and almost certainly does nothing for wireless. But yes, if starting afresh it would give you headroom for the future. But, I'll say it again, it won't solve in "issues".

I hate seeing schools throw money at new switches just expecting it to speed everything up. Day to day I feel it would be difficult to saturate a 2 x 1gb lag unless reimagining PC's etc.

[Jamo]

Monitor, monitor monitor!!!!

 

Install Cacti, or CheckMK or similar, can be done in a Linux VM for no cost at all, 1vCPU 1GB RAM is fine especially for small Cacti installations.

 

If you can stretch to CheckMK its a far more complete monitoring solution but the initial user setup is more.

 

I would certainly look to move the CCTV off your normal network, usually separate cabling or switch infrastructure is easiest, something unmanaged will be fine for them if they are just off at the side of the network in a LAN which isn't accessible from the main network. The will save a ton of bandwidth and you will probably find that the rest of the network can survive a multi year upgrade strategy (10G is most likely overkill, multi GB etherchannel is usually sufficient).

 

Work out where your bottlenecks are, I suspect they aren't where you expect. Everyone blames the interconnects, but without proof of where the issue lies you will be randomly replacing components forever. It could even be SAN or just simply design of the components. I've seen crazy things like routed iSCSI or distributed multi switch core's with incorrect spanning tree configuration causing havok. None of that required new hardware, just new thinking.

 

Also ++ for aruba switches, 38xx series are great for Core's if budget doesn't stretch to the chassis switches, plus the 2930s are great for 1G to the edge. There are 10G options if you MUST aggregate at 10G though I would definitely recommend a multi 1G etherchannel (LACP trunk to HP/Aruba nomenclature)

[netadmin]

We're a fairly small campus - we used Juniper EX3300 switches in our edge closets and an EX4300 as our core switch. We're looking at some building additions in the near future, so I'm thinking I'll add a Juniper EX4600 fiber switch when that time comes, as it can connect to the EX4300 as a single virtual chassis. The Juniper switches have been very reliable for us, and Juniper support has been top notch when issues came up with firmware upgrades a couple of times.