Recommend me a 10GbE switch

[localzuk]

So, I'm now planning our large-scale rebuild of IT across the new MAT. The goal is to centralise the "big stuff", so main servers will be here (possibility, depending on cost/funding, of having a redundant setup at one of the other schools too).

 

One thing I need is a reasonable 10GbE switch, in fact 2 of them.

 

1 will be used by the Hyper-V cluster to connect to a SoFS storage cluster, so will need a mere 4 10GbE ports (but we should have 8 just in case, for expansion).

The other, will be used as the access switch, with all the Hyper-V servers connected in - a mere 2 ports for the servers, and a couple to connect to the network! Ideally I'd like to keep these separated, to allow for a split of the load across switches.

 

They'd all be SFP+ connectors also.

 

I'd *love* to use a Meraki switch for the role, but their 10GbE stuff is 24 ports minimum, and costs an absolute fortune. So for this I will be looking elsewhere...

Edited January 27, 2016 by localzuk

[pantscat]

How about a Procurve 6600 with the appropriate modules in the back?

 

Meant the 5500... the 6600 is all 10gbe.

Edited January 27, 2016 by pantscat

[Snuffkins]

I like the Cisco 4500-X, that's what we use here, but those are our network core switches, with our server and storage connected from a couple of FIs.

 

You might be able to use the Cisco C3850. They do a 10gbe version, which i think starts at 12 ports.

 

Just checked and it's a WS-C3850-12XS, probably still a bit pricey though. Dollar rate isn't going to be helping at the moment

Edited January 27, 2016 by Snuffkins

[psydii]

I'd suggest the 5500 too with that small number of 10gbe ports required. However a 5900 might provide better long term value as you have more options to bring on new physical servers as you upgrade your server infrastructure in 3-6 years. (if you max out your 10gb ports now, it forces a cut over migration and potentially limits data transfers between your 2016 infrastructure and your 'not yet a twinkle in your eye' 2022 infrastructure) A decent core switch should last several server generations. Remeber the time taken to build a solution is often a substantial element of the cost, and time today is cheaper than time tomorrow.

 

I am intrigued though: What is the client/server architecture/mix like? Even with 1Gbe site to site I'd be reluctant to try to design a solution that has remote sites depend on a central hub for core services. WSUS saturates its 1gb/s uplink to our core when we release updates. Software deployments would do the same.

 

Or have you gone all Google ChromeBooks?

[Steve21]

As above, unless you're doing something like VDI at the remote sites I wouldn't want to take servers away as you're effectively running at Internet speeds (split between all sites!)

 

But in regards to switches why not get a blank 5400r chassis and just add one or two modules to start with, for example a 16 SFP+ with dual psu and management is about £7k (from a quote I got the other day)

 

Steve

[twin--turbo]

The 5500 is not Procurve it's Comware. I have 3 5700FF's would not exactly rave about them.

 

5400's ( are Procurve ) of which we have 5 in service, they are being resigned to edge port duty certainly don't rate the non "R" model and we recently discoverd you need 4PSU's installed in a 5412 to make it power redundant on all switch ports ( even before considering POE...

 

I still have a fond affection for a Cisco 4507R-E , but they now do many 1-2 U Units that can cut the mustard just as well. And the fixed config switches DO ( despite what many say ) have lifetime warranty.

 

TT

 

- - - Updated - - -

 

The 5500 is not Procurve it's Comware. I have 3 5700FF's would not exactly rave about them.

 

5400's ( are Procurve ) of which we have 5 in service, they are being resigned to edge port duty certainly don't rate the non "R" model and we recently discoverd you need 4PSU's installed in a 5412 to make it power redundant on all switch ports ( even before considering POE...

 

I still have a fond affection for a Cisco 4507R-E , but they now do many 1-2 U Units that can cut the mustard just as well. And the fixed config switches DO ( despite what many say ) have lifetime warranty.

 

TT

[FN-GM]

We also use the Cisco Catalyst 4500-X switches in our MAT. We have 6 of them. Really good switches, we stack them using VSS and they haven't skipped a beat.

 

For access switches we are using Catalyst 2900-X or 3750-X. Love the 3750-X, dual PSU, Power stacking a L3 so loads of benefits in that itself.

Edited January 27, 2016 by FN-GM

[IanT]

Core: Cisco 3850 with C3850-NM-2-10G, 4 or 8 versions

Edge: Cisco 2960-X Series

 

Cisco Catalyst 3850 Series Switches Data Sheet - Cisco

[psydii]

I have 3 5700FF's would not exactly rave about them.

 

....

 

I still have a fond affection for a Cisco 4507R-E ,

 

TT

 

So what did you like about the Cisco and what left you feeling flat about the Comware kit?

 

(I note that for MATs it does seems that the general preference is for Cisco)

[twin--turbo]

So what did you like about the Cisco and what left you feeling flat about the Comware kit?

 

(I note that for MATs it does seems that the general preference is for Cisco)

 

5700...

 

It seems to be difficult or near impossible to get any decent SNMP data out of the HP which is a real PITA when we have monitoring of most of the rest of the network.

 

It has no visual status indicators for the overall health of the switch.

 

Only port 1/2 - 13/14 15/16-31/32 are labled

 

The port jacks don't have their own status indicators, they are all above the top row so it can be difficult to determine if a port is live.

 

Comware programming is half Cisco mind set, 1/8th Procurve and 3/8ths randomness... HP should bring the two together but I doubt it will ever happen, so interoperability with their ProCurve needs thought.

 

Trying to find decent accurate guides on anything to do with comware seems to be problematic, and some contradict others.

 

On one of our other parts of the network which is managed for us, we do have some Catalyst 2960's with PVSTP which are not compatible with MSTP. So I had to force the 5700 to be an edge port.

 

It just feels and looks a bit Zyxel too.

 

Having ASA's and Cisco Routers too I just find having 3 Platforms that are all not quite compatible a real pain.

 

 

 

Generally I found that CISCO was always well supported by 3rd Party developments. Procurve Less so, Commware less than that.

 

TT

[FN-GM]

we do have some Catalyst 2960's with PVSTP which are not compatible with MSTP

 

Are you sure about that? Im pretty sure the 2960's can do MSTP. It is a bit of a swine to setup compared to PVSTP but should be do able.

 

EDIT: just checked the 2960's can do MSTP. This is a screenshot from a 2960.

 

 

It just feels and looks a bit Zyxel too.

 

Do you use Zywall products?

Edited January 27, 2016 by FN-GM

[twin--turbo]

Are you sure about that? Im pretty sure the 2960's can do MSTP. It is a bit of a swine to setup compared to PVSTP but should be do able.

Do you use Zywall products?

 

Yeah, I think they can do MSTP, but for this section of the network we would have to go through design and evaluation to get that changed. And in fact as only 2 ports are used on the switch now ( 1 to a firewall, 1 to the 5700 switch) I am probably going to suggest we ditch the switch in the middle.

 

 

No Zy products, tried to use a switch on a customer site once when self employed. Never again.

 

 

Still have fond memories of our old D-Link DES-6000 core chassis that we installed at my 1st school, must have been circa 2002. was a real step up from out Planet 1U switch that kept locking up once a month!

[FN-GM]

Yeah, I think they can do MSTP, but for this section of the network we would have to go through design and evaluation to get that changed. And in fact as only 2 ports are used on the switch now ( 1 to a firewall, 1 to the 5700 switch) I am probably going to suggest we ditch the switch in the middle.

 

 

No Zy products, tried to use a switch on a customer site once when self employed. Never again.

 

 

Still have fond memories of our old D-Link DES-6000 core chassis that we installed at my 1st school, must have been circa 2002. was a real step up from out Planet 1U switch that kept locking up once a month!

 

I would probably ditch it then.

 

We are having a DDOS problem at the moment. I would say 25% of the attackers are coming from connections that use Zywalls, there must be some kind of problem with them.

[localzuk]

I am intrigued though: What is the client/server architecture/mix like? Even with 1Gbe site to site I'd be reluctant to try to design a solution that has remote sites depend on a central hub for core services. WSUS saturates its 1gb/s uplink to our core when we release updates. Software deployments would do the same.

 

Or have you gone all Google ChromeBooks?

 

The plan, as it stands, is to centralise as much as possible. Each branch school would have a single server with various VMs hosted on it, handling things such as SCCM DP, local DHCP (handling their scope), and DFS-R for their local users (plus a local print server) etc...

 

The idea being that they have quick access to the things they need locally, but also have a) the redundancy of having the central setup to fall back on if their local box fails, and b) they don't have to have an expensive server in every school - a relatively cheap box can do the job (especially when you consider the size of the schools). It also centralises everything, and ensures all the MAT staff can easily go to any of the schools, log in and have their documents ready for them (even if they're a little slower to grab, until they get added as a "local site" user).

[pow]

I've got a top-of-rack M7100 in the server room which is very good?

[Steve21]

The idea being that they have quick access to the things they need locally, but also have a) the redundancy of having the central setup to fall back on if their local box fails, and b) they don't have to have an expensive server in every school - a relatively cheap box can do the job (especially when you consider the size of the schools). It also centralises everything, and ensures all the MAT staff can easily go to any of the schools, log in and have their documents ready for them (even if they're a little slower to grab, until they get added as a "local site" user).

 

The thing is though unless you're linking directly to the other schools (in terms of private lines etc), and I'm assuming your net is what 100Mb? even if it's 1Gb that's all its going to increase your network traffic by as that's your bottle neck.

 

Guess it depends if you feel you're actually going to need that 10Gb within your own network, or whether you'd prefer for example your Meraki solution thats ease of management and at the end of the day the same connection to them externally

 

Steve

[localzuk]

The thing is though unless you're linking directly to the other schools (in terms of private lines etc), and I'm assuming your net is what 100Mb? even if it's 1Gb that's all its going to increase your network traffic by as that's your bottle neck.

 

Guess it depends if you feel you're actually going to need that 10Gb within your own network, or whether you'd prefer for example your Meraki solution thats ease of management and at the end of the day the same connection to them externally

 

Steve

 

The 10GbE is for the HyperV cluster. The majority of users of the actual equipment will be at our school (we're by far the largest of the lot). The other schools shouldn't particularly be throwing a lot of data back and forth to be honest. Things like updating the SCCM DP can be scheduled for off hours, DFS-R shouldn't be overly heavy either, as it can be "throttled" somewhat.

[Steve21]

Ah sorry, assumed when you said about centralising like running everything across it thus the 10Gb

 

If it's just for the cluster are you assuming 2 switches per role? e.g. stack for cluster, stack for access? Or happy with 1 on each if it's single failure point?

 

Steve

[localzuk]

Ah sorry, assumed when you said about centralising like running everything across it thus the 10Gb

 

If it's just for the cluster are you assuming 2 switches per role? e.g. stack for cluster, stack for access? Or happy with 1 on each if it's single failure point?

 

Steve

 

I'm drawing up 3 plans - "bare minimum", "this would be better" and "this would be best". So, having 2 for each role will be in the last one. 1 for each role in the middle plan, and 1 shared between roles in the first plan.

 

Obviously, it'll come down to what the PTB are willing to spend!

[cpjitservices]

Juniper EX4500

[CyberNerd]

An HP A5800 has 4 10GBe SFP+ ports and can take a module of another 4. We have 4 of these switches in a cluster with three modules giving us 28 10GBe ports - some of these are used in the cluster though. Remember you can also use these for FCoE which means you can offset the cost of the FC switching.

[njc235]

An HP A5800 has 4 10GBe SFP+ ports and can take a module of another 4. We have 4 of these switches in a cluster with three modules giving us 28 10GBe ports - some of these are used in the cluster though. Remember you can also use these for FCoE which means you can offset the cost of the FC switching.

The 5800 series is very popular for a number of reasons one of which is that they are at the top of the Lifetime warranty range. 4 x SFP+ out of the box reducing to 2 if you create a full IRF with multiples but as Cybernerd says, you can add a 4 x SFP+ module in the back. The 5820 is a full SFP+ switch which can bee added to an IRF but is very expensive and requires carepacks.

[duzzie]

We have just purchased some Brocade switches for our core. Have a look at these - BROCADE ICX 7250 or 7450. The 7250 was a good fit for us as they have 8 SFP+ ports along with 24 or 48 1GB RJ-45 ports which we couldn't find with any other manufacturer (without buying an empty 24 port chassis and filling it with SFP). The 7450 comes with 12 x 10GB RJ-45 Ports.

[localzuk]

I've been looking at this a little more, and its actually more likely we'll only need a single switch/pair of switches (depending on the level of resilience we opt for), as we're likely going to look at hyper-converged Storage Spaces Direct setup on the server side instead. That way the Hyper-V nodes would be the storage nodes also. The only data going back and forth on the storage side would be replication traffic.

 

Has anyone looked at the HP Aruba 3810M?

[wesleyw]

Dell 5548 has 2 x 10gb links at about 1.5k but a decent amount (24) on a 8024f (also has 4 x 10gb copper) for around 7-8k just need to pop in sfp+ modules at £150 a piece good for future expansion.