Just after some opinions here on the way people do things.
Out wiring topology is generally quite good. We have a central wiring cabinet, and a dedicated fibre link to each cabinet at gigabit. Some of our cabinets are quite large, one serves 3 ICT suites, loads of offices and classrooms and a large part of the wireless system. This used to be wired so the switches were chained off one-another, fibre went into first one, then copper gigabit into the next, into the next etc. This linked about 8 swithes or so in one line.
I wasn't very happy with this, seeing that if a switch failed then all the rest underneath it would also be affected, and also thought it created a bit of a bottle neck on the network, so I changed it this summer using a HP 2824 switch, 2x gigabit fibre coming into that from our main cabinet, then a gigabit line to each switch in said cabinet.
I've never really been taught best practice when dealing with wiring, I've always used my common sense, and I've now got the possibility of doing this in the second of our big cabinets. I was after peoples opinions as to whether it's actually worth it or not, or if there's any real benefit to changing it from the switchs chaining off one another, to a locallised gigabit switch to act as distrabution.
Any opinions gratefully received.
Cheers,
Mike.
Well the reduced bandwidth of the last switch down the line doesn't bear thinking about for a start.
Is there only a single pair of Fibres terminated in the large cabinet?
I would have thought that very unusual as a fibre "cable" tends to have a minumum of 3 pairs and if your pulling the cable you might as well terminate them all. I would use more of these fibre pairs to get more bandwidth and redundancy from your Core switch out to that cabinet. Fibre GBICs are cheap these days.
The set-up that you changed your first cabinet to sounds ideal - repeat that in all the others. Daisy-chained switches are best avoided wherever possible, especially (as Mark points out) from the point-of-view of the poor souls on the bottom switch. Your PCs should be as few steps from the server as possible, on as fast a line as possible, so get yourself a nice big fully-gig switch in each cabinet and then feed the individual switches off that.
The concept I go by is 'least hops'. There should be as few hops between the server and the end client as possible. Here we have a fibre going from the core to every switch. So, if one switch dies, only that switch is affected. It also means that bandwidth doesn't decrease depending on where you are in a chain.
So using an aggregator like you are doing reduces the number of hops considerably, and should increase network latency and bandwith to a good number of devices.
Also, when you say you have a fibre to each cab, is it only a 2 core fibre (ie, can it only support a single link)? If it has more cores, why not trunk them together to make a 2 or 4GB link back to the core? That way, when you have your switches connected to the 2824, they have 2 or 4 times the bandwidth available.
Full star topology:
Gbic core switch central comms cab with all other Gig switches linked by fibre (OM3) directly back to core switch. Server switch links via Gbic to core switch voila full gig capacity. When we move on to 10gig switches all we have to do is swap out the switches as the infrastructure is already in place.
That's exactly the way Becta recommends (dictates?Originally Posted by bossman
) you do it, and also the way corporates have their infrastructure
Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
We do exactly the same as you - i.e. have a cheapy spare switch sitting on the shelf which we can drop in should one go wrong somewhere. It's not a very impressive switch, so I would always swap it back out when the replacement arrived, but it does at least keep the network running during that time.
Our servers have 2 cables...You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
I still have all our old Dlink stuff, and cabling with which to hook it up should something fail. The warranty on our switches is 24 hour replacement though, so it should only be using the old tat for a short while...We do exactly the same as you - i.e. have a cheapy spare switch sitting on the shelf which we can drop in should one go wrong somewhere. It's not a very impressive switch, so I would always swap it back out when the replacement arrived, but it does at least keep the network running during that time.
"You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!"
*cough* Ours had dual gigabit with failover *cough*
Anyway, yes, we had a big procurve chassis switch in the server room which handled the core switching with fibre links to all the other cabinates in the school which were then in turn 100mbit to the desktop.
In the server room there was gigabit copper between all the switches for the ICT department (there were 6 or 7 of those i seem to remember) and the chassis switch (in the same cab). All the fibre terminated in the server room also.
We didn't have a fail over core switch in place in case of failures. There just wouldn't have been the room for it for a start! But there were lots of replacements for other switches hanging around and GBic modules aswell i think.
Pfft :-) You take my point, though. You cannot prevent every single possible point of failure (short of having a spare server mirrored and ready to go!) but you can aim to reduce the number of such points as far as is possible.Our servers have 2 cables...You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
We have also have a star topology with all switches linked directly to the core. The main cabinet which houses the core also has several access layer switches in it. These are linked to the core by 4GB trunks over 4x1gb Ethernet ports. Each server has dual(edit: two individual) 1GB network cards which are also trunked into the core switch.
We are just in the process of adding additional fiber links to the other cabinets which will take them up to two 1gb fiber links each (from the one they have now).
The only true way to get rid of the single point of failure is to use a full mesh topology which is extremely expensive. In this each distribution cabinet connects to all of the other cabinets this means that any cabinet can fail and be routed around by the network. You of course need your servers connected with fail over links to more than one switch to accomplish this.
Out of intrest and hopefully without derailing the thread too much.
For adapter teaming on your servers what do people use.
Load balancing by Windows TCP/IP
802.3 LAG Protocol (which obviosuly requires support on switches)
Or HP / CISCO Proprietry solutions?
I've had some problems in the past getting our Broadcom cards to negotiate LAG correctly. One of them abjectly refuses until checksum offload is disabled. I've always meant to do some throughput testing on the different solutions but never found the time.
With the dual port NICs its not that redundant anyway. Same card :P
Indeed! Thats why you should have 2 separate cards in your server. And 2 servers, for redundancy, and 2 switches, and 2 ups's connected to 2 separate power rings, and 2 aircon systems :P
By the time we're done we have enough equipment to send a shuttle into space... And no money.
There are currently 1 users browsing this thread. (0 members and 1 guests)
Posting Permissions
LinkBack URL
About LinkBacks
Reply With Quote