+ Post New Thread
Page 1 of 2 12 LastLast
Results 1 to 15 of 29
How do you do....it? Thread, How do you wire your cabinets? in Technical; Just after some opinions here on the way people do things. Out wiring topology is generally quite good. We have ...
  1. #1

    maniac's Avatar
    Join Date
    Feb 2007
    Location
    Kent
    Posts
    3,069
    Thank Post
    210
    Thanked 430 Times in 310 Posts
    Rep Power
    144

    How do you wire your cabinets?

    Just after some opinions here on the way people do things.

    Out wiring topology is generally quite good. We have a central wiring cabinet, and a dedicated fibre link to each cabinet at gigabit. Some of our cabinets are quite large, one serves 3 ICT suites, loads of offices and classrooms and a large part of the wireless system. This used to be wired so the switches were chained off one-another, fibre went into first one, then copper gigabit into the next, into the next etc. This linked about 8 swithes or so in one line.

    I wasn't very happy with this, seeing that if a switch failed then all the rest underneath it would also be affected, and also thought it created a bit of a bottle neck on the network, so I changed it this summer using a HP 2824 switch, 2x gigabit fibre coming into that from our main cabinet, then a gigabit line to each switch in said cabinet.

    I've never really been taught best practice when dealing with wiring, I've always used my common sense, and I've now got the possibility of doing this in the second of our big cabinets. I was after peoples opinions as to whether it's actually worth it or not, or if there's any real benefit to changing it from the switchs chaining off one another, to a locallised gigabit switch to act as distrabution.

    Any opinions gratefully received.

    Cheers,

    Mike.

  2. #2
    mark's Avatar
    Join Date
    Jun 2005
    Posts
    3,972
    Thank Post
    257
    Thanked 49 Times in 45 Posts
    Blog Entries
    2
    Rep Power
    46

    Re: How do you wire your cabinets?

    Well the reduced bandwidth of the last switch down the line doesn't bear thinking about for a start.

  3. #3

    Join Date
    Jun 2006
    Location
    Belfast, N\'Ireland
    Posts
    190
    Thank Post
    10
    Thanked 9 Times in 7 Posts
    Rep Power
    18

    Re: How do you wire your cabinets?

    Is there only a single pair of Fibres terminated in the large cabinet?

    I would have thought that very unusual as a fibre "cable" tends to have a minumum of 3 pairs and if your pulling the cable you might as well terminate them all. I would use more of these fibre pairs to get more bandwidth and redundancy from your Core switch out to that cabinet. Fibre GBICs are cheap these days.

  4. #4
    enjay's Avatar
    Join Date
    Apr 2007
    Location
    Reading, Berkshire, UK
    Posts
    4,488
    Thank Post
    282
    Thanked 196 Times in 167 Posts
    Rep Power
    76

    Re: How do you wire your cabinets?

    The set-up that you changed your first cabinet to sounds ideal - repeat that in all the others. Daisy-chained switches are best avoided wherever possible, especially (as Mark points out) from the point-of-view of the poor souls on the bottom switch. Your PCs should be as few steps from the server as possible, on as fast a line as possible, so get yourself a nice big fully-gig switch in each cabinet and then feed the individual switches off that.

  5. #5

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,973
    Thank Post
    519
    Thanked 2,505 Times in 1,945 Posts
    Blog Entries
    24
    Rep Power
    841

    Re: How do you wire your cabinets?

    The concept I go by is 'least hops'. There should be as few hops between the server and the end client as possible. Here we have a fibre going from the core to every switch. So, if one switch dies, only that switch is affected. It also means that bandwidth doesn't decrease depending on where you are in a chain.

    So using an aggregator like you are doing reduces the number of hops considerably, and should increase network latency and bandwith to a good number of devices.

    Also, when you say you have a fibre to each cab, is it only a 2 core fibre (ie, can it only support a single link)? If it has more cores, why not trunk them together to make a 2 or 4GB link back to the core? That way, when you have your switches connected to the 2824, they have 2 or 4 times the bandwidth available.

  6. #6

    bossman's Avatar
    Join Date
    Nov 2005
    Location
    England
    Posts
    3,943
    Thank Post
    1,199
    Thanked 1,071 Times in 762 Posts
    Rep Power
    330

    Re: How do you wire your cabinets?

    Full star topology:
    Gbic core switch central comms cab with all other Gig switches linked by fibre (OM3) directly back to core switch. Server switch links via Gbic to core switch voila full gig capacity. When we move on to 10gig switches all we have to do is swap out the switches as the infrastructure is already in place.

  7. #7
    Oops_my_bad's Avatar
    Join Date
    Jan 2007
    Location
    Man chest hair
    Posts
    1,738
    Thank Post
    438
    Thanked 53 Times in 50 Posts
    Rep Power
    30

    Re: How do you wire your cabinets?

    Quote Originally Posted by bossman
    Full star topology:
    Gbic core switch central comms cab with all other Gig switches linked by fibre (OM3) directly back to core switch. Server switch links via Gbic to core switch voila full gig capacity. When we move on to 10gig switches all we have to do is swap out the switches as the infrastructure is already in place.
    That's exactly the way Becta recommends (dictates? ) you do it, and also the way corporates have their infrastructure

    Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case

  8. #8
    enjay's Avatar
    Join Date
    Apr 2007
    Location
    Reading, Berkshire, UK
    Posts
    4,488
    Thank Post
    282
    Thanked 196 Times in 167 Posts
    Rep Power
    76

    Re: How do you wire your cabinets?

    Quote Originally Posted by TeddyKGB
    Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
    You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!

    We do exactly the same as you - i.e. have a cheapy spare switch sitting on the shelf which we can drop in should one go wrong somewhere. It's not a very impressive switch, so I would always swap it back out when the replacement arrived, but it does at least keep the network running during that time.

  9. #9

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,973
    Thank Post
    519
    Thanked 2,505 Times in 1,945 Posts
    Blog Entries
    24
    Rep Power
    841

    Re: How do you wire your cabinets?

    Quote Originally Posted by NickJones
    Quote Originally Posted by TeddyKGB
    Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
    You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!
    Our servers have 2 cables...

    We do exactly the same as you - i.e. have a cheapy spare switch sitting on the shelf which we can drop in should one go wrong somewhere. It's not a very impressive switch, so I would always swap it back out when the replacement arrived, but it does at least keep the network running during that time.
    I still have all our old Dlink stuff, and cabling with which to hook it up should something fail. The warranty on our switches is 24 hour replacement though, so it should only be using the old tat for a short while...

  10. #10
    Joedetic's Avatar
    Join Date
    Jan 2006
    Location
    Walsall
    Posts
    1,316
    Thank Post
    6
    Thanked 13 Times in 13 Posts
    Rep Power
    22

    Re: How do you wire your cabinets?

    "You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!"

    *cough* Ours had dual gigabit with failover *cough*

    Anyway, yes, we had a big procurve chassis switch in the server room which handled the core switching with fibre links to all the other cabinates in the school which were then in turn 100mbit to the desktop.

    In the server room there was gigabit copper between all the switches for the ICT department (there were 6 or 7 of those i seem to remember) and the chassis switch (in the same cab). All the fibre terminated in the server room also.

    We didn't have a fail over core switch in place in case of failures. There just wouldn't have been the room for it for a start! But there were lots of replacements for other switches hanging around and GBic modules aswell i think.

  11. #11
    enjay's Avatar
    Join Date
    Apr 2007
    Location
    Reading, Berkshire, UK
    Posts
    4,488
    Thank Post
    282
    Thanked 196 Times in 167 Posts
    Rep Power
    76

    Re: How do you wire your cabinets?

    Quote Originally Posted by localzuk
    Quote Originally Posted by NickJones
    Quote Originally Posted by TeddyKGB
    Only potential pitfall is you have a single point of failure - ie your core switch. But just keep one spare and configured ready just in case
    You can never totally eradicate the single-point of failure, even if you had some fail-over on the switches, the server still only has one Ethernet cable in it after all!
    Our servers have 2 cables...
    Pfft :-) You take my point, though. You cannot prevent every single possible point of failure (short of having a spare server mirrored and ready to go!) but you can aim to reduce the number of such points as far as is possible.

  12. #12

    SYNACK's Avatar
    Join Date
    Oct 2007
    Posts
    11,206
    Thank Post
    876
    Thanked 2,730 Times in 2,309 Posts
    Blog Entries
    11
    Rep Power
    782

    Re: How do you wire your cabinets?

    We have also have a star topology with all switches linked directly to the core. The main cabinet which houses the core also has several access layer switches in it. These are linked to the core by 4GB trunks over 4x1gb Ethernet ports. Each server has dual(edit: two individual) 1GB network cards which are also trunked into the core switch.

    We are just in the process of adding additional fiber links to the other cabinets which will take them up to two 1gb fiber links each (from the one they have now).

    The only true way to get rid of the single point of failure is to use a full mesh topology which is extremely expensive. In this each distribution cabinet connects to all of the other cabinets this means that any cabinet can fail and be routed around by the network. You of course need your servers connected with fail over links to more than one switch to accomplish this.

  13. #13

    Join Date
    Jun 2006
    Location
    Belfast, N\'Ireland
    Posts
    190
    Thank Post
    10
    Thanked 9 Times in 7 Posts
    Rep Power
    18

    Re: How do you wire your cabinets?

    Out of intrest and hopefully without derailing the thread too much.

    For adapter teaming on your servers what do people use.

    Load balancing by Windows TCP/IP

    802.3 LAG Protocol (which obviosuly requires support on switches)

    Or HP / CISCO Proprietry solutions?

    I've had some problems in the past getting our Broadcom cards to negotiate LAG correctly. One of them abjectly refuses until checksum offload is disabled. I've always meant to do some throughput testing on the different solutions but never found the time.

  14. #14
    DMcCoy's Avatar
    Join Date
    Oct 2005
    Location
    Isle of Wight
    Posts
    3,476
    Thank Post
    10
    Thanked 500 Times in 440 Posts
    Rep Power
    114

    Re: How do you wire your cabinets?

    With the dual port NICs its not that redundant anyway. Same card :P

  15. #15

    localzuk's Avatar
    Join Date
    Dec 2006
    Location
    Minehead
    Posts
    17,973
    Thank Post
    519
    Thanked 2,505 Times in 1,945 Posts
    Blog Entries
    24
    Rep Power
    841

    Re: How do you wire your cabinets?

    Quote Originally Posted by DMcCoy
    With the dual port NICs its not that redundant anyway. Same card :P
    Indeed! Thats why you should have 2 separate cards in your server. And 2 servers, for redundancy, and 2 switches, and 2 ups's connected to 2 separate power rings, and 2 aircon systems :P

    By the time we're done we have enough equipment to send a shuttle into space... And no money.

SHARE:
+ Post New Thread
Page 1 of 2 12 LastLast

Similar Threads

  1. How do YOU wire your data cabinets?
    By Oops_my_bad in forum How do you do....it?
    Replies: 35
    Last Post: 17th July 2007, 10:37 AM
  2. Cabinets For Grabs
    By Ste_Harve in forum Wireless Networks
    Replies: 0
    Last Post: 28th June 2007, 03:34 PM
  3. SERVER CABINETS
    By alan-d in forum Our Advertisers
    Replies: 6
    Last Post: 1st May 2007, 01:38 PM
  4. Server Cabinets
    By mortstar in forum Hardware
    Replies: 33
    Last Post: 20th April 2007, 01:54 PM
  5. SERVER CABINETS
    By alan-d in forum Hardware
    Replies: 30
    Last Post: 15th September 2006, 08:47 PM

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •