New fiber backbone and switches brainstorm with me!!!

[ChrisH]

Im looking at getting fiber in and making all links and inter switch links a gig.

I currently have it cobbled together with a supposed copper gig link between the cabinets and all the switches in the stacks are usually 100 meg between each cascading off the switch with the gig ports.

I have all 3com stuff. Im trying to keep this fairly cheap (as cheap as getting fiber in goes anyways).

I was looking at one switch in each cabinet with 2 GBIC slots for fiber and the rest 24 port with 2 gig ports for uplinking to other switches.

Ideas/advice please? Also any reccomendations for people doign fiber in Lancs?

 

I currently have 4 cabinets but im going to consolidate 2 as there is only a room between them both. So 2 runs of fiber.

[sahmeepee]

If you've got the 3com switches with expansion slots in the back you can use the 3com stacking modules to stack them and I'm pretty sure the backplane speed is >> 100Mbit.

 

I guess it depends if your current 3com kit is managed stuff or the cheaper unmanaged type. If it's not managed, it might be worth taking a look around at other cheaper kit from newer players doing slightly less featureful gigabit switches with web admin etc..

[ChrisH]

Current 3 com stuff is managed. No expansion slots on most and one pair is connected with a matrix cable.

[Ric_]

The HP Procurve gear that I have is superb - with a selection of interfaces (web, command line and menu driven). I've never liked the 3COM stuff since I had all those problems at my last place.

 

I use gig fibre to interconnect (SX mini-GBIC modules) and currently have 2x 4104gl chasis (1x 6-port mini-GBIC and 1x 6-port 1000-baseT in each), 2x 2650 (with one mini-GBIC) and 1x 2824 (24 gig ports ). All these also allow for trunking, VLANs, etc. (and they look quite nice).

 

I got a quote recently off a guy called Eddy Cooper from Ampec (tel 01257 271679) - he seemed to know his stuff and was rather helpful.

[ninjabeaver]

I'm with Ric_ on this one. We've got HP PRoCurve throughout the school now, apart from one or two I've still got to get around to replacing.

 

I'd now swear by these switches (2524). I've also got the 4108 with the GBIC and 1000 baseT as well. Awesome gear, and so easy to manage via web interface.

 

Plus lifetime warranty. WOOT!

 

My main peice of advice is to name the switch with the room location as you go around. Me and my techie have got this task to do after it's all been installed.

[ChrisH]

All my current switches are named according to location and I even did a quick and dirty website using the switch images to link to their web admin.

@Ric those chassis things are expensive!! £700 for the chasis (which I could live with) but then it cost another £700 for a 24 x 10/100 module eeeekkk 8O

[Ric_]

@ChrisH: I'm not sure why you would get the 10/100 module cos (as you have seen) it is ridiculously expensive. The other modules are a lot less.

 

The 2824 cost about £1000 and I didn't buy the 2650s but I don't think that they cost too much (relatively speaking - and you would probably buy the 2626 anyway).

[DMcCoy]

I'm with Ric_ on this one. We've got HP PRoCurve throughout the school now, apart from one or two I've still got to get around to replacing.

 

I'd now swear by these switches (2524). I've also got the 4108 with the GBIC and 1000 baseT as well. Awesome gear, and so easy to manage via web interface.

 

Plus lifetime warranty. WOOT!

 

My main peice of advice is to name the switch with the room location as you go around. Me and my techie have got this task to do after it's all been installed.

 

Isn't it odd that we all end up with the same switches

 

I've got a 4108, 4104, 2524 and 2626(?) at the moment.

 

I first got one a few years back becasue they support fast etherchannel (FEC - Cisco protocol).

 

Unfortunatley the 4108 will only do 6 trunks and I need 7 now, so I'm going to upgrade to the 5308.

 

Don't you find the 4104/8 a funny size though? It needs a 500mm cabinet and even then has to sit right at the front.

 

David

[ChrisH]

@Ric I like the look of the HP 2626. I shall investigate more later. Packed with a lot more features than the 3 Com for about another £30.

[ninjabeaver]

All my current switches are named according to location and I even did a quick and dirty website using the switch images to link to their web admin.

 

What an awesome idea. Nice one mate. I'll start that on that after the holiday, which I'm absolutley gasping for.

[ChrisH]

Im actually having a holiday myself this time round. Four days in Amsterdam with the g/f rather than workign all the way through as usual.

[Westbrooke]

Thought I'd chip in with a few design comments. Don't know what level of knowledge you are starting from, so sorry if any of this is beneath you!

 

Fibre is great for links between cabinets. In theory you can achieve much the same with Cat6 links these days, but they are a pain in the a**e! (although in some setups I use both as redundant links).

 

One thing you need to look at is whether to install OM2 or OM3 grade fibre. They will both work, but OM3 can be pushed to 10Gb. You also need to think about how many cores to pull (2 cores per connection). I normally get 8 core fibre installed, because so much of the cost is paying the bloke to lay it, and then you get spare capacity.

 

It sounds as if you are buying a lot of new kit, as opposed to adding some small upgrades to the existing setup (although I assume you will be re using some kit)

 

Ideally, you should aim for a simple collapsed backbone model with a good core switch feeding a number of edge switches. If possible, you need to avoid a second level of cascade in the cabinet, so if you have multiple switches either go for multiple separate uplinks (can get pricey) or make sure they are properly stacked. (note that some modern swithches can be stacked using the GB uplink ports) The core switch should supply all the edge switches and the servers. Everything else comes off edge switches.

 

The trick to selecting the kit is to look at the backplane speed. These rate from about 1Gb for an edge switch up to about 256G for a Cisco core switch. There is obviously no point feeding a dozen GB links into a switch with a 4 Gb backplane as you'll simply run out of bandwidth.

 

I'd agree with previous posters on the HP stuff - very nice. I've also used 3Com for years on very large networks and it also works well and can be very cost effective. In both cases you need to make sure you have the right switches.

 

Oh, and you mentioned keeping the cost down. the biggest trick is to use fixed configuration switches (ie the ones with little or no expansion) but get the right ones. you may a lot for the upgrade capacity.

 

If you let me know the number of active (I assume 10/100) ports you want in each cabinet and the number of (Gb) server connects, I'll have a look at the 3Com and HP options for you.

[ChrisH]

I will need 2 cabinets of 96 and one of 70. Thanks for the help

[Westbrooke]

Any idea how many gigabit (or lower) connections you'll need at the core for servers etc?

 

Also, do you think you will just need layer 2 switching (standard ethernet) or are you going to need layer 3 (an expensive add on unless you are starting to use routed subnets or VLANs. This includes using VoIP!)

[ChrisH]

I dont think we will need layer 3 switches I will get away with layer 2. Once everything is sorted were probably looking at 6 or so servers and the Firewall/Filter will use 2 ports so at least 7 for the servers.

[ChrisH]

Ok Im looking into this again as some money has been made available.

Also since we have double the members as well Im hoping for some good advice .

 

Basically Im going to have the central cabinet in the server room and some 8 core fiber links to two other cabinets.

For the other 2 cabinets Im looking at having some HP Procurve 2650 These have 48 10/100 and 2 1000/minigbic.

 

Is it worth me using both GBIC slots on each switch to feed from the fiber and set up any traffic rules (if possible) or will one suffice?

 

As for the server room end im not sure I need about 70 points for clients and then say about 10 gig ports for servers and 8 gbic slots if I fully utilise all the cores on the to 8 core fibers running to it.

 

I was thinking either :

 

Scenario 1

HP Procurve 2824

this has 20 gig ports which will do for the servers and trunking to other switches in the cabinet and 4 GBICs for the fibre.

 

2 HP Procurve 2650 These have 48 10/100 and 2 1000/minigbic.

 

to serve the clients and handle the other fiber (if needed)

 

or

 

Scenario 2

 

HP Procurve 2650 These have 48 10/100 and 2 1000/minigbic.

 

To serve most of the clients. ^^

 

Hp Procurve 4140gl this has 40 10/100/100 ports which would be used for the server and the clients that cant go on the 2650. This with another 6 port GBIC module to handle the rest of the fiber if needed.

 

I think scenario 2 sounds the best but then im not sure if this is right for this implementation.

 

Thoughts advice all welcome.

Be nice!!!

 

Cheers

 

Chris

[ChrisH]

Oh and if anyone can reccomend someone to do the Fiber in Lancashire or a national company I would be gratefull.

[Geoff]

Ok, first thing. Don't bother going faster than 1Gbit with PCI based systems. The PCI bus will not go any faster than 1.3Gbit.

 

Secondly. Western Business Systems in Blackpool are a good firm for fibre. They have all the testing gear and what not. Give them a ring on 01253 298111.

[plexer]

Just to add to this we run HP Procurve throughout now with my latest upgrade. With a 4108gl as the core switch and 2524's in the cabinets stacked with 2324's below. Before I upgraded by putting a 2524 in each cabinet I had no management except for the core switch.

 

Fibre link's to all but 2 cabinets which are gig copper.

 

I use network notepad to produce interactive diagrams of the network and ict suites. This allows you to pick an object i.e one of your switches and with a right click to either telnet to it or go to the web management of it.

 

I also have diagrams of my ict suites with pictures of the computers all labelled with the correct names and then with a right click can vnc view any of them. Handy when someone says "It's the 3rd one from the window" etc...

 

http://www.networknotepad.com

 

Ben

[ChrisH]

When you say dont go faster than 1 gigabit on the links did you take into account some of the switches have 48 ports? Thats why I was asking would one gig link be sufficient for a 48 porter or should I use both if I can.

[wesleyw]

Depends on the sort of traffic but in general I've had no problems with just the single gig link, if you plug in both to the main chassis you'll get a loopback effect and network availability will cease within a few minutes there are settings to stop this but I can't think of name presently (Sunday Morning, good drinking session last night etc ) I do remember it was mentioned in another forum topic on this site though!

 

Wes

[ChrisH]

Spanning tree?

[wesleyw]

Yeah that's it! Shows what alcohol does to the brain lol 8O Sadly I learned this the hard way by not realising I'd connected a series of switches into a loopback, what fun that was! Thankfully no-one saw this as I quickly unplugged the offending cables. :oops:

 

Wes

[ChrisH]

Thats a good point about the loopback effect now I have thought about it but I assumed something could be done with a Vlan or something else to handle the traffic to avoid such situations.

 

Edit Typical just found it first time

[ChrisH]

Oh whats HP like on warranty returns for switches? I was wondering whether or not to get a spare....