New Server Room

[Soulfish]

Hi Everyone,

 

Since I started here a couple of months ago I've been looking to move the server rack into a dedicated server room (out of the first aid room/disabled lift room!). After talking to my line manager and the caretaker we finally think we have a workable solution (partitioning part of the boiler room to give us a dedicated server room!), and now we're talking to SMT to get approval to go ahead.

 

As part of this move we're looking to also sort out our currently very poor UPS setup and put in some new racks. I was wondering what rack manufacturers people recommend? Which ones do you find easy to work with and give you all the nice cable routing features you want

 

Also what UPS setups do people have. We'd be starting from scratch, and it looks like there won't be enough cash to put in a single (well 2 to give redundancy ) UPS for the room so I'm looking at what other schools are currently doing.

 

I'm also looking for suggestions on environmental monitoring. We're not putting in A/C to begin with since the boiler room is around 15C all year round, but I was told if temperature/humidity is a problem then we'll get it installed. So a decent system to monitor this and other things like water/smoke detection would be great

 

Finally anyone got any suggestions or hints/tips for putting in a new server room? We're currently hoping to get the room ready by the summer to allow us to move everything over once the school breaks up, and at the same time allow us to lay new fibre to all the comms cabinets. If anyone has any suggestions for companies that do cable installs in the south east (sussex) then that would also be greatly appreciated

 

John

[localzuk]

Our UPS solution isn't ideal either, so I can't really advise on that. What I will say is that you should put A/C in from the outset. It may be 15 degrees now, but when you stick some servers in there, it'll soon heat up. Especially if it is a closed room. We have a dozen machines in our room, and without A/C it'd be hitting 40 degrees in there.

[FN-GM]

Will a holier room next door not be hot for servers

[Linfit]

Would agree totally about the air-con.

You will need it as soon as the servers are up and running, and it will be easier to fit it during other building work than retro-fitting it later.

 

As for UPS, redundancy is the key. Better to have several smaller capacity than one to service the whole lot, expecially if you have redundant power supplies in the servers.

[alextreadwell]

The Aircon failed in our server room this summer with only 5 servers 5 switches and 2 ups running in there and as server rooms go its quite large. We came back in on monday (it had failed over weekend) and you could have cooked an egg on the plaster board wall, the other tech opened the door and burn his hand on the door handle and thats not next to a boiler room, other consideration is when boiler starts leaking and your server room is under water will the insurance pay out not the ideal place to have it.

[torledo]

As for UPS, redundancy is the key. Better to have several smaller capacity than one to service the whole lot, expecially if you have redundant power supplies in the servers.

 

can you explain why ? what do you know about the redundancy of units sufficiently sized and designed to provide backup power and power filtering for all servers ?

 

have you not heard of a 2N configuration ?

 

For the OP, the big four server vendors all make good equipment racks. My preference would be hp or ibm.

 

apc also do a good range, but most rack vendors have a number of cable management options so nothing stands out in that regard. some racks are wider to accomodate cable routing channels, but i don't think those are necessary in most cases.

 

i would say split rear doors are a nice feature, and a size below 42u and above 24u is a good size for getting under doors, getting to top equipment in the rack and having enough room to accomodate a small server estate. hp's 36u racks are possibly the most convenient height.

[wesleyw]

2N configuration?

 

Wes

[torledo]

2N configuration?

 

Wes

 

sorry, i meant N+1 when using a single ups.

 

2N is the configuration for having dual ups. It's not the case that you need dual UPS for a redundant configuration with servers that have dual power supplies. you could have dual power strips but they can both go into a suitably sized ups. infact many larger orgs use a single big ups with an automatic transfer switch upstream connected to mains power and generator for power source redundancy.

 

there may be a cost implication for going with a larger single ups vs two or more smaller ups but there's enough redundancy.

[Soulfish]

While I'd love to stick A/C in now it's something SMT want to hold off on until we know more about the temps in the room with the servers in there. The current location doesn't have A/C either and is currently sitting at around 30C (without any of the new servers being switched on!), which is one of the major considerations in moving the servers.

 

Having never been near our boiler when it's on I've been assured that the room stays cold throughout winter and summer, and I've also been assured that the water pipes are on the opposite side of the (very) large boiler room. The floor is being raised due to a large concrete plinth that is too difficult to remove, so the servers will be at a much higher height than the boiler room so hopefully if the water pipes do burst we'll have some time to respond before the room floods totally!

 

Which UPS manufacturers (other than APC) are people using? Or is it just worth sticking to the APC range of UPS equipment?

[somabc]

What is a 2N configuration?

 

For non redundant UPS systems, a 2N configuration can be used to achieve high levels of availability. In a 2N configuration, the load is connected to two UPSs via an Automatic Transfer Switch (ATS). Each UPS is connected to two separate AC lines (each UPS receives power from a separate power source) and the ATS monitors the power quality from both the primary and redundant UPS and automatically switches the load to the redundant UPS when it detects a potential disruption to the load, such as a UPS failure or replacement.

[timzim]

I've got to agree with Alex - why would you want to site your servers next to your boiler room? I presume the boiler room is on the ground floor so what is the drainage like around the room?

 

In terms of H&S I'm not sure anyone called in to fix a leaking boiler would be particularly enthusiastic to know you've got a whole lot of electrical equipment in the room adjacent either

 

Sounds like someone needs to do a risk assessment...

[srochford]

With or without aircon, you need some kind of environmental monitor so that if the room gets too hot then systems get shut down cleanly. You can also get the electricians to install the aircon supply in such a way that if it fails, the power for the servers gets cut off. UPS will then take over but they will then shut down the servers before everything melts.

 

There's a whole raft of monitoring stuff which you can get that just rack mounts and gives SNMP info but I found this at Dartmount college which gives a rather simple DIY system. the web site refers to US suppliers but I think homechip sell all the bits you need

[john]

I would look at the APC stuff (well the MGE stuff seen as MGE own APC) I'm still partial to the MGE onces which look very impressive.

[Batman]

The Aircon failed in our server room this summer with only 5 servers 5 switches and 2 ups running in there and as server rooms go its quite large. We came back in on monday (it had failed over weekend) and you could have cooked an egg on the plaster board wall, the other tech opened the door and burn his hand on the door handle and thats not next to a boiler room, other consideration is when boiler starts leaking and your server room is under water will the insurance pay out not the ideal place to have it.

 

We have a lot more stuff than that... 20 servers, a big stack of switches... two ups units. Our air con got switched off. The lino on the floor melted, the feet of the racks are now sunk about a centimetre into the floor! We were extremely lucky, the metal covers came off the sprinklers so had the heat risen a few more degrees it could have been very messy.

 

All the servers except one powered themselves off when they got too hot, and all except one came back on with no problems... yup... it had to be the SIMS server that went down. You can guarantee if something's gonna break it's gonna be SIMS!

[sdc]

We have Riello UPS devices (two - one in each server rack). They're excellent - they include digital readouts on the front panels, giving a huge amount of information (more is available via the management software), and for our 17 servers, most of which are pretty powerful, we get two hours' runtime after power failure. I've set them up to power down when only 15 minutes remains in the UPS batteries. Servers with dual power supplies are fed from both UPSes.

 

This is the model we've gone for: Dialog Dual

 

We purchased from UPS Systems PLC, who were very competitive and most helpful.

 

Hope that helps!

 

PS - we also have APC SmartUPSes in critical comms cabinets.

[Andrew_C]

In terms of H&S I'm not sure anyone called in to fix a leaking boiler would be particularly enthusiastic to know you've got a whole lot of electrical equipment in the room adjacent either

Oh for pity's sake! Do you not think that a boiler room will have plenty of power in it anyway? How do you think pumps run, or boilers are controlled?

 

If SMT won't spring for A/C up front, insist on some decent air handling fans. In the UK it is seldom over 30C outside, so pulling fresh air in will help keep temps under control. These (two for redundancy) would need sensors to detect failure, but that is trivial to achieve.

[torledo]

Oh for pity's sake! Do you not think that a boiler room will have plenty of power in it anyway? How do you think pumps run, or boilers are controlled?

 

If SMT won't spring for A/C up front, insist on some decent air handling fans. In the UK it is seldom over 30C outside, so pulling fresh air in will help keep temps under control. These (two for redundancy) would need sensors to detect failure, but that is trivial to achieve.

 

great post. stumbled across an article yesterday about using fresh air ahu's to save a bundle on A/C costs....but presumably the feasability of fresh air cooling would reach a limit depending on size of the room, total btu etc. ? is fresh air cooling a better than nothing approach.

[torledo]

We have Riello UPS devices (two - one in each server rack). They're excellent - they include digital readouts on the front panels, giving a huge amount of information (more is available via the management software), and for our 17 servers, most of which are pretty powerful, we get two hours' runtime after power failure. I've set them up to power down when only 15 minutes remains in the UPS batteries. Servers with dual power supplies are fed from both UPSes.

 

This is the model we've gone for: Dialog Dual

 

We purchased from UPS Systems PLC, who were very competitive and most helpful.

 

Hope that helps!

 

PS - we also have APC SmartUPSes in critical comms cabinets.

 

how many battery expansion packs do you have to provide 2 hours of runtime for 17 servers ?

[timzim]

Oh for pity's sake! Do you not think that a boiler room will have plenty of power in it anyway? How do you think pumps run, or boilers are controlled?

 

Durrr, apart from the boiler's electrics being fairly waterproof (waterproof servers??) a boiler room has to have (by law) an electrical isolator switch. Even if you configured this to also turn off power to the server room there's a whole bunch of UPSs in there keeping the lot live.

[powdarrmonkey]

Durrr, apart from the boiler's electrics being fairly waterproof (waterproof servers??)

 

 

You said earlier 'in the room adjacent'. I'd say a wall was pretty waterproof myself.

 

a boiler room has to have (by law) an electrical isolator switch. Even if you configured this to also turn off power to the server room there's a whole bunch of UPSs in there keeping the lot live.

 

No, there's a bunch of UPSs keeping the device side of the system live. The boiler room side is dead as the dodo.

Edited April 29, 2009 by powdarrmonkey

[timzim]

The boiler room side is dead as the dodo.

 

Unless there's a foot of water on the floor (i.e the boiler's leaked). That's my point.

[Andrew_C]

Durrr, apart from the boiler's electrics being fairly waterproof

Have you ever seen a boiler with IP anything rated wiring? I certainly haven't. All a bit academic as the OP says

The floor is being raised due to a large concrete plinth that is too difficult to remove, so the servers will be at a much higher height than the boiler room so hopefully if the water pipes do burst we'll have some time to respond before the room floods totally!

[pgrant]

We fitted our server room out with open racks which be purchased from cablenet. They were a good quality and could supply cable trays and rings as required. They also supplied a rack mounted tft screen with built in 16 port KVM

 

We use a largish APC ups which has extra batteries added to give an extended run time with a nework interface and envirmental monitoring. It was oversized when we purchased it but additional servers have been fitted unexpectely this year and any further servers will need to go on a second.

 

The server are connected to the UPS through a APC Switched PDU this means we can control the start up sequence as we have a server that appears to be reliant on another being up. It can also be used to perform a hard reset with out a visit to the server room.

 

The room has AC which is massivly overrated but it did not add to much to the price to go for the larger one. It is attualy a heat pump so it could be used for heating (not that we want that) because of this there is a subsidy availiable for fitting one.

 

We have a Lascar EL-USB-RT this logs temperature and humdidty and sends an email alert out if the room goes out of range and cost about £35.

 

Finaly if all else has failed there is a temperature sensor that if the room gets really hot cuts the power. The ups will then start a safe shutdown of the room.

[Butuz]

Lol they fresh-aired my server room when I originally managed to get it built. It took 5 years to get it built so I didnt kick up a hissy fit over air con. It sook 5 mins to melt the entire world though....

 

Biggest make ever. 4 Portable air con units later I am still pushing for reliable, powerful, permanant air con.... *sigh*

 

Go for £2k worth of air con now IMHO.

 

Butuz

[Marci]

Spring for aircon now rather than later... ambient aircooling is insufficient... fresh air cooling adds further risks over winter, especially if you shut servers down over Christmas hols etc, unless you can completely close the vents to outdoors...

 

If SMT are against this, there is plenty of research out there online to back up the need. Find a way to hint to them that they're being blatantly ignorant by skimping on it, without being as blunt as I tend to be when I deal with them

 

Enviro monitoring on the cheap: http://www.t-balancer.com/english/bng.htm

Analog sensors can be positioned to monitor and record to CSV files ambient temps, internal case temps etc etc. Can also be used to trigger 12v devices on / off (eg: additional case fans, or solenoids to additional 240v rack fans) - can report all back to MBM5 or similar free mobo monitoring software, and can be used to trigger shutdowns based on temps of any sensor attached to it that you specify.

Edited April 29, 2009 by Marci