Build your own server.

[ICT_GUY]

Has anyone here built their own server?

 

I am thinking of it for a few reasons.

 

I am responsible for maintenance of the server, if I build myself it's easy to stock up on a spare motherboard and drives.

 

ECC memory is cheap for self build. £77 for 12 gigs.

 

I get fleeced for drives for dedicated servers.

 

I could happily build a very reliable server with spares for £1500 inc VAT with an intel server mobo, zeon processor 12 gigs of ram, 4 300gb 15k sas drives and a dedicated sas controller with a huge cache.

 

It should be noted that I am a competent hardware builder and have had poor experience with previous warranty problems. I tend to prefer to fix things myself as it's usually quicker. I have included a spare motherboard and hard disk in the above. If there was a hardware issue I am sure that I could rebuild the server within a couple of hours though with the components selected I doubt that it would be a problem. Though I would budget for 2 x PSUs.

 

So has anyone done this and Is my thinking straight? Comparable HP quote came in over £2k and didn't have sas drives.

 

All thoughts appreciated.

Edited October 24, 2012 by ICT_GUY

[glennda]

take a look at http://www.edugeek.net/forums/hardware/103262-post-about-building-google-like-servers-their-school.html

[ICT_GUY]

I was thinking of adding a ssd for the boot partition.

[VeryPC_Ed]

Hey ICT_GUY,

 

I think there are pros and cons for building your own kit, if your confident in building and supporting a self build then it's a great way to save money if your happy to do it.

 

Some people like the safety net of having someone else do all the sourcing, building (and testing) for them and then handling any warranty / support as well.

 

I'd be interested in the spec of the HP quote you got as maybe able to do something comparable (Supermicro / Intel) and cheaper, plus we don't gouge people on the disk costs

 

I'll pm my details if you want to send some info over.

[glennda]

Well yes - Custom build is good option. there is quite a few different ways to do it.

 

what os you going to run ontop?

 

Toby

[ICT_GUY]

I'm going to go for server 2012.

 

The warranty for the disks is 5 years everything else is 1 year but nothing is so expensive it can't be replaced easily (I would keep a spare mobo). Processor spares will be easy to come by as will hard disks.

 

Testing might be an issue but an intel board is pretty damn standard. HP controller cards are easy to use. Building and maintaining I don't see a problem but this is why I am asking for advice! :-)

[Duke]

All thoughts appreciated.

 

Eh, I think it's all about the pros and cons that vary on your dependence on the hardware and the budget available to you. I've built my own servers in the past, but only for testing and development purposes. Once a server/service goes into production it's put on a pre-built server with a warranty. I wouldn't run production-level services on something that I'd built myself unless budget restrictions forced me to, but that's not to say you shouldn't or that it can't be done.

 

Just playing devil's advocate for a minute:

 

Pros

• Cheaper
  
• Easier to service yourself
  
• Can customise to your exact spec
  
• Pay off-the-shelf prices for components, not inflated server manufacturer prices
  
• Can keep hot/cold spares on-hand
  

 

 

Cons

• No HCL or guarantees of components properly working together
  
• No support if you cannot fix an issue yourself
  
• No guarantees of spare parts being available, or information on their expected lifecycle
  
• No central place for hardware/support issues, you need to contact individual component manufacturers yourself and arrange RMAs with separate suppliers
  
• No SLA's in place if you need a hardware/software fix and do not have the required equipment on-hand
  

 

I don't doubt that you're a perfectly capable system builder, but what happens when your RAID controller decides it doesn't like the firmware on your SAS disks, and starts to corrupt data, get read errors, or mistakenly report drives as failed? Is that something you have experience dealing with? Do you think you'll get quick support from the RAID card or disk manufacturer? Is it not quite likely that LSI (or whoever) will blame Seagate (or whoever), and Seagate will blame LSI, and you'll be stuck in the middle with no fix? If I buy a pre-built server from Dell, all the hardware will be properly tested together, and if I have any issues I go straight to Dell who will either supply a fix themselves, or find out which component manufacturer needs to fix it.

 

What happens when you're running lots of critical services on this server and you run out of component X replacements, only to find out that manufacturer Y no longer makes component X, and the next replacement component uses a different chip which would require rebuilding the server? E.g. you run out of motherboards, and the next model available uses a completely different chipset that your OS won't support. I have no reason not to believe that Dell will be able to supply me with compatible hardware for the life of the server. They've done so for servers 6-7 years old, and have a vested interest in components that have a long lifecycle, whereas consumer hardware manufacturers often don't.

 

What happens when you get a hardware issue that you just can't figure out? I'm not saying you're not very skilled, but even the best of us get a problem we just can't fix without more information or support from the supplier of the hardware/software. Who will you go to when you've bought all the components individually, and none of the manufactures guaranteed that their component X will work with someone else's component Y? A Dell server is designed and tested for all the components inside to work together, and if I do get a weird issue then it's their support department's job to fix it.

 

Now, I've only used Dell as an example because that's what we use here. I have obviously exaggerated the issues you might face and the level of support we get from Dell, just to make a point.

 

I'm not saying you can't or shouldn't do it, but you asked for all thoughts.

 

EDIT: There's been a few posts since I started writing this and you've pointed out the 1 year warranty on some components. I can usually have 3-year as a base warranty on a server, and keep extending it upwards of 5 years. How do you and your users feel about running hardware that's out of warranty, even if you do keep a spare of every component on hand? What about the costs involved with replacing them out of your own pocket (well, the school's pocket!).

 

Chris

Edited October 24, 2012 by Duke

[synaesthesia]

We built our own servers - intel boards, intel chassis and redundant PSUs. Very powerful, loads of RAM and literally a 3rd the price of anything from HP or Dell.

[ICT_GUY]

Thank you for your input. Yes I had a problem with the firmware on my HP server. HP were less than useless, and it was only after I found out about there being a firmware issue did I get to the bottom of the problem.

 

Putting together the sas controller and array is my biggest concern.

[Duke]

Thank you for your input. Yes I had a problem with the firmware on my HP server. HP were less than useless, and it was only after I found out about there being a firmware issue did I get to the bottom of the problem.

 

Putting together the sas controller and array is my biggest concern.

 

Hmm, I can see why you'd be tempted down the self-build route if you've had bad experiences with support!

 

Anyway, I certainly wasn't aiming to criticise, just giving you some food for thought.

[Achandler]

You can get custom build and warranties from other manufacturers anyway with warranties, the like of Broadberry come to mind. They are often cheaper then Dell and HP, who unless you buy what they want to sell you, you end up with a huge price increase.

 

I often find nowadays though, that most server hardware and even desktop hardware lasts for longer then it use to, if you don't buy the cheapest stuff you can. Also depending on what your servers are used for, depends on the need for support. If I was building say 4 machines to be virtual hosts and connected to a SAN in a fail over cluster. Then the need for immediate support is reduced because in theory you can keep running on only 3/4 or even 2/4 servers so you have more time to fix the problems.

[ICT_GUY]

Hmm, I can see why you'd be tempted down the self-build route if you've had bad experiences with support!

 

Anyway, I certainly wasn't aiming to criticise, just giving you some food for thought.

 

No criticism taken! Anything that makes me think is good. I have had bad experience with support, but that's HP for you. Half the price of dell for a reason!

[glennda]

No criticism taken! Anything that makes me think is good. I have had bad experience with support, but that's HP for you. Half the price of dell for a reason!

 

You say that - my experience is completely the other way around in terms of costs and support

[ICT_GUY]

You can get custom build and warranties from other manufacturers anyway with warranties, the like of Broadberry come to mind. They are often cheaper then Dell and HP, who unless you buy what they want to sell you, you end up with a huge price increase.

 

I often find nowadays though, that most server hardware and even desktop hardware lasts for longer then it use to, if you don't buy the cheapest stuff you can. Also depending on what your servers are used for, depends on the need for support. If I was building say 4 machines to be virtual hosts and connected to a SAN in a fail over cluster. Then the need for immediate support is reduced because in theory you can keep running on only 3/4 or even 2/4 servers so you have more time to fix the problems.

 

It's a single main server, we are a smallish school running 100 pc's. The self build will be a file server and run AD & DHCP I will probably rebuild one of our old servers as an application server.

 

Failure is my biggest concern which is my thinking for the second mobo. Ram rarely goes bad, neither do processors. A quality mobo is probably going to last 5 years without a hitch. Power supplies are a pain, but self build = picking up a spare easily.

 

So I agree totally with your view.

[synaesthesia]

Spend wiseley on redundancy for power supplies, a decent UPS to back it up, decent backup solutions, a decent RAID setup without failover (and buy in a couple of extra drives for hot-swap) and you're good to go. Buy in decent quality RAM on known quantity motherboards such as Intel and you'll get the power of a decent warranty anyway.

 

From experience do not skimp on the chassis.

We tend to use 3ware raid controllers, like the 9650SE - basic but quick enough even for our large secondary.

[sted]

we used to build servers (and workstations) for schools we support but it became a pain and in cases esp workstations it was cheaper to buy a hp/fuji/whatever and resell than make our own (granted this was because we had to "charge" for build time) servers we used to do intel boards etc but it as a company wasnt worth it with having to cover waranty so we moved to hp and now to dell (as hp drive waranty is pants and they changed it with no notice and they do seem to die and cost a packet to replace with real drives though ive never had an issue pulling the drive out of the caddy and putting a non hp drive in the caddy)

 

As to ssd boot drive on the server why? its not like the server should be rebooted often and once in windows there probably isnt much on the c drive that needs or would benefit from the speed (again thats my opinion and if you have a big sql db on c drive then there may be a benefit but if all you have there is windows and netlogon/print shares seems overkill)

[dhicks]

What happens when you're running lots of critical services on this server and you run out of component X replacements, only to find out that manufacturer Y no longer makes component X, and the next replacement component uses a different chip which would require rebuilding the server?

 

That's why you run virtual machines - the underlying hardware doesn't really matter.

 

Depending on where in country you are, people like Maplin now do a same-day delivery service - here in central London I think they advertise a 90 minute delivery time. If a component fails and you need a new one you simply phone and buy one and it turns up.

[Duke]

You say that - my experience is completely the other way around in terms of costs and support

 

I think I'm in the middle - we've found Dell to be cheaper than HP on the servers we buy and I've had great support from Dell but poor support from HP.

 

Something I meant to mention in my first post but forgot - what happens if/when you decide to leave and your replacement has to manage these servers? If you document everything really well then it's not too bad, but how many of us have the time to write proper documentation for our network and what if they're not as comfortable with server hardware as you are? Again, just trying to raise some thoughts.

 

Chris

[Duke]

That's why you run virtual machines - the underlying hardware doesn't really matter.

 

Depending on where in country you are, people like Maplin now do a same-day delivery service - here in central London I think they advertise a 90 minute delivery time. If a component fails and you need a new one you simply phone and buy one and it turns up.

 

Oh definitely, if you have redundancy at either the OS or Application level then many of the points I raised become less of an issue. I got the impression that this was going to be a physical server, but if it's a VMware/Hyper-V/Xen host then I'm happy to be corrected.

 

Re: Maplin - What's their stock like on server components? I've never used them for PC hardware before, but do they sell things like LSI RAID cards? Only concern would be if they're out of stock on something, but I suppose that's why you keep your own spares in-house.

[jamesfed]

Have built the following servers for work

 

CCTV Server - AMD Phenom II powered and 4x 2TB HDDs for video - cost about £650 and runs a treat (Inside a new Storage Server | my world of IT)

 

Backup Server - Had a Chenbro case and a load of 1TB HDDs donated to us only issue is it had one of the super old Core2Duos that wasn't 64bit - spent about £280 on a new AMD A6, 8GB RAM, mobo and SSD to boot off - it runs like the wind! (Time for a new backup server | my world of IT)

 

RDS Server Farm - Serious custom builds here - 2U cases with 2x mobos in each, fantastic design IMO that I would repeat time after time (Series: Custom RemoteFX RDS Farm « my world of IT)

 

However........for serious work I am sticking to HP

[ICTLTD]

 

Depending on where in country you are, people like Maplin now do a same-day delivery service - here in central London I think they advertise a 90 minute delivery time. If a component fails and you need a new one you simply phone and buy one and it turns up.

 

Problem is in 1-2 years it is unlikely you would be able to get the same MB so a failed MB or CPU could lead to buying a new CPU, Mainboard and Ram (should server boards in X years have different RAM as standard). For about £1750-1800 you could have a Dell tower server with 3Y NBD Warranty for the spec you wanted. I think the security of that may be worth the extra IMHO. (That is using a Current Gen Xeon E5 CPU)

Edited October 24, 2012 by ICTLTD

[SYNACK]

Problem is in 1-2 years it is unlikely you would be able to get the same MB so a failed MB or CPU could lead to buying a new CPU, Mainboard and Ram (should server boards in X years have different RAM as standard). For about £1750-1800 you could have a Dell tower server with 3Y NBD Warranty for the spec you wanted. I think the security of that may be worth the extra IMHO. (That is using a Current Gen Xeon E5 CPU)

 

I agree from a sheer lack of stuffing around POV but most of those issues can be removed by running the hosts as VM hosts and using that to abstract the hardware and storage. I have wasted enough time pissing about with desktop rubbish when it is messing with just one user though and appreciate reliability. Sure desktop gear may be 1/3 to 1/4 the cost but you need to buy three to four times as much to get the same reliability and burn so much power in the attempt you may as well have just brought decent gear to start with.

[localzuk]

We have 3 custom servers - 1 in use, 2 as spares. Its out phone server, and is an ITX box. These will be retired soon though, and the server virtualised and a gateway device purchased to act as the phone line interface instead.

 

With server hardware being so cheap now, putting so much effort into custom building is kind of pointless IMO. Ok, I can't say our latest servers were cheap (I didn't spec or buy them though), but ones that should have been bought would've been about 4k a piece for dual 6 core processors, 100GB RAM and a couple of SAS drives. And they can run everything that we need, retiring 10 old servers in one go.

 

Plus you get the advantage of easy warranty replacements - if your motherboard dies under warranty in a custom build, you'll more than likely end up with a different model sent to you instead - the savings are not really there.