Server Spec for Virtualization

[Iain]

Hi all,

 

I've just been given the go ahead for updating our servers, and have decided on the virtualization route for consolidation. I am after peoples opinions on what spec servers I should get for hosting the VMs.

 

First a bit of background, we are a 3 form primary with approx, 480 users, and 160 PCs / Laptops. We currently have 9 physical boxes serving our network, with all the usual roles: DC x 2, DHCP, DNS, WSUS, App / Print Server, Antivirus, MS SQL, MIS, Exchange, Web Server, and Spam / Web filter, etc.

 

I'm eventually looking at virtualizing all these roles, although I'm planning on keeping one of the DCs as a physical box.

 

Our current storage usage is around 300GB inclusive of user areas / profiles / exchange / shared areas / WSUS etc. but excluding OS space requirements.

 

Given our current space usage, and projected growth, I cannot really justify the cost of going down the SAN route (although I would like to in the future). So I'm thinking local replicated storage (drbd, etc.) which should also allow for failover in case of hardware failure.

 

Reading through similar threads, I have cobbled together the following spec on which to host the VMs:

 

Dell PowerEdge R710 x 2

 

2x Intel Xenon E5530

16GB (RDIMM)

PERC H700 RAID

Raid 1 (SAS) For host OS

Raid 5 (SAS) - undecided on size) For VM storage

iDRAC6 Enterprise

Embedded Gigabit NIC 4 Port

2x Intel Gigabit NIC 4 Port

 

Now it's been a while since I've had to spec any hardware, and I don't have too much experience of server virtualization, so would appreciate any input that anyone can give.

 

Appologies for the verbose post!

 

Thanks,

 

Iain.

[strawberry]

if your going to virtualise you need to look at some sort of redundency, if your using hyper v for isntance using a cluster would give you redunancy and options for moving vm's

[john]

In terms of the Servers to host the VMs, I've got dual Quad Core Xeons, 24GB ram, Mirrored 10K SAS Drives for the base OS (Citrix XenServer in my situation), 4 NICs (so I can use 1 for Management, 2 for Network and 1 for the SAN connection), they also have full remote iLO (Remote Management in HP Lingo) couple with a 3 year on-site warranty.

 

For my SAN, I will be using NFS to access the storage, and am using a Sun S7110 4TB version, which by what you have said isn't an option, but you could got for a smaller product from somoene like Drobo, Qnap or Netgear perhaps?

[tmcd35]

I'd say your Dell spec's look fine. I would question the need for 12 NICs in 1 server? I'd also advocate using a cheap iSCSI SAN rather than internal RAID-5 storage for the VM's.

 

In terms of Spec, the only thing I'd double check is that there is enough CPU/RAM to run all the core VM's required to keep the school running on one host if one of the servers failed.

[apaton]

Hi all,

 

Dell PowerEdge R710 x 2

...

Raid 1 (SAS) For host OS

Raid 5 (SAS) - undecided on size) For VM storage

 

Why waste two whole expensive SAS disk for OS?, if using VMware ESXi/ESX then you don't need an isolated disk for the OS/Hypervisor/Console VM. Just use all the disks (Consider RAID 5 or 50), you will get better performance and greater capacity for your VM's.

 

Andy

[Iain]

Thanks for the responses so far. I'm hoping to achieve reduncancy, and live migration by getting two servers, and replicating their local storage using drbd, then set up a HA cluster using xen / pacemaker / openais. Andy, good point about using one raid 5 array with more disks - something I need to consider, thanks. tmcd35, I was thinking that many nics, as I was planning on using bonding to achieve redundancy / higher throughput for the drbd and pacemaker links, but yes I think I may have over speced on that one.

[Duke]

My main suggestions would be more RAM and centralised storage. I'd look at 48GB RAM and the Oracle S7000 Unified Storage personally. If possible I'd also strongly recommend three servers rather than two - generally people will run their virtual hosts at fairly high capacity (if you're not close to maxing them out then you're not really getting all the benefits of virtualisation), if you have two hosts and one fails then there won't be enough RAM/CPU in the other box to handle everything. Three, or n+1, hosts makes this easier to manage in the event of a failure.

 

Local storage and drbd may well work, but I'd really, really, really recommend a SAN (iSCSI) or NAS (NFS).

 

I've worked with Andy (apaton) before and can very much recommend his services if you've got some spare budget.

 

Chris

[teejay]

What virtualisation platform are you looking at? If VMWare then locallly stored disk images are limited to 250Gb, which might not suit your needs.

Have a word a chat with Dell, they do some cheapish iSCSI stuff and you could take out all the hdd's and Raid controller on the servers and run the VM software from flash which is an option on the R710's, which might end up costing not a lot more.

Memory is a bit low, VMWare will overcommit memory but it's not ideal to do this too much, look at 32Gb+. Hyper-V if I recall cannot overcommit memory, not sure on XenServer.

As for NIC connections, I've done the same as you and do use them all :-)

I really would recommend going with an iSCSI SAN, you really will regret it if you don't as it adds so much more flexibility. Also, stick with the free virtualisation stuff to start with, it does pretty much all you need in a school.

[3s-gtech]

With VMWare, I bought an HP DL385 G5. For the slightly less demanding roles it plays I stuck with internal RAID-5 SAS, but the host OS boots from an internal USB memory stick - plenty quick enough, but also very cheap and easy. Just make sure it's a good one!

[zag]

I've written about this in other threads but for me I would get 3/4 cheap dell rack servers and build the redundancy into them rather than getting 1 or 2 epic servers.

 

You can spec one up on dell that easily comes under £1,000 with 16gb of RAM.

[Iain]

Thanks Chris, Yes I really would like a SAN setup however, I can't really justify the cost of a redundant SAN solution. Although I have considered taking two of the newer existing servers and putting in a load of disks and builing a SAN from them using openfilter. As for the virtualisation platform, I'm going down then XEN route.

[maark]

Have been using R710 similar spec to yours with hyper v and they have been great - got 4 virtual servers on each at the moment and they are hardly breaking a sweat.

[tmcd35]

Thanks Chris, Yes I really would like a SAN setup however, I can't really justify the cost of a redundant SAN solution. Although I have considered taking two of the newer existing servers and putting in a load of disks and builing a SAN from them using openfilter. As for the virtualisation platform, I'm going down then XEN route.

 

I would say, depending on drive speeds, etc, that retro-fitting an old server as a SAN box would be better than using local storage. I wonder somtimes if we go overboard on systems redundancy? Redundancy costs and although ideal do you really need a redundant SAN box? If it goes down, so long as you keep good backups, you shrug your shoulders and do your best to get it back in a day or two.

 

Instinctively I dislike the idea of mirroring drives across servers like that. Either they are going to be constantly mirroring the data and you'll be taking one hell of a performance hit both across the network and on the drives themselves as the servers struggle to keep the images in sync. Or the images on the two drives are going to be out of sync by half hour/hour/day/how ever often you set them to resync. If one server goes down there will be inconsitances in the VM images and data loss.

[Iain]

I would say, depending on drive speeds, etc, that retro-fitting an old server as a SAN box would be better than using local storage. I wonder somtimes if we go overboard on systems redundancy? Redundancy costs and although ideal do you really need a redundant SAN box? If it goes down, so long as you keep good backups, you shrug your shoulders and do your best to get it back in a day or two.

 

Unfortunatly I think I would be for the chop, if I put into place a system that could potentially leave the school without it's entire network for a couple of days! Also, my concerns for retrofitting the old server with drives are that it only takes the old U320 SCSI drives and I'm not sure they'll give enough performance for a SAN.

 

One of my reasons for thinking local replicated storage was that, looking at our usage, most of it seems to be data reads rather than writes, so I wouldn't have thought drbd replication would have too much trouble keeping up.... something for me to think about further though. Thanks.

[Duke]

I wonder somtimes if we go overboard on systems redundancy? Redundancy costs and although ideal do you really need a redundant SAN box? If it goes down, so long as you keep good backups, you shrug your shoulders and do your best to get it back in a day or two.

 

In an ideal world I say redundant everything, with the dream being something like the diagram below (yeah I know, MS Paint - it's Friday!)

 

http://www.hardenhuish.wilts.sch.uk/avatar/redundant.jpg

 

The two SANs and VMware clusters are in different locations, obviously in reality there would be more switches and redundant network links on everything. (clustered) vCentre monitors the clusters and handles failover between the hosts, the SANs have clustered heads and do synchronous replication with snapshots, everything's on UPS, etc. etc.

 

However, we're on school budgets and it's not an ideal world so most people aren't going to manage this.

 

My concern with doing virtualisation on a budget is that you're putting all your eggs in one basket and actually (potentially) making reliability worse. At present, if one server fails you lose one server and a few services. If you're hosting your entire network on two virtual hosts and a SAN then if one of the hosts goes down (and you don't have to auto-failover, or the other host can't handle everything by itself) or the SAN dies then you've lost the entire network. Sounds like a step backwards rather than a step forwards?

 

Chris

Edited July 9, 2010 by Duke

[brockbarn]

Post edited due to unauthorised advertising, as per the AUP you agreed to on sign up. Please note only sponsors can use the forums in a commercial capacity.If you wish to enquire about sponsoring please contact us.

 

Dos_Box

Edited August 6, 2010 by Dos_Box