Server Virtualisation..The options? Your opinions & feedback...

[MYK-IT]

We are looking at the possibility of moving over to server virtualisation as and when we get budget to replace our old servers.

Whilst we understand the principles of virtualisation and its benefits etc., it is still a bit of a mind field.

 

In an ideal world, 3 physical servers with 2 SAN appliances would be great, but I have to be realistic (financially) and I am therefore looking for a modular based approach / solution.

 

Possible solution:

2 servers with a lot of RAM and local storage. (As an example; R710/DL380, 8 HDDs, 96GB RAM)

Then as budget becomes available we could purchase a SAN (with HDDs or migrate the HDDs across from the servers).

 

The area that I would appreciate feedback on, is the actual virtualisation process.

 

We have Hyper-V and VMware ESXi (free) installed on 2 computers for testing purposes.

 

Hyper-V

We have successfully configured a couple of virtualised servers. What concerns me is that it is a very big footprint (basically a whole server OS) where you install additional Server OS on top. Applying updates/service packs to the Hyper-V server would mean rebooting and downtime of all P/V servers.

 

What are the recommendations / solutions for backing up the virtual servers?

Snap shots? DPM or use existing backup software to backup VHD files?

 

If we had 2 (or more) physical servers how would we introduce redundancy if one physical server failed?

For Hyper-V would they need to be clustered?

I realise that by having a SAN that would assist with this scenario.

 

VMware vSphere Hypervisor (ESXi)

Although free, we have been impressed with VMware ESXi, small footprint on the HDD and its remote configuration console.

We have again, successfully installed a couple of virtualised servers.

 

What are the recommendations / solutions for backing up the virtual servers?

 

If we had 2 (or more) physical servers how would we introduce redundancy if one physical server failed?

Would we need to purchase VMware vCentre Server to do this? Or is it achievable (but not as easy/instant) with the free version?

I know that the full version, VMware vCentre Server can move virtualised servers to a different physical server in a matter of seconds (with a SAN appliance) and can offer more features.

 

Basically, what I am trying to ask is "What options are available for virtualisation? Including backing up and redundancy!” comparing Hyper-V, VMware ESXi and VMware vCentre Server.

As a guide, we would be trying to replace 5 ML350 G3/G4 servers initially (DCs, DNS, DHCP, Printer Server, AV Server, WSUS Server)

 

I would also appreciate any feedback on how you have virtualised (remembering we don't have an unlimited budget or any come to think of it!) using the (non)free virtualisaton solutions along with any pros/cons you have come across.

 

Thanking you in advance

[stevenlong1985]

Im in the same Boat too! its a huge mind field, I'm currently thinking the best option for me its HyperV route with AMD as you get more cores for the same price either the HP ProLiant DL385 G7 Server or PowerEdge R715 or R815 (two of the same model with a SAN). looking a ball park figure in my mind 25k to 30k. with WMare "I think" the main cost comes from the yearly support which is a lot of money

[TheScarfedOne]

Stay tuned to my blog on here, and links over on Microsoft Schools blog as I will be detailing our solution using HyperV and two R710s, 1 MD3200 SAN. Didnt cost the earth either - I had a very limited budget too!

 

A couple of quick answers tho... yes, cluster your Hyper V. Why? Well, you can then do the rebooting of Physical servers with no downtime. You can let HyperV migrate them around for you. We had this demonstrated perfectly when we had a UPS power board fail taking out a server. Yes, the VM OSs will need updating, but let WSUS take care of that, and reboot for the ones that need it out of hours by scheduled tasks.

 

A quick whistly stop, but any qs message me. There will be articles soon on this. If you are in the southwest - or can get here, you are welcome to come and take a look.

[jamesreedersmith]

Look at XEN

[ChrisH]

Talking of foot prints and Server 2008 does server core Hyper-V run full server 2008 guests? I assume so? When I tried server core recently I set a core and a full VM installing at the same time, the core was asking me to change the admin password while the full server was still halfway through copying files.

[mrbios]

Additionalto your small footprint note on vmware, you can actually run your servers completely hard drive free if you store your VMs on iSCSI and have an internal pen drive slot.

 

We're running a few VM hosts off driveless servers with an internal pen drive, no performance lost with this method and no need to splash out on drives

[MYK-IT]

Additionalto your small footprint note on vmware, you can actually run your servers completely hard drive free if you store your VMs on iSCSI and have an internal pen drive slot.

 

We're running a few VM hosts off driveless servers with an internal pen drive, no performance lost with this method and no need to splash out on drives

 

I had read about VMware being able to be installed like that, even on a SD Card installed internally.

HDD-less physical servers would mean money towards a SAN.

 

What version of VMWare are you using? and backup methods?

 

Thanks

[mrbios]

I had read about VMware being able to be installed like that, even on a SD Card installed internally.

HDD-less physical servers would mean money towards a SAN.

 

What version of VMWare are you using? and backup methods?

 

Thanks

 

Backup Exec 2010 R3 is doing the backups with the vmware license It's a little bit problematic at times but that's partially my fault for constantly breaking backup exec

 

We're using vSphere 4, apparently 5 is more expensive so i doubt we'll move to that any time soon. My opinion of vmware is that it's fantastic, so many possibilities, the only drawback is that its bells and whistles come at a cost.

[gshaw]

If you can get VMWare with a 3yr support bundle it'll work out a bit cheaper and means you're sorted for a good length of time. We also have our hypervisor installed on internal memory card (DL380 G7) so that's 6 less spinning disks over the 3 hosts

 

Hyper-V will do the job but I trust VMWare as a platform more. Also with the hosts if you can stretch to 3 it's a better configuration as if one goes down you're not putting 100% of the load on a single box...

 

One other point about the backup, Microsoft makes a neat package with Hyper-V \ DPM 2010 as a fully supported 100% Microsoft setup (at minimal cost for education) whereas VMWare would need a Backup Exec or Veeam which offer their own additional features but at a price.

 

As I found out this summer, there's no right or wrong path, just what you feel more comfortable with and how much you have in the budgets...

Edited October 4, 2011 by gshaw

[ChrisH]

I have my Esxi servers running of Pen drives too. I have some spare as well ready to put in if one goes wrong. Some people really don't like the idea of that single point of failure but it works fine for us.

[hfnistelrooy]

I am looking to virtualise one of our servers (file server so if anything goes wrong nothing major as long as my backups are working) now i have tested VMWare ESXi and its light, my question is VSphere asks for a licence - do I have to purchase VSphere separately to be able to manage the VMs?

 

Also is there ANY way that the VMs can be view on the actual server ?

[dhicks]

Possible solution: 2 servers with a lot of RAM and local storage. (As an example; R710/DL380, 8 HDDs, 96GB RAM)

 

That's a good solution for a school - at no point is your need for processing power likely to outstrip your need for storage. A standard Windows network with client machines doing all the processing mostly needs lots of network storage space, you aren't running a processor-heavy web server farm. If you have the money to get a new server in the first place you might as well get one capable of a decent mount of local storage.

 

You might want to get a separate NAS device as your main storage for your user's files - I would simply make my own, but if you don't like that idea then Qnap devices are pretty resonable. Ideally, have two NAS devices, in separate physical locations, one with smaller, faster storage (maybe proper hardware RAID) as your main file server, the other with larger, slower storage as a backup server. Do regular (twice a day?) syncs between the two and set the backup server up as a read-only version of the live file server, that way people can restore their own files without having to bother you.

 

If your main servers can have local storage for the VM images you'll have no performance issues from the storage being on the network someplace. I prefer the open source version of Xen as a virtualisation platform as it can run both Windows and Linux VMs, but can also run Linux VMs in paravirtualised mode which makes it quite efficient. If you have two servers you want to mirror, one as a stand-by for the other, you can do that very easily with DRBD, but bear in mind that you shouldn't mirror things like Domain Controllers - they have their own duplication method and it's best to let them get on with it.

 

Processor-wise, I prefer AMD as all their processor range supports hardware virtulisation - even their low-power Brazos chipset, which would be perfect for a NAS device.

[garethEds]

IN the same boat here.

 

I asked on another forum post what hardware people were using. Many replied but I almost fainted at some of the stuff. Where the heck to people get their cash from? Where do schools get their funds?

 

So at the moment we are looking at getting a recon server from @ict-direct with two 500Gb drives in that we will mirror. We will install 2008R2 on there with the free VMWare ESXi software. I've already run the VMWare Converter on one of our servers and it took the image without any issues (it was a non critical server with three partitions).

 

The servers we will be virtualising are not mission critical and will just hold SuccessMaker Enterprise, Sibelius License server, Impero. At the moment because we have an embarrassment of riches after leaving RM and going back to the LEA for our network. We've gone from 4 RM DC servers down to one fileserver and authentication across the broadband network to the central LEA DC servers. It works well.

 

Our Leccy bill is huge as a school so I think pulling all the servers down into one will work for now. I may eventually get another server and virtualise our pCounter server - but this is in the thinking stage.

 

I'll then sell our RM servers (which still work) - or will make up a super 32 bit server.

 

I have no cash or funds in the central ICT budget :-)

 

As for backing them up I'm going to rely on the mirroring that the server will have but will look at backing up somehow.

 

[Edit: As a side note I'm interested in the mirroring of servers. If I bought two servers how would I have fall-over if one of them should die?]

 

Gareth

[dhicks]

Sibelius License server

 

In my experience, it does have a tendancy to crash and take the whole virtual server with it - not sure if that's anything to do with virtualisation as such or wether it just conking out on a regular basis is what it does.

 

As for backing them up I'm going to rely on the mirroring that the server will have but will look at backing up somehow.

 

I'm sure others will point out that mirroring is not a backup - delete a file on one half of the mirror and the mirroring system quickly and effciently deletes it on the other half, too. If by "mirroring" you mean RAID of some description then that's a good start, but ideally you should have some kind of diasaster recovery facility (server conks out, how do you restore it / get another to take over) and file/email/etc backup so people can restore stuff they delete by accident.

 

Edit: As a side note I'm interested in the mirroring of servers. If I bought two servers how would I have fall-over if one of them should die?

 

My experience is with Linux servers, in which case you install DRBD to mirror harddrives between two machines (like having a RAID-1 array split over separate machines) and have the second machine take over the running of the VM should the first one stop for some reason. You can do that manually, or use Hearbeat to do automatic failover for you. Some VM systems will also transfer live, running VMs between different physical machines for you, but as a failover solution for a school this is probably overkill - a service coming back up a few minutes after a server fails while a new VM boots is generally perfectly acceptable.

[webman]

In my experience, it [sibelius] does have a tendancy to crash and take the whole virtual server with it - not sure if that's anything to do with virtualisation as such or wether it just conking out on a regular basis is what it does.

 

One virtual server we have runs various licensing apps, including Sibelius, and works without a flaw. That's Windows 2003 on XenServer attached to a Sun 7110 SAN.

[garethEds]

In my experience, it does have a tendancy to crash and take the whole virtual server with it - not sure if that's anything to do with virtualisation as such or wether it just conking out on a regular basis is what it does.

 

Ohhhhhhh - that's not good. I'll monitor the situation when it happens. It will be the latest Sibelius version - 7. Ordered it the other day.

 

I'm sure others will point out that mirroring is not a backup - delete a file on one half of the mirror and the mirroring system quickly and effciently deletes it on the other half, too. If by "mirroring" you mean RAID of some description then that's a good start, but ideally you should have some kind of diasaster recovery facility (server conks out, how do you restore it / get another to take over) and file/email/etc backup so people can restore stuff they delete by accident.

 

No - my mistake. I know what you mean. So really is it not worth me mirroring the drives? What if I put the OS on one drive and the virtual machines on another drive? Then look at backup solutions? Or stick it all on one drive and get some backup software to back it up to the second drive? Or should I just bite the bullet and backup to a seperate NAS box? As you can guess I'm new to this virtual stuff. Our main fileserver gets backed up across the broadband network every night with the files being sent to LEA Towers. It's been a long time since I had to worry about AIT3/4 tape backups on our RM DCs :-)

 

My experience is with Linux servers, in which case you install DRBD to mirror harddrives between two machines (like having a RAID-1 array split over separate machines) and have the second machine take over the running of the VM should the first one stop for some reason. You can do that manually, or use Hearbeat to do automatic failover for you. Some VM systems will also transfer live, running VMs between different physical machines for you, but as a failover solution for a school this is probably overkill - a service coming back up a few minutes after a server fails while a new VM boots is generally perfectly acceptable.

 

I thought about looking at the Zen stuff but the Physical to Virtual application costs some money (not sure how much). The heartbeat sounds interesting as well. Will research that tomorrow.

 

Thanks for your reply,

 

Gareth

[Mr.Ben]

I'm only a small school, but i have been using Hyper-V since it appeared in 2008.

 

My setup is 2x Hyper-V hosts running 4 Virtual Servers each. Both hosts have local RAIDed storage, with 2 partitions (one for the OS, one for Virtual Machines). I have 4x network cards in each server and 3 are used between the 4 virtual machines. Approximate cost of the servers was £3k each

 

We don't have a SAN, and Hyper-V currently lacks some of the more advanced features that vSphere and Zen have. i am looking forward to Windows Server 8, which will catchup with some of these features

 

Backups are done to an iSCSI drive using DPM, and can be restored in a couple of hours.

[mrbios]

I'm only a small school, but i have been using Hyper-V since it appeared in 2008.

 

My setup is 2x Hyper-V hosts running 4 Virtual Servers each. Both hosts have local RAIDed storage, with 2 partitions (one for the OS, one for Virtual Machines). I have 4x network cards in each server and 3 are used between the 4 virtual machines. Approximate cost of the servers was £3k each

 

We don't have a SAN, and Hyper-V currently lacks some of the more advanced features that vSphere and Zen have. i am looking forward to Windows Server 8, which will catchup with some of these features

 

Backups are done to an iSCSI drive using DPM, and can be restored in a couple of hours.

 

Just like to point out that you'd probably find little to no performance change with a single network card in each server rather than one per VM, and vlans setup where required a 1GB link to a host is more than sufficient in the vast majority of cases.

[dhicks]

Ohhhhhhh - that's not good. I'll monitor the situation when it happens. It will be the latest Sibelius version - 7.

 

Admitedly my experience with Sibelious is now a couple of years old - I imagine it works fine now.

 

So really is it not worth me mirroring the drives?

 

Hang on - what sort of "mirroring" are we talking about? For any server, having your local storage be a RAID array is a good idea. If you're buying a new server you should probably aim for something with proper hardware RAID (not Windows-only fake RAID that needs a driver, but a proper dedicated RAID card - that'll probably cost you around £300). Maybe have a RAID 1 array for smaller, faster OS partitions and a RAID 6 array for data storage.

 

If you mean mirroring accross the network, I think an backup solution is more practical for a traditional file server - there's little point in having a live block-level mirror of a file server unless you really need super-reliable uptime, your resources are better utilised making a backup that can restore people's deleted files.

 

I thought about looking at the Zen stuff but the Physical to Virtual application costs some money (not sure how much).

 

It might be cheaper, quicker and simpler not to bother - simply set up new Windows servers as virtual machines. A DC should be easy enough as you just have it replicate data accross, and things like printer servers and applications servers can be moved accross bit by bit.

[nicholab]

Dot hill might be able to do something in your budget I would also consider getting my servers second had from ETB-tech so I could spend more on the SAN. My person view is that most school should be able to run their server infrastructure in one 42u Rack.

[pantscat]

I've never had any issues with Sibelius licence server running on a Win2003 VM on an ESX 4.1 host.

 

You'll probably be quite surprised once you get into as to how "affordable" server virtualisation can be (I was in a position whereby I was comparing it with replacing approximately 8 physical servers which were all rather old/stupidly specced/not-a-real-server). I'm running two hosts with a dedicated SAN and as the discussions with suppliers went on the prices came down further and further. Plus it helps if you can be flexible when you purchase - if you can buy when it's the hardware manufacturer's quarter end you'll probably be able to get some very good prices. I did with HP and NetApp...

[MYK-IT]

Stay tuned to my blog on here, and links over on Microsoft Schools blog as I will be detailing our solution using HyperV and two R710s, 1 MD3200 SAN. Didnt cost the earth either - I had a very limited budget too!

 

Hi, any update on this blog TheScarfedOne?

 

As far as our virtualisation proposal is concerned, we have seemed to narrowed it down so far to 2x R710s, 1x MD3200i with potentially VMware Sphere 5 Essentials Plus.

 

But there are still other outstanding concerns/issues being:

 

Amount of Hosts (nodes)

Whilst 2 nodes are great, many have suggested n+1 (as if 1 node died, could the other node cope with the additional virtualised servers?)

 

Backups

Majority of solutions are now D2D. We are currently using Backup Exec 12 (combined with a Dell TL2000 LTO4 Tape Library) so the backup software would need updating anyway.

 

But with Backup Exec 2010 agents in the region of £700 per (virtualised) server, that is a huge chuck of the budget.

Alternatively, Microsoft DPM looks very appealing and cheap and I know that it works fine with Hyper-V, but I assume it could be used with VMware? (install DPM backup agent on each virtualised server combined VMware/NAS snapshots?)

 

I was planning on using our existing backup server (Dell 2950) for the (newer) backup solution, but since it would be D2D (then 2T) the current local storage capacity would not match that of the proposed SAN - nightmare!

This then gets me thinking; would we then need either an additional SAN (for backup) or another storage appliance of same (or more) capacity.

 

Domain Controller

Much debate online whether you 'should' still have a physical Domain Controller or not. Pro/Cons with both.

 

I've thought about having a virtualised Domain Controller on each node (but on local storage) so if the SAN or other host goes down the domain is still available.

Alternatively having a low spec 1U server (Dell R310) purely for a physical Domain Controller.

I was planning on the Dell 2950 Server as a Domain Controller, but found out that Microsoft DPM cannot be installed on a Domain Controller!

 

Hyper-V/VMware

I've not ruled out Hyper-V, so still open to suggestions/recommendations etc from people using it.

 

...and breath and relax..

I fully appreciate the benefits of virtualising servers but as mentioned there is no one solution fits all, but there is huge pressure (on the budget) to ensure that all areas are covered and to get it right.

Virtualisation is meant to reduce the amount of physical kit, but before you know it you could easily end up with 3 hosts, 1 domain controller, 1 (or 2) SANS, backup host (with additional storage).

 

Sorry, rant over.

Edited November 23, 2011 by MYK-IT

[m25man]

We run 3 node Hyper-v clusters, ReadyNAS 12 Disk iscsi arrays

Servers are HP dual quads or hex core 32/48 GB 4 or 6 Nic ports

Dedicated L3 10GBe switch for node management and iSCSI

Backup using BE2010 Hyper-V edition

We settled for DL380's as they are generally quieter than their 1u counterparts.

2008R2 Datacenter for the hosts as there is no limit on the number of VMs per host having a pair of 146Gb SAS drives means the OS can run the full GUI which makes things pretty easy to manage from virtually anywhere.

These configs have been thrashed and over the past two years on we haven't had a single issue.

 

We are seriously considering dumping the BE as DPM is so much cheaper on a schools agreement and it integrates with SCCM & SCOM 2012

[glennda]

I run a 5 host system using linux kvm and an Oracle File System cluster. connected via multiple trunked 1gb network cards. Hosts are a mixture of DL380 G5's and DL360 G7's with a range from 16gb to 64Gb ram each.

 

From my stress testing i did KVM came out quicker then vmware

[RTFM]

We have just put in a new san, 3 hosts and 2 switches.

 

The San is a NetApp FAS 2040 with 15k SAS drives for server storage and SATA drives in a seperate disk shelf for 'slow' storage.

 

The hosts are all HP DL360 G7 with 48gb RAM each, opteron processor, quad nic's, redundant PSU etc.

 

The switches are HP Procurve.

 

We then have a 4th host in another building which is a HP G6 and has an MSA60 and auto loading tape drive attached to it so we can put backups to tape using Backup Exec 2010 and put them into a fire proof safe in a third building.

 

The hosts are running ESX with VMWare on it.

 

Got installed last week and is very sweet!