VMware, Xen, Hyper-V - Your Experiences?

[Duke]

Hi all,

 

I've been playing with Xen, VMware ESX and Hyper-V for a little while and I was just wondering if there seems to be a consensus of which is most popular in a school environment? If you are using one of these virtualisation technologies in a school/education system or are likely to soon, would you mind voting in the poll to show your preference?

 

If you look at threads like this one then you can see a huge use of VMware in more enterprise-level setups, and certainly everything I've read and seen would suggest VMware ESX is the most powerful virtualisation technology out there.

 

However, here are my personal experiences with the three technologies:

 

Hyper-V - Disappointing, far more complex to set up than it needs to be, and only supports Windows (and SUSE Enterprise) guest VMs. Even in it's Core form, I personally find Server 2008 to be much more work to configure and manage than setting up a ESX or Xen host. If you want to do things properly then you need SCVMM (and a MS SQL server). Costs are good as it will be mostly 'free' for those who have MS volume licensing.

 

Xen - I considered Xen to be the underdog as it hasn't got the large deployments of VMware or the big name of Microsoft behind it, but so far it has really impressed me. It is so unbelievably easy to set up and use, the XenCenter console take two minutes to install and configure, and there's no dependence on a central management server or SQL database to control your setup. Does anyone know where the pool config is stored with Xen - presumably it's replicated across the hosts?

 

VMware ESX - I really wanted to love this product, but so far it hasn't impressed me that much. Maybe vSphere will be better? The ESX Hardware Compatibility List seems pickier than Xen (not a problem if you're buying new servers, but it was a real pain to have to buy new NICs to fit in brand new Dell desktops for testing), it requires a dedicated VCenter Server that needs an SQL database (you can't really virtualise this box so does it not become a single point of failure?), and seems far more awkward to set up than Xen.

 

My Xen setup was as simple as building a couple of hosts (five minute each), installing XenCenter (two minutes), configure a LUN on a SAN (two minutes), create a Pool on XenCenter, add the hosts and a Storage Repository from the SAN (five minutes), tick High Availability (two seconds) and I've got a failover-level setup...

 

With Hyper-V and ESX doing this seems a lot more complicated, or is it just me?

 

Many thanks,

Chris

Edited May 19, 2009 by Duke

[pete]

Poll fail - assumes we're only using one

 

We've two ESXi servers here, but we're moving to Xen on the new setup because they offer more before we have to start paying. I can't justify the cost of "proper" (live motion/HA/etc) VMware as more of our infrastructure goes virtual.

[Duke]

Poll fail - assumes we're only using one

 

Touché! I guess I should clarify and say what is your primary or preferred virtualisation platform.

 

If money was no-issue, which would you have chosen? I'm not in that position (unfortunately!) but I would hope I could find the budget if VMware was significantly better than Xen. I've looked at VMware's 2009 roadmap and watched the Fault Tolerance videos and it always impresses me, but I'm using the 60-day ESX trial right now and it's very difficult to understand why some things are configured the way they are.

 

I want to set up two hosts and have automatic failover between them. To do this I fully appreciate that I will need central storage between the two hosts (iSCSI SAN) and to enable HA. In Xen, I simply add the LUN to the pool and tick a box. So far in VMware I've had to make a cluster, add a VMkernel Network Interface and IP address on each host (why do I have to do this manually on each host when they already have NICs and IPs?), manually enable iSCSI on each host (why not enabled by default?) and I still cannot see where I add a central storage repository for the whole cluster that all the machines can access.

 

To ESX users - What am I missing here? If you've used Xen too, does the Xen way of doing things not seem much more logical? Am I just not fully appreciating the power and control that ESX gives you?

 

Cheers,

Chris

[Guest monkeyx]

We are in the process of switching from VMware server to Promox. It provides an integrated kernel build for openVZ and KVM, Proxmox also adds a great web front end to both these tools for common admin tasks.

 

Supports clustering and live migrations. Were looking at xen but then xensource version did not like our hardware and is not actually open source.

 

Also looked at the Hyper V, but has poor support for Linux and less functionality than Proxmox.

 

Take a look at proxmox is a great project

[tmcd35]

If money is no object then it has to be VMWare ESX all the way. It's definatly the most mature and feature complete of the server virtualisation systems. It's also very expensive.

 

We're currently heading down the Hyper-V route. Purely on the basis that we are a M$ only shop as far as the OS goes. We'll be moving to 2008kR2 and Win7 in due course. This leads to the question of why not Hyper-V?

 

When we virtualised my last school it was early days and VMWare was pretty much the only choice. We had to pay for it as ESXi wasn't available then. Xen seemed too much of a learning curve. Less in the way of management tools for Xen compared to today.

 

If I was to be virtualising my last school, starting a fresh, today then I think Proxmox is the most interesting of the virtualisation platforms out there today.

[pete]

Touché! I guess I should clarify and say what is your primary or preferred virtualisation platform.

 

If money was no-issue, which would you have chosen? I'm not in that position (unfortunately!) but I would hope I could find the budget if VMware was significantly better than Xen. I've looked at VMware's 2009 roadmap and watched the Fault Tolerance videos and it always impresses me, but I'm using the 60-day ESX trial right now and it's very difficult to understand why some things are configured the way they are.

 

I want to set up two hosts and have automatic failover between them. To do this I fully appreciate that I will need central storage between the two hosts (iSCSI SAN) and to enable HA. In Xen, I simply add the LUN to the pool and tick a box. So far in VMware I've had to make a cluster, add a VMkernel Network Interface and IP address on each host (why do I have to do this manually on each host when they already have NICs and IPs?), manually enable iSCSI on each host (why not enabled by default?) and I still cannot see where I add a central storage repository for the whole cluster that all the machines can access.

 

To ESX users - What am I missing here? If you've used Xen too, does the Xen way of doing things not seem much more logical? Am I just not fully appreciating the power and control that ESX gives you?

 

Cheers,

Chris

 

If money were no obstacle, I'd have chosen ESX - I've been using it for a couple of years, a bit more hardware is explicitly supported and the live motion is more polished, plus the vmware forums are miles better (IME) in terms of vmware people participating than the Citrix ones. There's more documentation and user discussions out in the wild and vendors understand it (as opposed to going "we don't support remote/ts installs" when you mention Citrix Xen) and support with their kit.

Edited May 19, 2009 by pete

[Duke]

Thanks for your replies! I must admit that I'd never heard of Proxmox so I'll look into it. I suppose that leads to another question though (and I think pete covers this) - how important do you find the level of support/community/implementation when looking at your virtualisation platform?

 

To me this is a really big issue, as essentially all your data is going to be tied into whatever platform you use. It's not like choosing between HP and Dell servers because at the end of the day it's all x86 standard hardware, whereas virtualisation is a much more proprietary thing. If I'm going to lock my data into a provider then I want to be 100% sure it's got a long-term future, it's already got a wide deployment base that I can go to for advice and configurations, and vendors are aware of it and support it as Pete mentioned.

 

ESX is definitely the winner here so far as I can see, do any of you have concerns about moving your whole infrastructure to a lesser-tested provider like Hyper-V or Proxmox?

 

Cheers,

Chris

[tmcd35]

ESX is definitely the winner here so far as I can see, do any of you have concerns about moving your whole infrastructure to a lesser-tested provider like Hyper-V or Proxmox?

 

Cheers,

Chris

 

None at all. As I said above everything here will be Hyper-V this time next year (hopefully). I've just finished converting some legacy VMWare .vmdk hard drive images (from the VMWare Server product, not ESX) to .vhd files, currently being used with M$ Virtual Server 2005R2 - ready for Hyper-V.

 

If I wanted to change platform at some point in the future, I don't expect it to be too difficult to convert the hard drive to another format.

 

Actually, I wouldn't be too surprised if a standard 'Virtual Hard Disk' format emerges some time soon. I know Suns VMx implementation can support both VMDK and VHD files. I believe Hyper-V also supports vmdk files, but cannot use them for live migrtration and I think there are other limitation on using vmdk's.

[pete]

WRT ease of changing virtualisation technology, any virtualisation vendor that doesn't offer a well-documented and supported means of converting vms from their competitors products is either daft or lacks confidence in their ability to gain market share from their competitiors.

[Duke]

If think if virtual disks become standardised then it'll be a big bonus for everyone. I suppose that until then I'm just a little cautious of converting the entire network to virtual machines with Provider A, that company going broke, then Provider B saying sorry they don't support that particular version of Provider A's virtual disks. The nice advantage of VMware there is that they're the biggest player so probably the safest bet on that front.

[Soulfish]

If think if virtual disks become standardised then it'll be a big bonus for everyone. I suppose that until then I'm just a little cautious of converting the entire network to virtual machines with Provider A, that company going broke, then Provider B saying sorry they don't support that particular version of Provider A's virtual disks. The nice advantage of VMware there is that they're the biggest player so probably the safest bet on that front.

 

That's one of the reasons I discounted Virtual Iron quite early on. They were a very small outift that were trying to compete with Citrix XenServer and VMWare, and while the product (which was also based on Xen) seemed good I couldn't justify going with them. That and there pricing was more expensive then Citrix . Of course now they've been bought out by Oracle so they're probably a pretty safe bet for the time being, provided Oracle doesn't axe the products!

 

I choose to go with Citrix for our Virtualisation requirements as while VMWare is great the cost was just too high compared to XenServer Enterprise. Given that Citrix seems to be in a fairly strong position with their thin client and app/desktop virtualisation systems I wasn't too bothered about them not being as big as VMWare in the server VM marketplace as I believe overall the company is here to stay for the foreseeable future.

 

The VMWare community certainly seems to be bigger than the Citrix one, but the Citrix forums are still fairly good and I believe given time and now that they're starting to put out some products that really can keep up with VMWare we'll start to see some growth in who's using it and the associated third party support sites

[Guest monkeyx]

ESX is definitely the winner here so far as I can see, do any of you have concerns about moving your whole infrastructure to a lesser-tested provider like Hyper-V or Proxmox?

 

Proxmox is a web front front to openVZ and KVM. openVZ is very mature solution for Linux virtualisation. KVM is the less mature but does run 2003/2008/Vista/XP. It does not make full use of hardware virtualisation at this stage though.

 

All 3 projects have great documentation and a resonsive community to provide help. All 3 projects have full source code available so are more open than the commercial distributions!

 

There are also some great guides on how to convert vmware discs etc.

[ranj]

I managed to find this comparison of ESXi and Hyper V, It is a ball of laughs, its done by someone at Vmware but its so funny when he talks about the amount of times you have to restart a server with hyper-V compared to ESXi.

 

It focuses on setting up the hypervisors from bare metal to working hypervisor.

 

Check it out, worth the time and effort!

 

Hyper V - side-by-side comparison

[dhicks]

Proxmox is a web front front to openVZ and KVM. openVZ is very mature solution for Linux virtualisation. KVM is the less mature but does run 2003/2008/Vista/XP. It does not make full use of hardware virtualisation at this stage though.

 

Monkeyx (and anyone else): are you coming along to the open source virtualisation presentation/discussion at the OSS Schools Unconference? See:

 

Open Source Virtualisation | Open Source Schools

 

I was planning to turn up as I'm interested in hearing someone else's experiences of running Xen as their virtualisation system. I'm inclined to stick with Xen for the moment as I've only just got the last server converted to it last week, but I'd be interested in hearing more about Proxmox / OpenVZ / KVM for future reference. I'd also be interested to hear about other people's experiences or suggestions as regards disk storage for virtual machine images - I use DRDB mirrored volumes, but it'd be worth seeing what open source SAN systems are in use out there.

 

--

David Hicks

[binky]

We are still in debate about this, we are planning to get 3/4x Servers connected to an iSCSI san with HA and motion, but the biggest difference between VMWare and Xen (apart from price) is no consilidated backup in Xen! If anyone else uses Xen how do you efficiently backup your machines?

[Butuz]

Waiting for servers and SAN to arrive to do some performance testing.

 

At the moment though am looking more towards Xen as the free version is easy to use and easy to manage multiple hosts (looking top use 2 hosts to start off with and then move to 4), unfortunatly we can not afford the HA and motion features.

 

ESXi you can only manage one host at a time for free, which is not useful.

 

Hyper V I have not really looked at to be honest.

 

Butuz

[dhicks]

how do you efficiently backup your machines?

 

We run Xen on top of Linux LVM, so we can take a snapshot of a running filesystem at any time. Most VMs are split into OS and data drives so we can take an image of an OS drive any time we make a configuration change and data is backed up in the normal way (in our case, a script that runs on each server and copies files over to a central file server every night). I'm sure you could also figure out something involving a binary diff between DD images of VM drive volumes, though.

 

--

David Hicks

[Marc]

With Xen-server free you get the xenmotion free (works only with shared storage) but not the HA. The latter is part of the essentials pack.

 

I'm dying to tryout xen-server properly with a view of replacing our linux host/ VMware server 1.0 setup I built last year but I have hit a significant snag in a lack of VT extension enabled core2 duos for testing!

 

Until I can get something that will run windows guests it's prevented me from seriously looking at xen.

[Theblacksheep]

VMware ESX it requires a dedicated VCenter Server that needs an SQL database (you can't really virtualise this box so does it not become a single point of failure?), and seems far more awkward to set up than Xen.

 

You can VM the vCenter (and SQL) box no problems. Have a search in my 'VMware vSphere' post for an explanation as to why its not a problem or why a single VM doesn't have to be a single point of failure.

 

If you have HCL hardware, its not a problem to setup at all, but I cant compare it to Xen.

Edited June 1, 2009 by Theblacksheep

[Duke]

I managed to find this comparison of ESXi and Hyper V, It is a ball of laughs, its done by someone at Vmware but its so funny when he talks about the amount of times you have to restart a server with hyper-V compared to ESXi.

 

Obviously this is a hugely biased video, but it does make some good points. Hyper-V seems (to me) a huge pain to install. The Hypervisor install should be quick and simple as it's only a virtualisation layer for the hardware. ESX and Xen show this perfectly as they can be set up in under ten minutes easily, Hyper-V is... frustrating. I'd like to say the video exaggerates the steps required, but it's actually pretty similar to my own experiences.

 

Theblacksheep - I think I'm finally starting to get my head around VMware. What seemed like an overly complicated feature set now makes a lot more sense. If you're used to playing with Xen, you just accept that your single physical NIC will happily do everything and there's no real configuration of your hosts needed. Once you figure out all the settings in ESX then you start to appreciate all the enterprise-level features of having dedicated, redundant paths for all your connections on separate physical and virtual networks to keep your traffic flowing properly.

 

Cheers,

Chris

[artfulmatt]

Bungay High School are having an open day for IT Managers/techs to come along and look at Xendesktop/server in action etc.....

 

Citrix will be there as will Getech (a company in Ipswich that specialise in Virtualisation)

 

If your in the Norfolk/Suffolk area you are more than welcome to come and have a look/play

 

Its on the Wednesday 17th June

 

Please email [email protected] for more details

[Duke]

Bungay High School are having an open day for IT Managers/techs to come along and look at Xendesktop/server in action etc.....

 

I won't be able to make this but it's a really kind offer and I hope someone who can attend is able to post their impressions of what sounds like a fairly large Xen install. I've just got my vSphere trial running properly so it's interesting to see how the two compare.

 

Cheers,

Chris

[artfulmatt]

I won't be able to make this but it's a really kind offer and I hope someone who can attend is able to post their impressions of what sounds like a fairly large Xen install. I've just got my vSphere trial running properly so it's interesting to see how the two compare.

 

Cheers,

Chris

 

Well its not "large" Xen install - only 20 machines at the moment, but we have capacity for the moving the whole school across in the future (we have a cycle to follow of replacements)

 

The day has been changed to the 24th JUNE

Edited June 3, 2009 by artfulmatt
forgot something