Xen Virtualisation - Who Is Using It In Production?

[Richie1972]

Later this year we are building a new server core and I'm going down the road of virtualisation, redundant core switches and a dual SAN with High Availability (Openfiler with DRBD).
So far my tests have all revolved around Linux Xen (which is proving to be very easy, stable and nice to setup). My problem, at the moment, is that my boss is having doubts about it and talking about just putting in standard, basic, servers
What I'm looking for is a list of schools, companies, etc that use it and are happy using it (some online documentation to back it up would be excellent).
I, personally, don't see the problem as the virtual clients will mostly be running Windows 2008 server - but I think his concerns are that I'm looking at something very much "non-standard" (he's worried that when BSF takes over I'll be kicked out or re-deployed elsewhere - not to mention the concerns of what would happen if I had an accident).
Assurances that the whole setup will be fully documented (with "noddy guides" and pictures too!!!) haven't completely satisfied him, so any help very much appreciated

[kmount]

Hi Richie,

I'm toying with Xen currently; finding it more robust than openvz but I've not done too much with it as yet.

[AngryTechnician]

A friend of mine is the MD of Memset, a successful web hosting company who use Xen extensively to provide virtual 'dedicated' servers.

Among other things, they host Private Eye's website

[powdarrmonkey]

We have everything except our MIS on Xen in production, and that will be too when we can find a good time to do it. What do you want to know? - visit if you like.

[Richie1972]

@AngryTechnician: thanks I'll use that as a reference
@powdarmonkey: thanks for the offer, but I'm really just looking for the info to make my case. I'm looking to try SIMS virtualised too

I'm trying to build up a good written case to present to my boss to allay his fears that I'm building on something not tried and tested, reliable and industry used.

[j17sparky]

Amazons EC2 compute cloud (i think im right in saying one of the biggest network of VMs on the planet) is built on xen

[SYNACK]

dhicks is using something simmilar and may be a good person to talk to when he is next on:

Your Virtual Setups

[tom_newton]

A friend of mine is the MD of Memset, a successful web hosting company who use Xen extensively to provide virtual 'dedicated' servers.

Among other things, they host Private Eye's website

They also host one of the loadbalanced servers for our website on xen IIRC. Can ask our sysadmin if it would help the OP?

[Ric_]

If mainstream support is a problem, you could always go down the Citrix XenServer route (which I have done). I'm not sure how 'supported' your SAN solution would be. You could always download 'Express' and try it out.

[Richie1972]

I'm not worried so much about the paid support side of things - I find that the ppl on here and other forums (fora ????) are very helpful.
I'm making sure everything is fully documented so that anyone new is easily able to keep things going
As I've said, I'm really just trying to reassure the powers that be that Xen is used in industry (and, hopefully, other schools) and is stable, etc

[dhicks]

As I've said, I'm really just trying to reassure the powers that be that Xen is used in industry (and, hopefully, other schools) and is stable, etc

As others have pointed out, Xen is used by Amazon and other similar large "cloud" providers. Xen use seems to be at the larger, more scalable end of the market - the kind of thing where you have thousands of VMs scattered around the globe. Tools like Puppet and Enomalism exist to manage large-scale Xen deployments. Xen has the massive advantage that you don't have to pay for it, or even consider licensing - it saves all this palavar of wading through license agreements to find out which copies of your software you can run where.

We run Xen on top of CentOS 5.1. Installation consisted of checking the "virtualisation" option on the install screen. That was it. Then you just create a config file for each VM (easy stuff, just tell it what disks to use, how much RAM to assign, stuff like that) and away it goes. "Standard" computers with single network cards assigned work easily, you can have a VM up and running in about 5 minutes.

Things I've found fiddly:

Installing drivers. You're using a customised Linux kernel, which generally works just fine, but can be a bit of a palaver if you have an exotic RAID controller or whatever. I never did get our (very cheap, eBay-bought) SATA "RAID" controller working, and it took me ages to get through the installation of a SCSI card (mainly, it turns out, because all the disks were dead anyway - explains why I couldn't seem to mount them...). My advice is simply to buy proper (£300+) hardware RAID cards that the machine will see as a single SATA disk, which is a good idea anyway as you'll get better performance. Either that or buy motherboards with 8 or 10 SATA controllers.

Similar to the above problem with drivers, installing DRBD proved a bit fiddly, too. The problem is that CentOS' packaged copy of DRBD seems to be set up to work with the non-Xen kernel. I eventually figured out that all I had to do was type "yum install drbd", then copy over the DRBD driver from a previously installed location and update the modules list. All quite easy in the end, but took a bit of working out (I have the details written down at work, I'll post them at some point). You also have to remember to configure CentOS' firewall to let DRBD traffic through.

Windows 2003 and 2008 work just fine under DRBD in HVM mode, assuming you have a processor that supports virtualisation extensions. I couldn't get Windows NT 4 to work (kept blue-screening a few seconds after booting from the install CD ISO) so instead I ran it under QEMU, which works just fine (and surprisingly fast). I have still to figure out exactly how to get networking working with QEMU - I was working on it last thing Friday, I hope to have it sorted tomorrow.

I couldn't get our SIMS (Windows 2003) server to transfer wholesale to a Xen VM, it simply refused to boot. It could well be that I didn't try long enough (I found out that SIMS now runs on Server 2008, so I figure I can just install a new 2008 VM instead) or that there needs to be some cunning conversion/transference done. That's the kind of thing where you could maybe hire in some paid support (a consultant for a day to transfer accross your existing Windows servers) or buy some conversion software to help you (Citrix are bound to be able to flog you something).

I can't quite get my head around networking with multiple network cards in a VM. The idea with networking is that create a "bridge", basically a software-simulated switch, which you then "plug" other network connections into. Choosing the "virtualisation" install option on CentOS sets all this up for you for one connection per VM - it all just happens. However, I wanted to have our Internet gateway and web server machines be VMs so I could move them between ADSL connections if one went down. An Internet gateway/filter/firewall needs two connections, of course - one to the outside Internet and one to the local network. Setting up the second network bridge and getting all the connections plugged in to the right places wound up utilising some scripts, which seems a bit complex just for a network connection. Again, it could well be that I've got the wrong idea somewhere (or, er, just not read the manual...).

I've trundled along fine with Xen so far (as of Friday I'm up to 10 Xen machines in our "cloud"). As with any virtualisation system, it's probably best to have all shiny-new servers with nice RAID cards and multi-core, virtualisation-supporting processors stuffed with lots of RAM. However, our system consists of some very, very old and crusty servers (I think one is from 1999) and some very, very cheap servers (£99 Dell specials) and everything still works, so the performance overhead certainly can't be huge.

--
David Hicks

[Richie1972]

I'm already testing Xen on Centos 5.2 on a quad core desktop and a dual core athlon x2 (at home). I've had XP, Centos (32&64 bit), Win2k3, Win2k8 all running fine, but obviously not tried connecting client machines yet. I've even had VMs installed on iSCSI targets attached to the xen server (ran really slow, but I did have Openfiler sat on an old Compaq P3 733MHz lol).
I might even play around with Vista and Windows 7...

I want to try to build a Windows 2008 SIMS setup, but I believe there are problems with getting updates if you are already running a SIMS server - I'll also have to dig out the SIMS installation docs (once I find them).

I'm in the middle of building a test for Openfiler with DRBD and Heartbeat (on two Compaq P3 733's - so fast test lol). However, with money constraints looming, I'm not sure how much of this I'll be able to build - I might have to try to split some of the project over a couple of years (BSF permitting).

The servers I'm going to be using will have "proper" on board RAID (server class boards, so should be too) as opposed to the fake RAID you get on desktop computers.

More than likely I am going to sit our ADSL/cable link behind a firewall and just point our virtual gateway (Linux with Websense7) at it to save having to dedicate a LAN connection to the ADSL link.

One thing I'm not able to test until I get the hardware is Xen "live" (and not so live) migration - there are various documents on how to do it, though.