What can/should I Virtualise?

[TheFopp]

Over the summer I'll be getting a couple of new servers to replace our two oldest, and I'm interested in a): the idea of using virtual servers on these to help cut down the number of servers I have, and b): the idea of being able to restore a server very quickly in the event of a server dying. So I'd like to hear from those of you who have already done this with your thoughts on our setup.

Currently we have 6 servers:

1. DC, DNS, DHCP, WSUS, IIS, McAfee Updates
2. DC, DNS, Fileserver (110GB data)
3. SQL (sims + a couple of smaller databases we've built) (20GB data)
4. Moodle (10GB data)
5. Exchange, Ranger Outpost, DC (I know it shouldn't be, but the guys who did the install did it this way and it hasn't caused any problems - touch wood)
6. just one IIS instance

The 2 oldest servers I will be replacing are 1 & 2. So here are my thoughts on virtualising

New servers
1. VM1 - DC, DNS, DHCP
VM2 - WSUS, McAfee Updates, IIS (from both IIS machines as WSUS needs it anyway)

2. VM1 - DC, DNS
VM2 - Fileserver
VM3 - Ranger Outpost

Old servers
3. VM1 - SQL
4. Moodle
5. Exchange

Does that look reasonable?
Does it make sense to virtualise the fileserver when it holds so much data - and will increase in datasize? (taking into account that we can't afford a SAN)
Is SQL happy on a VM?
What about Moodle, would that be happy as a VM on the same box as SQL, thereby reducing the number of servers required to 4?

VMWare or Virtual Server (free editions - can't afford ESX)

I also know you need plenty of memory on the host servers to run a number of VMs. I'm looking at HP DL380s for the servers. If I go with more than 4GB of RAM fro each server, do I need to run a different version of Server 2003 (currently running Server 2003 Standard) to access all the available memory?

Any advice or thoughts would be most welcome.

Cheers
Adrian

[Ric_]

VMWare or MS... I think that the general consensus is down to personal choice (although if you use VMWare you can upgrade to a higher product without having to convert images).

RAM... If you are sticking with Windows as your base OS, x64 edition of Server 2003 standard will do the job for up to 8GB RAM - after this you need Enterprise or if you stick with 32bit. Of course, you could run Linux as your base OS. There are also licensing considerations for virtualisation and I'm not the best to describe these.

Your spread of services doesn't look too bad, although I would separate out IIS from your update stuff since WSUS can be a bit funny.

[TheFopp]

Your spread of services doesn't look too bad, although I would separate out IIS from your update stuff since WSUS can be a bit funny.

I was under the impression that WSUS needs IIS to run - web-based admin centre.

[Dos_Box]

I have my Sophos and WSUS servers virtualised. Oh, and the KS3 testing platform too

[Geoff]

Also remember MS Virtual Server is really poor at running Linux VMs. So if you want to put anything in running under Linux, you should probably go for VMWare.

[ZeroHour]

I am curious about hypervisor vs. vmware free as options rather then virtual server as once hypervisor comes out I think that will probably be EOL for virtual server.

[tmcd35]

We are currently in the process of moving our entire server farm onto two ESX servers. I am currently building a third Windows based VMWare server as an extra backup server from one of the old, but still very good, servers that has now been virtualised.

We are also in the process of moving everything across to an iSCSI SAN environment - very existing times.

The best advise - you can never do enough research and planning! And even with all the research an planning and testing, unexpected things will go wrong. But I think in the long run it has been a worth while project here.

A very long way down the road we want to split our virtual servers up more, so that the MIS has it's own virtual server, the maths Kaelidos program as it own VM, we'll have individual dedicate VM's for DHCP, DNS, etc - or at least thats the plan.

Server 20003 SE will access 8Gb using PAE, basically memory paging, but it's not ideal. For more than 4Gb you really need a 64bit OS. For a VMWare box with >4Gb ram I'd go linux. In deed the only reason the machine I'm building now is running Windows is because I want it as a back up SAN server as well as a backup VMWare server and the SAN software runs on Windows only.

For licencing, remember each VM is in effect a seperate physical machine and needs all the licences any seperate physical machine would need - OS, CAL's, whatever. If you have a licence for Windows 2003 Server Enterprise R2 (must be R2), than you can run up to 4 VM's from the one licence.

[srochford]

When you dcpromo the box running Exchange down to a member server, Exchange will break - I've now forgotten the details (I think it's OWA that falls over) but it's messy (I've done it :-()

Either leave it as it is or put in a second Exchange server, move all the mailboxes to it, remove Exchange from the DC, dcpromo down, put Exchange back, move the mail back.

You definitely want a 64 bit OS to let you get at lots of RAM. The standard version will work but the Enterprise version of the server will allow you up to 4 VMs at no extra cost. If you're not going till the summer I'd be tempted to see if 2008 Hyper-V has been released by then and go with it.

[dhicks]

the idea of being able to restore a server very quickly in the event of a server dying.

I'm using Xen (also free, built-in to Ubuntu Server, although I'm having a few fiddly installation issues). I'm setting up DRDB to clone virtual file systems at the block level, so as data is written to one filesystem it is cloned across the network to another one. If one machine goes down another should be able to start right up with something like 800ms of downtime (seemingly... I'll believe it when I see it!).

> VM1 - DC, DNS, DHCP

I'm using Linux VMs, so there's no licensing involved, but I was rather thinking I'd have a separate VM for each of those. Doesn't one version of Windows Vista/Server 2008 let you run four virtual machine copies or something?

> Does it make sense to virtualise the fileserver when it holds so much
> data - and will increase in datasize? (taking into account that we can't
> afford a SAN)

The file system and the VM itself will be separate. You can even assign a VM to run off of a number of physical harddrives. Personally, I'm still of the opinion that a SAN is a waste of money for what any school is likely to do with it. SANs let you detach storage from processing power, so you can add more "costly" processing power as you like, at the cost of needing a whole bunch more equipment to communicate between the SAN and the processing kit. Most school's probably actually use very little processing power, mostly what they need is disk I/O performance, so you might as well have the processing power tightly-coupled to the storage (i.e. "a server with a nice big RAID array") and ditch the communications overhead. Processors are cheap these days, anyhow. Failover can be handled very nicely (when I get it to compile...) by block-level filesystem cloning (see DRDB, above).

> Is SQL happy on a VM?
> What about Moodle, would that be happy as a VM on the same box as
> SQL, thereby reducing the number of servers required to 4?

You can even get Moodle on a ready-made VM, so yes, it runs on a VM just fine (note that it's one of the things that'll need a bit more processing/RAM power - PHP works by smooshing together pages of text). Not sure about SQL - see if there are any ready-made SQL appliances around, that'll at least tell you if it's suitible to run on a VM or not.

> VMWare or Virtual Server (free editions - can't afford ESX)

Xen!

Stuff your servers with as many ethernet ports as you can - you can get four-on-one PCI Express cards. This means you can use an aggregated connection to the switch, so maybe a 4Gbps link, and maybe a direct link to another server if you want to use something like DRDB to clone filesystems.

--
David Hicks

[torledo]

We are currently in the process of moving our entire server farm onto two ESX servers. I am currently building a third Windows based VMWare server as an extra backup server from one of the old, but still very good, servers that has now been virtualised.

We are also in the process of moving everything across to an iSCSI SAN environment - very existing times.

The best advise - you can never do enough research and planning! And even with all the research an planning and testing, unexpected things will go wrong. But I think in the long run it has been a worth while project here.

A very long way down the road we want to split our virtual servers up more, so that the MIS has it's own virtual server, the maths Kaelidos program as it own VM, we'll have individual dedicate VM's for DHCP, DNS, etc - or at least thats the plan.

Server 20003 SE will access 8Gb using PAE, basically memory paging, but it's not ideal. For more than 4Gb you really need a 64bit OS. For a VMWare box with >4Gb ram I'd go linux. In deed the only reason the machine I'm building now is running Windows is because I want it as a back up SAN server as well as a backup VMWare server and the SAN software runs on Windows only.

For licencing, remember each VM is in effect a seperate physical machine and needs all the licences any seperate physical machine would need - OS, CAL's, whatever. If you have a licence for Windows 2003 Server Enterprise R2 (must be R2), than you can run up to 4 VM's from the one licence.

I think it's more practicaly to have application isolation environments for things such as dhcp, dns and you're maths thingy than dedicated seperate VM's with seperate OS instances. Using something like Parallels/Virtuozzo containers for instance.

As you've said it's one for down the road....so you've got plenty of change to change you're mind

[DMcCoy]

I'm using Linux VMs, so there's no licensing involved, but I was rather thinking I'd have a separate VM for each of those. Doesn't one version of Windows Vista/Server 2008 let you run four virtual machine copies or something?

From the windows perspective you need a server/exchange etc license for each host that it's used on unless it is only moved once every 180 days. This affects VMotion and the Xen equivalent. I bought datacenter licenses as it has unlimited virtualisation rights. The enterprise one has up to 4 additional copies of server 2003 but *only* on the host which holds the base license.

So if you plan to migrate as single instance of exchange between 10 blades more frequently than once every 180 days then you could potentially need 10 licenses for that single instance! Some may say this is due to the fact that MS don't have an equivilent application yet :P

[tmcd35]

I
> Does it make sense to virtualise the fileserver when it holds so much
> data - and will increase in datasize? (taking into account that we can't
> afford a SAN)

The file system and the VM itself will be separate. You can even assign a VM to run off of a number of physical harddrives. Personally, I'm still of the opinion that a SAN is a waste of money for what any school is likely to do with it. SANs let you detach storage from processing power, so you can add more "costly" processing power as you like, at the cost of needing a whole bunch more equipment to communicate between the SAN and the processing kit. Most school's probably actually use very little processing power, mostly what they need is disk I/O performance, so you might as well have the processing power tightly-coupled to the storage (i.e. "a server with a nice big RAID array") and ditch the communications overhead. Processors are cheap these days, anyhow. Failover can be handled very nicely (when I get it to compile...) by block-level filesystem cloning (see DRDB, above).

I have two ESX boxes attached to our SAN and a third VMWare (free vers.) going online soon. Depending on what we do with the MIS a fourth server may be added to the SAN. Also our File Server, thats not yet virtual, has been given some temporary SAN space to tide it over during the migration process.

I have, in the short time it's been in place, found that the SAN has given us greater scope/freedom with storage provisioning amongst servers. Extending drive space to individual servers and adding more storage to the SAN are now task that are quite simple to do. Where has adding more storage to an existing physical server can be quite a troublesome task.

SANs don't have to cost the earth - A Promise VTrack, SATA-II drives and SANMelody software!

I think it's more practicaly to have application isolation environments for things such as dhcp, dns and you're maths thingy than dedicated seperate VM's with seperate OS instances. Using something like Parallels/Virtuozzo containers for instance.

I've been looking into Application Vitualisation for general application deployment across the LAN. I can't find a system that's cheap/good enough for what I want

I still prefer separate VM's for separate tasks. Currently ony VM/Physical box may provide 6/7 different services - DHCP, DNS, AD, Printing, Apps, Files, etc.

If one of the hosted service forces you into restarting the entire server - all the service are lost.

Consider a Printer not responding and the Print Spooler freezing when you try to restart the service. Now imagine the MIS being on that machine. Yo restart the machine to get the printers back and loose the MIS in the process!

[dhicks]

Extending drive space to individual servers and adding more storage to the SAN are now task that are quite simple to do. Where has adding more storage to an existing physical server can be quite a troublesome task.

Admittedly I'm coming at this from a Linux point of view, where you run virtual machines on top the logical volume manager (which itself runs on top of mdadm software RAID - yes this would be more efficient with a decent hardware controller, but you only get so much for a £99 server). It's like having a "virtual SAN", but minus the expensive bits.

Dedicated VMs for DNS/DHCP/etc: there must be some ultra-small Linux distribution out there that take something like 64MB of RAM and can just run DHCP or whatever? No need for a whole mega-distribution with GUI and everything. DamnSmall could probably do it (and that has a GUI!).

--
David Hicks

[tmcd35]

Dedicated VMs for DNS/DHCP/etc: there must be some ultra-small Linux distribution out there that take something like 64MB of RAM and can just run DHCP or whatever? No need for a whole mega-distribution with GUI and everything. DamnSmall could probably do it (and that has a GUI!).

--
David Hicks

I like this mans thinking. Yep, Ideally, I'd replace most service currently being bundled together on the over stretched servers we have at the moment into multiple Linux mini-OS's dedicated to the task. No need to run Windows for DHCP, DNS, possibly printing and file sharing, and likely a few other services as well.

[TheFopp]

> Does it make sense to virtualise the fileserver when it holds so much
> data - and will increase in datasize? (taking into account that we can't
> afford a SAN)

The file system and the VM itself will be separate. You can even assign a VM to run off of a number of physical harddrives. Personally, I'm still of the opinion that a SAN is a waste of money for what any school is likely to do with it. SANs let you detach storage from processing power, so you can add more "costly" processing power as you like, at the cost of needing a whole bunch more equipment to communicate between the SAN and the processing kit. Most school's probably actually use very little processing power, mostly what they need is disk I/O performance, so you might as well have the processing power tightly-coupled to the storage (i.e. "a server with a nice big RAID array") and ditch the communications overhead. Processors are cheap these days, anyhow. Failover can be handled very nicely (when I get it to compile...) by block-level filesystem cloning (see DRDB, above).
--
David Hicks

Not sure I follow you there. If I create a Virtual Server instance and port my current fileserver (C: Windows D: Files) surely the virtual server would have to hold all the files for the fileserver so the image for the VS would be 100GB in size.

My question is, is it a good idea to have all of those files (100GB +) held within the Virtual Server file?