What can/should I Virtualise?

[dhicks]

Not sure I follow you there. If I create a Virtual Server instance and port my current fileserver (C: Windows D: Files) surely the virtual server would have to hold all the files for the fileserver so the image for the VS would be 100GB in size.

My question is, is it a good idea to have all of those files (100GB +) held within the Virtual Server file?

Again, I'm maybe used to a Linux/Unix point of view here, but I thought Windows -based virtual machine systems could also give VMs direct access to individual harddrives (or RAID volumes). You could assign a VM a file for storage of its operating system and a separate file (or a whole harddrive, or several harddrives) to store its data (the files it's going to be serving). That way the virtual machine itself can be detached from its data if wished. This is what I've done with all my Xen VMs (I've given them a third drive fro swap space, too). I run VirtualBox on my Windows PC at home, I'll have a look through that and see what backing stores it can use as "harddrives" for VMs.

--
David Hicks

[TheFopp]

Again, I'm maybe used to a Linux/Unix point of view here, but I thought Windows -based virtual machine systems could also give VMs direct access to individual harddrives (or RAID volumes). You could assign a VM a file for storage of its operating system and a separate file (or a whole harddrive, or several harddrives) to store its data (the files it's going to be serving). That way the virtual machine itself can be detached from its data if wished. This is what I've done with all my Xen VMs (I've given them a third drive fro swap space, too). I run VirtualBox on my Windows PC at home, I'll have a look through that and see what backing stores it can use as "harddrives" for VMs.

--
David Hicks

Yes, I get the idea of a seperate file for each Partition in effect. I would have one file for the OS Partition and another file for the Files partition but what I'm asking is there any practical limit as to how big you should let this get eg. is practical to have a Virtual file that would hold my 100GB+ of files - or is there a size limit I shouldn't go past. This file would obviously be stored on whatever Disk hardware i had in the host server, but is it a good idea to have a single file this size or potentially much bigger from a performance point of view.

[localzuk]

Yes, I get the idea of a seperate file for each Partition in effect. I would have one file for the OS Partition and another file for the Files partition but what I'm asking is there any practical limit as to how big you should let this get eg. is practical to have a Virtual file that would hold my 100GB+ of files - or is there a size limit I shouldn't go past. This file would obviously be stored on whatever Disk hardware i had in the host server, but is it a good idea to have a single file this size or potentially much bigger from a performance point of view.

I think what dhicks was getting at is that you don't need to use a file at all. You should be able to assign physical partitions directly to your virtual machines. For that sort of number of files I would never consider using a file, the performance will suck.

[TheFopp]

I think what dhicks was getting at is that you don't need to use a file at all. You should be able to assign physical partitions directly to your virtual machines. For that sort of number of files I would never consider using a file, the performance will suck.

Ah... so the actual 100GB of user-accessible files I want to share from the file server aren't actually virtualised.... as performance would suck, they are left on the hard disks as normal. So really the idea of being able to bring a Virtual server back online quickly onto another host server should the original host hardware die is not applicable in this case as I'd still have to do a full restore of all the userfiles using Backup Exec or whatever as normal, (presuming the Files and virtual server were all running on the same server) it would just be the virtual OS partition in effect that I could get back running quickly.

Am I starting to get it?!

If this is the case what about things like SQL then, What would be the best way to do Virtual Server of this?

[Ric_]

@TheFopp: What you would do is mirror the files on the second piece of hardware (or somewhere else).

[TheFopp]

@TheFopp: What you would do is mirror the files on the second piece of hardware (or somewhere else).

...

and then if disaster happens just bring the Virtual OS online on the second piece of hardware and point it the mirrored files.

Ok.. getting there.

So really my initial plans aren't viable as I also need a server mirroring the files incase of disaster. So its not really going to cut down on servers by the time I'm done. I can see why people go for the SAN option with this. 2 SANS different locations mirroring each other. Virtual servers pointing to files on the SAN.. easy to load up another instance on a different server pointing at the mirrored SAN if neccesary.. it all makes sense.

But what if you are my position with my current setup (and not enough funds for the SAN option).

What would those of you who are already virtualised recommend I do with my current setup in order to get less wasted power from the current servers, hopefully cut down on the number we are running and get good redundancy via the use of Virtualisation.

PS. I know I must be sounding a bit dumb when it comes to this, but I am just exploring all my options .... thanks for all the input so far.

[DMcCoy]

If you are going to start tying Virtual OS to physical drives then you are going to make life very difficult if you want to move things. You really want to use the virtual disk options, performance will be a bit lower, but if it's that critical whatever you are virtualising shouldn't be. A san is really only to become useful when it has a cluster file system too.

[localzuk]

If you are going to start tying Virtual OS to physical drives then you are going to make life very difficult if you want to move things. You really want to use the virtual disk options, performance will be a bit lower, but if it's that critical whatever you are virtualising shouldn't be. A san is really only to become useful when it has a cluster file system too.

If you are going to virtualise 100GB+ of files then performance will be more than a 'bit lower'. It will be downright awful.

[DMcCoy]

If you are going to virtualise 100GB+ of files then performance will be more than a 'bit lower'. It will be downright awful.

It depends on what you are using, I've got around 1TB of virtual disk files, no performance issues.

Assigning a raw disk for a VM, unless it's a large database really is shooting yourself in the foot with respect to one of the largest advantages of virtualisation. Plus you need one disk for very VM. If you don't have one per vm then you are going to have locking issues on the drive anyway, so multiple disk files make little difference.

[torledo]

If you are going to start tying Virtual OS to physical drives then you are going to make life very difficult if you want to move things. You really want to use the virtual disk options, performance will be a bit lower, but if it's that critical whatever you are virtualising shouldn't be. A san is really only to become useful when it has a cluster file system too.

Have you had a play with any of the available clustered file systems on you're san ? If so, what would you recommend.

There are quite a lot large SAN implementation without any sort of clustered FS implementation, when you say a SAN only becomes useful with a clustered FS are you specifically referring to clustered hardware setups such as an ESX farm connected to a SAN fabric ?

[DMcCoy]

A SAN becomes much more useful when you can share the virtual disks between hosts without having to move them, but for this you need a file system that can have multiple hosts accessing the LUN. While a SAN is still useful outside of virtualisation for its other features, it's this sharing that is important when virtualising.

I'm currently using ESX 3 with vmfs 3 file system on all my LUNs, all hosts share the same LUNs so I can run any VM on any host within 60 seconds. Care is still needed when formating the lun and virtual disks to make sure they are properly aligned, you can lose up to around 10% perfomance if you don't.

[dhicks]

and then if disaster happens just bring the Virtual OS online on the second piece of hardware and point it the mirrored files.

Exactly.

So really my initial plans aren't viable as I also need a server mirroring the files incase of disaster.

No! You still only need two! Split your VMs evenly between the two and have each one mirror the file systems of the other. If one machine conks out the other one can take the whole load for a while, just a bit slower.

Note that this kind of implies you have the better sort of server here - decent hardware RAID, decent number of fast harddrives, probably best to have a dedicated network link between the two servers. Don't worry, though, you're saving way more money by not buying a SAN!

I can see why people go for the SAN option with this. 2 SANS different locations mirroring each other. Virtual servers pointing to files on the SAN.. easy to load up another instance on a different server pointing at the mirrored SAN if neccesary.. it all makes sense.

Don't do it! Don't be seduced by the dark side! SANs are good if you're Amazon, Google, a render farm, the human genome project, etc - people for whom the amount of CPU (or RAM I/O, anyway) power they need outweighs the disk I/O speed they need, and who are doing this kind of thing on a large enough scale that adding something like fibre channel links doesn't make much difference, percentage wise, to the final amount they spend. A school doesn't need that kind of CPU power (and, if you do decide you need a batch-job render farm or whatever, remember you've got several labs of PCs sitting idle from 4pm until 9am...) - the main problem is disk I/O speed and network speed.

We're having the exact same debate that comes up repeatedly in computer science - offload processing to a dedicated, more loosely-coupled device, or keep things tightly-coupled and reduce the cost/communications overhead. The same thing has happened with mathematical operations (CPUs, then external floating point units, then integrated chips, now physics cards), graphics (graphics? terminals! ah - vectors! ooh - bitmaps! CGA/EGA/VGA - external cards! Onboard! External! Onboard for laptops! External cards again! Onboard again! TWO External cards!), sound (bleeper! DAC! Onboard! Soundblaster! Onboard! MIDI! Onboard MIDI! USB! Onboard! Surround sound! Onboard again!), etc, etc...

Note that this isn't a all-SANs-are-bad rant, it's just that I don't think they're suitible for any but maybe the very largest of school installations. Admittedly, this might change over a few years if things become more web-based (I figure web servers need more processing power than straight file servers), but I doubt it as I figure processors will just keep getting cheaper anyway.

But what if you are my position with my current setup (and not enough funds for the SAN option). What would those of you who are already virtualised recommend I do with my current setup in order to get less wasted power from the current servers, hopefully cut down on the number we are running and get good redundancy via the use of Virtualisation.

Mirrored servers, as above. I'd recommended spending more on them than we did (£99 each...), though.

--
David Hicks

[DMcCoy]

The problem with that is you may well increase the amount of hardware you have for something thats primarily for consolidation.

SAN is not about speed! It's about connectivity, features and consolidation.

[tmcd35]

Although I'm planning on having all our drives run within vmdk files on our SAN, I can see how using RAW disks might work. Unless I'm missing something, you can...

Have a C: drive as a vmdk that you can restore/rerun anywhere. That VM can have a RAW disk assigned to it, drive D: for user data. If that RAW disk is on a SAN then the VM can be restarted on any host that as access to that SAN and it will be able to see the D: drive.

So for disaster recovery. If the server hosting the VM dies, you can restore/run the VM on another server quickly and it will have access to the data. If the user data was lost, a new D: drive can be assigned/attached to the VM and the data should be restored from your standard weekly/daily data backups.

Just because a server is within a virtual environment is no excuse for not keeping standard tape/ext. hdd backups of data.

In this scenario, mirroring SANs, file servers, etc is just plain over kill in a school environment IMHO. But SANs themselves, they provide a very good and useful method of separating data storage from physical hardware. And as I have said above, they don't need to cost the ten's of thousands you'd perhaps expect.

SAN is not about speed! It's about connectivity, features and consolidation.

Exactly!

[tmcd35]

Okay, lacking a bit of Netiquette, two post on a row, but just spotted this...

But what if you are my position with my current setup (and not enough funds for the SAN option).

What would those of you who are already virtualised recommend I do with my current setup in order to get less wasted power from the current servers, hopefully cut down on the number we are running and get good redundancy via the use of Virtualisation.

PS. I know I must be sounding a bit dumb when it comes to this, but I am just exploring all my options .... thanks for all the input so far.

Just thought I'd say one, probably very important thing, along the lines of the planning/research advise I gave earlier.

If you decide to go down the virtualisation route - spend money now, don't do a half job, you'd regret it - we are!
virtualise
If you are going to proper I'd say choose between the free Xen and the expensive VMWare ESX and do the job right first time.

We bought 1 ESX server and a RAID array hooked up as DAS. We then bought a 2nd ESX server and found we couldn't connect it to the DAS, we had used both available connections. We then bought a SAN server to control the DAS over an iSCSI SAN so all required servers can use the storage space. The headaches this piece-mill approach has caused just isn't funny.

I'm currently moving VM stored on the RAID from LUN's presented as DAS on the original ESX server to LUNS on the same RAID box that are presented to the same ESX server but this time via the iSCSI SAN.

We should have bought both servers and the SAN day 1!