SAN and virtualisation

[MK-2]

We've been asked to look into server upgrades and moving the school forwards and are considering SANs and virtual servers.

Has anyone used/implemented/seen a VM server SAN solution? If so, what costs were you quoted?

We're only exploring possibilities right now, so might still end up buying standard servers instead of going the VM route, but want to know all options.

 

At present we have:

1 x ISA server

2 x DC (1 of which is user data file server too)

1 x Exchange / AV / SQL / MIS (all on one)

1 x Backup

1 x Filtering

1 x intranet

1 x terminal server (only used for 2 clients so doesn't need to be powerful)

 

What would people suggest in terms of virtualising and using a SAN? could we get exchange/DC/file server VMs on to one physical server with about 32GB ram, or how would others lay it out?

 

Cheers

 

Mike

[glennda]

We currently run KVM on Ubuntu as the host running on 2 Dl380 G5's with a Fiber Channel HP MSA2000. These two servers are dual quad core with 64gb Ram. (we can run around 14 servers from these including fs/dc's/sims) These two would pretty much be able to run the entire system that you have. Keep the Backup server as a physical box aswell as one of your DC's (just incase the vm hosts goes down) The filtering I would also setup it on a physical box and put it between the network and the internet so there is no way around going through it (enless you want to be able to).

 

But there are multiple hyper visor software such as HyperV, ESX from Vmware and Xen Plus KVM so its very much go out and see demonstrations contact local schools etc see what they do. If you where not so fare away I would say you can come here!

[Duke]

I've been working on this same project here for about 3 years now, it's mainly held back by budget but we have one SAN and two ESXi hosts so far. At some point we'll (ideally) have mirrored SANs and two VMware clusters in different locations on site. We should be able to virtualise pretty much all our servers (keep a PDC physical, probably Exchange and SQL too for now).

 

I'd rather not re-type everything, if you'd like to chat then PM me some contact details and I can either email you some documents I've written (business proposals, plans, etc.) or we can have a chat on the phone (probably easier).

 

Cheers,

Chris

[glennda]

I went to a VMware thing with this company IT Managed Services, Managed IT Services, Outsourcing - ADA when I was at my last place - they have an office in birmingham so might be able to drive up and show you a few different things

[dhicks]

what costs were you quoted?

 

I doubt you need a SAN in a school setup - disk I/O is going to be your main bottleneck, our server VMs hardly ever went over about 10% processor useage. SANs are for sharing a central pool of storage around multiple, expensive-processor servers, but if your main use of disk space is as a file store (or you just operate at a scale than is likely for the average school) then you're better off simply having some large file servers.

 

Mirroring of file stores can be easily accomplished via DRBD. I used Xen for virtualisation at my last job, although I'm setting up with KVM for my home server. I would keep the file server and TS server as separate hardware, but everything else can be virtualised.

 

--

David Hicks

[Duke]

I doubt you need a SAN in a school setup [...]

 

How do you do a clustered VMware (or any HV) install without centralised storage though? From my point of view, having proper fail-over and redundancy is one of the biggest benefits of virtualisation - if a single failed server still takes out my services (because the storage is gone so another host can't start up the VMs) I'm not getting one of the main benefits of going virtual.

 

Just saying.

[dhicks]

How do you do a clustered VMware (or any HV) install without centralised storage though?

 

You use DRBD to mirror volumes on your VM hosts. Disk storage is cheaper these days than the equipment needed (fibre channel, etc) to get local-disk-like performance from networked-connected disk arrays.

 

--

David Hicks

[Duke]

Sorry - missed that bit!

 

Must admit I don't know much about DRBD, when using it for virtualisation do you do synchronous mirroring or do you essentially do a scheduled snapshot?

[MK-2]

we have been quoted for the following server as a VM host. would this be sufficient to run say a DC, exchange server and apps/ghost server?

i know in an ideal world (as all the companies quoting keep saying) we could do with more memory, etc. but we are on a budget so cost is a real factor.

 

Xeon E5620 2.4Ghz,

Additional Processor,

32GB Ram,

2 x 146GB SFF 6Gbps 15k SAS Hard Drives,

6 x 600GB SFF 6Gbps 10k SAS Hard Drives,

M5014 SAS RAID Controller,

Redundant Power & Fans,

3 Year Onsite 4 hour response 9 x 5 Warranty

 

also, at the moment we are looking at Hyper-V. I'm guessing if we factored in the cost of VMWare it would be way too expensive so what are peoples thoughts on Hyper-V too?

[Duke]

I know VMware ESXi will definitely run off an SD card, will Hyper-V? If it does (or if you can go with VMware), drop the 2x146GB drives (which I assume are for the hypervisor) and put in a card reader, then install ESXi (or whatever) to an SD card. Instant cost saving.

 

Other than that the spec looks okay, although Exchange is generally disk intensive and isn't as happy about being virtualised as other servers (same goes for SQL). It certainly can be done though, but I'd check your IO requirements beforehand.

 

Here's the spec I'm aiming for with our hosts:

 

Dell PowerEdge R710

2x Xeon L5520 (2.26GHz)

48GB (12x 4GB)

Dual Port Broadcom Gb NIC

Quad Port Intel Gb NIC

DVD Drive

Dual 870W PSU

On-board SD Card (1GB)

iDRAC6 Enterprise

Ready Rack Rails (no CMA)

3 Year Support

 

£4.3k-ish.

[Butuz]

Don't forget you can use the Free version of ESXi - it works a treat for £0. Of course you don't get the load balancing, automatic failover, advanced backup and are limited to 2 physical CPU's and 256gb ram per box but frankly 90% of schools can make do with that IMHO.

 

Using it here with 3 x ESXi hosts and 2 x SANS and very pleased. Don't get me wrong in an ideal world I would like to upgrade the ESXi to Advanced but that is something we can purchase later as and when funds become available. You can easily virtualise your entire network and have an excellent performing and highly avilable network using the free version.

 

Butuz

Edited October 11, 2010 by Butuz

[Duke]

3 x ESXi hosts and 2 x SANS

 

Same here with 2x ESXi hosts and 1x SAN (all right for some! ).

[Butuz]

Bah well the third box is poo (just using it to host a couple of V workstations), and my two sans cost much less than your one san

 

Alright for some tut tut!

 

Butuz

[MK-2]

considering this is our current setup, how would you implement it in terms of what is virtualised, what stays physical, etc? As if SQL and exchange are virtualised, and backup was left physical, and if file server was phsical due to disk intensiveness, we're left with pretty much nothing to virtualise.

 

1 x ISA server

2 x DC (1 of which is file server too)

1 x Exchange / AV / SQL / MIS (all on one)

1 x Backup

1 x Filtering

1 x intranet

1 x terminal server (only used for 2 clients so doesn't need to be powerful)

[Duke]

Without knowing the exact details it's hard to be sure, but on the face of it I'd be happy to virtualise:

 

Intranet (Windows web server? Should virtualise easily)

Filtering (Unless it needs a physical connection to a router this should virtualise fine)

1x DC (Always keep one DC physical)

ISA (Can't see any reason why this should be a problem)

TS (Don't know much about this, but it should virtualise perfectly well as some thin clients use virtual servers)

 

Any budget for centralised storage, even if it's home-built? That way you could use it for the virtual machine hosting (NFS or iSCSI) and for the file server (CIFS), thus meaning your DCs wouldn't have any other roles. Could you move AV onto one of the other virtual boxes too? I see no problem with keeping Exchange and SQL on a physical box until you can budget for something that'll definitely handle the IO requirements. I assume backup needs to stay physical because it has a tape drive of some kind?

 

Chris

[dhicks]

Must admit I don't know much about DRBD, when using it for virtualisation do you do synchronous mirroring or do you essentially do a scheduled snapshot?

 

Sorry, missed your post the other day - DRBD does synchronous mirroring, it duplicates the data live as it's written to disk. You can specify exactly how it does that, too - you can esentially define whether it does write-back or write-through. I found the DRBD documentation to be clear and complete, but it's worth taking some time to go through and check your setup before going live as performance issues can affect system stability of VMs.

 

--

David Hicks

[BIST]

If you want a chat about all vmware options drop me a message.

 

VMware do you have very nice options, Compare VMware vSphere Kits for Cloud Computing, Server and Data Center Virtualization

 

Vsphere Essentials - Doesnt require shared storage , is really cheap and licences 3 hardware appliances and vcenter essentials

Vsphere Essentials Plus -Does require shared storage , is decent entry level into shared virtual enviroment, licences 3 hardware appliances and vcenter essentials

 

In regards to hardware it is completely dependent on how many virtual enviroments you are going to be running on each server and also the amount of data being processed. Vmware is like a dog, the more you give it the more it gives back

 

Shaun