Virtualization Project

[lafleur1977]

Hello,

 

I am currently looking into virtualizing our servers and I need some help on the kind of setup that would be best.

 

First a little background. We are a large high school with about 1500 users. We have 12 physical and 6 virtual Windows Servers (WS 2008 to WS2012R2) with roles such as AD Domain Controller, DHCP, File Services, Print Services, SIMS, Remote Desktop Services, WDS, WSUS, Sophos Enterprise Console. Most of the servers are around 5 years old and getting ready for replacement. I do have some knowledge of VMware ESXi since a company helped us migrate from our in house Exchange server to Office 365 last year, and they set us up with a Virtual Host Server with a few VM's on (DC/Mail Relay/Remote access (RemoteApp and Desktop Connection). Since then I have dabbled with ESXi and setup another virtual host on an older server to learn the installation process and setup a couple of VM's (WSUS and a web server) so I feel pretty comfortable with ESXi.

 

I have been reading up about other peoples scenarios and what kind of solutions people are implementing. For example, many people are dismissing SAN's and suggesting 2 or 3 hosts with local storage and then either replicating to another host via software such as veaam and/or doing backups to a good quality NAS. I am keen to stick with VMware due having a bit of knowledge with this and the large support it has. I am conscious of ESXi's limitations with things like backups and I will also need to consider how I can backup the VM environment as I go from physical to virtual. I manually perform windows server backups and we currently use BackupExec for file backups to a B2D target and then to tape which is something I would like to improve perhaps with Veamm. Would it be worth purchasing VMware licences perhaps starting with essentials?

 

A couple of other queries:

 

- Workloads/IOPS - I have heard about IOPS but don't fully understand it and how people calculate this? How is it best to go about sizing up the current workloads to then figure out the spec of the host servers?

 

- Our file server is currently physical and is connected to a SAN (for users file storage) and a DAS for B2D backups. Would you recommend setting up a virtual file server and use the local storage from the host? I am not keen on the SAN as it had a double hard drive failure a little while ago. We have about 2.5 TB of user data on our file server so I will be needed quite alot of disk space!

 

- Where to install ESXi? To the local disks or SD card? Alot of people seem to be recommending installing ESXi to an SD card to keep the OS and datastore separate however I cannot find any information as to why. The company who setup our first virtual host server did not say anything about this and installed it to disk.

 

- RAID level? I have seen lots of people dismissing RAID5 and recommending RAID10.

 

With the 12 current physical servers and 6 virtual servers I would be looking at around 18 VM's in total, and perhaps a few more as I would like to separate some of the services out on to different servers. How many hosts would be needed and what kind of setup would you suggest? I will be looking for as much CPU, memory and storage as I can afford but our budgets are not huge. Realistically I would probably need a solution for under 15k and for it to maybe be phased over a couple of years. The first Virtual host I mentioned that was setup for us is a Dell R720 - 2x Intel Xeon 6 core CPU / 32 GB RAM / 600 GB storage which I could upgrade and can use this as one host. The second host I setup myself I would be looking to decommission and move the 2 VM's off it.

 

Thanks for any help and suggestions. Sorry if my explanation is hard to understand. If I have missed anything let me know,

 

Thanks

[peterp]

We went through very similar when we moved from CC3 to vanilla, we also virtualised.

We got in touch with a company called Ergo (IT solutions for education and public sector from Ergo Computing) who provided the hardware, and handled the migration.

The system was built to our specs. It is all based on VMware esxi with Veeam being used for the backups.

 

We couldn't be more happy. I'd suggest talking with them over your needs, they are very good.

[lafleur1977]

We went through very similar when we moved from CC3 to vanilla, we also virtualised.

We got in touch with a company called Ergo (IT solutions for education and public sector from Ergo Computing) who provided the hardware, and handled the migration.

The system was built to our specs. It is all based on VMware esxi with Veeam being used for the backups.

 

We couldn't be more happy. I'd suggest talking with them over your needs, they are very good.

 

Thanks for your reply. I was thinking of involving some IT companies but I am just concerned about the additional costs. I will have a look at Ergo and give them a call. Are you using free ESXi with veaam? Veaam told me I needed to use paid ESXi to use it. Also, I have been looking at veeam a little bit and was wondering where is best to install it? E.g. on a VM or a separate physical server?

[peterp]

We have virtualised pretty much everything.

3 physical hosts, VM's for everything else. We are on the paid ESXi, with auto movement of virtual machines if hosts go down.

I'll send you a PM with some more details.

[3s-gtech]

For ESXi installation, having the OS installed on a separate USB stick or SD card just gives some separation between storage. It means that you can easily wipe/upgrade the ESXi installation with no effect on the storage, you don't have to set aside a fiddly partition on your disks for it, etc. I'm sure there are other advantages too (the relative lack of writes to the stick means they last well in the job).

 

RAID, IMO stick to RAID 10. It's expensive on disks, but offers good resilience and is quick. RAID 5 plus SQL databases often leads to poor performance, and RAID 5 has a nasty habit of failing (URE) during a rebuild, losing the lot.

[lafleur1977]

For ESXi installation, having the OS installed on a separate USB stick or SD card just gives some separation between storage. It means that you can easily wipe/upgrade the ESXi installation with no effect on the storage, you don't have to set aside a fiddly partition on your disks for it, etc. I'm sure there are other advantages too (the relative lack of writes to the stick means they last well in the job).

 

RAID, IMO stick to RAID 10. It's expensive on disks, but offers good resilience and is quick. RAID 5 plus SQL databases often leads to poor performance, and RAID 5 has a nasty habit of failing (URE) during a rebuild, losing the lot.

 

Thanks, 3s-gtech. That makes sense to use a USB/SD Card with the points you mentioned. I will definitely look to do that when setting up the new Hosts. I agree with you on RAID10 over RAID5. I have experienced myself that pitfall with RAID5.

[maark]

If you have a Microsoft EES agreement then you might want to consider using hyper v - very cheap - pay for datacenter 2012 and you can install as many virtual servers as you like on that hardware. I have been using hyper v in similar sized school on Dell R710 and R720 with no issues for several years. I would prefer using several hosts rather than SAN as SAN can be very pricey and can also give you a single point of failure unless you have 2 - even more pricey.

[lafleur1977]

If you have a Microsoft EES agreement then you might want to consider using hyper v - very cheap - pay for datacenter 2012 and you can install as many virtual servers as you like on that hardware. I have been using hyper v in similar sized school on Dell R710 and R720 with no issues for several years. I would prefer using several hosts rather than SAN as SAN can be very pricey and can also give you a single point of failure unless you have 2 - even more pricey.

 

Yes we do have an EES but its only for desktop OS/CALS/Office. My manager never likes to add the server licences onto it. Are you saying to purchase 2012 datacentre x1 to add onto the EES annually and then I can create as many 2012 datacentre VM's as I like? I have recently been buying 2012R2 standard, which allows for 2 VM's per licence. I must admit I haven't tried Hyper-v yet, mainly because we have started down the VMware road.

 

I was preferring to use several hosts rather than a SAN. I want to get rid of our current SAN due to age/hardware failure. We can't afford a new one and definitely can't afford two. So, if I was to use several hosts how many might you recommend for around 20 VM's? If I was to use several hosts with local disks, how would I be best to tackle the file server? Its currently physical and connected to a 5TB SAN for user data, about 60% full. If I setup a VM for it, would I just create one big virtual hard disk or multiple ones for the data and store this on the host server (therefore requiring alot of space on the hosts)?

 

Thanks

[Steve21]

Are you saying to purchase 2012 datacentre x1 to add onto the EES annually and then I can create as many 2012 datacentre VM's as I like?

 

As many on a single host yes, so could run 99999999 if single host supports it.

 

Using 2 hosts = 2 data centre etc.

 

STeve

[lafleur1977]

As many on a single host yes, so could run 99999999 if single host supports it.

 

Using 2 hosts = 2 data centre etc.

 

STeve

 

Ah, I see. Cheers.

[maark]

I would say you need four hosts for 20 virtual machines - depends how much memory and disk space you have.

Worth having some leeway so you can replicate between hosts in case one host fails. Or you can replicate to older servers as a disaster recovery. Veeam is good - I am only using the free version for disaster recovery and easy backup before doing big upgrades etc.

[Steve21]

Personally I always like 3-4, but obviously depends on budget wise and spec of the hosts. Currently running 36 virtual servers on 2 HOSTs until we can afford more

 

Steve

[lafleur1977]

I think 3 would be possible and then means all my eggs aren't in one basket. 4 might be a stretch on budget but could look at an extra one in the future. Serversource have some great offers on and at the moment they are doing buy one get one free. The R720 I currently have I could upgrade too.

[detjo]

Here's what I would do:

Split the VMs between two hosts and have an additional two hosts set up as replicas. That's four physical servers. I would go with the option of installing Server 2012R2 Datacenter on the hosts and use Hyper-V for virtualisation. This will allow you to install all the VMs you need at no extra cost, and Veeam supports it as is. Replication can be done thru Hyper-V (and is a piece of cake to set up) or Veeam.

 

 

If using Hyper-V:

 

1. Note that it's recommended [required ??] to have matching hardware for replication.
   
2. I imagine the 2 x 6 core CPU would be enough for the VMs (depends on CPU spec.)
   
3. Might need to look at the 32Gb memory. This may be enough, even with 10 VMs per host, but it depends on what the VMs will be doing.
   
4. 600Gb space will almost certainly need increasing
   

 

I would be tempted to get a bunch of large capacity host disks for storage. SAN double disk failure isn't appealing! and could be expensive to replace. You could always keep the SAN for old/archive stuff(?).

 

As for RAID, this is a really down to you and what best fits with your environment and cost. RAID10 might offer better fault tolerance, but it also loses a lot of space. With replica hosts and Veeam backup of entire VM disks, I'm not so sure this is so much of an issue. Personally, I have had RAID5 set up in all the schools I have worked in (most set up by previous techs) and I have never yet seen one fail.

 

 

FYI: Things you might not know about 2012R2 Hyper-V if you haven't used it, but might be nice to know for you to consider all your options:-

 

• No Windows activation is required for the VM guest OS
  
• Assign 'Dynamic Memory' to VMs so it only uses what it needs
  
• Assign CPU weighting control to VMs
  
• Differencing disks (NOT recommended for DCs)
  
• Live VM migration
  
• Replication intervals down to 30secs
  

[kmount]

Make sure you ask whoever you talk to to run a Capacity Planning exercise on your current physical + virtual servers to provide you with the evidence behind their specification on CPU/Mem/Disk+IOPs so you know it's based on real information and fit for your current + predicted future needs.

 

I'd be very surprised if you need more than 3 hosts, the number of CPU cores and RAM capacities now make less hosts more grunt feasible. This is especially important when it comes to licensing VMware and View which both have 'Essentials' bundles which are aimed at 3 hosts (6CPUs) for a low cost. That fourth host is where it becomes expensive so I'd recommend ramping up the spec of your 3 to cover you at n+1 with some spare over the top.

 

I have a customer running 50+ VMs across 3 hosts no problem at all, they're a HE/FE college and it runs a bunch of stuff. The CPU RDY%/WAIT% is the important bit when it comes to whether you need another host if you have VMs waiting for CPU; in this case it doesn't happen because there's plenty of grunt.

 

Re Hyper-V 2012 R2, a lot of customers are starting to take it seriously and it will continue to gain market share for all of the reasons the posters above pointed out so definitely worth investigating your options.

 

Definitely definitely go with a Capacity Planning exercise though, guessing isn't good enough!

 

Have worked with Ergo in the past, nice chaps.

Edited April 3, 2015 by kmount

[detjo]

This Hyper-V Youtube vid is worth a watch: