maximum CPU's per VM on vSphere 4

[Avalon]

I am having a problem with a VM on vpshere 4. To give a bit of background we have two similar VM's running Server 2003 standard, which both had 6 CPU's allocated in vCenter.

 

For some strange reason after a routine restart of one of the VM's it will not boot with an error message complaining about the number of CPU's. I have contacted VMware support and they said that only the Enterprise version supports more than 4 CPU. What i don't understand is why both VM's were happy running with 6 CPU's and now after restarting one of the servers there is a problem.

 

Also, surely there must be a way of assigning these CPU's as cores rather physical processors. So I could allocate 8 CPU's to a VM which are configured as four cores per processor.

 

Has anybody experienced anything similar?

[Tallwood_6]

You allocate vcpu's which are cores esentially cores rather than a whole CPU and if you allocate more than one it should simply add an addional core to the existing CPU which is showing under the host, or at least this is how it shows up for me. Its not really best practice to have multiple vcpu's added to a single host, why do you need that many vcpus added to a host?

[Avalon]

its not best practice to add more than one vCPU to a VM? Never heard that before.

 

I am testing a terminal server running as a VM. With 4 CPU's allocated the server cpu usage is critical, with 6 CPU's it runs perfectly.

[Tallwood_6]

I dont know the full reasoning but providing you dont over allocate the number of cores available in the host its not really an issue. I believe essentailly when you allocate a core to a host by default it is not fixed so when a host needs processing resources it has to basically queue for that resource to be made available, if you have 6 vcpus alloacted to a host then it has to wait untill all six cores are ready before it can execute.

[apeo]

It is possible to reserve resources BTW.

 

I really dont get why its not best practice to have multiple vCPUs for a VM unless its beyond what the host has (as you say).

[Tallwood_6]

I'm sure someone who is done VCP or similar will give a more detailed explaination. I have one VM which has 2 vcpu's myself but found i got decent enough performance out of a single VCPU on the others and i'm sure the performance scales differently adding a vcpu vs a phyical core or processor to a server.

[Theblacksheep]

I am having a problem with a VM on vpshere 4. To give a bit of background we have two similar VM's running Server 2003 standard, which both had 6 CPU's allocated in vCenter.

 

For some strange reason after a routine restart of one of the VM's it will not boot with an error message complaining about the number of CPU's. I have contacted VMware support and they said that only the Enterprise version supports more than 4 CPU. What i don't understand is why both VM's were happy running with 6 CPU's and now after restarting one of the servers there is a problem.

 

Also, surely there must be a way of assigning these CPU's as cores rather physical processors. So I could allocate 8 CPU's to a VM which are configured as four cores per processor.

 

Has anybody experienced anything similar?

 

Doesnt help you much but what is it running that you need 6 vcpus?

 

Our UAG box is the most intensive with 3, our 2 exchange 2010 boxes each with 2.

 

 

tallwood is spot on, 6 cores needs to wait for all 6 cores to be available at the same time.

 

If you have 8 (2 4cores), esx will use 1 (1/8)... your 6 vcpu VM can run free on the 7 available cores (7/8). Add in a 2 core VM and ESX will have to wait until 2 cores are free at any one time, with only 1 slot free it'll be fighting with your 6 core machine for CPU scheduling.

 

You can force ESX to use specific CPUs for its cores, you can tell a VM not to share a core with anything else (or VMs). You can also tell a VM to use cores or groups of cores (0, 2-4, 3-7). However keep in mind ESX itself needs a core (not hyperthreaded).

 

Enterprise Plus license is required for 8vcpu, you can test this by reinstalling ESXi on one of your hosts and putting it in evaluation mode. It is possible to run 2 CPUs with 4 cores each but theres no difference between multiple cores and multiple cpus per core.

Edited May 9, 2011 by Theblacksheep

[apeo]

I'm sure someone who is done VCP or similar will give a more detailed explaination. I have one VM which has 2 vcpu's myself but found i got decent enough performance out of a single VCPU on the others and i'm sure the performance scales differently adding a vcpu vs a phyical core or processor to a server.

 

I'd be interested to find out.

 

And as far as i know just because you say its x number of vCPUs, it doesnt quite equate to x number of core (as you say it scales differently). My SQL VM needs atleast 4 vCPUs or some queries just wont run/timeout.

[Arthur]

It's not best practice to add more than one vCPU to a VM? Never heard that before.

I thought everyone knew this. VMware used to recommend using as few vCPUs as possible (see quote from the vSphere v4.0 PDF linked below), but this advice has changed slightly for vSphere v4.1.

 

Performance Best Practices for VMware vSphere v4.1

Performance Best Practices for VMware vSphere v4.0

 

Use as few virtual CPUs (vCPUs) as possible. For example, do not use virtual SMP if your application is single-threaded and will not benefit from the additional vCPUs.

 

Even if some vCPUs are not used, configuring virtual machines with them still imposes some small resource requirements on ESX:

 

• Unused vCPUs still consume timer interrupts.
  
• Maintaining a consistent memory view among multiple vCPUs consumes resources.
  
• Some older guest operating systems execute idle loops on unused vCPUs, thereby consuming resources that might otherwise be available for other uses (other virtual machines, the VMkernel, the console, etc.).
  
• The guest scheduler might migrate a single-threaded workload amongst multiple vCPUs, thereby losing cache locality.
  

This translates to real CPU consumption from the point of view of ESX. For additional information see KB article 1730, listed in “Related Publications” on page 8.

[Soulfish]

I am having a problem with a VM on vpshere 4. To give a bit of background we have two similar VM's running Server 2003 standard, which both had 6 CPU's allocated in vCenter.

 

For some strange reason after a routine restart of one of the VM's it will not boot with an error message complaining about the number of CPU's. I have contacted VMware support and they said that only the Enterprise version supports more than 4 CPU. What i don't understand is why both VM's were happy running with 6 CPU's and now after restarting one of the servers there is a problem.

 

Also, surely there must be a way of assigning these CPU's as cores rather physical processors. So I could allocate 8 CPU's to a VM which are configured as four cores per processor.

 

Has anybody experienced anything similar?

 

It sounds like you were using Enterprise Plus trial licensing or similar and now that the trial has expired it's refusing to bring the VM backup. As others have said don't assign more vCPUs than you need. This is true for all VM platforms (VMware/Citrix/MS/KVM etc), as the VM will have to wait for the same number of physical cores to be available for it to run anything. Best practise is to start with 1vCPU unless you really know you're going to need more. Having done the VMware VCP and Citrix CCA courses its the same explanation from both

 

I've never seen a virtual machine need more than 4 vCPU, and to be honest particularly with things like terminal servers it's much better to bring up extra servers and spread the load than to try and add resources to a heavily used server.

[apeo]

Ah i see.

[Butuz]

I think people are getting confused with VMWares recommendations.

 

They are saying it is best practice keep the number of vCPUs to the minimum required in order to be as efficient with physical CPU resources as possible. I.e. do not allocate 8vCPUs to a simple web server that barely even uses 1vCPU.

 

What they Are NOT saying is "do not use multiple vCPUs ever" and this is the impression many people seem to get. If you have a vMachine that requires multiple vCPUs then that is fine - use as many vCPUs as you need.

 

In my case I have 4 vCPU allocated to each of my Terminal Servers. They work fine, barely any different to physical. The actual CPU overhead due to using VMware is almost nothing its negligible.

 

Happy vCPU'ing!!

 

Butuz

[bio]

This info might help:

 

A single vCPU virtual machine will only use one CPU. A 8 core machine could be simultaneously be running 7 VM's no problem leaving the 8th core free for the host's processes, for example the scheduler.

That last bit is what is important when deciding how many processors to assign to a VM. The host's scheduler is what is used to decide who runs what when.

Lets say you have a dual core machine. You run one VM. If you create that VM with one vCPU, the host only has to pull off a running process off one core to run the VM. If you create it with 2 vCPU's, then the host has to pull off the processes on both cores to run that VM as both cores need to be made available to the VM at the same time. This means just for the host to do anything, it has to pull the VM off the processor. This is where a performance problem occurs. However, if your host had a quad core machine, a 2vCPU VM does not cause the same problem.

On a 8 core machine, you could simultaneously run 4 2vCPU VM's. One of them is going to be scheduled off when the host needs to be doing something, but really that is going to happen eventually. However, that still may not be optimal because 2 cores must be used by each running VM.

In general always create VM's so that the number of vCPU's is less then the number of real CPUs/cores. So on a dual core, in general don't create a 2 vCPU VM. The more real CPUs/Cores you have, the safer for everyones performance it is to create 2 vCPU VM's, though you might still find that single vCPU VMs are still the optimal case. So the basic rule could be stated along the lines, always create single vCPU VMs unless you have special reasons not to.

Now, we all know that hard and fast rules are rarely always right. If a VM is doing something that genuinely can benefit from 2 processors AND has low IO needs, you might see performance increases with creating a 2 vCPU VM on a Dual Core machine, but these cases are outside the norm.

You should also have in the back of your mind that VMware FT currently supports only VMs with just one vCPU. The decision to give every VM multiple vCPUs would block any opportunity to leverage VMware FT. Also, the more CPUs you allocate to the VM, the harder it is for DRS to find an opportunity to perform a VMotion migration to move the VM to a better ESX host.

 

Another interresting read for vsphere 4.1 : http://www.boche.net/blog/index.php/2010/07/25/vsphere-4-1-multicore-virtual-cpus/

 

bio..

Edited May 11, 2011 by bio

[SkreeM1980]

I've never seen a virtual machine need more than 4 vCPU, and to be honest particularly with things like terminal servers it's much better to bring up extra servers and spread the load than to try and add resources to a heavily used server.

 

Totally agree, more servers is a better option, just out of curiosity are you using 32 or 64bit VM's for your guests. We had some amazing results with 64Bit TS's, so may more users on a server with more available resources.