VLANs in Virtualisation

[Dave_O]

1. again, in your school. Push some heavy 3D or movie files over that and your calcs go out the window

 

2. OMG you have taken seporation to the next level in that, I know that memory dedupe exists but that kind of seporation is very specialised. I hate to think of the queues your VMs have to wait in to get access to the network cards propper and all the traffic on virtual DMA mapping.

 

Impressive storage but 8GB shared between that many hosts still provides a bottleneck. We also have a hp 5412 which is a nice bit of equipment but again, every school and implementation is different.

 

1. We do and it doesn't. Do you have figures you can share that show otherwise?

2. 48 VMs split across 3 ESX that's an average of 16 VMs per host. Given there are 4 nics trunked per host that's theoretically 4 VMs per nic. Now given that there are 6 cores per CPU ie 12 per host that's 12 cores for 16 machines lets say worst case 2 machines sharing one core so the core has to cycle 2 machines for the 4 nics. Let also look at a worst case where 2 file servers are on the same host using the same core and the same nic and are offering 100Mb files to 2 different users. They could then only offer half of the card's 1Gb capability. Explain to me how that represents a bottleneck?

3. 8Gb across 3 hosts a bottleneck??? Not sure what you're talking about here, the SAN fibre or something else? If it's the fibre you're talking about then trust me when I say it really doesn't get any better than that. Look at the screen shot below. This is the V7000 during a school day (usually no more than 3000 IOPS) where I set 3 SAN Veeam backups going (all day) as well, just to see what effect this had on performance and the user experience. Bear in mind that the V7000 (in this setup) is capable of serving 56000 IOPS and has 8Gb cache per controller. It had no effect, no latency in delivering files web pages etc.

 

Edited March 30, 2013 by Dave_O

[Dave_O]

No-one has mentioned 10GbE fibre. My host servers have 10GbE ports onboard, and the storage server has it as an extra card. They plug into an 8 port 10GbE module in our HP 5406zl switch, which was full. So, our purchasing decision was to buy the 1GbE versions of the servers and then trunk things, or just spend the £2k on a 10GbE module for the core, and then we used copper 10GbE cables (gbics formed into the cable).

 

Worked out cheaper for us than going 1GbE.

I stand corrected 10GbE

[SYNACK]

1. We do and it doesn't. Do you have figures you can share that show otherwise?

2. 48 VMs split across 3 ESX that's an average of 16 VMs per host. Given there are 4 nics trunked per host that's theoretically 4 VMs per nic. Now given that there are 6 cores per CPU ie 12 per host that's 12 cores for 16 machines lets say worst case 2 machines sharing one core so the core has to cycle 2 machines for the 4 nics. Let also look at a worst case where 2 file servers are on the same host using the same core and the same nic and are offering 100Mb files to 2 different users. They could then only offer half of the card's 1Gb capability. Explain to me how that represents a bottleneck?

3. 8Gb across 3 hosts a bottleneck??? Not sure what you're talking about here, the SAN fibre or something else? If it's the fibre you're talking about then trust me when I say it really doesn't get any better than that.

 

1. no, our system is different and almost certainly smaller but not all systems or HD video classes are the same.

2. It is not about the cores, NIC traffic has to travel on the virtual bus and then go into the pool of avalible devices, if you have 16 hosts or even 4 competing for one NIC at some point your NIC queues are going to get expanded if nothing else. The CPUs are another matter, If they are Core I level then there are several hardware DMA channels built in to the cpu. If you are using older CPUs or even have a stack of hosts on each making DMA alls then you could end up with network and other hardware traffic taking the long way round through the CPU instead of direct hardware communication. We are not scaleing to to two users here but dozens or hundreds per instance as that level of seporation implies.

3. You have 4GB comming our of every host which is 12GB/s vs 8GB/s of storage bandwidth for everything going out to the hosts and into the VMs, if nothing else there is potential. It also does get better, 10GB/s iSCSI or even better teamed 10GB/s. There is also the option of multiple SANs to spread out the bandwidth.

 

Now, don't get me wrong, your system probably runs great and may have addressed most or all of the contention issues but you can't take your system and apply it to every system everywhere. You don't know what everyone else is doing or requireing. It may not even be bandwidth but realtime, ultra time sensitive stuff. Anyhow I am just saying that your single use case does not constitute proof that everyone else is somehow wrong or unjustified.

[Dave_O]

1. no, our system is different and almost certainly smaller but not all systems or HD video classes are the same.

2. It is not about the cores, NIC traffic has to travel on the virtual bus and then go into the pool of avalible devices, if you have 16 hosts or even 4 competing for one NIC at some point your NIC queues are going to get expanded if nothing else. The CPUs are another matter, If they are Core I level then there are several hardware DMA channels built in to the cpu. If you are using older CPUs or even have a stack of hosts on each making DMA alls then you could end up with network and other hardware traffic taking the long way round through the CPU instead of direct hardware communication. We are not scaleing to to two users here but dozens or hundreds per instance as that level of seporation implies.

3. You have 4GB comming our of every host which is 12GB/s vs 8GB/s of storage bandwidth for everything going out to the hosts and into the VMs, if nothing else there is potential. It also does get better, 10GB/s iSCSI or even better teamed 10GB/s. There is also the option of multiple SANs to spread out the bandwidth.

 

Now, don't get me wrong, your system probably runs great and may have addressed most or all of the contention issues but you can't take your system and apply it to every system everywhere. You don't know what everyone else is doing or requireing. It may not even be bandwidth but realtime, ultra time sensitive stuff. Anyhow I am just saying that your single use case does not constitute proof that everyone else is somehow wrong or unjustified.

 

1. OK lets set up real world test that we can use for other people to try and share the information. How about we have a video file that is made available to students in a group lets say a group of 20 in a class that they all play simultaneously 10 minutes into the lesson to avoid login and logout profile issues. It's a bit of a rough figure but it will give an indication of peoples systems in a real world sense. We collect information on the % bandwidth usage of each of the ports used by the switch(es) directly servicing the ESX servers (in with other normal school traffic) and the IOPS of the storage (if you can). Then rather than talking about it and dismissing specific examples we have some hard facts (across a number of school) that actually mean something.

 

2. See below. I'm not sure how long you have been working in a school and with virtualisation and also what collected date you are basing your conclusions on (I'm sure your "theoretical" propositions have some basis in fact) but I would suggest you information bears little resemblance to reality.

 

3. You really need think about the context in which you are offering these scenarios. Schools do not have tens of thousands of users logging in simultaneously, there are probably 3-400 maximum at any one time. You just don't need massive network or storage bandwidth to service this number of concurrent users. Again if you have evidence to the contrary then please share it.

[SYNACK]

1. OK lets set up real world test that we can use for other people to try and share the information. How about we have a video file that is made available to students in a group lets say a group of 20 in a class that they all play simultaneously 10 minutes into the lesson to avoid login and logout profile issues. It's a bit of a rough figure but it will give an indication of peoples systems in a real world sense. We collect information on the % bandwidth usage of each of the ports used by the switch(es) directly servicing the ESX servers (in with other normal school traffic) and the IOPS of the storage (if you can). Then rather than talking about it and dismissing specific examples we have some hard facts (across a number of school) that actually mean something.

 

2. See below. I'm not sure how long you have been working in a school and with virtualisation and also what collected date you are basing your conclusions on (I'm sure your "theoretical" propositions have some basis in fact) but I would suggest you information bears little resemblance to reality.

 

3. You really need think about the context in which you are offering these scenarios. Schools do not have tens of thousands of users logging in simultaneously, there are probably 3-400 maximum at any one time. You just don't need massive network or storage bandwidth to service this number of concurrent users. Again if you have evidence to the contrary then please share it.

 

1. sounds practical

2. Oh patronising... Look up the CPU architecture and performance metrics on network cards it is based in fact.

3. as above I am only responding to the scaling that you put forward as an example, that kind of seporation is scaled for thousands of users, I am going on your setup. I am fully aware of the fact that most schools are not that big.

 

Again I am just providing alternatives, your view of the one true way just pushes my buttons, you do not allow for any system or schenario that is not your own.

[Dave_O]

1. sounds practical

2. Oh patronising... Look up the CPU architecture and performance metrics on network cards it is based in fact.

3. as above I am only responding to the scaling that you put forward as an example, that kind of seporation is scaled for thousands of users, I am going on your setup. I am fully aware of the fact that most schools are not that big.

 

Again I am just providing alternatives, your view of the one true way just pushes my buttons, you do not allow for any system or schenario that is not your own.

 

2. Yes couldn't resist that one, especially when I note you are half a world away (or at least I hope you are!)

3. My way is the true way! You should know that from reading my other posts! Seriously though there is a lot of mis-information out there (especially from vendors) about what is needed in a school scenario. What I don't want is schools spending an inordinate amount money on a solution that is totally over spec'ed and unnecessary and not fit for purpose. Hence my constant rants about VDI

[SYNACK]

2. Yes couldn't resist that one, especially when I note you are half a world away (or at least I hope you are!)

3. My way is the true way! You should know that from reading my other posts! Seriously though there is a lot of mis-information out there (especially from vendors) about what is needed in a school scenario. What I don't want is schools spending an inordinate amount money on a solution that is totally over spec'ed and unnecessary and not fit for purpose. Hence my constant rants about VDI

 

2. Oh I am everywhere, that scratching at the windows, the creak of the floorboard, that's me

 

3. Fair enough. We don't have 10GBe either but there may well be a place for it, VDI is an interssting subject too, but not overly economical till really cheap client hardware shows up with full acceleration handoff.

[ConradJones]

No-one has mentioned 10GbE fibre. My host servers have 10GbE ports onboard, and the storage server has it as an extra card. They plug into an 8 port 10GbE module in our HP 5406zl switch, which was full. So, our purchasing decision was to buy the 1GbE versions of the servers and then trunk things, or just spend the £2k on a 10GbE module for the core, and then we used copper 10GbE cables (gbics formed into the cable).

 

Worked out cheaper for us than going 1GbE.

 

+ 1 cable vs 6

[Dave_O]

I have been using Veeam One over the last couple of days and remembered this thread. The attached image shows the Network usage from 7:40am to 2pm today Thursday. Just a normal day in toyland. Bear in mind this represents network usage across all virtual machines not just a few. Not sure what happened at 13:18 but even that only peaks at 60MBps. At that time there were 323 logged in users and it was the middle of a lesson so could be anything...

 

[localzuk]

I've had our storage server task manager open for a while today, and it averages around 10Mbps, however, it can and does peak at > 1Gbps at times. I had it hit 1.5Gbps last night.

[Dave_O]

Was that during a backup routine?

[localzuk]

Was that during a backup routine?

 

No, that was during a complete reboot of all VMs.

[Dave_O]

Yes that would do it!! I don't normally see that amount of activity unless I am rebuilding a VM pool through the VMWare View broker!

 

I do have to ask, why? If it's VMware then possibly tools upgrade following patching. For Microsoft stuff I don't know is there an equivalent patch procedure. Or do you just like putting the systems through its paces!