mjk Posted March 3, 2017 Posted March 3, 2017 Nah, 8MBps would be 64Mbps, which if the transfer is to a 100Mbit connected device, would be a reasonable speed. The original post did say "gig ports"
localzuk Posted March 3, 2017 Posted March 3, 2017 The original post did say "gig ports" Ah, I missed that it was the OP. Yeah, definitely something wrong then!
PotNoodleTech Posted March 3, 2017 Posted March 3, 2017 Yeh. OP needs to fill us in on the setup, server spec, switches etc maybe we can figure something out!
howartp Posted March 3, 2017 Author Posted March 3, 2017 Yeh. OP needs to fill us in on the setup, server spec, switches etc maybe we can figure something out! Destination is 2008 R2, quad core Intel Xeon 64bit with 10GB RAM, 2 Broadcom NetXtreme II GigE ports. Origin is 2008 R2, dual core Intel Xeon (vSphere 5.5 vm) with 6Gb RAM, 1 VMXNET3 virtual NIC based on Intel 82580 Gigabit NICs. VMware estate is all on a Cisco 3750 which has four 1GB copper ports trunked together to the HP Core 5406zl. File transfer was running on the destination server. We have same problem with speeds between server VMs and the Veeam Backup server which is in another building, on two 1GB fibre ports trunked between HP Core and a HP 2530-24G POE switch. Peter
howartp Posted March 3, 2017 Author Posted March 3, 2017 Typical - I'm now getting 24 MB/s so I presume I was getting 8MB/s yesterday.
PotNoodleTech Posted March 6, 2017 Posted March 6, 2017 My gut is telling me this is likely a disk issue rather than a network one - what kind of disk/raid config you got on both the VMware setup and the destination server? Do you have battery backup'd cache on the raid controllers?
Oaktech Posted March 6, 2017 Posted March 6, 2017 It could be a disk issue, but if it's also a problem between VMs I wonder if the OP needs a nic driver update. He's already said there is no option to turn off VMQ, so I wonder if a nic driver update will give that option (or fix the issue completely like my last Lenovo server update did)
Sagima Posted March 6, 2017 Posted March 6, 2017 It could be a disk issue, but if it's also a problem between VMs I wonder if the OP needs a nic driver update. He's already said there is no option to turn off VMQ, so I wonder if a nic driver update will give that option (or fix the issue completely like my last Lenovo server update did) I'd still run the powershell command to turn it off even if it wasn't exposed in the gui. It does seem a strong possibility. --- Get-NetAdapterVmq then for all listed as true Set-NetAdapterVmq -Name "NIC Name" -Enabled $False --- There's probably a command line option to do the same. I believe VMQ should be off for 1Gig and under connections and on for 10Gb and over iirc (I don't know about 2.5Gb and 5Gb as I have never needed to look into it as I don't have switches or NICs that support those speeds) 1
howartp Posted March 6, 2017 Author Posted March 6, 2017 My gut is telling me this is likely a disk issue rather than a network one - what kind of disk/raid config you got on both the VMware setup and the destination server? Do you have battery backup'd cache on the raid controllers? VMware is on an IBM SAN with everything RAID5. Some datastores are fast, small capacity SAS disks and some are slower, high capacity SATA disks; SQL and DC's are on the SAS disks, general data is on the SATA disks. This data is from the SATA disks. It's highly likely both the NAS and the Backup server are RAID5 if I've set them up. (I think the NAS was my manager before I took his job) It could be a disk issue, but if it's also a problem between VMs I wonder if the OP needs a nic driver update. He's already said there is no option to turn off VMQ, so I wonder if a nic driver update will give that option (or fix the issue completely like my last Lenovo server update did) I'd still run the powershell command to turn it off even if it wasn't exposed in the gui. It does seem a strong possibility. --- Get-NetAdapterVmq then for all listed as true Set-NetAdapterVmq -Name "NIC Name" -Enabled $False --- There's probably a command line option to do the same. I believe VMQ should be off for 1Gig and under connections and on for 10Gb and over iirc (I don't know about 2.5Gb and 5Gb as I have never needed to look into it as I don't have switches or NICs that support those speeds) I'll try between VMs and see what speeds we're getting - don't think I've tried that recently. I tried the powershell last week but the server didn't have high enough version - i'll have a look further.
Sagima Posted March 6, 2017 Posted March 6, 2017 You can upgrade powershell by updatinging the windows management framework. This will update it to version 5 for Server 2008 R2 or later https://www.microsoft.com/en-us/download/details.aspx?id=50395
PotNoodleTech Posted March 6, 2017 Posted March 6, 2017 VMware is on an IBM SAN with everything RAID5. Some datastores are fast, small capacity SAS disks and some are slower, high capacity SATA disks; SQL and DC's are on the SAS disks, general data is on the SATA disks. This data is from the SATA disks.[/Quote] It would be useful if you used a disk benchmarking something along the lines of crystaldiskmark works well as it's small, straightforward and can immediately identify disk issues e.g. that write cache is not enabled due to not having battery backup cache on your raid card?
howartp Posted March 6, 2017 Author Posted March 6, 2017 SAS Disk, SATA Disk, NAS server and Backup server images in that order.
howartp Posted March 6, 2017 Author Posted March 6, 2017 You can upgrade powershell by updatinging the windows management framework. Done - but needs reboot before I can try it
mjk Posted March 6, 2017 Posted March 6, 2017 At risk of repeating myself, I did give an example of exactly how you should measure your network speeds before you go looking for other issues. Ignore me by all means but VMware are offering exactly the same advice : https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2052119
PotNoodleTech Posted March 7, 2017 Posted March 7, 2017 The disks speeds don't look to be an issue so I would go with mjk idea. hey mjk it only takes 5 minutes to run crystal disk mark, well worth doing to identify any obvious disk issues. I've seen people looking at network issues for days and when they eventually run disk benchmarks their disks were incapable of writing at greater than 10MB/s (crap controller, no cache, no writeback cache, SATA disks)!!
howartp Posted March 7, 2017 Author Posted March 7, 2017 I did give an example of exactly how you should measure your network speeds before you go looking for other issues. No you didn't - you suggested I "use a network diagnostic tool to get the number" which means nothing to me. Now that you've given me a set of testable commands, I'm more than happy to try them and report back. The nc command test on the VMware link you gave me is giving ~20 MbTX/s and ~950 MbRx/s on both the vmk2 kernel port and the vmnic3 vSwitch port. The vSwitch that all the VMs are connected to has three physical NICs on it - vmnic2, vmnic3 and vmnic4 - but there's generally only traffic showing on vmnic3 - shouldn't I expect traffic on all three vmnics most of the time? (There is some traffic on the other 2 but not much) Peter
ITGURU Posted March 7, 2017 Posted March 7, 2017 All depends on the type of files being copied. one large file will always have a better throughput.
mjk Posted March 7, 2017 Posted March 7, 2017 No you didn't - you suggested I "use a network diagnostic tool to get the number" which means nothing to me. Now that you've given me a set of testable commands, I'm more than happy to try them and report back. The nc command test on the VMware link you gave me is giving ~20 MbTX/s and ~950 MbRx/s on both the vmk2 kernel port and the vmnic3 vSwitch port. The vSwitch that all the VMs are connected to has three physical NICs on it - vmnic2, vmnic3 and vmnic4 - but there's generally only traffic showing on vmnic3 - shouldn't I expect traffic on all three vmnics most of the time? (There is some traffic on the other 2 but not much) Peter Sorry if it wasn't clear : my very first post had a netcat command posted into it. Things are pointing to a network issue if the transmit speeds are down. I would try plugging in another machine (a laptop or something) into the port the vmware sever is in and trying nc between two machines ont eh same switch https://sourceforge.net/projects/nc110/
localzuk Posted March 7, 2017 Posted March 7, 2017 I'd use iperf to test network speeds. I second this one. Used it a lot with our VPNs to locate problems.
mjk Posted March 7, 2017 Posted March 7, 2017 ... it's why I didn't bang on about netcat too much and tried to talk about a 'generic network tool'. iperf is good too.
howartp Posted March 7, 2017 Author Posted March 7, 2017 Sorry if it wasn't clear : my very first post had a netcat command posted into it. Sorry - I did see that post originally but I'm not a Linux person so those commands went completely over my head - I hadn't realised it was you that posted it when I was looking back to your subsequent post. I seem to think I've used iPerf before so I'll have a play with that. Peter
howartp Posted March 8, 2017 Author Posted March 8, 2017 (edited) iPerf is giving me 800 Mbits/sec both ways from VM to NAS. NAS to Backup is 940 Mbits/sec both ways. VM to Backup is 650 Mbits/sec Backup to VM is 880 Mbits/sec Edited March 8, 2017 by howartp
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now