Frodo_Baggins Posted February 25, 2022 Posted February 25, 2022 Bit stuck on this one, any thoughts on where to look next? End user (21H1) reports latency when opening files from and navigating to network shares (When seeing the issue in person it looks they are re-indexing every time they go to a sub folder? both shares with issues are on the same server, other shares on other servers/hosts are fine.) General performance on the VM is not great and laggy but not seeing high usage or spikes in resource usage when users report and performance on the host is good (seemingly no pattern in reporting time.) Lots of Event ID: 1020 which relate to a delay in opening the files and most mornings this week Event ID: 1032 but didn't collect a log/dmp due to "Stalled I/O" Uninstalled the last batch of Windows updates on the server and after a long restart the issue remains I've got Sophos Intercept Advanced X installed but not had any issues on any other servers... Found this thread but not had chance to look through it yet as it looks like a bit of a minefield Server 2019 network performance - Windows Server - Spiceworks Any thoughts or rough ideas on where to look outside of moving the VM to another host?
TwistedHelixis Posted February 25, 2022 Posted February 25, 2022 is the issue with just one user? what are those shares like if the users logs in from a different laptop?
Frodo_Baggins Posted February 25, 2022 Author Posted February 25, 2022 Thanks, All users on site are effected, performance is the same on both laptops and PC's. Wont admit to being an expert on Hyper V but VMQ and RSC is enabled on the VM (can see quite a few threads online saying to turn these off on in 2019 due to performance issues)
pantscat Posted February 25, 2022 Posted February 25, 2022 Might seem odd - but check the power plan is not set to Balanced. I've experienced disk latency issues with VMs that weren't set to "High Performance". 2
foofighterjim Posted February 25, 2022 Posted February 25, 2022 Thanks, All users on site are effected, performance is the same on both laptops and PC's. Wont admit to being an expert on Hyper V but VMQ and RSC is enabled on the VM (can see quite a few threads online saying to turn these off on in 2019 due to performance issues) We had something broadly similar a few years ago and it was our AV causing the issue (or rather my poor configuration of it). Basically we had on demand file scanning active on the clients and the server policies. This was obviously fine when we were testing, but once everyone logged on and the server was scanning every file being accessed this obviously caused quite a queue for disk read / write attempts. 1
Frodo_Baggins Posted February 25, 2022 Author Posted February 25, 2022 We had something broadly similar a few years ago and it was our AV causing the issue (or rather my poor configuration of it). Basically we had on demand file scanning active on the clients and the server policies. This was obviously fine when we were testing, but once everyone logged on and the server was scanning every file being accessed this obviously caused quite a queue for disk read / write attempts. Coming from Sophos and having the same before I immediately thought the same, scan on read and write are turned on so will keep an eye on that - Thanks!
Frodo_Baggins Posted February 25, 2022 Author Posted February 25, 2022 Might seem odd - but check the power plan is not set to Balanced. I've experienced disk latency issues with VMs that weren't set to "High Performance". Just changed it as it was set to balanced, will monitor. Cheers!
Frodo_Baggins Posted March 1, 2022 Author Posted March 1, 2022 More reports today of the same issue, will keep updating this thread in case anyone else has the same issue! Have just logged a ticket with Sophos
Jawloms Posted March 1, 2022 Posted March 1, 2022 I had this a few years ago and it was simply the driver for the disks/RAID. It had put a default one in, when I installed HP's proper one it was fine.
Oaktech Posted March 1, 2022 Posted March 1, 2022 Is the network card a Broadcom one? You may need to disable VMQ
Frodo_Baggins Posted March 1, 2022 Author Posted March 1, 2022 Is the network card a Broadcom one? You may need to disable VMQ Thanks, It's HP/Intel - VMQ & RCS are disabled
Oaktech Posted March 3, 2022 Posted March 3, 2022 Thanks, It's HP/Intel - VMQ & RCS are disabled https://www.intel.co.uk/content/www/uk/en/support/articles/000005593/ethernet-products.html I had problems with an HP server's performance that turned out to be needing to turn off IPV4 Large Send offload, RSS and fine tuning other Offloading options. Can't remember now exactly what the combination was now! 1
Frodo_Baggins Posted March 3, 2022 Author Posted March 3, 2022 (edited) https://www.intel.co.uk/content/www/uk/en/support/articles/000005593/ethernet-products.html I had problems with an HP server's performance that turned out to be needing to turn off IPV4 Large Send offload, RSS and fine tuning other Offloading options. Can't remember now exactly what the combination was now! Thanks will give this a look, banging my head against the wall a bit If you dont mind and still have access could you DM the settings you have on the network adapter - Just disabled IPV4 Large Send Offload (I enabled a similar setting called "Large SMB MTU" when testing) Edited March 3, 2022 by Frodo_Baggins
Oaktech Posted March 3, 2022 Posted March 3, 2022 (edited) Thanks will give this a look, banging my head against the wall a bit If you dont mind and still have access could you DM the settings you have on the network adapter - Just disabled IPV4 Large Send Offload (I enabled a similar setting called "Large SMB MTU" when testing) Unfortunately I don't have access to that server any more... Let me check the settings on an HP VM Host I do have access to. *Edit* I've just checked - the only HP I have access to has a Broadcom chipset. Edited March 3, 2022 by Oaktech 1
Frodo_Baggins Posted March 3, 2022 Author Posted March 3, 2022 Looks like it could be a failing disk on the san, a lot of 149 & 147 events 1
Oaktech Posted March 3, 2022 Posted March 3, 2022 Any logging/errors on the SAN itself? My EMC web console gave a load of SMART warnings when one of the disks failed, but I had to log into it directly.
Frodo_Baggins Posted March 3, 2022 Author Posted March 3, 2022 Any logging/errors on the SAN itself? My EMC web console gave a load of SMART warnings when one of the disks failed, but I had to log into it directly. No warnings, only alerts for firmware updates
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now