bergmbe

Well that gives me great hope that having switched to that will fix our issues as well. I need to download and learn to use IOMeter so I can stress test it, but my job's other responsibilities (part time IT guy here) have gotten it the way. People don't tend to give me as much freedom to troubleshoot when things (to their eyes) are working just fine.

Definitely not specific to ESXi 5.1 or earlier. I have ESXi 5.5 Build 1474528 and still experienced this problem consistently. Currently I'm in the process of verifying stability after switching the network adapters from VM's E1000 to VMXNET3 to see if that fixes it. If you're not experiencing the problems, would you mind checking to see what network adapters you've assigned to the VM running 2012R2? Are they E1000's, or VMXNET3? Or something else? If 2WheelTinker's VMXNET3 supposition doesn't fix it, I intend to implement the fix detailed in the technet link above. So far, after a few days of stability, I haven't had any issues with the network adapter change. But I also haven't had time to stress test it to see for sure if it's fixed.

From the technet thread courtesy of Vitis_Vinifera "RESOLVED! OK People! I hope I can help you everyone out with this solution MS have provided me. I have been dealing closley with the Network Team, in particular the most skilled escalation tech of Asia Pacific who happened to be an expert in SMB. My system has been stable for around 10 days now. My Environment vmware ESXi 5.1.0 build 1123961 Server 2012 R2 My Symptoms Random network dropouts approx 2-4 per day, cannot access any file shares \\servername\share, \\severname\c$ etc During the dropout server 2003 boxes can access the shares, all OS's above 2003 cannot (indicating there is an issue with SMB2 and above). For more info on SMB versions see Windows Server 2012 R2: Which version of the SMB protocol (SMB 1.0, SMB 2.0, SMB 2.1, SMB 3.0 or SMB 3.02) are you using? - Jose Barreto's Blog - Site Home - TechNet Blogs Restarting would fix the issue immediately MS have recommended two components to this fix, however, with the vm driver fix applied I was still experiencing the issue, it wasn’t until I made the change to the srv2.sys that the fix became permanent. Vmware driver; MS believe that this driver (vsepflt) could have been conflicting with the srv2.sys driver. To disable follow this article. VMware KB: Degraded Windows network file copy performance after full ESXi 5 VMware Tools installation Srv2.sys; In my opinion this is the actual change that fixed the issue. The srv2.sys driver controls SMB 2 traffic at the kernel level, in operating systems pre Server 2012 R2 the driver is set to auto start. Microsoft have changed the functionality for Server 2012 R2 to ‘start on demand’, this seems to not be starting gracefully when a request is made on SMB 2 or above. To change srv2.sys to auto start, open cmd and type sc config srv2 start=auto a reboot will be required after running this command. When I talked to the lead network tech about this fix, referenced him to this article, how many people it was affecting etc, he advised that in all cases experiencing this issue changing the srv2.sys to auto start has worked 100% of the time. Why haven’t MS release an official patch I asked – because there have not been enough cases to warrant an official fix. I’m very interested in if this fixes your issues, please mark this as an answer if I bring you success! Best of luck!"

Thanks 2wheel, I'll definitely give this a shot this weekend. I've since moved the bulk of our files to a very simple Samba fileshare on Ubuntu Server 12.04 LTS. Works like a dream, but I don't have the know how to properly set up user based access in command line, so I'm planning to move back to 2012R2 when I get a fix in place. I left a small fileshare and print sharing on the VM running 2012R2, so I'll switch the NICs over from E1000 to VMXNET3 and see if it solves it on our end as well. I'll report back next week once I've had a chance to test it.

You're going to laugh, but at the moment I'm rebuilding the infrastructure from the ground up so that's my excuse. We have a nearly pristine Windows Server 2012 R2 install with no antivirus currently running. Individual workstations are mostly running microsoft security essentials for now.

Same here. Reoccurred this morning, going to see how today goes. If it changes the frequency of problems at all. In the meantime, I'll proceed with moving all of our files over to Ubuntu LTS and getting a samba fileshare up. It's not an ideal solution, but at least I won't have to reboot it every few hours.

I had previously disabled smb 2/3 on our server anyway, thinking it might be related given that it seemed to fix similar problems in 2008R2. All smb connections are at 1.5. If I'm understanding correctly, leasing wasn't introduced until SMB 2 or 2.1. If I'm still having the issues, it seems (logically) like disabling leasing wouldn't necessarily help. That said, I'll probably try re-enabling smb 2/3 later today or tonight and then disabling leasing. Because I'm flat out of ideas at this point. ETA: Had our clients begin losing access, after 1/3rd of the company had lost access, I opted to re-enable SMB2 & 3 in powershell, then add the registry key DJL recommended prior to rebooting the server to restore access. I'll let you know if/when we have the same problem again. After looking at this a second time: http://support.microsoft.com/kb/2696547 I noticed that SMB Leasing isn't listed as being disabling when you turn smb2 & 3 off, so it's possible that my previously disabling smb 2 & 3 wouldn't have also disabled leasing. I've since verified with get-smbserverconfiguration in powershell that leasing is, in fact, disabled now.

Any way to filter the results so it doesn't list listening ports, but only active ones? I did a netstat -? in powershell and looked up a few things, but didn't see an easy way to filter for that in the results.

I'm only seeing a few entries at a time using :389, and no discernible difference in the before, during and after file share issues. What I am seeing, and this may be normal as I haven't used netstat often enough to know if this is normal, is seeing a TON of entries overall. Pages and pages of them, most for TCP 127.0.0.1:randomport# showing state: esablished, or UDP 0.0.0.0:randomport#, or UDP [::]:randomport# both of the latter two showing *.* as their state. Perhaps you could tell me what those mean? Because that's an awful long list. PS: If you have time to explain, assume I'm a total admin noob.

@cogrady84 For what it's worth, disabling SMB2/3 seems to have stopped me from having to reboot workstations. It still stalls out when they go to access a shared drive after having a stalled save, but it seems to eventually reconciles itself after anywhere from 20 seconds to ~3 minutes. It's not a solution, but it allowed me to keep my system up in most cases. This only applies to Windows XP and Windows 7 home/pro Basically if someone has a problem saving, I do this and it seems to avoid a workstation reboot: Let Excel/Word/whatever chug along and try try try in the background. It leaves the 'saving' prompt up with a cancel button, which I do not hit. Attempt to open the shared drive. Wait for the shared drive to properly display contents - usually 40 seconds to 2 minutes. Once it properly displays contents, navigate back to the program trying to save, hit cancel, wait for it become responsive again, and the try saving again. Obviously this is by no means a solution, but believe it or not it's the only way I've been able to avoid a reboot on a workstation. Which is (unfortunately) necessary when they have 15 documents open that need to be saved. Here's the kicker... I get through that entire process, and it can take up to 5 minutes... and the only events I see in the logs on either the server or workstation relate to a credential negotiation (Event 4776) that returns a successful result (Event 4624). Once in a while, there's a 2012 network error event on the server, and of course it always registers apphang events related to excel/word/etc and explorer if that froze up for too long trying to display contents of a network share.

ETA: After re-reading the whole thread, I suspect mine is a separate issue from what's being discussed here. I'll try posting my own question later. Leaving what I wrote before in case I'm wrong... I'm not positive the problem I'm having is exactly the same as what's described here. Here's the reason I'm not sure this is related to what you guys are discussing: All of our clients don't fail at once. It's one client at a time, not all at once. And often as not, after a little waiting, the client WILL eventually reconnect to the file share. Not always, but usually. The problem is, once one person is having the problem, it often occurs on other client machines and the only sure way to fix it all at once is just to reboot the server. Details below: Single Server: VMWare ESXi 5.5 build 1474528 with Windows Server 2012 R2 (Domain Controller) running DHCP, DNS, AD, File and Printer sharing Client computers: Mostly Windows XP, some Windows 7. For us, the problem manifests as a stall on a client pc (Most often Windows 7, very rarely Windows XP) when we go to save files in Office 2007 to the mapped network drive that points to the server. It doesn't happen every time, and the length of time a computer has had the file open doesn't seem to affect it either. That makes me think it isn't an opportunistic locking problem. It doesn't appear to be limited to office files, but that's where we see it most frequently as that's the most frequent type of file in use. When we try to access the mapped network drive during the stalled save, it stalls opening the window for a while, but eventually opens in most cases. Often as not, you can then find a way to pull office out of its stall and save properly it properly. It creates a .tmp file in the directory that was being saved to sometimes as well, which isn't unusual for Office. During these stalls, a ping command to both the IP and the name of the server works fine. NSLookup works fine. Address resolution seems to work fine for everything on the network. Here's the weird thing: Eventviewer isn't showing any consistent errors across these instances besides the app hang errors and credentials being submitted and verified by the server. The server isn't showing any consistent errors either. The events that are being logged on a fairly regular basis are 1001 (hang) and 4648 (log on, resolves successfully) I see a simultaneous event on the server, ID 4776 credential validation, and then 4624 logon successful. Once in a while, I also see a warning - ID2012 System Log warning indicating a network error during transmitting/receiving data. I've tried swapping out our network devices (switches & gateways) and that didn't solve the issue. I've tried disabled smb2/3 on the server and the client machines on account of reports that it could cause issues just like this in Office during saves to mapped network drives. No dice so far. This happens at least once a day, and it's driving me bonkers. The ONLY real fix that gives me a few hours of peace is to reboot the server. Thankfully, it reboots inside of 3 minutes, but it destroys workflow in the office on anyone using shared files. (That's everyone in the office.) Now that I know what service to try restarting (I'd previously tried restarting a bunch of services related to SMB and file sharing, but hadn't tried restarting the server service.) I'll give that a shot next time it happens and see if I can avoid a reboot. I'm going on a month and a week of troubleshooting this, and I've pulled out just about all my hair. Very much looking forward to that list of services they disabled. I can't get funding from my company to get outside help or open a case with Microsoft. So you guys are my best, last hope. Interesting links worth browsing related to all the searches I've done trying to figure out what the hell is causing this, some of which come from this thread. Whitepaper on Opportunistic Locking and possible file corruption problems: Opportunistic Locking and Read Caching on Microsoft Windows Networks SMB commands for enabling/disabling smb 1, 2/3 on various operating systems: How to enable and disable SMBv1, SMBv2, and SMBv3 in Windows Vista, Windows Server 2008, Windows 7, Windows Server 2008 R2, Windows 8, and Windows Server 2012 Current hotfixes for 2012 & 2012R2 related to file sharing issues: List of currently available hotfixes for the File Services technologies in Windows Server 2012 and in Windows Server 2012 R2