Always-On VPN - Connections keep terminating

[CHiLL]

I've had multiple reports from different users that our Microsoft Always-On VPN keeps disconnecting when they're working remotely, which had been running fine (for the most part) since we deployed it in 2019. The time ranges that have been reported range from every few minutes, to every 15 minutes or so. I've checked the Application Event Log for an affected laptop and noticed this entry (which also appears on other affected user's devices):

CoId={3D750570-366A-0001-B505-753D6A36D901}: The user \ dialed a connection named SJW Always-On VPN which has terminated. The reason code returned on termination is 631.

From what I can tell, this affects remote users more than it affects internal users. By that, I mean if I manually connect the VPN when the device is on-site, then it seems to remain connected for long periods of time. This is only a more recent issue and I'm not sure what's causing it.

 

Our setup:

Clients: Windows 10 21H2 (19044.2486)

VPN Server: Windows Server 2022 21H2 (20348.1487)

Firewall: Sophos XG managed by Wave9

VPN Configuration: Device tunnel, certificate based (PEAP), Ikev2

Edited February 6, 2023 by CHiLL

[aac]

Have you looked at IP fragmentation as we initially found AOVPN quite unreliable but this seemed to stabilize it for us...

 

https://directaccess.richardhicks.com/2019/02/11/always-on-vpn-and-ikev2-fragmentation/

[CHiLL]

Have you looked at IP fragmentation as we initially found AOVPN quite unreliable but this seemed to stabilize it for us...

 

https://directaccess.richardhicks.com/2019/02/11/always-on-vpn-and-ikev2-fragmentation/

Thanks. I didn't know about that and have now implemented it. However, it hasn't resolved the issue for us. I have an SLT laptop that this is happening to and I've also upgraded it to Windows 10 22H2, along with the IKEv2 fragmentation enabled on the server. I have it connected via hotspot from my phone.

 

I've just witnessed the connection terminate in person for the first time, about 10 minutes into the session. I have the Remote Access Management Console open on the server and can see the current connections. I suddently saw the connection disappear from the connected clients list. However, checking the laptop shows the VPN says it's still connected. Checking the event logs of the client laptop, there are absolutely no logs at the time I witnessed the client disconnect (11:48 in the instance as I write this post). There are also no event logs on the VPN server or NPS server for that time either.

[Michael]

Here's a general summary of error 631 which may help!

[Michael]

I'd say more than likely this is probably firewall related in some capacity. For example (as I configure), the Windows Firewall is disabled on the LAN, but enabled off the LAN as one area to investigate.

 

You could try with a subset of users disabling the Windows Firewall off the LAN and see whether this narrows down the problem.

[Oaktech]

We've been seeing some similar issues recently which seem to be network interface related - particularly with Intel ProSET and Realtek devices. Latest drivers from the manufacturer seem to be alleviating it for us. Interestingly, some of the 'latest' drivers from the manufacturer are several versions behind those on the device and available through windows update, but the drivers from WU seem to have features missing which I think might be causing the issue.

[free780]

Thanks. I didn't know about that and have now implemented it. However, it hasn't resolved the issue for us. I have an SLT laptop that this is happening to and I've also upgraded it to Windows 10 22H2, along with the IKEv2 fragmentation enabled on the server. I have it connected via hotspot from my phone.

 

I've just witnessed the connection terminate in person for the first time, about 10 minutes into the session. I have the Remote Access Management Console open on the server and can see the current connections. I suddently saw the connection disappear from the connected clients list. However, checking the laptop shows the VPN says it's still connected. Checking the event logs of the client laptop, there are absolutely no logs at the time I witnessed the client disconnect (11:48 in the instance as I write this post). There are also no event logs on the VPN server or NPS server for that time either.

 

I've seen this happen to multiple devices at the same time.

I've seen it happen to a test laptop. What happens is the route on the VPN Server isn't created so the VPN is connected however there is no route. I can't seem to find anyone else having similar issues.

What seem to make things worse is using auto assignment for client IP rather than DHCP. Mainly to do with client affinity to IP addresses which become important for per user firewall rules.

 

I'm looking at moving to using a 3rd party firewall IPSEC VPN.

 

Disconnections can happen due to ISPs/Poor WI-Fi etc.

[Oaktech]

 

Disconnections can happen due to ISPs/Poor WI-Fi etc.

 

We've noticed that users on TalkTalk connections are far more likely to experience consistent AOVPN issues than any of the other big providers. We've also noticed that on Sky branded routers VPN traffic is the first to be dropped if the router gets busy.

[free780]

We've noticed that users on TalkTalk connections are far more likely to experience consistent AOVPN issues than any of the other big providers. We've also noticed that on Sky branded routers VPN traffic is the first to be dropped if the router gets busy.

 

I was on Talk Talk for a day due to SSE moving to it. Seems like a lot of proxying going on causing a lot of issues.

 

we found we needed our staff the add the VPN Server address here.

 

https://www.sky.com/help/diagnostics/sky-broadband-shield/cant-use-a-vpn

[MatthewL]

More than likely an end user router the cause from experience, these so "filtering/shield" features are an issue. Used to happen years ago on the Linksys routers with VPN's too, used to be a tick box that was needed to be changed.

[MartinRouterKing]

I found users using IKEv2 had more issues than people using SSTP.

 

In the end we allowed both so if users had IKEv2 issues we just swapped them to SSTP.

 

You can actually automate this.

[free780]

I found users using IKEv2 had more issues than people using SSTP.

 

In the end we allowed both so if users had IKEv2 issues we just swapped them to SSTP.

 

You can actually automate this.

 

Yes you can do this. SSTP isn’t as secure as IKEv2 but means the user can connect from networks that allow https but not udp 4500. You can tie Conditional Access if using user auth which can secure things further.