I need some help with our VPN solution, if you would be so kind :)
The full error message is
Remote Access error 789 - The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer
This only happens on one single users PC, and only when he is on his home network. No other clients is having issues.
I've tried to change the network profile from Public to Private. Did not help. I've also tried to change it to Domain network, and that didn't help either.
We've got Symantec Antivirus on all machines, but again, there hasn't been any troubles like this on ANY other client.
The firewall SEEMS to be turned off when on Private and Domain network, but not on Public. But this is controlled by the antivirus.
I haven't got immediate access to his router, and would prefer if this could be leaved "untouched"
The connection is a L2TP/IPsec, requires encryption, with a Pre-shared key. The clients use a batch, which is using rasdial to connect.
The server itself, is a Meraki firewall.
Please, ask any questions if you have any.
25 Answers
It can be a protocol incompatibility (AES/3DES, etc).
- Check the firewall settings
- UDP port 500 and 4500 should be NATed and 1701 forwarded
- Run services.msc as administrator.
- Find “IKE and AuthIP IPsec Keying Modules” and “IPsec Policy Agent” Check the status, right click to “restart” if it states “started” If the “started” option is disabled, enable it.
- Right click and scroll down and click on Properties Select the “Startup type” and change it to “Automatic” and save Restart your VPN and it should work smoothly now as the protocol settings should reset to default
- If that does not work, you will have to manually set the encryption method for both server and clients so they are compatible.
From: Fix Windows 10 VPN error 789 connection failed due to security issues:
Remote Access error 789 pops up when your system is not properly set up to connect to an L2TP server, thus the connection attempt fails even before you establish a connection with the server.
It is also linked to incorrect configuration of your operating system like Windows 10 in this case. This generic error is thrown when the IPSec negotiation fails for the L2TP/IPSec connections.
Other possible causes include:
- L2TP based VPN client (or VPN server) is behind NAT.
- Wrong certificate or pre-shared key is set on the VPN server or client.
- Machine certificate or trusted root machine certificate is not present on the VPN server.
- Machine Certificate on VPN Server does not have ‘Server Authentication’ as the EKU.
Here are solutions you can use to fix Windows 10 VPN error 789 on your computer.
- Reset network adapter
- Check the certificate
- Re-enable IPSec on your computer
Before trying any of these solutions, ensure that L2TP and IPSec pass-through options are enabled from your router. If you configured your VPN service manually, then make sure you use the preshared key.
Reset network adapter
Right-click Start and select Device Manager.
Find Network adapters and click to expand the list
Identify your network adapter and right-click on it, then select Uninstall.
- Click OK.
- Restart your computer. The device will reinstall and should reset it to default settings.
If this doesn’t fix error 789, try the next solution.Check the certificate
Ensure the correct certificate is used both on the client and the server side. In case Pre Shared Key (PSK) is used, ensure that the same PSK is configured on the client side, and the VPN server machine.
Re-enable IPSec on your computer
Right-click Start and select Run.
Type
services.mscFind 'IKE and AuthIP IPSec Keying Modules'.
Find 'IPSec Policy Agent'.
Check the status. If it says 'started' click to restart. If the 'started' option is disabled, enable it.
- Double-click on each of the two.
Select Startup type.
Change it to Automatic.
Save the changes.
- Restart your VPN service.
Once you have done all the steps above carefully, the VPN should work smoothly as the protocol settings have been reset to default. If, however, it doesn’t work, you have to manually set the encryption method both for the server and the client side, in order for them to be compatible.
In case you have a user-specific issue on your computer yet you still get error 789 after trying any of the above solutions, you can also contact the customer care or tech support team for your specific VPN provider and share the details for further assistance.
1In my case, the network administrator changed the vpn server configuration, so I got error 789, after changing my vpn config from VPN Type L2TP to the correct PPTP or Automatic, it worked.
Though this solution is implied by a number of other answers, the only thing I had to do is described by the third option on this blog. I copy it here in case the link goes bad:
- IKE and AuthIP IPsec Keying Modules disabled: Solution: This occurs most often when 3rd party VPN software has been installed and disables the IKEEXT service. This can be re-enabled by navigating in Windows to Control Panel > Administrative Tools > Services. Find the service named “IKE and AuthIP IPsec Keying Modules” and open it. Change the Startup type to “Automatic”. it may be necessary to remove the 3rd party VPN software.
In my case, I didn't have to uninstall any 3rd party VPN software. I happened to be running Windows 10 (1803) at the time.
The January 2022 Cumulative Update for Windows 10 can interfere with IKEv2 VPN connections on some version of Windows 10 [KB5009543 (OS Builds 19042.1466, 19043.1466, and 19044.1466)]. The update has not been fixed as of January 20, 2022 so it seems the only remedy is to uninstall and block it for now.
This blog post suggests downloading the Windows Update Troubleshooter if you don't already have it.
Uninstall KB5009543 by going to:
- Control Panel -> All Control Panel Items -> Programs and Features, then,
- click 'view installed updates'
- select the update for KB5009543,
- click 'Uninstall' (at top of installed updates list).
- click 'restart now' to reboot your computer.
Run the above mentioned troubleshooter:
click next,
click Hide updates to †see a list of available updates and select KB5009543.
†note: you must have automatic updates turned on in settings for this to be work.
Hopefully one of the next cumulative updates will repair this issue.
2
