"Error 789" error message when you use an L2TP VPN client through a Windows 2000 Terminal Services client session (326751)


The information in this article applies to:

  • Microsoft Windows 2000 Advanced Server
  • Microsoft Windows 2000 Server
This article was previously published under Q326751

SYMPTOMS

When you try to use a virtual private network (VPN) connection through a Microsoft Windows 2000 Terminal Services client session, you may receive the following error message:
Error: 789 "The L2TP connection attempt failed because the security layer encountered a processing error during initial negotiations with the remote computer"
However, you can successfully establish the VPN connection from the Terminal server console.

CAUSE

This issue may occur if both of the following conditions are true:
  • You establish a client session by using Windows 2000 Terminal Services.
  • You try to establish a VPN connection by using Layer Two Tunneling Protocol (L2TP) from the Terminal server to connect to a Windows 2000 Server-based computer that is configured as a VPN server.

RESOLUTION

To resolve this issue, you can use a preshared key on both ends of the VPN connection. The L2TP/IPSec feature supports gateway-to-gateway VPN implementations by using a preshared key for Internet Key Exchange (IKE) authentication.

Note Microsoft does not support or recommend the use of a preshared key for IKE authentication on remote access L2TP/IPSec client connections. However, Windows 2000 is compliant with IKE Request for Comments (RFC) 2409.

MORE INFORMATION

To implement the preshared key authentication method for use with an L2TP/IPSec connection, follow these steps:
  1. Add the ProhibitIpSec registry value to both Windows 2000 Server-based endpoint computers, and then restart both computers.
  2. Manually configure an IPSec policy on both the Windows 2000 Server-based computers before you try to establish an L2TP/IPSec connection between them through a Terminal Services client session.

STATUS

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section of this article.

REFERENCES

For additional information about configuring L2TP, click the following article numbers to view the articles in the Microsoft Knowledge Base:

240262 How to configure a L2TP/IPSec connection using pre-shared key authentication

248711 Mutual authentication methods supported for L2TP/IPSec

Modification Type:MajorLast Reviewed:3/30/2004
Keywords:kbprb KB326751 kbAudITPRO
©2004 Microsoft Corporation. All rights reserved.