IPSec Does Not Secure Kerberos Traffic Between Domain Controllers (254728)


The information in this article applies to:

  • Microsoft Windows 2000 Server
  • Microsoft Windows 2000 Advanced Server
  • Microsoft Windows 2000 Professional
This article was previously published under Q254728
IMPORTANT: This article contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following article number to view the article in the Microsoft Knowledge Base:

256986 Description of the Microsoft Windows Registry

SYMPTOMS

The IP Security Protocol (IPSec) does not secure Kerberos or RSVP traffic between Windows 2000 domain controllers, even when the IPSec policy filter is configured to match all IP traffic between the two IP addresses. IPSec, when it is configured to secure all traffic between domain controllers, still lets Kerberos traffic appear as Kerberos traffic on the wire, unprotected by IPSec.

CAUSE

Some types of traffic are exempted by design from being secured by IPSec, even when the IPSec policy specifies that all IP traffic should be secured. The IPSec exemptions apply to Broadcast, Multicast, RSVP, IKE, and Kerberos traffic. Kerberos is a security protocol itself, can be used by IPSec for IKE authentication, and so was not originally designed to be secured by IPSec. Therefore, it is exempt from IPSec filtering. For details about these exemptions, please refer to the following Microsoft Knowledge Base article:

254949 Client-to-Domain Controller and Domain Controller-to-Domain Controller IPSec Support

RESOLUTION

To resolve this problem, obtain the latest service pack for Windows 2000. For additional information, click the following article number to view the article in the Microsoft Knowledge Base:

260910 How to Obtain the Latest Windows 2000 Service Pack

No operational affect on Kerberos was observed during internal testing when this fix was applied and Kerberos was IPSec-protected. If you notice problems with Kerberos traffic being secured, but domain trusts not working as a result, please contact Microsoft Product Support Services.

The purpose of this fix is to enable full IPSec protection of all Unicast traffic between two domain controllers, which can now include the Kerberos traffic with the registry key set. This fix is intended to be used on each domain controller, not on Windows 2000 Professional clients. The information in the following Microsoft Knowledge Base article still applies:

254949 IPSec support for client-to-domain controller traffic and domain controller-to-domain controller traffic

You should use this fix in conjunction with firewall rules that allow only IPSec and IKE traffic through. For additional information, click the article number below to view the article in the Microsoft Knowledge Base:

233256 How to Enable IPSec Traffic Through a Firewall

During the boot process, a few packets may be sent before the IPSec driver is initialized and has the IPSec policy fully processed. A properly configured firewall that allows only IKE and IPSec protocols can prevent this non-IPSec traffic from going to inappropriate networks.

To enable a server to be promoted as a child of a remote domain, set a local IPSec policy that uses certificate authentication. Test the IPSec security association by using Ipsecmon.exe and Ping, or some other method of generating traffic to the remote domain controller. If the IPSec security association is successfully established, all traffic to the remote domain should be protected. The server should be able to join the remote domain, and Dcpromo, Kerberos cross-domain trusts, and normal RPC-based directory replication should all work.

After you apply this hotfix and add the following registry key, you can control the exempt behavior for RSVP and Kerberos with IPSec filter rules. Setting this new registry entry to 1 causes these protocols to be filtered.

WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.

To add the registry key:
  1. Start Registry Editor (Regedt32.exe).
  2. Click the following registry key:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IPSEC

  3. On the Edit menu, click Add Value, and then add the following value: Value Name: NoDefaultExempt (note that this name is case sensitive)
    Data Type: REG_DWORD
    Data Value: 0 or 1

    • 0: Default exemptions apply (default)
    • 1: RSVP and Kerberos are not exempted (only IKE, Multicast, and Broadcast are exempted)

STATUS

Microsoft has confirmed that this is a problem in Microsoft Windows 2000. This problem was first corrected in Windows 2000 Service Pack 1.

MORE INFORMATION

For additional information about how to install Windows 2000 and Windows 2000 hotfixes at the same time, click the article number below to view the article in the Microsoft Knowledge Base:

249149 Installing Microsoft Windows 2000 and Windows 2000 Hotfixes

You may have to install a hotfix that is in Windows 2000 SP3 to use Kerberos filtering over IPSec. For additional information, click the article number below to view the article in the Microsoft Knowledge Base:

309304 IP Security Transport Mode with Encryption May Drop Fragmented Packets

Modification Type:MinorLast Reviewed:4/14/2006
Keywords:kbbug kbfix kbQFE kbWin2000SP1Fix KB254728
©2004 Microsoft Corporation. All rights reserved.