Excess Padding May Cause IPSec ESP Packet Loss with Third-Party Implementations (276360)

SYMPTOMS

After you successfully establish an IP Security Protocol (IPSec) Encapsulating Security Protocol (ESP) security association (either a transport or tunnel) between a computer that is running Windows 2000 Service Pack 1 (SP1) and a third-party IPSec implementation, certain packet sizes may be dropped upon being received on the Windows 2000 SP1-based computer. Other ESP traffic for the same IPSec security association is received normally. If you uninstall SP1, the problem does not occur; all IPSec ESP traffic flows correctly.

The problem does not occur when you use IPSec ESP to secure traffic between Windows 2000 (the original retail release) and Windows 2000 SP1 (see the "Cause" section); it occurs only with some third-party implementations of IPSec.

The IPSec driver may silently discard a packet and increment the "Received packets discarded" IP protocol counter. When you use IPSec, the netstat -s command may show the "Packets Received" counter not to equal the "Received Packets Delivered" counter. You can see the total number of received packets by using the Netmon traffic analyzer tool. Take snapshots of the netstat -s output immediately before and after the Netmon sniff. You see the received ESP packets with Netmon as received, but a packet may then be dropped by IPSec such that TCP or other upper-layer protocols behave as though it did not receive the packet.

This problem is known to affect Layer 2 Tunneling Protocol (L2TP)/IPSec virtual private network (VPN) tunnels with Cisco IOS VPN gateways (the 71xx series) causing the tunnel to disconnect. Other IPSec products that send ESP packets padded with 8 bytes or more are also affected.

For L2TP/IPSec tunnels between Windows 2000 SP1-based computers and Cisco IOS gateways, you can verify that this problem is occurring by watching the PPP send log on the Cisco IOS gateway and matching it with the PPP receive log from Windows 2000 SP1. You may see the Cisco gateway send a PPP data frame that is not listed as being received in the Windows 2000 SP1 PPP log.

CAUSE

In the IPSec ESP format, original IP packets may require padding to align the packet payload to a 64-bit boundary that is required for DES or 3DES block cipher encryption operations. In Window 2000 (the original retail release), IPSec ESP processing adds 0 to 7 bytes of padding on sending, and accepts 0 to 9 bytes of padding on receiving. This was tested as being interoperable with as many manufacturers as possible before release.

The security checking on padding was tightened to allow only 0 to 7 bytes of padding for received ESP packets. Consequently, the IPSec driver in SP1 discards ESP packets if they are received with 8 or more bytes of padding. This causes certain packets that are sent by third-party IPSec implementations to be dropped. This condition was not detected in interoperability tests before SP1 was released.

Although the acceptable padding length is not required to be more than that which is necessary to align the original data to the nearest 64-bit boundary, interoperability is better served with this fix that allows 0 to 255 bytes of padding to be accepted because senders may use up to 255 bytes. At the time this article was written, no known third-party implementations regularly send more than 8 bytes of padding.

For additional information, see RFC 2406, section 2.4, at the following Web RFC Web site:

http://www.ietf.org/rfc/rfc2406.txt

Microsoft provides third-party contact information to help you find technical support. This contact information may change without notice. Microsoft does not guarantee the accuracy of this third-party contact information.

RESOLUTION

To resolve this problem, obtain the latest service pack for Windows 2000. For additional information, click the following article number to view the article in the Microsoft Knowledge Base:

260910 How to Obtain the Latest Windows 2000 Service Pack

The English version of this update should have the following file attributes or later:

   Date        Time    Version        Size    File name
   ----------------------------------------------------
   05/09/2000  11:28a  5.0.2195.1569  33,616  Fips.sys
   08/23/2000  03:06p  5.0.2195.2103  69,552  Ipsec.sys

MORE INFORMATION

This hotfix contains both the fix for the ESP 8-byte padding issue and FIPS 140-1 certified compatibility for IPSec through the addition of the Fips.sys file.

For additional information about FIPS 140-1 certification, click the article number below to view the article in the Microsoft Knowledge Base:

272173 The Microsoft Windows 2000 Kernel Mode Cryptographic Module Has Received the FIPS 140-1 Status

For additional information about how to install Windows 2000 and Windows 2000 hotfixes at the same time, click the article number below to view the article in the Microsoft Knowledge Base: