Error message when you use Service Broker or database mirroring to connect to an instance of SQL Server 2005: "Connection handshake failed" (912412)

The information in this article applies to:

Microsoft SQL Server 2005 Standard Edition
Microsoft SQL Server 2005 Developer Edition
Microsoft SQL Server 2005 Enterprise Edition
Microsoft SQL Server 2005 Workgroup
Microsoft SQL Server 2005 Express Edition

Bug #: 413389 (SQLBUDT)

SYMPTOMS

When you use Service Broker or database mirroring to connect to an instance of Microsoft SQL Server 2005, you receive the following error messages:

Connection handshake failed. An OS call failed: (5) 5(error not found). State 87.

Connection handshake failed. An OS call failed: (5) 5(Access is denied). State 87.

This issue occurs when you use certificate-based authentication for the connection.

CAUSE

This issue occurs because the discretionary access control list (DACL) of the folder that contains the RSA keys is set incorrectly. Therefore, the account that runs the instance of SQL Server 2005 does not have sufficient permissions to access certificate pair keys.

The following folder contains the RSA keys:

%ALLUSERSPROFILE%\Application Data\Microsoft\Crypto\RSA\MachineKeys

RESOLUTION

To resolve this issue, you must manually set the DACL of the folder to the default permissions. To do this, follow the steps that are listed in Microsoft Knowledge Base article 278381. For more information, click the following article number to view the article in the Microsoft Knowledge Base:

278381 Default permissions for the MachineKeys folders

Modification Type:	Major	Last Reviewed:	12/27/2005
Keywords:	kbprb KB912412 kbAudDeveloper