PRB: Connectivity Problems Using 16-Bit Multiprotocol Net-Lib (169623)

SYMPTOMS

You may encounter connectivity problems when you use the 16-bit multi- protocol network library (Dbmsrpc3.dll) on a computer running Windows NT. When this happens, the DB-Library connection fails with OS error 0 "ConnectionOpen(RPCOpen())". If you use ODBC, connections fail with the following error:

Connection Failed.
SQL State '01000'
SQL Server Error:0
[Microsoft][ODBC SQL Server Driver][DBMSRPC3] ConnectionOpen(RPCOpen().

Connection Failed.
SQL State '08001'
SQL Server Error:11
[Microsoft][ODBC SQL Server Driver][DBMSRPC3] General network error.
Check your network document.

Sometimes, the client application gets a general protection fault (GPF) due to the conflict.

WORKAROUND

To work around this problem, copy the Security.dll file for the 16-bit Windows RPC from the Mssql\bin directory (if you are using SQL Server 6.5) or the Sql60\bin directory (if you are using SQL Server 6.0) to the %Windows%\System directory. Be careful not to overwrite the Windows NT Security.dll file that is in the System32 directory. After making the change, you need to restart the client computer that is running Windows NT.

MORE INFORMATION

The 16-bit multiprotocol Net-Library uses the 16-bit Windows RPC extension service provided by the following DLLs:

    Dnetapi.dll      Netapi.dll       Rpcrt1.dll      Rpcns1.dll
    Rpc16c1.dll     Rpc16c3.dll     Rpc16c3x.dll     Rpc16c4.dll
    Rpc16c5.dll     Rpc16c6.dll     Rpc16dg3.dll    Rpc16dg6.dll
   Security.dll

These files come with the 16-bit SQL Server tools, By default, they are installed in the Sql60\bin or Mssql\Bin directory. The Security.dll file for the 16-bit Windows RPC may conflict with an existing Windows NT file with the same name if the 16-bit Windows Security.dll file is not present in the System or application directory. In such a case, the 32-bit Windows Security.dll file in the System32 directory is found first, and incorrectly loaded. As a result, the WOW subsystem is corrupted, and connection problems for applications using the 16-bit multiprotocol Net-Library occur. However, the conflict is not obvious all the time. That is, if ISQL/W is the first 16-bit application run during the current Windows NT session, Windows NT searches the application directory first and correctly loads the 16-bit Windows Security.dll. Because all 16-bit applications run in the same WOW subsystem by default (a separate WOW subsystem is created for each 16-bit application only if the application is started in a separate memory space), the applications works fine.