How To Debug OLE Server Applications Using MSVC (151074)

MORE INFORMATION

Debugging an In-Process OLE Server Application

An OLE client application interacting with an in-process OLE server application is simply loading an OLE server DLL in the client application address space. To debug such an in-process server, standard DLL debugging techniques can be used. For more information, please see the following article in the Microsoft Knowledge Base:

85221 Debugging a Dynamic-Link Library (DLL) in Windows

Debugging an Out-of-Process OLE Server Application

In an OLE client application interacting with an out-of-process OLE server application, the debugging involves crossing process spaces, which makes it much more difficult. Following are few techniques that can be used to debug out-of-process OLE server applications:

• Setup a hardcoded breakpoint in the server, and when the breakpoint hits in the server code, the Microsoft Visual C++ (MSVC) debugger is launched. Then step through server code and add breakpoints at locations of interest in server code.
  
  Add the following line of code in the source code, where a hardcoded breakpoint is needed.
  
  DebugBreak();
• Setup a breakpoint in the container/client application, then step into the code where the server code is called. The action of stepping into the code causes a new version of the MVSC debugger to be launched with the Server debug information. Then you can step through the server code and add breakpoints at locations of interest in the server code. To configure this, check in the MSVC option, the Tool Option, the debug tab, the Just-in-time debugging, and the OLE RPC debugging checkboxes that need to be checked. This technique requires source code and a debug version of the client application as well. For more information, please see the following article in the Microsoft Knowledge Base:
  
  154116 Tutorial: Debugging OLE Applications
  
  

• Launch the server application from the client and then find the process id of the server application using PView. Launch the MSVC -p <process ID>. This launches the MSVC option that attaches itself to the running server application. However, this technique is not useful in debugging the startup code of the server application. For more information, please see the following article in the Microsoft Knowledge Base:
  
  120707 How to Debug an Active Process in Visual C++ 2.0
  
  
• If the server application has registered the ClassFactory object with the system as single use (CoRegisterClassObject with CLS_SINGLEUSE flag), you can run the server application from the MSVC option as stand alone. To simulate the server being launched from the container/client application, you need to specify the /Embedding /Automation program arguments as applicable in the MSVC debug options and run the server application as stand alone. The /Embedding and /Automation switches do the appropriate server initialization, and register the class factory as if launched from the container/client application. Because the server's class factory is single use and not already connected, when the container/client application tries to hook up to the server, the container connects to the running server application in the debugger, and debugging is easier.
• If the server application registers the ClassFactory object with the system as multiple use (CoRegisterClassObject with CLS_MULTIPLEUSE flag), run the server application from the MSVC option as stand alone. Since such servers register the ClassFactory on startup, you do not need to specify /Embedding /Automation program arguments in the MSVC debug options. When the container/client application tries to connect to the server, it connects to the instance of the server application in the debugger because the server's class factory is registered multiple use, and debugging is easier.