PRB: Self-Referencing Dynamic Web Pages That Mask 404 Errors Can Cause Loop in Content Analyzer (281749)

IMPORTANT: This article contains information about modifying the registry. Before you modify the registry, make sure to back it up and make sure that you understand how to restore the registry if a problem occurs. For information about how to back up, restore, and edit the registry, click the following article number to view the article in the Microsoft Knowledge Base:

256986 Description of the Microsoft Windows Registry

SYMPTOMS

When you use Content Analyzer to crawl a Web site that uses Allaire ColdFusion, you must add a registry key and value so that Content Analyzer recognizes the ColdFusion (.cfm) pages. After you add the registry key and value, when you try to crawl the Web site, the crawl does not stop, and the page count for the Web site exceeds the number of physical pages.

CAUSE

ColdFusion does not return an HTTP 404 "File not found" error when a page is not found. In addition, if the generic page that is returned references another broken link, Content Analyzer follows this broken link, which creates a loop.

RESOLUTION

To resolve this problem, configure ColdFusion to return an HTTP 404 error with the error page. This informs Content Analyzer that the page doesn't exist. If you do not configure ColdFusion to return an HTTP 404 error, there is no way to find the broken links.

WORKAROUND

To work around this problem, correct the reference to the broken link on the 404 error page that ColdFusion returns.

MORE INFORMATION

WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.

For Content Analyzer to recognize .cfm files, you must add the following registry entry:

Start Registry Editor (Regedt32.exe).
Locate the following key in the registry:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Site Server\3.0\Analysis\CA\
On the Edit menu, click Add Key, and then add the following registry key:
MIME
Open the MIME key.
On the Edit menu, click Add Value, and then add the following registry value:
Value Name: ExtraHTMLExts
Data Type: REG_SZ
Value: CFM
Quit Registry Editor.

The third-party products that are discussed in this article are manufactured by companies that are independent of Microsoft. Microsoft makes no warranty, implied or otherwise, regarding the performance or reliability of these products.