A POST or PUT request may fail when you use the HttpWebRequest class to send lots of data on a computer that is running the .NET Framework (908573)


The information in this article applies to:

  • Microsoft .NET Framework 1.1
  • Microsoft .NET Framework 1.0

SYMPTOMS

When you use the HttpWebRequest class to send lots of data by using a POST or PUT request, the request may fail on a computer that is running the Microsoft .NET Framework. Additionally, you may receive an out-of-memory or time-out error message.

If you do not receive an out-of-memory or time-out error message, you may notice that the application that uses the HttpWebRequest class uses lots of memory. When you use Performance Monitor to monitor the application that uses the HttpWebRequest class, the Private Bytes count will continue to increase as data is sent. Therefore, you may also experience slow performance on the computer and in other applications because increased memory and resource utilization occurs.

Note The amount of data that can be uploaded by default will vary according to the memory and the resources that are available on the computer.

CAUSE

This issue occurs because the .NET Framework buffers the outgoing data by default when you use the HttpWebRequest class.

WORKAROUND

To work around this issue, set the HttpWebRequest.AllowWriteStreamBuffering property to false. However, you may receive an error message that is similar to the following when you set the HttpWebRequest.AllowWriteStreamBuffering property to false:
This request requires buffering of data for authentication of redirection to be successful.
To successfully send lots of data by using a POST or PUT request when the HttpWebRequest.AllowWriteStreamBuffering property is set to false, use one of the following methods, depending on the authentication method that you want to use.

Anonymous authentication

If the Web server is configured to use Anonymous authentication, set the HttpWebRequest.AllowWriteStreamBuffering property to false. No other changes are needed.

Basic authentication

If the Microsoft Internet Information Services (IIS) Web server is configured to use Basic authentication, and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you must send a HEAD request to pre-authenticate the connection before you send the POST or PUT request. You must also set the HttpWebRequest.PreAuthenticate property to true. Then, send the POST or PUT request, and then receive the response. To do this, use code that is similar to the following code example.
public void test(Uri URL)
		{
			HttpWebRequest WRequest;
			HttpWebResponse WResponse;
			//preAuth the request
			// You can add logic so that you only pre-authenticate the very first request.
			// You should not have to pre-authenticate each request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = new NetworkCredential(user, password);
			WRequest.PreAuthenticate = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "HEAD";
			WRequest.Timeout = 10000;
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			WResponse.Close();
			// Make the real request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = new NetworkCredential(user, password);
			WRequest.PreAuthenticate = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "POST";
			WRequest.AllowWriteStreamBuffering = false;
			WRequest.Timeout = 10000;
			FileStream ReadIn = new FileStream(c:\\testuploadfile.txt, FileMode.Open, FileAccess.Read);
			ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file.
			WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file.
			Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments.
			int DataRead = 0;
			Stream tempStream = WRequest.GetRequestStream();
			do
			{
				DataRead = ReadIn.Read(FileData,0,2048);
				if (DataRead > 0) //we have data
				{
					tempStream.Write(FileData,0,DataRead); 
					Array.Clear(FileData,0, 2048); // Clear the array.
				}
			}while(DataRead > 0);
	
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			// Read your response data here.
			// Close all streams.
			ReadIn.Close();
			tempStream.Close();
			WResponse.Close();
		}
Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

Integrated Windows authentication

You can configure a computer on which IIS is installed to respond by using Negotiate or NTLM Windows authentication. If IIS is configured to use Negotiate for Windows authentication, the client can use either Kerberos or NTLM to authenticate. If IIS is configured to use NTLM authentication, only NTLM authentication can be used, and Kerberos authentication is not supported.

If Negotiate with Kerberos authentication is used, use the following workaround. The workaround will fail if NTLM is used.

Negotiate with Kerberos authentication

If the IIS Web server is configured to use Negotiate authentication and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you must send a HEAD request to pre-authenticate the connection before you send the POST or PUT request. You must also set the HttpWebRequest.PreAuthenticate property to true. Additionally, you may have to set the HttpWebRequest.UnsafeAuthenticatedConnectionSharing property to true. Then, send the POST or PUT request, and then receive the response. To do this, you can use code that is similar to the following code example.

Note This workaround will fail if the client cannot use Kerberos with Negotiate authentication. You also must make sure that the HttpWebRequest.KeepAlive property is set to true. By default, the setting for the HttpWebRequest.KeepAlive property is true. The logic for Kerberos and Basic authentication is almost the same.
public void test(Uri URL)
		{
			HttpWebRequest WRequest;
			HttpWebResponse WResponse;
			CredentialCache myCredCache = new CredentialCache();
			myCredCache.Add(URL,"Negotiate",(NetworkCredential) CredentialCache.DefaultCredentials);
			// Pre-authenticate the request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = myCredCache;
			// This property must be set to true for Kerberos authentication.
			WRequest.PreAuthenticate = true;
			// Keep the connection alive.
			WRequest.UnsafeAuthenticatedConnectionSharing = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "HEAD";
			WRequest.Timeout = 10000;
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			WResponse.Close();
			// Make the real request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = myCredCache;
			// This property must be set to true for Kerberos authentication.
			WRequest.PreAuthenticate = true;
			// Keep the connection alive.
			WRequest.UnsafeAuthenticatedConnectionSharing = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "POST";
			WRequest.AllowWriteStreamBuffering = false;
			WRequest.Timeout = 10000;
			FileStream ReadIn = new FileStream("c:\\ testuploadfile.txt ", FileMode.Open, FileAccess.Read);
			ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file.
			WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file.
			Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments.
			int DataRead = 0;
			Stream tempStream = WRequest.GetRequestStream();
			do
			{
				DataRead = ReadIn.Read(FileData,0,2048);
				if (DataRead > 0) // We have data.
				{
					tempStream.Write(FileData,0,DataRead); 
					Array.Clear(FileData,0, 2048); // Clear the array.
				}
			}while(DataRead > 0);
	
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			// Read your response data here.
			// Close all streams
			ReadIn.Close();
			tempStream.Close();
			WResponse.Close();
		}
Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

NTLM authentication

If the IIS Web server is also configured to use NTLM authentication with Windows Integrated authentication, and you must set the HttpWebRequest.AllowWriteStreamBuffering property to false, you can set the authentication type to NTLM in the client code. After you configure IIS to use both Negotiate and NTLM authentication and you set the authentication type to NTLM in the client code, you can configure how IIS handles authentication requests by setting the AuthPersistSingleRequest property in the IIS metabase to false.

Note For more information about how to configure IIS to support both Negotiate and NTLM authentication, see the "References" section.

You must also send a HEAD request to pre-authenticate the connection before you send the POST request and set the HttpWebrequest.UnsafeAuthenticatedConnectionSharing property to true. Then, set the HttpWebRequest.PreAuthenticate property to false. Finally, send the POST or PUT request, and then receive the response. To do this, use code that is similar to the following code example.
public void test(Uri URL)
		{
			HttpWebRequest WRequest;
			HttpWebResponse WResponse;
			CredentialCache myCredCache = new CredentialCache();
			myCredCache.Add(URL,"NTLM",(NetworkCredential) CredentialCache.DefaultCredentials);
			// Pre-authenticate the request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = myCredCache;
			// For NTLM authentication, you must set the following property to true
			// so the connection does not close.
			WRequest.UnsafeAuthenticatedConnectionSharing = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "HEAD";
			WRequest.Timeout = 10000;
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			WResponse.Close();
			// Make the real request.
			WRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
			// Set the username and the password.
			WRequest.Credentials = myCredCache;
			// For NTLM authentication, you must set the following property to true
			// so the connection does not close.
			WRequest.UnsafeAuthenticatedConnectionSharing = true;
			WRequest.UserAgent = "Upload Test";
			WRequest.Method = "POST";
			WRequest.AllowWriteStreamBuffering = false;
			WRequest.Timeout = 10000;
			FileStream ReadIn = new FileStream("c:\\ testuploadfile.txt", FileMode.Open, FileAccess.Read);
			ReadIn.Seek(0, SeekOrigin.Begin); // Move to the start of the file.
			WRequest.ContentLength = ReadIn.Length; // Set the content length header to the size of the file.
			Byte[] FileData = new Byte[ReadIn.Length]; // Read the file in 2 KB segments.
			int DataRead = 0;
			Stream tempStream = WRequest.GetRequestStream();
			do
			{
				DataRead = ReadIn.Read(FileData,0,2048);
				if (DataRead > 0) // We have data.
				{
					tempStream.Write(FileData,0,DataRead); 
					Array.Clear(FileData,0, 2048); // Clear the array.
				}
			}while(DataRead > 0);
	
			WResponse = (HttpWebResponse)WRequest.GetResponse(); 
			// Read your response data here.
			// Close all streams.
			ReadIn.Close();
			tempStream.Close();
			WResponse.Close();

		}
Note Depending on how the application is designed, you may not have to pre-authenticate each request by sending a HEAD request.

REFERENCES

For more information about the HttpWebRequest.AllowWriteStreamBuffering property, visit the following Microsoft Developer Network (MSDN) Web site:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfsystemnethttpwebrequestclassallowwritestreambufferingtopic.asp

For more information about the HttpWebRequest.PreAuthenticate property, visit the following MSDN Web site:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetHttpWebRequestClassPreAuthenticateTopic.asp

For more information about the HttpWebRequest.UnsafeAuthenticatedConnectionSharing property, visit the following MSDN Web site:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/cpref/html/frlrfSystemNetHttpWebRequestClassUnsafeAuthenticatedConnectionSharingTopic.asp

For more information about Microsoft Negotiate authentication, visit the following MSDN Web site:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/secauthn/security/microsoft_negotiate.asp

For more information about Microsoft NTLM authentication, visit the following MSDN Web site:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/secauthn/security/microsoft_ntlm.asp

For more information about the AuthPersistSingleRequest property in the IIS metabase, visit the following Microsoft Web site:

http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/b0b4ec5c-74f8-43e9-ac64-d8b852568341.mspx

For more information about delays in response to a Web service request that may occur when you use NTLM authentication, click the following article number to view the article in the Microsoft Knowledge Base:

331020 Delay in response to a Web service request when you use NTLM authentication

For more information about how to configure IIS to use both Kerberos and NTLM authentication, click the following article number to view the article in the Microsoft Knowledge Base:

215383 How to configure IIS to support both the Kerberos protocol and the NTLM protocol for network authentication

For more information about issues that may occur when you negotiate Kerberos authentication after you upgrade to Microsoft Internet Explorer 6, click the following article number to view the article in the Microsoft Knowledge Base:

299838 Unable to negotiate Kerberos authentication after upgrading to Internet Explorer 6

For more information about how to modify the metabase to control authentication persistence, click the following article number to view the article in the Microsoft Knowledge Base:

318863 How to modify the AuthPersistence metabase entry controls when clients are authenticated

Modification Type:MinorLast Reviewed:4/26/2006
Keywords:kbtshoot kbprb KB908573 kbAudDeveloper
©2004 Microsoft Corporation. All rights reserved.