INFO: Internet Explorer Always POSTs Unicode Data as UTF-8 (303612)

MORE INFORMATION

When you deal with Globalization and character sets, it is important to distinguish between character sets and character set encodings. A character set is a mapping of numeric values to characters in a character repertoire. A character set encoding is a specific bit representation of the integer values in the character set. Unicode is a 16-bit character set with several different encodings, including UCS-2, UTF-16, UTF-7, and UTF-8. Web developers typically specify the encoding of the page (and thus, by implication, the charset) by using the META charset value, as follows:

<META Name="Content-Type" Value="text/html;charset=utf-16">
				

Typically, Internet Explorer encodes POST data in accordance with the page encoding. If you write a Japanese page that uses the Shift-JIS Japanese character encoding, Internet Explorer submits the POST data in Shift-JIS. If the page uses the Unicode character set, however, it encodes the submission as UTF-8, even if the encoding used is, for example, UTF-16. This is because many Web servers (including IIS) cannot process UTF-16 surrogates, which are special Unicode extensions that allow you to use 32 bits to address a character instead of the usual 16.

Note that this rule applies no matter whether the form uses an ENCTYPE of application/x-www-form-urlencoded or multipart/form-data.

As of the writing of this article, the format of Unicode POST transmissions is currently not dictated by any standard; however, working drafts by the World Wide Web Consortium (W3C) indicate a move toward UTF-8 as the standard Unicode encoding for the Web. Developers should use UTF-8 for all Unicode data that they send to and receive from the browser.

Developers who are also using SQL Server need to use one of the remedies that are suggested in the following Knowledge Base article for storing UTF-8 data in SQL Server's UCS-2 Unicode fields: