UTF-8, UTF-16 - ASP VB MSSQL Server - It just doesn't work.

[InvestorTrade]

When I POST using UTF-16 on the web using ASP and VBScriipt
(not .Net), to a MSSQL Server 2000, my data gets stored in weird
characters. My application is used worldwide (Chinese, Japanese,
German, Spanish...) all sharing the same database, but each location
stores their information using their own language. When I retrieve
the data on the Web, the characters look fine - but using SQL Query
Analyzer the characters are all mangled.

I also have an offline application (written in HTA, VBScript, and
Javascript) that allows my users to upload their data at a later
point
via a CSV file created using UNICODE, which appears to store the data
properly under the MSSQL Server (I confirm this by using Query
Analyzer)


To make things worse, the Web application will display properly the
data that it stored with the POST, but the data that was uploaded
though the CSV file displays incorrectly ( '???' characters). The
effect appears to be reversed when I use MSSQL Query Analyzer to
inspect the data - all the data that was uploaded using the CSV file
appears correct, but all the POST data is mangled.


I've placed under all the Web Pages forms and POST action pages:


<meta http-equiv="Content-Type" content="text/html; charset=utf-16">


I've also made all my forms to have:
<form accept-charset='utf-16'... >


I'm using Chinese in conjunction with English to troubleshoot my
problem.

 

[Anthony Jones]

When I POST using UTF-16 on the web using ASP and VBScriipt
(not .Net), to a MSSQL Server 2000, my data gets stored in weird
characters. My application is used worldwide (Chinese, Japanese,
German, Spanish...) all sharing the same database, but each location
stores their information using their own language. When I retrieve
the data on the Web, the characters look fine - but using SQL Query
Analyzer the characters are all mangled.

I also have an offline application (written in HTA, VBScript, and
Javascript) that allows my users to upload their data at a later
point
via a CSV file created using UNICODE, which appears to store the data
properly under the MSSQL Server (I confirm this by using Query
Analyzer)


To make things worse, the Web application will display properly the
data that it stored with the POST, but the data that was uploaded
though the CSV file displays incorrectly ( '???' characters). The
effect appears to be reversed when I use MSSQL Query Analyzer to
inspect the data - all the data that was uploaded using the CSV file
appears correct, but all the POST data is mangled.


I've placed under all the Web Pages forms and POST action pages:


<meta http-equiv="Content-Type" content="text/html; charset=utf-16">


I've also made all my forms to have:
<form accept-charset='utf-16'... >


I'm using Chinese in conjunction with English to troubleshoot my
problem.

Try adding this to the top of the ASP page receiving the Form post:-

<%@codepage=1200%>

The codepage is what the ASP uses to determine how a characters in the post
are encoded.
The assumption is that this is the same codepage as it would use to generate
its output.

When the correctly received CSV data when sent from ASP would have been
encoded as the current system codepage (probably 1252 or something like
that). But your meta Content-Type header says its unicode hence it appears
corrupted.

OTH, when you sent the already corrupted SQL data to the client the reverse
of the corruption occurs and hence to the client it looks ok (its
complicated).


My general advice on this is. Save all pages as UTF-8.

Include this in the top all pages:-

<%@codepage=65001 %>
<%
Response.CharSet = "UTF-8"
%>


In this way all static is content is UTF-8, all dynamically generated
content is encode to UTF-8, all form posts are assumed to be UTF-8 and the
browser is informed that its getting UTF-8.

BTW, don't bother with the meta Content-Type tag. Its purposes is to
pretent to be the Content-Type header when a HTM file is opened from a file
system. Since ASP is sending a real HTTP response you may as well set those
headers for real. text/html is the default content-type and the
Response.CharSet adds the ;charset=UTF-8 to it.

 

[InvestorTrade]

Thanks a lot! this worked like a charm!

For anyone else having problems out there - follow the post from:
Anthony Jones.

Thanks mate.