printing character ' and " in asp using vbscript

Discussion in 'ASP General' started by S N, Mar 16, 2008.

  1. S N

    Paul Randall Guest

    Thanks for posting one of your problematic characters. I think one of the problems is that a number of distinct concepts, such as charset, font, and locale, are being blurred together. At first you mentioned having problems with the single and double quote characters, and defined them as apostrophe character ' and double quote ". Recently you posted the ‘ character, which I assume is what you meant by an apostrophe character. It certainly looks a lot like what I would call a single quote, but if you put my single quote ('), and your single quote together, you can see that they are different: (‘'). Well, maybe you can see the difference, and maybe you can't. It all depends on what font the characters are being displayed in. I think in general, if a font does not contain a glyph for a character, then it displays a square or rectangular box for that character. I think most fonts contain glyphs for all characters in the range Chr(32) to Chr(127). Many fonts contain glyphs for characters in the range Chr(128) to Chr(255) too. Many fonts also include glyphs for some characters in the range ChrW(256) to ChrW(65535), which are Unicode characters. My knowledge of Unicode is limited, so some of my terminology may not be technically correct, and I would appreciate being corrected.

    Copy the code below into a .vbs file and run it. You will get two message boxes. The first message box will contain two lines:

    Hello *'΄‘* Unicode
    ΄‘

    The first line contains a mixture of what might be considered Unicode and non-Unicode characters. The three characters between the asterisks (*) might all be considered single quotes, but only the first one is Chr(39), the character I consider a single quote. The second one is ChrW(900), and the third one is your single quote, ChrW(8216).

    The second line displays what is left of the first line after removing all characters whose AscW value is less than 255.

    I included the ChrW(900) character because it illustrates how differently certain characters may be handled.

    The second message box contains info about the two Unicode characters:

    1 ΄ 63 ? 900 ΄
    2 ‘ 145 ‘ 8216 ‘

    The six columns contain the following:
    1) i (position within the string)
    2) Mid(s, i, 1) the character at position i.
    3) Asc(Mid(s, i, 1)) value of the character, sometimes and sometimes not.
    4) Chr(Asc(Mid(s, i, 1))) Character associated with the reported Asc value.
    5) AscW(Mid(s, i, 1)) Unicode value of the character.
    6) ChrW(AscW(Mid(s, i, 1))) Character associated with the reported AscW value.

    The Asc function almost always returns an 8-bit value, and AscW returns a 16-bit value. For certain Locales, Asc returns the same 16-bit value as AscW. See the scripting help file for info on the GetLocale and SetLocale functions. The thing to note is that depending on Locale, for some Unicode characters, the Asc function returns returns 63, a value that corresponds to a question mark, and for others it returns a value under 256 that displays the same character as is displayed by the Unicode character. So ChrW(900) maps to a question mark but ChrW(8216) maps to Chr(145). I don't have any examples that would produce the inverted question mark you talked about in your early posts.

    Your posts talk about a number of code pages and charsets, like 65001 and utf-8 and iso-8859-1. I believe that charset 65001 represents all characters as fixed-length two-byte values, so it can handle all the thousands of standard Unicode characters. UTf-8 is a variable length encoding that uses one to four bytes to represent a character. It can handle all the characters that charset 65001 can handle. Charset iso-8859-1 can only handle 256 8-bit characters.

    I think you should build a little standalone VBScript that displays many of your problematic characters in something like the six columns I did above, and post the result. Perhaps we can figure out a way to fix the problem after you show us what the problem is. It might help if you tell us your Locale number too. Control-C can be used to copy the text from a message box.

    Option Explicit
    Dim i, j, s, sMsg
    s = "Hello *'" & ChrW(900) & "‘* Unicode"
    msgbox s & vbcrlf & sKeepOnlyUnicode(s)
    s = sKeepOnlyUnicode(s)

    For i = 1 To Len(s)
    sMsg = sMsg & i & vbTab & Mid(s, i, 1) & vbTab & _
    Asc(Mid(s, i, 1)) & vbTab & Chr(Asc(Mid(s, i, 1))) & vbTab & _
    AscW(Mid(s, i, 1)) & vbTab & ChrW(AscW(Mid(s, i, 1))) & vbCrLf
    Next 'i
    MsgBox sMsg

    Function sKeepOnlyUnicode(sAnyString)
    'Returns sAnyString with only Unicode [actually, all
    ' characters outside the range ChrW(0) to
    ' ChrW(255)] being kept. VBScript strings are made
    ' up of 16-bit characters so they can handle a
    ' lot of Unicode stuff.
    With New RegExp
    .Global = True
    .Pattern = "[\u0000-\u00FF]"
    sKeepOnlyUnicode = .Replace(sAnyString, "")
    End With
    End Function 'sKeepOnlyUnicode(sAnyString)


    -Paul Randall
    i changed the codepage tp 65001 and charset to utf-8, then the question mark ? showing earlier, has changed to the rectangle as shown below.
    ‘
    the database field also shows the same character stored in it.
    please help.

    My guess is that they are not " " but are ‘ “ †typically cut'n'pasted in from Microsoft Word.

    These are still in the Windows-1252 range of characters but are not strictly in the iso-8859-1 set.

    Don't use http-equiv meta tags use real headers instead.

    IOW ditch the meta tags and include this:-

    <%Response.CharSet = "Windows-1252"%>

    I'm not hopeful because you are probably using IE and IE will treat ISO-8859-1 as Windows-1252 anyway.

    Always use Server.HtmlEncode on values retrieved from the Database. Stop mucking about with any other approach.

    If that doesn't work view the html source from the browser. What is the server actually sending.

    Another alternative is stop using Windows-1252.

    Save your pages as UTF-8 change the codepage at the top of the page to 65001 and include Response.CharSet = "UTF-8" in your page.

    BTW, Have you looked at the field content directly using the DB management tool?


    --
    Anthony Jones - MVP ASP/ASP.NET
    i am attaching the sample code. actually i am printing from a field in access database. the text entered in the database contains single quotes and double quotes. when i try to print them using response.write, the double quotes are getting replaced with question marks. i have tried the method of

    DataPrep = Replace(DataPrep, """", "&quot;")

    still problem remains.

    i also tried
    response.write(server.htmlencode(myrs(3))) ' where myrs is adodb recordset

    still the problem remains

    i am also attaching the header lines from my asp page

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
    <%@LANGUAGE="VBSCRIPT" CODEPAGE="1252"%>


    <HTML><HEAD>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    <meta http-equiv="Content-Language" content="en-us" />

    the problem is still not solved

    please help
     
    Paul Randall, Mar 24, 2008
    #21
    1. Advertisements

  2. From your posts, I'd say that wordcleaner isn't doing what you expect -
    every quote you've posted as a paste from your text is a curly open quote
    which is what I'd expect a copy and paste direct from Word to include.

    Dan

    S wrote on Mon, 24 Mar 2008 21:40:13 +0530:
     
    Daniel Crichton, Mar 27, 2008
    #22
    1. Advertisements

  3. S N

    S N Guest

    i have checked the access database. when i open the access table, there also i am finding the rectangular block whereever i expect apostrophe .

    also i have started using server.htmlencode for retrieving values from the database. but it displays the new line characters and paragraph characters (<BR> and <p> notations) stored in the text field as such. meaning instead of using these characters as commands for new line it is displaying them as it is, ie as "<BR>" and "<p>". in this way the paragraph boundaries has gone.

    please help me with the above two problems.

    Thanks for posting one of your problematic characters. I think one of the problems is that a number of distinct concepts, such as charset, font, and locale, are being blurred together. At first you mentioned having problems with the single and double quote characters, and defined them as apostrophe character ' and double quote ". Recently you posted the ‘ character, which I assume is what you meant by an apostrophe character. It certainly looks a lot like what I would call a single quote, but if you put my single quote ('), and your single quote together, you can see that they are different: (‘'). Well, maybe you can see the difference, and maybe you can't. It all depends on what font the characters are being displayed in. I think in general, if a font does not contain a glyph for a character, then it displays a square or rectangular box for that character. I think most fonts contain glyphs for all characters in the range Chr(32) to Chr(127). Many fonts contain glyphs for characters in the range Chr(128) to Chr(255) too. Many fonts also include glyphs for some characters in the range ChrW(256) to ChrW(65535), which are Unicode characters. My knowledge of Unicode is limited, so some of my terminology may not be technically correct, and I would appreciate being corrected.

    Copy the code below into a .vbs file and run it. You will get two message boxes. The first message box will contain two lines:

    Hello *'΄‘* Unicode
    ΄‘

    The first line contains a mixture of what might be considered Unicode and non-Unicode characters. The three characters between the asterisks (*) might all be considered single quotes, but only the first one is Chr(39), the character I consider a single quote. The second one is ChrW(900), and the third one is your single quote, ChrW(8216).

    The second line displays what is left of the first line after removing all characters whose AscW value is less than 255.

    I included the ChrW(900) character because it illustrates how differently certain characters may be handled.

    The second message box contains info about the two Unicode characters:

    1 ΄ 63 ? 900 ΄
    2 ‘ 145 ‘ 8216 ‘

    The six columns contain the following:
    1) i (position within the string)
    2) Mid(s, i, 1) the character at position i.
    3) Asc(Mid(s, i, 1)) value of the character, sometimes and sometimes not.
    4) Chr(Asc(Mid(s, i, 1))) Character associated with the reported Asc value.
    5) AscW(Mid(s, i, 1)) Unicode value of the character.
    6) ChrW(AscW(Mid(s, i, 1))) Character associated with the reported AscW value.

    The Asc function almost always returns an 8-bit value, and AscW returns a 16-bit value. For certain Locales, Asc returns the same 16-bit value as AscW. See the scripting help file for info on the GetLocale and SetLocale functions. The thing to note is that depending on Locale, for some Unicode characters, the Asc function returns returns 63, a value that corresponds to a question mark, and for others it returns a value under 256 that displays the same character as is displayed by the Unicode character. So ChrW(900) maps to a question mark but ChrW(8216) maps to Chr(145). I don't have any examples that would produce the inverted question mark you talked about in your early posts.

    Your posts talk about a number of code pages and charsets, like 65001 and utf-8 and iso-8859-1. I believe that charset 65001 represents all characters as fixed-length two-byte values, so it can handle all the thousands of standard Unicode characters. UTf-8 is a variable length encoding that uses one to four bytes to represent a character. It can handle all the characters that charset 65001 can handle. Charset iso-8859-1 can only handle 256 8-bit characters.

    I think you should build a little standalone VBScript that displays many of your problematic characters in something like the six columns I did above, and post the result. Perhaps we can figure out a way to fix the problem after you show us what the problem is. It might help if you tell us your Locale number too. Control-C can be used to copy the text from a message box.

    Option Explicit
    Dim i, j, s, sMsg
    s = "Hello *'" & ChrW(900) & "‘* Unicode"
    msgbox s & vbcrlf & sKeepOnlyUnicode(s)
    s = sKeepOnlyUnicode(s)

    For i = 1 To Len(s)
    sMsg = sMsg & i & vbTab & Mid(s, i, 1) & vbTab & _
    Asc(Mid(s, i, 1)) & vbTab & Chr(Asc(Mid(s, i, 1))) & vbTab & _
    AscW(Mid(s, i, 1)) & vbTab & ChrW(AscW(Mid(s, i, 1))) & vbCrLf
    Next 'i
    MsgBox sMsg

    Function sKeepOnlyUnicode(sAnyString)
    'Returns sAnyString with only Unicode [actually, all
    ' characters outside the range ChrW(0) to
    ' ChrW(255)] being kept. VBScript strings are made
    ' up of 16-bit characters so they can handle a
    ' lot of Unicode stuff.
    With New RegExp
    .Global = True
    .Pattern = "[\u0000-\u00FF]"
    sKeepOnlyUnicode = .Replace(sAnyString, "")
    End With
    End Function 'sKeepOnlyUnicode(sAnyString)


    -Paul Randall
    i changed the codepage tp 65001 and charset to utf-8, then the question mark ? showing earlier, has changed to the rectangle as shown below.
    ‘
    the database field also shows the same character stored in it.
    please help.

    My guess is that they are not " " but are ‘ “ †typically cut'n'pasted in from Microsoft Word.

    These are still in the Windows-1252 range of characters but are not strictly in the iso-8859-1 set.

    Don't use http-equiv meta tags use real headers instead.

    IOW ditch the meta tags and include this:-

    <%Response.CharSet = "Windows-1252"%>

    I'm not hopeful because you are probably using IE and IE will treat ISO-8859-1 as Windows-1252 anyway.

    Always use Server.HtmlEncode on values retrieved from the Database. Stop mucking about with any other approach.

    If that doesn't work view the html source from the browser. What is the server actually sending.

    Another alternative is stop using Windows-1252.

    Save your pages as UTF-8 change the codepage at the top of the page to 65001 and include Response.CharSet = "UTF-8" in your page.

    BTW, Have you looked at the field content directly using the DB management tool?


    --
    Anthony Jones - MVP ASP/ASP.NET
    i am attaching the sample code. actually i am printing from a field in access database. the text entered in the database contains single quotes and double quotes. when i try to print them using response.write, the double quotes are getting replaced with question marks. i have tried the method of

    DataPrep = Replace(DataPrep, """", "&quot;")

    still problem remains.

    i also tried
    response.write(server.htmlencode(myrs(3))) ' where myrs is adodb recordset

    still the problem remains

    i am also attaching the header lines from my asp page

    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
    <%@LANGUAGE="VBSCRIPT" CODEPAGE="1252"%>


    <HTML><HEAD>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    <meta http-equiv="Content-Language" content="en-us" />

    the problem is still not solved

    please help
     
    S N, Apr 6, 2008
    #23
  4. i have checked the access database. when i open the access table,
    If access is showing the wrong character that indicates the data is corrupt.

    If the field contains HTML (which it appears it does if it has <br> and <p>
    elements that you expect to be honors) then you should not be using
    Server.HTMLEncode. It has to be assumed that a field containing HTML is
    already HTML encoded.

    This is a long thread, I can't remember if you indicated how the data
    arrived in the DB in the first place.
     
    Anthony Jones, Apr 6, 2008
    #24
  5. S N

    S N Guest

    i had copied the data from a word file and using control-c i had pasted it
    into a richtext textbox in my asp form.

    also please tell me how to ensure that regardless of whether the data is
    html encoded or not, my server.htmlencode should work alright.
     
    S N, Apr 6, 2008
    #25
  6. I haven't read the whole thread, but pasting directly from Word into a rich
    text box is asking for trouble. I recommend pasting from Word into Notepad,
    then taking the result and pasting it into the Rich text box. Word
    sometimes does odd things with what should be "double quotes". And it
    retains a whole load of Word-specific formatting - often over-riding your
    carefully crafted css.
     
    Mike Brind [MVP], Apr 7, 2008
    #26
  7. S N

    S N Guest

    also please tell me how to ensure that regardless of whether the data is
    html encoded or not, my server.htmlencode should work alright.
     
    S N, Apr 7, 2008
    #27
  8. As Anthony said, if you are entering html code into the database with the
    idea that this takes effect when you pull it back to a web page, you do not
    want to server.htmlencode it. Since you are using a Rich Text Editor, I am
    assuming that this will apply html tags to the text on entry, and you want
    them to act on the output.

    What you really want to do is to make sure no javascript or clientside
    vbscript gets injected. One way to do this is just to reject any input that
    contains the string "<script>" in it during your server-side validation.
     
    Mike Brind [MVP], Apr 7, 2008
    #28
  9. S N

    S N Guest

    you have guessed it very correctly that i am entering html code into the database (like table tags <td> <tr> in particular) with the
    idea that this takes effect when it is pulled back to a web page, and hence i dont want to server.htmlencode it.

    can you suggest a server side validation script which does as indicated below by you. else can you suggest an alternate method of achieving the above (ensuring the table tags get translated into tables on the client side).

    please help.
     
    S N, Apr 11, 2008
    #29
  10. 'input is the posted content from the Rich Text Editor

    If InStr(input, "<string>") > 0 Then
    'reject it
    Else
    'process it
    End If
     
    Mike Brind [MVP], Apr 11, 2008
    #30
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.