Array of Bytes to Unicode chars (ISO-8859-1)

Discussion in 'C Programming' started by abhi147@gmail.com, Jul 25, 2006.

  1. Guest

    Hi ,
    I want to convert an array of bytes like :

    {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}

    into Unicode character with ISO-8859-1 standard.

    Can anyone help me .. how should I go about doing it ?

    Thanks
     
    , Jul 25, 2006
    #1
    1. Advertising

  2. P.J. Plauger Guest

    <> wrote in message
    news:...

    > I want to convert an array of bytes like :
    >
    > {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >
    > into Unicode character with ISO-8859-1 standard.
    >
    > Can anyone help me .. how should I go about doing it ?


    1) Use the file attached below to write your own mapping function.

    2) Use our C library and an appropriate setlocale call, such as
    setlocale(LC_ALL, en_CA) to make mbstowcs do it for you.

    HTH,

    P.J. Plauger
    Dinkumware, Ltd.
    http://www.dinkumware.com

    ------

    #
    # Name: ISO/IEC 8859-1:1998 to Unicode
    # Unicode version: 3.0
    # Table version: 1.0
    # Table format: Format A
    # Date: 1999 July 27
    # Authors: Ken Whistler <>
    #
    # Copyright (c) 1991-1999 Unicode, Inc. All Rights reserved.
    #
    # This file is provided as-is by Unicode, Inc. (The Unicode Consortium).
    # No claims are made as to fitness for any particular purpose. No
    # warranties of any kind are expressed or implied. The recipient
    # agrees to determine applicability of information provided. If this
    # file has been provided on optical media by Unicode, Inc., the sole
    # remedy for any claim will be exchange of defective media within 90
    # days of receipt.
    #
    # Unicode, Inc. hereby grants the right to freely use the information
    # supplied in this file in the creation of products supporting the
    # Unicode Standard, and to make copies of this file in any form for
    # internal or external distribution as long as this notice remains
    # attached.
    #
    # General notes:
    #
    # This table contains the data the Unicode Consortium has on how
    # ISO/IEC 8859-1:1998 characters map into Unicode.
    #
    # Format: Three tab-separated columns
    # Column #1 is the ISO/IEC 8859-1 code (in hex as 0xXX)
    # Column #2 is the Unicode (in hex as 0xXXXX)
    # Column #3 the Unicode name (follows a comment sign, '#')
    #
    # The entries are in ISO/IEC 8859-1 order.
    #
    # Version history
    # 1.0 version updates 0.1 version by adding mappings for all
    # control characters.
    #
    # Updated versions of this file may be found in:
    # <ftp://ftp.unicode.org/Public/MAPPINGS/>
    #
    # Any comments or problems, contact <>
    # Please note that <> is an archival address;
    # notices will be checked, but do not expect an immediate response.
    #
    0x00 0x0000 # NULL
    0x01 0x0001 # START OF HEADING
    0x02 0x0002 # START OF TEXT
    0x03 0x0003 # END OF TEXT
    0x04 0x0004 # END OF TRANSMISSION
    0x05 0x0005 # ENQUIRY
    0x06 0x0006 # ACKNOWLEDGE
    0x07 0x0007 # BELL
    0x08 0x0008 # BACKSPACE
    0x09 0x0009 # HORIZONTAL TABULATION
    0x0A 0x000A # LINE FEED
    0x0B 0x000B # VERTICAL TABULATION
    0x0C 0x000C # FORM FEED
    0x0D 0x000D # CARRIAGE RETURN
    0x0E 0x000E # SHIFT OUT
    0x0F 0x000F # SHIFT IN
    0x10 0x0010 # DATA LINK ESCAPE
    0x11 0x0011 # DEVICE CONTROL ONE
    0x12 0x0012 # DEVICE CONTROL TWO
    0x13 0x0013 # DEVICE CONTROL THREE
    0x14 0x0014 # DEVICE CONTROL FOUR
    0x15 0x0015 # NEGATIVE ACKNOWLEDGE
    0x16 0x0016 # SYNCHRONOUS IDLE
    0x17 0x0017 # END OF TRANSMISSION BLOCK
    0x18 0x0018 # CANCEL
    0x19 0x0019 # END OF MEDIUM
    0x1A 0x001A # SUBSTITUTE
    0x1B 0x001B # ESCAPE
    0x1C 0x001C # FILE SEPARATOR
    0x1D 0x001D # GROUP SEPARATOR
    0x1E 0x001E # RECORD SEPARATOR
    0x1F 0x001F # UNIT SEPARATOR
    0x20 0x0020 # SPACE
    0x21 0x0021 # EXCLAMATION MARK
    0x22 0x0022 # QUOTATION MARK
    0x23 0x0023 # NUMBER SIGN
    0x24 0x0024 # DOLLAR SIGN
    0x25 0x0025 # PERCENT SIGN
    0x26 0x0026 # AMPERSAND
    0x27 0x0027 # APOSTROPHE
    0x28 0x0028 # LEFT PARENTHESIS
    0x29 0x0029 # RIGHT PARENTHESIS
    0x2A 0x002A # ASTERISK
    0x2B 0x002B # PLUS SIGN
    0x2C 0x002C # COMMA
    0x2D 0x002D # HYPHEN-MINUS
    0x2E 0x002E # FULL STOP
    0x2F 0x002F # SOLIDUS
    0x30 0x0030 # DIGIT ZERO
    0x31 0x0031 # DIGIT ONE
    0x32 0x0032 # DIGIT TWO
    0x33 0x0033 # DIGIT THREE
    0x34 0x0034 # DIGIT FOUR
    0x35 0x0035 # DIGIT FIVE
    0x36 0x0036 # DIGIT SIX
    0x37 0x0037 # DIGIT SEVEN
    0x38 0x0038 # DIGIT EIGHT
    0x39 0x0039 # DIGIT NINE
    0x3A 0x003A # COLON
    0x3B 0x003B # SEMICOLON
    0x3C 0x003C # LESS-THAN SIGN
    0x3D 0x003D # EQUALS SIGN
    0x3E 0x003E # GREATER-THAN SIGN
    0x3F 0x003F # QUESTION MARK
    0x40 0x0040 # COMMERCIAL AT
    0x41 0x0041 # LATIN CAPITAL LETTER A
    0x42 0x0042 # LATIN CAPITAL LETTER B
    0x43 0x0043 # LATIN CAPITAL LETTER C
    0x44 0x0044 # LATIN CAPITAL LETTER D
    0x45 0x0045 # LATIN CAPITAL LETTER E
    0x46 0x0046 # LATIN CAPITAL LETTER F
    0x47 0x0047 # LATIN CAPITAL LETTER G
    0x48 0x0048 # LATIN CAPITAL LETTER H
    0x49 0x0049 # LATIN CAPITAL LETTER I
    0x4A 0x004A # LATIN CAPITAL LETTER J
    0x4B 0x004B # LATIN CAPITAL LETTER K
    0x4C 0x004C # LATIN CAPITAL LETTER L
    0x4D 0x004D # LATIN CAPITAL LETTER M
    0x4E 0x004E # LATIN CAPITAL LETTER N
    0x4F 0x004F # LATIN CAPITAL LETTER O
    0x50 0x0050 # LATIN CAPITAL LETTER P
    0x51 0x0051 # LATIN CAPITAL LETTER Q
    0x52 0x0052 # LATIN CAPITAL LETTER R
    0x53 0x0053 # LATIN CAPITAL LETTER S
    0x54 0x0054 # LATIN CAPITAL LETTER T
    0x55 0x0055 # LATIN CAPITAL LETTER U
    0x56 0x0056 # LATIN CAPITAL LETTER V
    0x57 0x0057 # LATIN CAPITAL LETTER W
    0x58 0x0058 # LATIN CAPITAL LETTER X
    0x59 0x0059 # LATIN CAPITAL LETTER Y
    0x5A 0x005A # LATIN CAPITAL LETTER Z
    0x5B 0x005B # LEFT SQUARE BRACKET
    0x5C 0x005C # REVERSE SOLIDUS
    0x5D 0x005D # RIGHT SQUARE BRACKET
    0x5E 0x005E # CIRCUMFLEX ACCENT
    0x5F 0x005F # LOW LINE
    0x60 0x0060 # GRAVE ACCENT
    0x61 0x0061 # LATIN SMALL LETTER A
    0x62 0x0062 # LATIN SMALL LETTER B
    0x63 0x0063 # LATIN SMALL LETTER C
    0x64 0x0064 # LATIN SMALL LETTER D
    0x65 0x0065 # LATIN SMALL LETTER E
    0x66 0x0066 # LATIN SMALL LETTER F
    0x67 0x0067 # LATIN SMALL LETTER G
    0x68 0x0068 # LATIN SMALL LETTER H
    0x69 0x0069 # LATIN SMALL LETTER I
    0x6A 0x006A # LATIN SMALL LETTER J
    0x6B 0x006B # LATIN SMALL LETTER K
    0x6C 0x006C # LATIN SMALL LETTER L
    0x6D 0x006D # LATIN SMALL LETTER M
    0x6E 0x006E # LATIN SMALL LETTER N
    0x6F 0x006F # LATIN SMALL LETTER O
    0x70 0x0070 # LATIN SMALL LETTER P
    0x71 0x0071 # LATIN SMALL LETTER Q
    0x72 0x0072 # LATIN SMALL LETTER R
    0x73 0x0073 # LATIN SMALL LETTER S
    0x74 0x0074 # LATIN SMALL LETTER T
    0x75 0x0075 # LATIN SMALL LETTER U
    0x76 0x0076 # LATIN SMALL LETTER V
    0x77 0x0077 # LATIN SMALL LETTER W
    0x78 0x0078 # LATIN SMALL LETTER X
    0x79 0x0079 # LATIN SMALL LETTER Y
    0x7A 0x007A # LATIN SMALL LETTER Z
    0x7B 0x007B # LEFT CURLY BRACKET
    0x7C 0x007C # VERTICAL LINE
    0x7D 0x007D # RIGHT CURLY BRACKET
    0x7E 0x007E # TILDE
    0x7F 0x007F # DELETE
    0x80 0x0080 # <control>
    0x81 0x0081 # <control>
    0x82 0x0082 # <control>
    0x83 0x0083 # <control>
    0x84 0x0084 # <control>
    0x85 0x0085 # <control>
    0x86 0x0086 # <control>
    0x87 0x0087 # <control>
    0x88 0x0088 # <control>
    0x89 0x0089 # <control>
    0x8A 0x008A # <control>
    0x8B 0x008B # <control>
    0x8C 0x008C # <control>
    0x8D 0x008D # <control>
    0x8E 0x008E # <control>
    0x8F 0x008F # <control>
    0x90 0x0090 # <control>
    0x91 0x0091 # <control>
    0x92 0x0092 # <control>
    0x93 0x0093 # <control>
    0x94 0x0094 # <control>
    0x95 0x0095 # <control>
    0x96 0x0096 # <control>
    0x97 0x0097 # <control>
    0x98 0x0098 # <control>
    0x99 0x0099 # <control>
    0x9A 0x009A # <control>
    0x9B 0x009B # <control>
    0x9C 0x009C # <control>
    0x9D 0x009D # <control>
    0x9E 0x009E # <control>
    0x9F 0x009F # <control>
    0xA0 0x00A0 # NO-BREAK SPACE
    0xA1 0x00A1 # INVERTED EXCLAMATION MARK
    0xA2 0x00A2 # CENT SIGN
    0xA3 0x00A3 # POUND SIGN
    0xA4 0x00A4 # CURRENCY SIGN
    0xA5 0x00A5 # YEN SIGN
    0xA6 0x00A6 # BROKEN BAR
    0xA7 0x00A7 # SECTION SIGN
    0xA8 0x00A8 # DIAERESIS
    0xA9 0x00A9 # COPYRIGHT SIGN
    0xAA 0x00AA # FEMININE ORDINAL INDICATOR
    0xAB 0x00AB # LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
    0xAC 0x00AC # NOT SIGN
    0xAD 0x00AD # SOFT HYPHEN
    0xAE 0x00AE # REGISTERED SIGN
    0xAF 0x00AF # MACRON
    0xB0 0x00B0 # DEGREE SIGN
    0xB1 0x00B1 # PLUS-MINUS SIGN
    0xB2 0x00B2 # SUPERSCRIPT TWO
    0xB3 0x00B3 # SUPERSCRIPT THREE
    0xB4 0x00B4 # ACUTE ACCENT
    0xB5 0x00B5 # MICRO SIGN
    0xB6 0x00B6 # PILCROW SIGN
    0xB7 0x00B7 # MIDDLE DOT
    0xB8 0x00B8 # CEDILLA
    0xB9 0x00B9 # SUPERSCRIPT ONE
    0xBA 0x00BA # MASCULINE ORDINAL INDICATOR
    0xBB 0x00BB # RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
    0xBC 0x00BC # VULGAR FRACTION ONE QUARTER
    0xBD 0x00BD # VULGAR FRACTION ONE HALF
    0xBE 0x00BE # VULGAR FRACTION THREE QUARTERS
    0xBF 0x00BF # INVERTED QUESTION MARK
    0xC0 0x00C0 # LATIN CAPITAL LETTER A WITH GRAVE
    0xC1 0x00C1 # LATIN CAPITAL LETTER A WITH ACUTE
    0xC2 0x00C2 # LATIN CAPITAL LETTER A WITH CIRCUMFLEX
    0xC3 0x00C3 # LATIN CAPITAL LETTER A WITH TILDE
    0xC4 0x00C4 # LATIN CAPITAL LETTER A WITH DIAERESIS
    0xC5 0x00C5 # LATIN CAPITAL LETTER A WITH RING ABOVE
    0xC6 0x00C6 # LATIN CAPITAL LETTER AE
    0xC7 0x00C7 # LATIN CAPITAL LETTER C WITH CEDILLA
    0xC8 0x00C8 # LATIN CAPITAL LETTER E WITH GRAVE
    0xC9 0x00C9 # LATIN CAPITAL LETTER E WITH ACUTE
    0xCA 0x00CA # LATIN CAPITAL LETTER E WITH CIRCUMFLEX
    0xCB 0x00CB # LATIN CAPITAL LETTER E WITH DIAERESIS
    0xCC 0x00CC # LATIN CAPITAL LETTER I WITH GRAVE
    0xCD 0x00CD # LATIN CAPITAL LETTER I WITH ACUTE
    0xCE 0x00CE # LATIN CAPITAL LETTER I WITH CIRCUMFLEX
    0xCF 0x00CF # LATIN CAPITAL LETTER I WITH DIAERESIS
    0xD0 0x00D0 # LATIN CAPITAL LETTER ETH (Icelandic)
    0xD1 0x00D1 # LATIN CAPITAL LETTER N WITH TILDE
    0xD2 0x00D2 # LATIN CAPITAL LETTER O WITH GRAVE
    0xD3 0x00D3 # LATIN CAPITAL LETTER O WITH ACUTE
    0xD4 0x00D4 # LATIN CAPITAL LETTER O WITH CIRCUMFLEX
    0xD5 0x00D5 # LATIN CAPITAL LETTER O WITH TILDE
    0xD6 0x00D6 # LATIN CAPITAL LETTER O WITH DIAERESIS
    0xD7 0x00D7 # MULTIPLICATION SIGN
    0xD8 0x00D8 # LATIN CAPITAL LETTER O WITH STROKE
    0xD9 0x00D9 # LATIN CAPITAL LETTER U WITH GRAVE
    0xDA 0x00DA # LATIN CAPITAL LETTER U WITH ACUTE
    0xDB 0x00DB # LATIN CAPITAL LETTER U WITH CIRCUMFLEX
    0xDC 0x00DC # LATIN CAPITAL LETTER U WITH DIAERESIS
    0xDD 0x00DD # LATIN CAPITAL LETTER Y WITH ACUTE
    0xDE 0x00DE # LATIN CAPITAL LETTER THORN (Icelandic)
    0xDF 0x00DF # LATIN SMALL LETTER SHARP S (German)
    0xE0 0x00E0 # LATIN SMALL LETTER A WITH GRAVE
    0xE1 0x00E1 # LATIN SMALL LETTER A WITH ACUTE
    0xE2 0x00E2 # LATIN SMALL LETTER A WITH CIRCUMFLEX
    0xE3 0x00E3 # LATIN SMALL LETTER A WITH TILDE
    0xE4 0x00E4 # LATIN SMALL LETTER A WITH DIAERESIS
    0xE5 0x00E5 # LATIN SMALL LETTER A WITH RING ABOVE
    0xE6 0x00E6 # LATIN SMALL LETTER AE
    0xE7 0x00E7 # LATIN SMALL LETTER C WITH CEDILLA
    0xE8 0x00E8 # LATIN SMALL LETTER E WITH GRAVE
    0xE9 0x00E9 # LATIN SMALL LETTER E WITH ACUTE
    0xEA 0x00EA # LATIN SMALL LETTER E WITH CIRCUMFLEX
    0xEB 0x00EB # LATIN SMALL LETTER E WITH DIAERESIS
    0xEC 0x00EC # LATIN SMALL LETTER I WITH GRAVE
    0xED 0x00ED # LATIN SMALL LETTER I WITH ACUTE
    0xEE 0x00EE # LATIN SMALL LETTER I WITH CIRCUMFLEX
    0xEF 0x00EF # LATIN SMALL LETTER I WITH DIAERESIS
    0xF0 0x00F0 # LATIN SMALL LETTER ETH (Icelandic)
    0xF1 0x00F1 # LATIN SMALL LETTER N WITH TILDE
    0xF2 0x00F2 # LATIN SMALL LETTER O WITH GRAVE
    0xF3 0x00F3 # LATIN SMALL LETTER O WITH ACUTE
    0xF4 0x00F4 # LATIN SMALL LETTER O WITH CIRCUMFLEX
    0xF5 0x00F5 # LATIN SMALL LETTER O WITH TILDE
    0xF6 0x00F6 # LATIN SMALL LETTER O WITH DIAERESIS
    0xF7 0x00F7 # DIVISION SIGN
    0xF8 0x00F8 # LATIN SMALL LETTER O WITH STROKE
    0xF9 0x00F9 # LATIN SMALL LETTER U WITH GRAVE
    0xFA 0x00FA # LATIN SMALL LETTER U WITH ACUTE
    0xFB 0x00FB # LATIN SMALL LETTER U WITH CIRCUMFLEX
    0xFC 0x00FC # LATIN SMALL LETTER U WITH DIAERESIS
    0xFD 0x00FD # LATIN SMALL LETTER Y WITH ACUTE
    0xFE 0x00FE # LATIN SMALL LETTER THORN (Icelandic)
    0xFF 0x00FF # LATIN SMALL LETTER Y WITH DIAERESIS
     
    P.J. Plauger, Jul 25, 2006
    #2
    1. Advertising

  3. Simon Biber Guest

    wrote:
    > Hi ,
    > I want to convert an array of bytes like :
    >
    > {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >
    > into Unicode character with ISO-8859-1 standard.
    >
    > Can anyone help me .. how should I go about doing it ?


    Let me get this straight. Your array of bytes contains ISO 8859-1
    characters, and you want to convert those characters into Unicode
    characters?

    Simply copy them into the low byte of your Unicode characters!

    /* unsigned char contains at least 8 bits, so
    it's enough for iso8859-1 */
    unsigned char iso88591[] =
    {79,104,-37,-66,24,123,30,-26,
    -99,-8,80,-38,19,14,-127,-3};

    /* unsigned long contains at least 32 bits, so
    it's enough for Unicode */
    unsigned long unicode[sizeof iso88591];
    size_t i;
    for(i = 0; i < sizeof iso88591; i++)
    {
    /* The bitwise 'and' with 255 does nothing in most cases
    but may be necessary if CHAR_BIT > 8 as the negative
    initialisers above will wrap to values over 255 */

    unicode = iso88591 & 255;
    }

    Now it's an array of unsigned long, containing the Unicode equivalent of
    each of your ISO-8859-1 characters.

    --
    Simon.
     
    Simon Biber, Jul 25, 2006
    #3
  4. Ian Malone Guest

    Simon Biber wrote:
    > wrote:
    >> Hi ,
    >> I want to convert an array of bytes like :
    >>
    >> {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >>
    >> into Unicode character with ISO-8859-1 standard.
    >>
    >> Can anyone help me .. how should I go about doing it ?

    >
    > Let me get this straight. Your array of bytes contains ISO 8859-1
    > characters, and you want to convert those characters into Unicode
    > characters?
    >
    > Simply copy them into the low byte of your Unicode characters!
    >


    Not quite, ISO-8859-1 includes characters with the most significant
    bit set (ie. 128-255). The low range matches (I believe, but could
    be wrong...) ASCII, the printable characters of which do map directly
    to Unicode. In general you need tables of corresponding characters
    to do this, see P J Plauger's post.

    --
    imalone
     
    Ian Malone, Jul 25, 2006
    #4
  5. Simon Biber Guest

    Ian Malone wrote:
    > Simon Biber wrote:
    >> wrote:
    >>> Hi ,
    >>> I want to convert an array of bytes like :
    >>>
    >>> {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >>>
    >>> into Unicode character with ISO-8859-1 standard.
    >>>
    >>> Can anyone help me .. how should I go about doing it ?

    >>
    >> Let me get this straight. Your array of bytes contains ISO 8859-1
    >> characters, and you want to convert those characters into Unicode
    >> characters?
    >>
    >> Simply copy them into the low byte of your Unicode characters!
    >>

    >
    > Not quite, ISO-8859-1 includes characters with the most significant
    > bit set (ie. 128-255). The low range matches (I believe, but could
    > be wrong...) ASCII, the printable characters of which do map directly
    > to Unicode. In general you need tables of corresponding characters
    > to do this, see P J Plauger's post.


    The Unicode character set was designed not only that 0-127 match ASCII,
    but also that 128-255 match ISO-8859-1. This works for UTF-16 and UTF-32
    but not, of course, UTF-8.

    Ref: http://en.wikipedia.org/wiki/ISO-8859-1
    "ISO-8859-1 was also incorporated as the first 256 code points of Unicode."

    Ref: http://en.wikipedia.org/wiki/Latin_Unicode
    "Unicode as of version 4.1 defines the following ranges for encoding the
    Latin alphabet and derived characters:
    * Basic Latin 0000–007F: identical to ASCII (0000–001F are control
    characters, 0020–003F are punctuation and Arabic numerals)
    * Latin-1 Supplement 0080–00FF: identical to ISO/IEC 8859-1
    (0080–009F are control characters, 00A0–00BF are currency
    symbols, punctuation and numerals)"

    --
    Simon.
     
    Simon Biber, Jul 25, 2006
    #5
  6. P.J. Plauger Guest

    "Simon Biber" <> wrote in message
    news:...

    > Ian Malone wrote:
    >> Simon Biber wrote:
    >>> wrote:
    >>>> Hi ,
    >>>> I want to convert an array of bytes like :
    >>>>
    >>>> {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >>>>
    >>>> into Unicode character with ISO-8859-1 standard.
    >>>>
    >>>> Can anyone help me .. how should I go about doing it ?
    >>>
    >>> Let me get this straight. Your array of bytes contains ISO 8859-1
    >>> characters, and you want to convert those characters into Unicode
    >>> characters?
    >>>
    >>> Simply copy them into the low byte of your Unicode characters!
    >>>

    >>
    >> Not quite, ISO-8859-1 includes characters with the most significant
    >> bit set (ie. 128-255). The low range matches (I believe, but could
    >> be wrong...) ASCII, the printable characters of which do map directly
    >> to Unicode. In general you need tables of corresponding characters
    >> to do this, see P J Plauger's post.

    >
    > The Unicode character set was designed not only that 0-127 match ASCII,
    > but also that 128-255 match ISO-8859-1. This works for UTF-16 and UTF-32
    > but not, of course, UTF-8.
    >
    > Ref: http://en.wikipedia.org/wiki/ISO-8859-1
    > "ISO-8859-1 was also incorporated as the first 256 code points of
    > Unicode."
    >
    > Ref: http://en.wikipedia.org/wiki/Latin_Unicode
    > "Unicode as of version 4.1 defines the following ranges for encoding the
    > Latin alphabet and derived characters:
    > * Basic Latin 0000–007F: identical to ASCII (0000–001F are control
    > characters, 0020–003F are punctuation and Arabic numerals)
    > * Latin-1 Supplement 0080–00FF: identical to ISO/IEC 8859-1
    > (0080–009F are control characters, 00A0–00BF are currency
    > symbols, punctuation and numerals)"


    Yep, you're right. If you look at the table I presented, it's
    obviously a one-to-one mapping. (We have so many tables like
    that in our locale code that I forgot this one is trivial.)

    So, assuming the OP's sequence of bytes is UTF-8 for codes no larger
    than 0xff, all you have to do is follow the algorithmic rules for
    converting UTF-8 to UCS-2 (which is the same as UTF-16 in this case).

    P.J. Plauger
    Dinkumware, Ltd.
    http://www.dinkumware.com
     
    P.J. Plauger, Jul 25, 2006
    #6
  7. posted:

    > Hi ,
    > I want to convert an array of bytes like :
    >
    > {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >
    > into Unicode character with ISO-8859-1 standard.
    >
    > Can anyone help me .. how should I go about doing it ?
    >
    > Thanks



    Perhaps something like:

    #include <wchar.h>

    void ToUnicode(wchar_t *pw, char const *pc)
    {
    while(*pw++ = *pc++);
    }

    int main(void)
    {
    char const str[] = "October is the tenth month of the year.";

    wchar_t buf[sizeof str * sizeof(wchar_t)];

    ToUnicode(buf,str);
    }

    --

    Frederick Gotham
     
    Frederick Gotham, Jul 25, 2006
    #7
  8. P.J. Plauger Guest

    "P.J. Plauger" <> wrote in message
    news:...

    > So, assuming the OP's sequence of bytes is UTF-8 for codes no larger
    > than 0xff, all you have to do is follow the algorithmic rules for
    > converting UTF-8 to UCS-2 (which is the same as UTF-16 in this case).


    Uh, never mind. He said ISO 8859-1, not UTF-8. It's just one-to-one,
    at least if you treat the "array of bytes" as unsigned. I'm now zero
    for two, I think...

    P.J. Plauger
    Dinkumware, Ltd.
    http://www.dinkumware.com
     
    P.J. Plauger, Jul 25, 2006
    #8
  9. wrote:
    >
    > I want to convert an array of bytes like :
    >
    > {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >
    > into Unicode character with ISO-8859-1 standard.
    >
    > Can anyone help me .. how should I go about doing it ?


    Your question is unclear. Are you saying that the current bit patterns
    in the array are ISO 8859-1 encodings, presumably represented as signed
    chars? (That is, that if I treat the bit patterns in the array as
    unsigned chars, then the low-order 8 bits of each are ISO 8859-1
    character values?)

    Assuming that, you first need to pick a type with at least 21 value
    bits to hold the Unicode characters. Life will usually be less
    confusing if you make it an unsigned type. Unsigned long will be big
    enough in all versions of Standard C. You then need to put the 8 low
    order bits from the each entry in the source array into the 8 low order
    bits in the destination entries, with zeroes in the higher bits. Use
    pointers to iterate over the arrays doing something like

    *targetp = *(unsigned char *)sourcep & 0xff;

    The values of ISO 8859-1 characters are the same as the values of the
    equivalent Unicode characters.
     
    J. J. Farrell, Jul 25, 2006
    #9
  10. "J. J. Farrell" <> writes:
    > wrote:
    >> I want to convert an array of bytes like :
    >>
    >> {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    >>
    >> into Unicode character with ISO-8859-1 standard.
    >>
    >> Can anyone help me .. how should I go about doing it ?

    >
    > Your question is unclear. Are you saying that the current bit patterns
    > in the array are ISO 8859-1 encodings, presumably represented as signed
    > chars? (That is, that if I treat the bit patterns in the array as
    > unsigned chars, then the low-order 8 bits of each are ISO 8859-1
    > character values?)
    >
    > Assuming that, you first need to pick a type with at least 21 value
    > bits to hold the Unicode characters. Life will usually be less
    > confusing if you make it an unsigned type. Unsigned long will be big
    > enough in all versions of Standard C. You then need to put the 8 low
    > order bits from the each entry in the source array into the 8 low order
    > bits in the destination entries, with zeroes in the higher bits. Use
    > pointers to iterate over the arrays doing something like
    >
    > *targetp = *(unsigned char *)sourcep & 0xff;
    >
    > The values of ISO 8859-1 characters are the same as the values of the
    > equivalent Unicode characters.


    Assuming sourcep is a char*, I think converting the char value to
    unsigned char is clearer than doing a pointer conversion:

    *targetp = (unsigned char)*sourcep & 0xff;

    The "& 0xff" may be unnecessary of CHAR_BIT==8.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Jul 25, 2006
    #10
  11. In article <> Ian Malone <> writes:
    ....
    > Not quite, ISO-8859-1 includes characters with the most significant
    > bit set (ie. 128-255). The low range matches (I believe, but could
    > be wrong...) ASCII, the printable characters of which do map directly
    > to Unicode. In general you need tables of corresponding characters
    > to do this, see P J Plauger's post.


    Look at the table that P. J. Plaugher posted and see that it starts with:
    0x00 0x0000
    and goes on to
    0xFF 0x00FF
    or something like that. The first 256 codes of Unicode are identical to
    ISO 8859-1.
    --
    dik t. winter, cwi, kruislaan 413, 1098 sj amsterdam, nederland, +31205924131
    home: bovenover 215, 1025 jn amsterdam, nederland; http://www.cwi.nl/~dik/
     
    Dik T. Winter, Jul 25, 2006
    #11
  12. Keith Thompson wrote:
    > "J. J. Farrell" <> writes:
    > > wrote:
    > >> I want to convert an array of bytes like :
    > >>
    > >> {79,104,-37,-66,24,123,30,-26,-99,-8,80,-38,19,14,-127,-3}
    > >>
    > >> into Unicode character with ISO-8859-1 standard.
    > >>
    > >> Can anyone help me .. how should I go about doing it ?

    > >
    > > Your question is unclear. Are you saying that the current bit patterns
    > > in the array are ISO 8859-1 encodings, presumably represented as signed
    > > chars? (That is, that if I treat the bit patterns in the array as
    > > unsigned chars, then the low-order 8 bits of each are ISO 8859-1
    > > character values?)
    > >
    > > Assuming that, you first need to pick a type with at least 21 value
    > > bits to hold the Unicode characters. Life will usually be less
    > > confusing if you make it an unsigned type. Unsigned long will be big
    > > enough in all versions of Standard C. You then need to put the 8 low
    > > order bits from the each entry in the source array into the 8 low order
    > > bits in the destination entries, with zeroes in the higher bits. Use
    > > pointers to iterate over the arrays doing something like
    > >
    > > *targetp = *(unsigned char *)sourcep & 0xff;
    > >
    > > The values of ISO 8859-1 characters are the same as the values of the
    > > equivalent Unicode characters.

    >
    > Assuming sourcep is a char*, I think converting the char value to
    > unsigned char is clearer than doing a pointer conversion:
    >
    > *targetp = (unsigned char)*sourcep & 0xff;


    Clearer perhaps, but also wrong in some environments given the
    assumptions I stated.

    > The "& 0xff" may be unnecessary of CHAR_BIT==8.


    Indeed. But why write it in a way that will be wrong if CHAR_BIT != 8,
    when it takes so little extra effort to make it portable?
     
    J. J. Farrell, Jul 26, 2006
    #12
  13. Ian Malone Guest

    Simon Biber wrote:
    > Ian Malone wrote:
    >> Simon Biber wrote:


    >>> Simply copy them into the low byte of your Unicode characters!
    >>>

    >>
    >> Not quite, ISO-8859-1 includes characters with the most significant
    >> bit set (ie. 128-255). The low range matches (I believe, but could
    >> be wrong...) ASCII, the printable characters of which do map directly
    >> to Unicode. In general you need tables of corresponding characters
    >> to do this, see P J Plauger's post.

    >
    > The Unicode character set was designed not only that 0-127 match ASCII,
    > but also that 128-255 match ISO-8859-1. This works for UTF-16 and UTF-32
    > but not, of course, UTF-8.
    >


    Sorry, my blood caffeine level was obviously dangerously low yesterday.
    Hallucinations that occurred involved the fact it won't fit into UTF8
    as you mention, and I somehow read low byte as 7 bits...

    --
    imalone
     
    Ian Malone, Jul 26, 2006
    #13
  14. On Tue, 25 Jul 2006 16:15:10 GMT, Frederick Gotham
    <> wrote:
    <snip>
    > Perhaps something like:
    >
    > #include <wchar.h>
    >
    > void ToUnicode(wchar_t *pw, char const *pc)
    > {
    > while(*pw++ = *pc++);
    > }
    >
    > int main(void)
    > {
    > char const str[] = "October is the tenth month of the year.";
    >
    > wchar_t buf[sizeof str * sizeof(wchar_t)];
    >

    The *sizeof(wchar_t) is unnecessary and wasteful. You need (only) the
    same number of wchar_t as there were chars in the source.

    > ToUnicode(buf,str);
    > }


    - David.Thompson1 at worldnet.att.net
     
    Dave Thompson, Aug 7, 2006
    #14
  15. Dave Thompson posted:

    >> char const str[] = "October is the tenth month of the year.";
    >>
    >> wchar_t buf[sizeof str * sizeof(wchar_t)];
    >>

    > The *sizeof(wchar_t) is unnecessary and wasteful. You need (only) the
    > same number of wchar_t as there were chars in the source.



    You're correct, I wrote the code quickly and sloppily. I should have written:

    char const str[] = "whatever";

    wchar_t buf[sizeof str];

    --

    Frederick Gotham
     
    Frederick Gotham, Aug 7, 2006
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?iso-8859-1?b?Sulq6Q==?=

    [EXCEL] sequential =?iso-8859-1?q?acc=E8s=2E?=

    =?iso-8859-1?b?Sulq6Q==?=, Jan 23, 2004, in forum: Perl
    Replies:
    0
    Views:
    720
    =?iso-8859-1?b?Sulq6Q==?=
    Jan 23, 2004
  2. Timiun
    Replies:
    1
    Views:
    527
    Joerg Jooss
    Dec 24, 2003
  3. Jonck
    Replies:
    3
    Views:
    2,460
    Jonck
    Oct 25, 2004
  4. Franck DARRAS
    Replies:
    12
    Views:
    654
    Jim Higson
    Aug 23, 2004
  5. Peter Jacobi
    Replies:
    13
    Views:
    854
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Aug 3, 2004
Loading...

Share This Page