UTF-8 locales

Discussion in 'C Programming' started by saroj.yadav@gmail.com, Mar 22, 2005.

  1. Guest

    As I understand it (correct me, if I am wrong) Unicode came into
    picture so that a document containing multiple language characters can
    be supported like somebody can write a document comparing Korean and
    Chinese in French language.

    Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
    HP, Solaris) platforms support Unicode by supporting language/region
    specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.

    Now in order to use UTF-8 for Japanese, I have to set locale to
    ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
    ko_KR.UTF-8.

    With this approach it's not possible to mix multiple language
    characters. Doesn't this defeat the whole purpose of Unicode ?
    Am I missing something ?

    Thanks in advance for any insight you can provide.
    , Mar 22, 2005
    #1
    1. Advertising

  2. In article <>,
    wrote:

    > As I understand it (correct me, if I am wrong) Unicode came into
    > picture so that a document containing multiple language characters can
    > be supported like somebody can write a document comparing Korean and
    > Chinese in French language.
    >
    > Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
    > HP, Solaris) platforms support Unicode by supporting language/region
    > specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.
    >
    > Now in order to use UTF-8 for Japanese, I have to set locale to
    > ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
    > ko_KR.UTF-8.
    >
    > With this approach it's not possible to mix multiple language
    > characters. Doesn't this defeat the whole purpose of Unicode ?
    > Am I missing something ?


    A locale describes much more than just the characters.

    There are for example the rules for formatting a number. Threethousand
    twohundred and six hundredths is 3,200.06 in one locale and 3.200,06 in
    another and 3'200.06 in a third locale.

    There are rules how to display monetary values. There are rules how to
    sort text. There are rules how to display dates. They all will be
    different between locales.
    Christian Bau, Mar 22, 2005
    #2
    1. Advertising

  3. wrote:
    > As I understand it (correct me, if I am wrong) Unicode came into
    > picture so that a document containing multiple language characters can
    > be supported like somebody can write a document comparing Korean and
    > Chinese in French language.
    >
    > Now, I am looking at all UNIX platforms and seems like all Unix (AIX,
    > HP, Solaris) platforms support Unicode by supporting language/region
    > specific UTF-8 locales like fr_FR.UTF-8, ja_JP.UTF-8, ko_KR.UTF-8 etc.
    >
    > Now in order to use UTF-8 for Japanese, I have to set locale to
    > ja_JP.UTF-8. To use UTF-8 for Korean, I have to set locale to
    > ko_KR.UTF-8.
    >
    > With this approach it's not possible to mix multiple language
    > characters. Doesn't this defeat the whole purpose of Unicode ?
    > Am I missing something ?
    >
    > Thanks in advance for any insight you can provide.
    >



    I am not certain how this works on Unix platforms. However I suspect
    that the locale has an effect on local conventions as the previous
    poster mentioned. I.e date formats and what not.

    You still should be able to display any language no matter what locale
    you have set, i.e in ja_JP.UTF-8 you should still be able to display
    French an Chinese and so on. Try it. ( You must have fonts capable of
    mapping those characters of course, if it doesn't work try changing font
    - look on the web for names of the fonts that support the glyphs that
    are needed )
    Spacen Jasset, Mar 23, 2005
    #3
  4. CBFalconer Guest

    wrote:
    >
    > As I understand it (correct me, if I am wrong) Unicode came into
    > picture so that a document containing multiple language characters
    > can be supported like somebody can write a document comparing
    > Korean and Chinese in French language.
    >
    > Now, I am looking at all UNIX platforms and seems like all Unix
    > (AIX, HP, Solaris) platforms support Unicode by supporting
    > language/region specific UTF-8 locales like fr_FR.UTF-8,
    > ja_JP.UTF-8, ko_KR.UTF-8 etc.


    .... snip ...

    This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
    solaris, etc. in its name. c.l.c deals with the actual portable
    language as defined in the C standard. As soon as you have to
    specify a system or compiler, you become off-topic.

    --
    "I conclude that there are two ways of constructing a software
    design: One way is to make it so simple that there are obviously
    no deficiencies and the other way is to make it so complicated
    that there are no obvious deficiencies." -- C. A. R. Hoare
    CBFalconer, Mar 23, 2005
    #4
  5. Kevin Bracey Guest

    In message <>
    CBFalconer <> wrote:

    > wrote:
    > >
    > > Now, I am looking at all UNIX platforms and seems like all Unix
    > > (AIX, HP, Solaris) platforms support Unicode by supporting
    > > language/region specific UTF-8 locales like fr_FR.UTF-8,
    > > ja_JP.UTF-8, ko_KR.UTF-8 etc.

    >
    > ... snip ...
    >
    > This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
    > solaris, etc. in its name. c.l.c deals with the actual portable
    > language as defined in the C standard. As soon as you have to
    > specify a system or compiler, you become off-topic.


    Which is of course why the whole locale system and wchar_t are effectively
    useless for portable programming. Everything about them is
    implementation-defined, so you can't even talk about them in comp.*.c :(

    Is there anywhere where one could discuss "best practice" or standardisation
    for these facilities?

    --
    Kevin Bracey, Principal Software Engineer
    Tematic Ltd Tel: +44 (0) 1223 503464
    182-190 Newmarket Road Fax: +44 (0) 1728 727430
    Cambridge, CB5 8HE, United Kingdom WWW: http://www.tematic.com/
    Kevin Bracey, Mar 23, 2005
    #5
  6. Michael Mair Guest

    Kevin Bracey wrote:
    > In message <>
    > CBFalconer <> wrote:
    >
    >
    >> wrote:
    >>
    >>>Now, I am looking at all UNIX platforms and seems like all Unix
    >>>(AIX, HP, Solaris) platforms support Unicode by supporting
    >>>language/region specific UTF-8 locales like fr_FR.UTF-8,
    >>>ja_JP.UTF-8, ko_KR.UTF-8 etc.

    >>
    >>... snip ...
    >>
    >>This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
    >>solaris, etc. in its name. c.l.c deals with the actual portable
    >>language as defined in the C standard. As soon as you have to
    >>specify a system or compiler, you become off-topic.

    >
    >
    > Which is of course why the whole locale system and wchar_t are effectively
    > useless for portable programming. Everything about them is
    > implementation-defined, so you can't even talk about them in comp.*.c :(
    >
    > Is there anywhere where one could discuss "best practice" or standardisation
    > for these facilities?


    Ack.
    I really would be interested in that, too.


    Cheers
    Michael
    --
    E-Mail: Mine is an /at/ gmx /dot/ de address.
    Michael Mair, Mar 23, 2005
    #6
  7. CBFalconer Guest

    Michael Mair wrote:
    > Kevin Bracey wrote:
    >> CBFalconer <> wrote:
    >>> wrote:
    >>>
    >>>> Now, I am looking at all UNIX platforms and seems like all Unix
    >>>> (AIX, HP, Solaris) platforms support Unicode by supporting
    >>>> language/region specific UTF-8 locales like fr_FR.UTF-8,
    >>>> ja_JP.UTF-8, ko_KR.UTF-8 etc.
    >>>
    >>>... snip ...
    >>>
    >>> This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
    >>> solaris, etc. in its name. c.l.c deals with the actual portable
    >>> language as defined in the C standard. As soon as you have to
    >>> specify a system or compiler, you become off-topic.

    >>
    >> Which is of course why the whole locale system and wchar_t are
    >> effectively useless for portable programming. Everything about
    >> them is implementation-defined, so you can't even talk about them
    >> in comp.*.c :(
    >>
    >> Is there anywhere where one could discuss "best practice" or
    >> standardisation for these facilities?

    >
    > Ack.
    > I really would be interested in that, too.


    I would consider comp.programming quite suitable for this sort of
    general internationalization discussion.

    --
    Chuck F () ()
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net> USE worldnet address!
    CBFalconer, Mar 23, 2005
    #7
  8. Guest

    Kevin Bracey wrote:
    > CBFalconer <> wrote:
    > > wrote:
    > > > Now, I am looking at all UNIX platforms and seems like all Unix
    > > > (AIX, HP, Solaris) platforms support Unicode by supporting
    > > > language/region specific UTF-8 locales like fr_FR.UTF-8,
    > > > ja_JP.UTF-8, ko_KR.UTF-8 etc.

    > >
    > > ... snip ...
    > >
    > > This is off-topic in c.l.c. Try a newsgroup with unix, posix, aix,
    > > solaris, etc. in its name. c.l.c deals with the actual portable
    > > language as defined in the C standard. As soon as you have to
    > > specify a system or compiler, you become off-topic.

    >
    > Which is of course why the whole locale system and wchar_t are
    > effectively useless for portable programming. Everything about
    > them is implementation-defined, so you can't even talk about
    > them in comp.*.c :(


    Yeah, amazing he doesn't see the irony of his own statement. The
    wchar_t types and functions are only portable in the very narrow scope
    of "I can compile that source code to do something" sense. But since
    the definition of wchar_t is so open ended, system dependent, and in
    fact *NOT* universal in any real sense, there's no way you can actually
    use it for any truly portable applications.

    And there's no way to somehow "augment" the specification, or use these
    types/function in some special way to *make* them portable. There are
    no ifdef tricks or anything like that -- you're just screwed. The
    whole wchar_t idea literally has no relevant functional content.

    C's whole misconception of what it means to be portable, is in fact
    what *prevents* it from being portable where it counts, on issues like
    localization.

    > Is there anywhere where one could discuss "best practice" or
    > standardisation for these facilities?


    Well more importantly, portability in general, really can't be
    discussed in comp.lang.c. I lurk on comp.programming, so you can
    always throw questions about portability there.

    ---
    Paul Hsieh
    http://www.pobox.com/~qed/
    http://bstring.sf.net/
    , Mar 27, 2005
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?YWxhbg==?=

    Datagrid sorting from different locales.

    =?Utf-8?B?YWxhbg==?=, Apr 21, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    417
    =?Utf-8?B?YWxhbg==?=
    Apr 21, 2005
  2. David List
    Replies:
    0
    Views:
    297
    David List
    Aug 6, 2003
  3. William Payne

    Correct use of locales?

    William Payne, Dec 13, 2003, in forum: C++
    Replies:
    3
    Views:
    397
    Morten Hanssen
    Dec 13, 2003
  4. Moof

    Python Locales under windows

    Moof, Oct 28, 2003, in forum: Python
    Replies:
    1
    Views:
    792
    =?UTF-8?B?0LTQsNC80ZjQsNC9INCzLg==?=
    Oct 28, 2003
  5. Ellen Herzfeld

    Bogus locales on Mac

    Ellen Herzfeld, Jul 14, 2004, in forum: Python
    Replies:
    3
    Views:
    378
    Ellen Herzfeld
    Jul 16, 2004
Loading...

Share This Page