How do I display unicode-paths?

Discussion in 'Python' started by Pettersen, Bjorn S, Oct 5, 2003.

  1. I've been trying to stay blissfully unaware of Unicode, however now it seems like it's my turn. From the outside it seems like a rather massive subject, so any pointers as to where I should _start_ reading would be appreciated. The usecase is a class:

    class path(object):
    ...
    def __str__(self):
    return self.pathstr.encode(???)

    the question is what to put at ??? to be most useful to programmers/end users?

    Background: I've got a directory with a file called 'bæ', B-AE, (which everyone knows is what Norwegian sheep say :). If I type u'bæ' at the Python command prompt, it returns:

    u'b\x91'

    (WinXP, 2.3.2, regular command window). With a little trial and error I found:

    >>> print u'b\x91'.encode('latin1')



    although

    >>> print u'b\x91'

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "e:\python23\lib\encodings\cp437.py", line 18, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
    UnicodeEncodeError: 'charmap' codec can't encode character '\u91' in position 1:
    character maps to <undefined>

    not sure I'm understanding this, and when I call:

    os.listdir(os.getcwdu())

    I get back

    u'b\xe6'

    which isn't making much sense either...

    help?!

    -- bjorn
     
    Pettersen, Bjorn S, Oct 5, 2003
    #1
    1. Advertising

  2. Pettersen, Bjorn S wrote:
    > I've been trying to stay blissfully unaware of Unicode, however now it seems like it's my turn. From the outside it seems like a rather massive subject, so any pointers as to where I should _start_ reading would be appreciated. The usecase is a class:
    >
    > class path(object):
    > ...
    > def __str__(self):
    > return self.pathstr.encode(???)
    >
    > the question is what to put at ??? to be most useful to programmers/end users?


    It is not possible to put something "most useful" here. If you want to
    convert a Unicode string to a byte string, you typically do so because
    you need to put data into a byte-oriented channel (such as a file, a
    socket, or a terminal). You *do* need to know the encoding of that
    channel to convert the Unicode object. Unfortunately, __str__ has no way
    of knowing where data will be put.

    If you can assume that the bytes returned are displayed to the user on
    the local system, you can try locale.getpreferredencoding(). This should
    work in most cases, with the notable exception of a cmd.exe window on MS
    Windows: the terminal uses an encoding *different* from the user's
    preferred encoding, and it is impossible to know in advance what that
    encoding will be.

    So in short: Just don't convert the Unicode string into a byte string,
    until you know exactly where it will be displayed.

    > Background: I've got a directory with a file called 'bæ', B-AE, (which everyone knows is what Norwegian sheep say :). If I type u'bæ' at the Python command prompt, it returns:
    >
    > u'b\x91'


    Unfortunately, this did not work very well: Try the same in IDLE, and
    you will get u'b\xe6'

    Unicode character U+00E6 really is LATIN SMALL LETTER AE:

    >>> u"\N{LATIN SMALL LETTER AE}"

    u'\xe6'

    Unicode character U+0091 is *not* AE, it is some (useless) control
    character. The problem is that the windows console was using MS CP850,
    but Python was interpreting it as Latin-1 (see PEP 263).

    Print u"\xe6" on the console should work fine, in Python 2.3 (since
    Python finds out, at the time of putting out the bytes, what the
    encoding of the console is).

    Regards,
    Martin
     
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Oct 5, 2003
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Robert Mark Bram
    Replies:
    0
    Views:
    4,048
    Robert Mark Bram
    Sep 28, 2003
  2. Noah
    Replies:
    5
    Views:
    848
  3. ygao

    unicode wrap unicode object?

    ygao, Apr 8, 2006, in forum: Python
    Replies:
    6
    Views:
    590
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Apr 8, 2006
  4. Gabriele *darkbard* Farina

    Unicode digit to unicode string

    Gabriele *darkbard* Farina, May 16, 2006, in forum: Python
    Replies:
    2
    Views:
    562
    Gabriele *darkbard* Farina
    May 16, 2006
  5. Ohad Lutzky

    Paths, gentleman, paths

    Ohad Lutzky, Nov 6, 2006, in forum: Ruby
    Replies:
    2
    Views:
    225
    David Vallner
    Nov 7, 2006
Loading...

Share This Page