how to print the GREEK CAPITAL LETTER delta under utf-8 encoding

Discussion in 'Python' started by =?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7z, May 29, 2007.

  1. =?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7z, May 29, 2007
    #1
    1. Advertising

  2. ÈËÑÔÂäÈÕÊÇÌìÑÄ£¬Íû¼«ÌìÑIJ»¼û¼Ò schrieb:
    > I lookup the utf-8 form of delta from the link.
    > http://www.fileformat.info/info/unicode/char/0394/index.htm
    >
    > and then I want to print it in the python ( I work under windows)
    >
    > #!/usr/bin/python
    > #coding=utf-8
    >
    > print "\xce\x94"
    >
    > but the result is not the 'delta' but an unknown character.


    I assume you print to the terminal (cmd.exe). This cannot work;
    the terminal (usually) does not interpret the characters in UTF-8.
    Instead, you should print a Unicode string, e.g.

    print u"\N{GREEK CAPITAL LETTER DELTA}"

    or

    print u'\u0394'

    This should work as long as your terminal supports printing
    the letter at all.

    Regards,
    Martin
     
    =?GB2312?B?Ik1hcnRpbiB2LiBMbyJ3aXMi?=, May 29, 2007
    #2
    1. Advertising

  3. On 5ÔÂ29ÈÕ, ÏÂÎç1ʱ34·Ö, "Martin v. Lo"wis" <> wrote:
    > ÈËÑÔÂäÈÕÊÇÌìÑÄ£¬Íû¼«ÌìÑIJ»¼û¼Ò schrieb:
    >
    > > I lookup the utf-8 form of delta from the link.
    > >http://www.fileformat.info/info/unicode/char/0394/index.htm

    >
    > > and then I want to print it in the python ( I work under windows)

    >
    > > #!/usr/bin/python
    > > #coding=utf-8

    >
    > > print "\xce\x94"

    >
    > > but the result is not the 'delta' but an unknown character.

    >
    > I assume you print to the terminal (cmd.exe). This cannot work;
    > the terminal (usually) does not interpret the characters in UTF-8.
    > Instead, you should print a Unicode string, e.g.
    >
    > print u"\N{GREEK CAPITAL LETTER DELTA}"
    >
    > or
    >
    > print u'\u0394'
    >
    > This should work as long as your terminal supports printing
    > the letter at all.
    >
    > Regards,
    > Martin


    yes, it could print to the terminal(cmd.exe), but when I write these
    string to file. I got the follow error:

    File "E:\Tools\filegen\filegen.py", line 212, in write
    self.file.write(data)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
    position 0
    : ordinal not in range(128)

    but other text, in which include "chinese characters" got from
    os.listdir(...), are written to the file OK. why?
     
    =?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7z, May 29, 2007
    #3
  4. > yes, it could print to the terminal(cmd.exe), but when I write these
    > string to file. I got the follow error:
    >
    > File "E:\Tools\filegen\filegen.py", line 212, in write
    > self.file.write(data)
    > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
    > position 0
    > : ordinal not in range(128)


    Yes, when writing to a file, you need to define an encoding, e.g.

    self.file.write(data.encode("utf-8"))

    You can use codecs.open() instead of open(),
    so that you can just use self.file.write(data)

    Alternatively, you can find out what sys.stdout.encoding is,
    and use that when encoding data for the terminal (falling back
    to "utf-8" when .encoding is not available on the file).

    > but other text, in which include "chinese characters" got from
    > os.listdir(...), are written to the file OK. why?


    Your version of Windows uses a code page that supports Chinese
    characters in the byte-oriented character set. These are normally
    encoded using the "mbcs" encoding (except that the terminal likely
    uses a different encoding). So if you use "mbcs" instead of "utf-8",
    you might be able to read the text as well.

    Regards,
    Martin
     
    =?GB2312?B?Ik1hcnRpbiB2LiBMbyJ3aXMi?=, May 29, 2007
    #4
  5. On 5ÔÂ29ÈÕ, ÏÂÎç3ʱ05·Ö, "Martin v. Lo"wis" <> wrote:
    > > yes, it could print to the terminal(cmd.exe), but when I write these
    > > string to file. I got the follow error:

    >
    > > File "E:\Tools\filegen\filegen.py", line 212, in write
    > > self.file.write(data)
    > > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
    > > position 0
    > > : ordinal not in range(128)

    >
    > Yes, when writing to a file, you need to define an encoding, e.g.
    >
    > self.file.write(data.encode("utf-8"))
    >
    > You can use codecs.open() instead of open(),
    > so that you can just use self.file.write(data)
    >
    > Alternatively, you can find out what sys.stdout.encoding is,
    > and use that when encoding data for the terminal (falling back
    > to "utf-8" when .encoding is not available on the file).
    >
    > > but other text, in which include "chinese characters" got from
    > > os.listdir(...), are written to the file OK. why?

    >
    > Your version of Windows uses a code page that supports Chinese
    > characters in the byte-oriented character set. These are normally
    > encoded using the "mbcs" encoding (except that the terminal likely
    > uses a different encoding). So if you use "mbcs" instead of "utf-8",
    > you might be able to read the text as well.
    >
    > Regards,
    > Martin


    Thanks a lot!
    I want to just use the utf-8. how could I convert my 'mbcs' encoding
    to the utf-8 and write it to the file?
    I have replaced the open() to codecs.open()

    but it still can not decode the 'mbcs', the error is as follow:

    File "E:\Tools\filegen\filegen.py", line 213, in write
    self.file.write(data)
    File "C:\Python25\lib\codecs.py", line 638, in write
    return self.writer.write(data)
    File "C:\Python25\lib\codecs.py", line 303, in write
    data, consumed = self.encode(object, self.errors)
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position
    32: ordinal
    not in range(128)
     
    =?gb2312?B?yMvR1MLkyNXKx8zs0cSjrM37vKvM7NHEsru8+7z, May 29, 2007
    #5
  6. En Tue, 29 May 2007 04:24:15 -0300, 人言è½æ—¥æ˜¯å¤©æ¶¯ï¼Œæœ›æžå¤©æ¶¯ä¸è§å®¶
    <> escribió:

    > On 5月29æ—¥, 下åˆ3æ—¶05分, "Martin v. Lo"wis" <> wrote:


    >> > yes, it could print to the terminal(cmd.exe), but when I write these
    >> > string to file. I got the follow error:

    >>
    >> > File "E:\Tools\filegen\filegen.py", line 212, in write
    >> > self.file.write(data)
    >> > UnicodeEncodeError: 'ascii' codec can't encode character u'\u0394' in
    >> > position 0
    >> > : ordinal not in range(128)

    >>
    >> Yes, when writing to a file, you need to define an encoding, e.g.
    >>
    >> self.file.write(data.encode("utf-8"))
    >>
    >> You can use codecs.open() instead of open(),
    >> so that you can just use self.file.write(data)
    >>

    > Thanks a lot!
    > I want to just use the utf-8. how could I convert my 'mbcs' encoding
    > to the utf-8 and write it to the file?
    > I have replaced the open() to codecs.open()
    >
    > but it still can not decode the 'mbcs', the error is as follow:
    >
    > File "E:\Tools\filegen\filegen.py", line 213, in write
    > self.file.write(data)
    > File "C:\Python25\lib\codecs.py", line 638, in write
    > return self.writer.write(data)
    > File "C:\Python25\lib\codecs.py", line 303, in write
    > data, consumed = self.encode(object, self.errors)
    > UnicodeDecodeError: 'ascii' codec can't decode byte 0xcc in position
    > 32: ordinal
    > not in range(128)


    Just to provide an example of what MvL already said:

    py> line = u"Delta=\u0394"
    py> f = open("data.txt","w")
    py> f.write(line.encode("utf8"))
    py> f.close()
    py> print repr(open("data.txt").read())
    'Delta=\xce\x94'

    py> import codecs
    py> f = codecs.open("data2.txt","w","utf8")
    py> f.write(line)
    py> f.close()
    py> print repr(open("data2.txt").read())
    'Delta=\xce\x94'

    --
    Gabriel Genellina
     
    Gabriel Genellina, May 29, 2007
    #6
  7. Re: how to print the GREEK CAPITAL LETTER delta under utf-8encoding

    tis 2007-05-29 klockan 09:05 +0200 skrev "Martin v. Lo"wis":
    > Yes, when writing to a file, you need to define an encoding, e.g.
    >
    > self.file.write(data.encode("utf-8"))
    >
    > You can use codecs.open() instead of open(),
    > so that you can just use self.file.write(data)


    If I for some reason can't open the object myself or needs encoding on
    other file-like objects, I think the following wrapper function is of
    use (it essentially does what codecs.open() does but takes a file-object
    instead of a filename):

    def filewrapper(f, encoding=None, errors='strict'):
    if encoding is None:
    return f

    info = codecs.lookup(encoding)
    srw = codecs.StreamReaderWriter(f, info.streamreader,
    info.streamwriter, errors)
    # Add attributes to simplify introspection
    srw.encoding = encoding
    return srw

    I find this especially useful for changing how stdout and friends does
    it's encoding, e.g:

    >>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
    >>> print u"åäö \N{GREEK CAPITAL LETTER DELTA}"


    Useful if you don't want to append .encode() to everything you print out
    that potentially can contain non-ascii letters.

    /Ragnar
     
    Ragnar Ouchterlony, May 29, 2007
    #7
  8. En Tue, 29 May 2007 15:16:52 -0300, Ragnar Ouchterlony
    <> escribió:

    > If I for some reason can't open the object myself or needs encoding on
    > other file-like objects, I think the following wrapper function is of
    > use (it essentially does what codecs.open() does but takes a file-object
    > instead of a filename):
    >
    > def filewrapper(f, encoding=None, errors='strict'):
    > if encoding is None:
    > return f
    >
    > info = codecs.lookup(encoding)
    > srw = codecs.StreamReaderWriter(f, info.streamreader,
    > info.streamwriter, errors)
    > # Add attributes to simplify introspection
    > srw.encoding = encoding
    > return srw
    >
    > I find this especially useful for changing how stdout and friends does
    > it's encoding, e.g:
    >
    >>>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
    >>>> print u"åäö \N{GREEK CAPITAL LETTER DELTA}"

    >
    > Useful if you don't want to append .encode() to everything you print out
    > that potentially can contain non-ascii letters.


    Isn't the same as codecs.EncodedFile?

    --
    Gabriel Genellina
     
    Gabriel Genellina, May 29, 2007
    #8
  9. Re: how to print the GREEK CAPITAL LETTER delta under utf-8encoding

    tis 2007-05-29 klockan 16:08 -0300 skrev Gabriel Genellina:
    > >>>> sys.stdout = filewrapper(sys.stdout, 'utf-8')
    > >>>> print u"åäö \N{GREEK CAPITAL LETTER DELTA}"

    > >
    > > Useful if you don't want to append .encode() to everything you print out
    > > that potentially can contain non-ascii letters.

    >
    > Isn't the same as codecs.EncodedFile?
    >


    No, codecs.EncodedFile() doesn't do exactly the same.

    If I understand it correctly, EncodedFile() takes a byte stream as input
    and produces another bytestream (with possibly different encoding) as
    output in both read and write.

    My function (as well as codecs.open()) decodes a byte stream for read
    and produces a unicode object or encodes a unicode object for write and
    produces a byte stream.

    At least, I were unable to create the same behaviour as my filewrapper()
    using EncodedFile(). If you are able to do that, I'm interested in how
    you do it.

    /Ragnar
     
    Ragnar Ouchterlony, May 29, 2007
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. dungping
    Replies:
    27
    Views:
    3,417
    Jukka K. Korpela
    Sep 2, 2005
  2. Replies:
    9
    Views:
    465
    Mike Duffy
    May 10, 2007
  3. srikanth.padava

    delta cycle?? (delta delay)

    srikanth.padava, Feb 28, 2008, in forum: VHDL
    Replies:
    0
    Views:
    1,129
    srikanth.padava
    Feb 28, 2008
  4. Replies:
    1
    Views:
    346
    Tim Roberts
    Nov 9, 2008
  5. Steven D'Aprano

    Re: Turnign greek-iso filenames => utf-8 iso

    Steven D'Aprano, Jun 12, 2013, in forum: Python
    Replies:
    17
    Views:
    496
    Steven D'Aprano
    Jun 13, 2013
Loading...

Share This Page