UnicodeEncodeError - a bit out of my element...

Discussion in 'Python' started by erikcw, Apr 11, 2007.

  1. erikcw

    erikcw Guest

    Hi all,

    I'm trying to parse an email message, but am running into this
    exception.

    Traceback (most recent call last):
    File "wa.py", line 336, in ?
    main()
    File "wa.py", line 332, in main
    print out['msg']
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in
    position 238: ordinal not in range(128)

    How can I decode/encode this string to print to stdout and send again
    in another email? Do I have to know what language the email is in?

    Thanks!
    Erik
     
    erikcw, Apr 11, 2007
    #1
    1. Advertising

  2. erikcw

    liupeng Guest

    I cut from Sam Python Phrasebook
    "Converting Unicode to Local Strings"

    import string locStr = "El "
    uniStr = u"Ni\u00F1o"
    print uniStr.encode('utf-8')
    print uniStr.encode('utf-16')
    print uniStr.encode('iso-8859-1')
    #Combine local and unicode results
    #in new unicode string
    newStr = locStr+uniStr
    print newStr.encode('iso-8859-1')
    #ascii will error because character '\xF1'
    #is out of range
    asciiStr = newStr.encode('iso-8859-1')
    asciiStr =asciiStr.translate(\
    string.maketrans('\xF1','n'), '')
    print asciiStr.encode('ascii')
    print newStr.encode('ascii')

    unicode_str.py

    Niño
    ÿN|I|ñ|o
    Niño
    El Niño
    El Nino
    Traceback (most recent call last):
    File "C:\books\python\CH2\code\unicode_str.py",
    line 19, in ?
    print newStr.encode('ascii')
    UnicodeEncodeError: 'ascii' codec can't encode
    character u'\xf1' in position 5: ordinal not in
    range(128)

    On Wed, Apr 11, 2007 at 08:16:07AM -0700, erikcw wrote:
    > Hi all,
    >
    > I'm trying to parse an email message, but am running into this
    > exception.
    >
    > Traceback (most recent call last):
    > File "wa.py", line 336, in ?
    > main()
    > File "wa.py", line 332, in main
    > print out['msg']
    > UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in
    > position 238: ordinal not in range(128)
    >
    > How can I decode/encode this string to print to stdout and send again
    > in another email? Do I have to know what language the email is in?
    >
    > Thanks!
    > Erik
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
    >


    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.6 (GNU/Linux)

    iD8DBQFGHQBTKeZs4sOrPzgRAtz6AJ0fivVr4nM/XKK8rN+JMSq+yQJEewCdHkVU
    ZGUTAD9I5FqLbdAFNsG8ZYw=
    =3Nek
    -----END PGP SIGNATURE-----
     
    liupeng, Apr 11, 2007
    #2
    1. Advertising

  3. erikcw

    Guest

    On Apr 11, 10:16 am, "erikcw" <> wrote:
    > Hi all,
    >
    > I'm trying to parse an email message, but am running into this
    > exception.
    >
    > Traceback (most recent call last):
    > File "wa.py", line 336, in ?
    > main()
    > File "wa.py", line 332, in main
    > print out['msg']
    > UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in
    > position 238: ordinal not in range(128)
    >
    > How can I decode/encode this string to print to stdout and send again
    > in another email? Do I have to know what language the email is in?
    >
    > Thanks!
    > Erik


    You'll need to do some encoding/decoding work. Check out the following
    links on unicode:

    http://www.reportlab.com/i18n/python_unicode_tutorial.html
    http://www.amk.ca/python/howto/unicode
    http://www.jorendorff.com/articles/unicode/python.html

    And here's a few links on parsing:

    http://docs.python.org/api/arg-parsing.html
    http://www.diveintopython.org/xml_processing/unicode.html

    Probably more information than you need, but it should help answer
    your question (and maybe any future questions about Unicode).

    Mike
     
    , Apr 11, 2007
    #3
  4. erikcw

    erikcw Guest

    On Apr 11, 11:35 am, liupeng <> wrote:
    > I cut from Sam Python Phrasebook
    > "Converting Unicode to Local Strings"
    >
    > import string locStr = "El "
    > uniStr = u"Ni\u00F1o"
    > print uniStr.encode('utf-8')
    > print uniStr.encode('utf-16')
    > print uniStr.encode('iso-8859-1')
    > #Combine local and unicode results
    > #in new unicode string
    > newStr = locStr+uniStr
    > print newStr.encode('iso-8859-1')
    > #ascii will error because character '\xF1'
    > #is out of range
    > asciiStr = newStr.encode('iso-8859-1')
    > asciiStr =asciiStr.translate(\
    > string.maketrans('\xF1','n'), '')
    > print asciiStr.encode('ascii')
    > print newStr.encode('ascii')
    >
    > unicode_str.py
    >
    > Niño
    > ÿN|I|ñ|o
    > Niño
    > El Niño
    > El Nino
    > Traceback (most recent call last):
    > File "C:\books\python\CH2\code\unicode_str.py",
    > line 19, in ?
    > print newStr.encode('ascii')
    > UnicodeEncodeError: 'ascii' codec can't encode
    > character u'\xf1' in position 5: ordinal not in
    > range(128)
    >
    > On Wed, Apr 11, 2007 at 08:16:07AM -0700, erikcw wrote:
    > > Hi all,

    >
    > > I'm trying to parse an email message, but am running into this
    > > exception.

    >
    > > Traceback (most recent call last):
    > > File "wa.py", line 336, in ?
    > > main()
    > > File "wa.py", line 332, in main
    > > print out['msg']
    > > UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in
    > > position 238: ordinal not in range(128)

    >
    > > How can I decode/encode this string to print to stdout and send again
    > > in another email? Do I have to know what language the email is in?

    >
    > > Thanks!
    > > Erik

    >
    > > --
    > >http://mail.python.org/mailman/listinfo/python-list

    >
    >
    >
    > signature.asc
    > 1KDownload


    I used the .encode("utf-8") method on the string and it fixed
    everything! Thanks for your help!
     
    erikcw, Apr 11, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    3
    Views:
    1,762
    Timothy Bendfelt
    Jan 19, 2007
  2. Replies:
    21
    Views:
    1,020
    Kevin Handy
    Jun 22, 2006
  3. Replies:
    9
    Views:
    978
    Juha Nieminen
    Aug 22, 2007
  4. akineko
    Replies:
    7
    Views:
    1,335
    akineko
    Dec 22, 2008
  5. Jeff.M
    Replies:
    6
    Views:
    178
    Lasse Reichstein Nielsen
    May 4, 2009
Loading...

Share This Page