elementtree w/utf8

Discussion in 'Python' started by Tim Arnold, Oct 25, 2007.

  1. Tim Arnold

    Tim Arnold Guest

    Hi, I'm getting the by-now-familiar error:
    return codecs.charmap_decode(input,errors,decoding_map)
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position
    4615: ordinal not in range(128)

    the html file I'm working with is in utf-8, I open it with codecs, try to
    feed it to TidyHTMLTreeBuilder, but no luck. Here's my code:
    from elementtree import ElementTree as ET
    from elementtidy import TidyHTMLTreeBuilder

    fd = codecs.open(htmfile,encoding='utf-8')
    tidyTree =
    TidyHTMLTreeBuilder.TidyHTMLTreeBuilder(encoding='utf-8')
    tidyTree.feed(fd.read())
    self.tree = tidyTree.close()
    fd.close()

    what am I doing wrong? Thanks in advance.

    On a related note, I have another question--where/how can I get the
    cElementTree.py module? Sorry for something so basic, but I tried installing
    cElementTree, but while I could compile with setup.py build, I didn't end up
    with a cElementTree.py file anywhere. The directory structure on my system
    (HPux, but no root access) doesn't work well with setup.py install.

    thanks,
    --Tim Arnold
    Tim Arnold, Oct 25, 2007
    #1
    1. Advertising

  2. On Thu, 25 Oct 2007 17:15:36 -0400, Tim Arnold wrote:

    > Hi, I'm getting the by-now-familiar error:
    > return codecs.charmap_decode(input,errors,decoding_map)
    > UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position
    > 4615: ordinal not in range(128)
    >
    > the html file I'm working with is in utf-8, I open it with codecs, try to
    > feed it to TidyHTMLTreeBuilder, but no luck. Here's my code:
    > from elementtree import ElementTree as ET
    > from elementtidy import TidyHTMLTreeBuilder
    >
    > fd = codecs.open(htmfile,encoding='utf-8')
    > tidyTree =
    > TidyHTMLTreeBuilder.TidyHTMLTreeBuilder(encoding='utf-8')
    > tidyTree.feed(fd.read())
    > self.tree = tidyTree.close()
    > fd.close()
    >
    > what am I doing wrong? Thanks in advance.


    You feed decoded data to `TidyHTMLTreeBuilder`. As the `encoding`
    argument suggests this class wants bytes not unicode. Decoding twice
    doesn't work.

    Ciao,
    Marc 'BlackJack' Rintsch
    Marc 'BlackJack' Rintsch, Oct 25, 2007
    #2
    1. Advertising

  3. Tim Arnold schrieb:
    > Hi, I'm getting the by-now-familiar error:
    > return codecs.charmap_decode(input,errors,decoding_map)
    > UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in position
    > 4615: ordinal not in range(128)
    >
    > the html file I'm working with is in utf-8, I open it with codecs, try to
    > feed it to TidyHTMLTreeBuilder, but no luck. Here's my code:
    > from elementtree import ElementTree as ET
    > from elementtidy import TidyHTMLTreeBuilder
    >
    > fd = codecs.open(htmfile,encoding='utf-8')
    > tidyTree =
    > TidyHTMLTreeBuilder.TidyHTMLTreeBuilder(encoding='utf-8')
    > tidyTree.feed(fd.read())
    > self.tree = tidyTree.close()
    > fd.close()
    >
    > what am I doing wrong? Thanks in advance.


    Being to clever for your own good.. sorry to say so. But
    TidyHTMLTreeBuilder takes the encoding for a reason: it expects a
    byte-string that it will decode itself.

    But you decode first, creating a unicode-object. When feeding that to
    the string-expecting feed-method, python attempts a conversion to a
    byte-string using the default-encoding.

    Not using codecs but a file instead should do the trick.

    diez
    Diez B. Roggisch, Oct 25, 2007
    #3
  4. Tim Arnold

    Tim Arnold Guest

    "Marc 'BlackJack' Rintsch" <> wrote in message
    news:-berlin.de...
    > On Thu, 25 Oct 2007 17:15:36 -0400, Tim Arnold wrote:
    >
    >> Hi, I'm getting the by-now-familiar error:
    >> return codecs.charmap_decode(input,errors,decoding_map)
    >> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa9' in
    >> position
    >> 4615: ordinal not in range(128)
    >>
    >> the html file I'm working with is in utf-8, I open it with codecs, try to
    >> feed it to TidyHTMLTreeBuilder, but no luck. Here's my code:
    >> from elementtree import ElementTree as ET
    >> from elementtidy import TidyHTMLTreeBuilder
    >>
    >> fd = codecs.open(htmfile,encoding='utf-8')
    >> tidyTree =
    >> TidyHTMLTreeBuilder.TidyHTMLTreeBuilder(encoding='utf-8')
    >> tidyTree.feed(fd.read())
    >> self.tree = tidyTree.close()
    >> fd.close()
    >>
    >> what am I doing wrong? Thanks in advance.

    >
    > You feed decoded data to `TidyHTMLTreeBuilder`. As the `encoding`
    > argument suggests this class wants bytes not unicode. Decoding twice
    > doesn't work.
    >
    > Ciao,
    > Marc 'BlackJack' Rintsch


    well now that you say it, it seems so obvious...
    some day I will get the hang of this encode/decode stuff. When I read about
    it, I'm fine, it makes sense, etc. maybe even a little boring. And then I
    write stuff like the above!

    Thanks to you and Diez for straightening me out.
    --Tim
    Tim Arnold, Oct 26, 2007
    #4
  5. Tim Arnold

    rzzzwilson Guest

    Tim Arnold wrote:
    <snip>
    > On a related note, I have another question--where/how can I get the
    > cElementTree.py module? Sorry for something so basic, but I tried installing
    > cElementTree, but while I could compile with setup.py build, I didn't end up
    > with a cElementTree.py file anywhere. The directory structure on my system
    > (HPux, but no root access) doesn't work well with setup.py install.
    >
    > thanks,
    > --Tim Arnold


    I had the same question a while ago .... and the answer is ElementTree
    is now
    part of the standard library.

    http://docs.python.org/lib/module-xml.etree.ElementTree.html

    Ross
    rzzzwilson, Oct 28, 2007
    #5
  6. Tim Arnold wrote:
    > On a related note, I have another question--where/how can I get the
    > cElementTree.py module? Sorry for something so basic, but I tried installing
    > cElementTree, but while I could compile with setup.py build, I didn't end up
    > with a cElementTree.py file anywhere.


    That's because it compiles into a binary extension module, not a plain Python
    module (mind the 'c' in its name, which stands for the C language here).

    I don't know what the standard library extension is under HP-UX, but look a
    little closer at the files that weren't there before, you'll find it.
    Depending on what you did to build it, it might also end up in the "build"
    directory or as an installable package in the "dist" directory.


    > The directory structure on my system
    > (HPux, but no root access) doesn't work well with setup.py install.


    That shouldn't be a problem as long as you keep the binary in your PYTHONPATH.

    As suggested before, if you have Python 2.5, you don't even need to install it
    yourself.

    Stefan
    Stefan Behnel, Oct 28, 2007
    #6
  7. Tim Arnold

    Tim Arnold Guest

    "Stefan Behnel" <> wrote in message
    news:...
    > Tim Arnold wrote:
    >> On a related note, I have another question--where/how can I get the
    >> cElementTree.py module? Sorry for something so basic, but I tried
    >> installing
    >> cElementTree, but while I could compile with setup.py build, I didn't end
    >> up
    >> with a cElementTree.py file anywhere.

    >
    > That's because it compiles into a binary extension module, not a plain
    > Python
    > module (mind the 'c' in its name, which stands for the C language here).
    >
    > I don't know what the standard library extension is under HP-UX, but look
    > a
    > little closer at the files that weren't there before, you'll find it.
    > Depending on what you did to build it, it might also end up in the "build"
    > directory or as an installable package in the "dist" directory.
    >
    >
    >> The directory structure on my system
    >> (HPux, but no root access) doesn't work well with setup.py install.

    >
    > That shouldn't be a problem as long as you keep the binary in your
    > PYTHONPATH.
    >
    > As suggested before, if you have Python 2.5, you don't even need to
    > install it
    > yourself.
    >
    > Stefan


    very nice--thanks. I saw the cElementTree.sl file, but didn't realize it
    would work as-is.
    thanks,
    --Tim
    Tim Arnold, Oct 29, 2007
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tim Arnold

    elementtree behavior

    Tim Arnold, Sep 27, 2004, in forum: Python
    Replies:
    2
    Views:
    252
    Fredrik Lundh
    Sep 27, 2004
  2. Jesper Olsen

    elementtree -

    Jesper Olsen, Oct 3, 2004, in forum: Python
    Replies:
    2
    Views:
    533
    Jesper Olsen
    Oct 4, 2004
  3. Stewart Midwinter

    xml file structure for use with ElementTree?

    Stewart Midwinter, Oct 9, 2004, in forum: Python
    Replies:
    7
    Views:
    363
    Stewart Midwinter
    Oct 11, 2004
  4. Kee Nethery
    Replies:
    12
    Views:
    2,064
    Stefan Behnel
    Jun 27, 2009
  5. gry
    Replies:
    2
    Views:
    712
    Alf P. Steinbach
    Mar 13, 2012
Loading...

Share This Page