Limited XML tidy

Discussion in 'Python' started by Toby White, Aug 23, 2005.

  1. Toby White

    Toby White Guest

    I have a program which produces well-formed XML documents,
    but takes several hours if not days to do so. It would
    be useful to be able to take the incomplete output and
    manipulate it as XML.

    Clearly, however, the incomplete output will not be well-
    formed, so before being able to manipulate it I need to
    make it wellformed.

    This is essentially fairly simple. I know it's well-formed
    up to now - all I need to do is close unclosed tags. So,
    I 've made a short function that will do this:

    #!/usr/bin/python

    import sys
    import xml.sax

    tagStack = []

    closingTags = ""

    class DodgyHandler(xml.sax.ContentHandler):
    def startElement(self, tag, attributes):
    tagStack.append(tag)

    def endElement(self, tag):
    tagStack.pop()


    class DodgyErrorHandler(xml.sax.ErrorHandler):
    def fatalError(self,exception):
    global closingTags
    tagStack.reverse()
    for tag in tagStack:
    closingTags += "</%s>" % tag
    return closingTags


    def finishXML(text):
    p = xml.sax.make_parser()
    p.setContentHandler(DodgyHandler())
    p.setErrorHandler(DodgyErrorHandler())

    for line in text:
    p.feed(line)
    p.close()

    text.append(closingTags)


    However - while this works for 90% of the cases I need,
    it fails in the case where my incomplete output stops in the
    middle of a tag (not to mention some other more arcane
    places I don't really care about).

    The problem is that when the sax handler raises an exception,
    I can't see how to find out why. What I want to do is for
    DodgyErrorHandler to do something different depending on
    where we are in the course of parsing. Is there anyway
    to get that information back from xml.sax (or indeed from
    any other sax handler?)

    Toby

    --
    Dr. Toby White
    Dept. of Earth Sciences, Downing Street, Cambridge CB2 3EQ. UK
    Email: <>
    Toby White, Aug 23, 2005
    #1
    1. Advertising

  2. In article <>, Toby White wrote:
    >

    [snip]

    I do similar stuff in the new (upcoming) version of Atox
    (atox.sf.net), which works with potentially ill-formed, partial XML
    (in the form of PYX events) internally, and can take partial,
    ill-formed XML as input.

    >The problem is that when the sax handler raises an exception,
    >I can't see how to find out why. What I want to do is for
    >DodgyErrorHandler to do something different depending on
    >where we are in the course of parsing. Is there anyway
    >to get that information back from xml.sax (or indeed from
    >any other sax handler?)


    What I ended up doing was using an SGML parser (sgmlop) instead. It's
    highly forgiving (even of illegal entities and the like) but gives me
    the information I need. Might be worth a look in your app too?

    >Toby


    --
    Magnus Lie Hetland
    http://hetland.org
    Magnus Lie Hetland, Aug 25, 2005
    #2
    1. Advertising

  3. Toby White

    Guest

    > The problem is that when the sax handler raises an exception,
    I can't see how to find out why. What I want to do is for
    DodgyErrorHandler to do something different depending on
    where we are in the course of parsing. Is there anyway
    to get that information back from xml.sax (or indeed from
    any other sax handler?)

    You can get raw location information, yes. See:

    http://www.xml.com/pub/a/2004/11/24/py-xml.html

    But I don't think this is enough for you. You also need recovery,
    which you're implementing in crude form.

    I tend to agree with Magnus that using an SGML parser might be your
    best bet. You might even be able to turn that SGML into XML using a
    tool such as James Clark's SX:

    http://www.jclark.com/sp/sx.htm

    --
    Uche
    http://copia.ogbuji.net
    , Aug 25, 2005
    #3
  4. Toby White

    Toby White Guest

    writes:

    >> The problem is that when the sax handler raises an exception,

    > I can't see how to find out why. What I want to do is for
    > DodgyErrorHandler to do something different depending on
    > where we are in the course of parsing. Is there anyway
    > to get that information back from xml.sax (or indeed from
    > any other sax handler?)
    >
    > You can get raw location information, yes. See:
    >
    > http://www.xml.com/pub/a/2004/11/24/py-xml.html
    >
    > But I don't think this is enough for you. You also need recovery,
    > which you're implementing in crude form.


    (If you're referring to the Locator objects), yes I'm aware
    that's possible. But what I want is not my location in the
    document, but for the parser to say "this is an error because
    I am in the middle of a tag & the document ended", or "I
    was in the middle of a text section and the document ended", or
    "I was in the middle of an attribute value and the document
    ended", etc, so that I can then construct a simple end to the
    document, inserting quote marks, finishing the tag, and closing
    all unclosed tags as appropriate.

    I have just realised that I might be able to grab the message
    that the exception gives me, look at the expat source code
    and work out what parsing events cause which error messages.
    Which is a bit round the houses, but I think ought to work.


    > I tend to agree with Magnus that using an SGML parser might be your
    > best bet. You might even be able to turn that SGML into XML using a
    > tool such as James Clark's SX:
    >
    > http://www.jclark.com/sp/sx.htm


    If I can't get my scheme above to work, I'll have a go. But I was
    hoping to do this without requiring additional packages. And in
    any case, it doesn't need to be perfectly robust. As long as it
    handles 99% of cases, I'll be happy.

    --
    Dr. Toby White
    Dept. of Earth Sciences, Downing Street, Cambridge CB2 3EQ. UK
    Email: <>
    Toby White, Aug 26, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric
    Replies:
    0
    Views:
    527
  2. Toby White

    Limited XML tidy

    Toby White, Aug 23, 2005, in forum: Python
    Replies:
    0
    Views:
    295
    Toby White
    Aug 23, 2005
  3. Replies:
    6
    Views:
    375
    Magnus Lycka
    Feb 27, 2006
  4. Ragnar
    Replies:
    9
    Views:
    1,502
    Andy Dingley
    Oct 27, 2006
  5. Ragnar
    Replies:
    13
    Views:
    593
    Joseph Kesselman
    Nov 8, 2006
Loading...

Share This Page