How to avoid Out of Memory Errors when dealing with a large XML file?

Discussion in 'Perl Misc' started by Saqib Ali, Jan 10, 2011.

  1. Saqib Ali

    Saqib Ali Guest

    Hi.

    I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

    I keep getting this error:

    "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
    global destruction."

    I'm using Windows XP. So I watched the task-management memory meter
    during the execution of this process. The PERL process chewed up a lot
    of the "available memory". But when the process died, it still showed
    about 216MB available memory.

    Is there anything I can do to work-around this problem? From reading
    responses to other similar questions, the only option may be to use a
    XML stream parser instead of one that builds the entire DOM tree
    internally.
     
    Saqib Ali, Jan 10, 2011
    #1
    1. Advertising

  2. Saqib Ali

    Chris Nehren Guest

    Re: How to avoid Out of Memory Errors when dealing with a large XMLfile?

    On 2011-01-10, Saqib Ali scribbled these curious markings:
    > Hi.
    >
    > I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()
    >
    > I keep getting this error:
    >
    > "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
    > global destruction."
    >
    > I'm using Windows XP. So I watched the task-management memory meter
    > during the execution of this process. The PERL process chewed up a lot
    > of the "available memory". But when the process died, it still showed
    > about 216MB available memory.


    Firstly, as someone or something has inflicted Windows on you, are you
    using Strawberry Perl like every sane Windows user (bit of an oxymoron,
    that)? Second, calling it "PERL" makes you look either ignorant or
    trollish, so don't do that. :)

    > Is there anything I can do to work-around this problem? From reading
    > responses to other similar questions, the only option may be to use a
    > XML stream parser instead of one that builds the entire DOM tree
    > internally.


    Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
    which provides a SAX interface via XML::LibXML::SAX.

    --
    Thanks and best regards,
    Chris Nehren
     
    Chris Nehren, Jan 11, 2011
    #2
    1. Advertising

  3. Saqib Ali

    Saqib Ali Guest

    FYI, someone posted a workaround here: http://osdir.com/ml/perl-beginners/2011-01/msg00182.html
    It worked for me.


    I'm using Perl provided by ActiveState.




    On Jan 11, 3:36 am, Chris Nehren <>
    wrote:
    > On 2011-01-10, Saqib Ali scribbled these curious markings:
    >
    > > Hi.

    >
    > > I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

    >
    > > I keep getting this error:

    >
    > > "Callback called exit at XML/XPath/Node/Element.pm at line 144 during
    > > global destruction."

    >
    > > I'm using Windows XP. So I watched the task-management memory meter
    > > during the execution of this process. The PERL process chewed up a lot
    > > of the "available memory". But when the process died, it still showed
    > > about 216MB available memory.

    >
    > Firstly, as someone or something has inflicted Windows on you, are you
    > using Strawberry Perl like every sane Windows user (bit of an oxymoron,
    > that)? Second, calling it "PERL" makes you look either ignorant or
    > trollish, so don't do that. :)
    >
    > > Is there anything I can do to work-around this problem? From reading
    > > responses to other similar questions, the only option may be to use a
    > > XML stream parser instead of one that builds the entire DOM tree
    > > internally.

    >
    > Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
    > which provides a SAX interface via XML::LibXML::SAX.
    >
    > --
    > Thanks and best regards,
    > Chris Nehren
     
    Saqib Ali, Jan 14, 2011
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Goldin

    Errors, errors, errors

    Mark Goldin, Jan 17, 2004, in forum: ASP .Net
    Replies:
    2
    Views:
    964
    Mark Goldin
    Jan 17, 2004
  2. George1776

    Out-of-memory errors and caching errors.

    George1776, Aug 28, 2006, in forum: ASP .Net
    Replies:
    2
    Views:
    1,298
    George1776
    Sep 14, 2006
  3. yong

    Dealing with a large integer

    yong, Mar 3, 2006, in forum: C Programming
    Replies:
    42
    Views:
    1,160
  4. brzak
    Replies:
    5
    Views:
    1,535
    Boogie
    Dec 1, 2008
  5. Replies:
    1
    Views:
    341
    Martin v. Löwis
    Jan 22, 2009
Loading...

Share This Page