Pyparsing performance boost using Python 2.6b1

Discussion in 'Python' started by Paul McGuire, Jun 19, 2008.

  1. Paul McGuire

    Paul McGuire Guest

    I just ran my pyparsing unit tests with the latest Python 2.6b1
    (labeled internally as Python 2.6a3 - ???), and the current 1.5.0
    version of pyparsing runs with no warnings or regressions.

    I was pleasantly surprised by the improved performance. The most
    complex parser I have is the Verilog parser, and I have just under 300
    sample input files, so the test gets a chance to run over a range of
    code and source inputs. Here are the lines/second parsing for the
    Verilog data (higher numbers are better):

    Python V2.5.1 Python V2.6b1
    base
    209.2 307.0

    with packrat optimization enabled
    349.8 408.0

    This is a huge percentage improvement, anywhere from 15-50%! I do not
    know what it is about 2.6 that runs so much faster, but given that
    packratting achieves somewhat less improvement, I would guess that the
    2.6 performance comes from some optimization in making function calls,
    or in GC of local variables when functions are completed (since
    packratting is a form of internal memoization of parse expressions,
    which thereby avoids duplicate calls to parsing functions).

    Using psyco typically gives another 30-50% performance improvement,
    but there is no psyco available for 2.6 yet, so I skipped those tests
    for now.

    -- Paul
     
    Paul McGuire, Jun 19, 2008
    #1
    1. Advertising

  2. Paul McGuire

    Guest

    On Jun 19, 12:40 pm, Paul McGuire <> wrote:
    > I just ran my pyparsing unit tests with the latest Python 2.6b1
    > (labeled internally as Python 2.6a3 - ???), and the current 1.5.0
    > version of pyparsing runs with no warnings or regressions.
    >
    > I was pleasantly surprised by the improved performance.  The most
    > complex parser I have is the Verilog parser, and I have just under 300
    > sample input files, so the test gets a chance to run over a range of
    > code and source inputs.  Here are the lines/second parsing for the
    > Verilog data (higher numbers are better):
    >
    >  Python V2.5.1   Python V2.6b1
    > base
    >  209.2                307.0
    >
    > with packrat optimization enabled
    >  349.8                408.0
    >
    > This is a huge percentage improvement, anywhere from 15-50%!  I do not
    > know what it is about 2.6 that runs so much faster, but given that
    > packratting achieves somewhat less improvement, I would guess that the
    > 2.6 performance comes from some optimization in making function calls,
    > or in GC of local variables when functions are completed (since
    > packratting is a form of internal memoization of parse expressions,
    > which thereby avoids duplicate calls to parsing functions).
    >
    > Using psyco typically gives another 30-50% performance improvement,
    > but there is no psyco available for 2.6 yet, so I skipped those tests
    > for now.
    >
    > -- Paul


    Tests was made on Windows ?
    Just try pybench on 2.6 and 2.5, and 2.6 is 17% better than 2.5 !
    Very nice !

    But I suspect the new compiler with Profile Guided Optimization (PGO)
    to be the main cause of this nice boost...
     
    , Jun 20, 2008
    #2
    1. Advertising

  3. Paul McGuire

    John Machin Guest

    On Jun 19, 8:40 pm, Paul McGuire <> wrote:
    > I just ran my pyparsing unit tests with the latest Python 2.6b1
    > (labeled internally as Python 2.6a3 - ???),


    Hi, Paul. If it says 2.6a3, that's what it is. Look at the thread of
    replies to Barry Warsaw's announcement of 2.6b1 ... [from memory]
    there was a delay expected before MvL would make a Windows msi
    available, and in the meantime the download page would point to "the
    alpha version".

    Cheers,
    John
     
    John Machin, Jun 20, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Richard Latter
    Replies:
    2
    Views:
    2,918
    Julie
    May 17, 2004
  2. Steve Knight
    Replies:
    2
    Views:
    780
    Steve Knight
    Oct 10, 2003
  3. Toby Bradshaw
    Replies:
    6
    Views:
    1,782
    Kai-Uwe Bux
    Jun 2, 2006
  4. Barry Warsaw

    RELEASED Python 2.6b1 and 3.0b1

    Barry Warsaw, Jun 19, 2008, in forum: Python
    Replies:
    0
    Views:
    278
    Barry Warsaw
    Jun 19, 2008
  5. Brian
    Replies:
    3
    Views:
    1,271
Loading...

Share This Page