Pyparsing performance boost using Python 2.6b1

[Paul McGuire]

I just ran my pyparsing unit tests with the latest Python 2.6b1
(labeled internally as Python 2.6a3 - ???), and the current 1.5.0
version of pyparsing runs with no warnings or regressions.

I was pleasantly surprised by the improved performance. The most
complex parser I have is the Verilog parser, and I have just under 300
sample input files, so the test gets a chance to run over a range of
code and source inputs. Here are the lines/second parsing for the
Verilog data (higher numbers are better):

Python V2.5.1 Python V2.6b1
base
209.2 307.0

with packrat optimization enabled
349.8 408.0

This is a huge percentage improvement, anywhere from 15-50%! I do not
know what it is about 2.6 that runs so much faster, but given that
packratting achieves somewhat less improvement, I would guess that the
2.6 performance comes from some optimization in making function calls,
or in GC of local variables when functions are completed (since
packratting is a form of internal memoization of parse expressions,
which thereby avoids duplicate calls to parsing functions).

Using psyco typically gives another 30-50% performance improvement,
but there is no psyco available for 2.6 yet, so I skipped those tests
for now.

-- Paul

 

[dwahli]

I just ran my pyparsing unit tests with the latest Python 2.6b1
(labeled internally as Python 2.6a3 - ???), and the current 1.5.0
version of pyparsing runs with no warnings or regressions.

I was pleasantly surprised by the improved performance.  The most
complex parser I have is the Verilog parser, and I have just under 300
sample input files, so the test gets a chance to run over a range of
code and source inputs.  Here are the lines/second parsing for the
Verilog data (higher numbers are better):

 Python V2.5.1   Python V2.6b1
base
 209.2                307.0

with packrat optimization enabled
 349.8                408.0

This is a huge percentage improvement, anywhere from 15-50%!  I do not
know what it is about 2.6 that runs so much faster, but given that
packratting achieves somewhat less improvement, I would guess that the
2.6 performance comes from some optimization in making function calls,
or in GC of local variables when functions are completed (since
packratting is a form of internal memoization of parse expressions,
which thereby avoids duplicate calls to parsing functions).

Using psyco typically gives another 30-50% performance improvement,
but there is no psyco available for 2.6 yet, so I skipped those tests
for now.

-- Paul

Tests was made on Windows ?
Just try pybench on 2.6 and 2.5, and 2.6 is 17% better than 2.5 !
Very nice !

But I suspect the new compiler with Profile Guided Optimization (PGO)
to be the main cause of this nice boost...

 

[John Machin]

I just ran my pyparsing unit tests with the latest Python 2.6b1
(labeled internally as Python 2.6a3 - ???),

Hi, Paul. If it says 2.6a3, that's what it is. Look at the thread of
replies to Barry Warsaw's announcement of 2.6b1 ... [from memory]
there was a delay expected before MvL would make a Windows msi
available, and in the meantime the download page would point to "the
alpha version".

Cheers,
John