Transformation with ``parser`` ast

[Malthe Borch]

(Note: repost from python-dev)

The ``compiler.ast`` module makes parsing Python source-code and AST
manipulation relatively painless and it's straight-forward to implement
a transformer class.

However, I find that the ``compiler.pycodegen`` module imposes a hard
limit on the length of functions since it calculates jump points in a
recursion.

I'm using this module to compile an XML dynamic template into Python
code (akin to "Mako") and functions may grow to a rather large size.

Now it seems that the ``parser`` module, which parses source code into
``parser.st`` trees does not have the same limitations, however, I could
not find a transformer class compatible with its tree structure.

What's the recommended way of working with the AST tree from the
``parser`` module?

\malthe

 

[Benjamin]

(Note: repost from python-dev)

The ``compiler.ast`` module makes parsing Python source-code and AST
manipulation relatively painless and it's straight-forward to implement
a transformer class.

However, I find that the ``compiler.pycodegen`` module imposes a hard
limit on the length of functions since it calculates jump points in a
recursion.

I'm using this module to compile an XML dynamic template into Python
code (akin to "Mako") and functions may grow to a rather large size.

Now it seems that the ``parser`` module, which parses source code into
``parser.st`` trees does not have the same limitations, however, I could
not find a transformer class compatible with its tree structure.

What's the recommended way of working with the AST tree from the
``parser`` module?

That's old. Use the shiny new ast module that has a higher level view
of the syntax. http://doc.python.org/library/ast.html should start you
off.