python bytecode grammar

[M1st0]

where I can find the grammar of python bytecode ? ( better if is in BCF
).

 

[Diez B. Roggisch]

where I can find the grammar of python bytecode ? ( better if is in BCF
).

There is no grammar for bytecodes - the are like assembly instructions.
And what's BCF supposed to mean - BNF is a form for grammars, BCF I
never heard of.

And besides that: tell us what you're after, we might help and suggest
other ways than creating/manipulating bytecode, as this is considered a
implementation detail that is not supposed to work across
implementations (like jython) and is subject to unannounced changes
between major revisions of python.

Diez

 

[M1st0]

Ops yes is BNF Bacus Normal Form if I am not wrong...

However......

I'am tryng to recognizing patterns in a bytecoded file in orderd to
optimize...

But I would like to "parse" i.e reconstruct it in something like a
tree..
in order to apply rules on a tree recursively.

I have seen compile.c in the Python dist...

 

[Diez B. Roggisch]

Ops yes is BNF Bacus Normal Form if I am not wrong...

However......

I'am tryng to recognizing patterns in a bytecoded file in orderd to
optimize...

But I would like to "parse" i.e reconstruct it in something like a
tree..
in order to apply rules on a tree recursively.

But bytecode is like assembly - there is no tree-structure. A opcode is
followed by a number of arguments, and oopcodes are in a seqence.

However the bytecode is _generated_ from the AST somehow. Maybe you can
work on that. Also take a look at psyco, it already does optimizations
for numeric calculations.

Besides: I serously doubt you can do much optimization on the
bytecodelevel itself, as it is very highlevel. The optimization efforts
like in psyco don't alter bytecode - they replace it....

Diez

 

[M1st0]

Ok I tried .

 

[Harry George]

Ops yes is BNF Bacus Normal Form if I am not wrong...

However......

I'am tryng to recognizing patterns in a bytecoded file in orderd to
optimize...

But I would like to "parse" i.e reconstruct it in something like a
tree..
in order to apply rules on a tree recursively.

I have seen compile.c in the Python dist...

I've never looked, but I'm assuming it is defined in the compiler and
in the interpreter, and hopefully documented elsehwere, like:
http://python.fyxm.net/peps/pep-0330.html

Related:
http://mail.python.org/pipermail/python-dev/2004-December/050542.html
http://search.cpan.org/search?query=python::bytecode&mode=all

 

[Terry Reedy]

where I can find the grammar of python bytecode ? ( better if is in BCF

I believe the top-level production is something like
BYTECODE := (OPCODE ARGS)*
See the documentation for the dis module for symbolic opcodes and
corresponding args.

Looking for patterns in bytecodes is one way CPython has been sped up.
Previous results have occasionally been reported on the Python development
list, archived somewhere at python.org. Examples I remember are the
frequencies of both individual bytecodes and of pair of codes.

For parse trees, you should look at the parser and compiler modules.

Terry J. Reedy

 

[Peter Dembinski]

I believe the top-level production is something like
BYTECODE := (OPCODE ARGS)*

ROTFL

 

[Terry Reedy]

ROTFL

Glad to make your day ;-)

I am aware that since ARGS depends on OPCODE, the above would lead to
context-dependent productions for ARGS. Upon thoroughly perusing
http://docs.python.org/lib/bytecodes.html
18.10.1 Python Byte Code Instructions

I discovered that there is at most one (non-stack) argument in the byte
stream (contrary to the possibly plural implication of "All of the
following opcodes expect arguments"). So the above could be written
context-freely as
BYTECODE := (NO_ARG_CODE | ARG_CODE TWO_BYTE_ARG)*

where the symbolic expansions of NO_ARG_CODE and ARG_CODE, with semantic
explanations, constitute the contents of the doc above.

Terry J. Reedy

 

[Peter Dembinski]

Glad to make your day ;-)

You are welcome.

"Asking stupid questions is the best way to get quick answers" :>

 

[Magnus Lycka]

Ops yes is BNF Bacus Normal Form if I am not wrong...

Backus Naur Form.

John Backus and Peter Naur first used it to describe ALGOL around 1960.
See e.g.
http://cui.unige.ch/db-research/Enseignement/analyseinfo/AboutBNF.html

 

[Bengt Richter]

But bytecode is like assembly - there is no tree-structure. A opcode is
followed by a number of arguments, and oopcodes are in a seqence.

However the bytecode is _generated_ from the AST somehow. Maybe you can
work on that. Also take a look at psyco, it already does optimizations
for numeric calculations.

Besides: I serously doubt you can do much optimization on the
bytecodelevel itself, as it is very highlevel. The optimization efforts
like in psyco don't alter bytecode - they replace it....

Well, Raymond Hettinger wrote a decorator to optimize functions in various ways,
rewriting the byte code, and I wrote another decorator inspired by that.
His is in the recipes at
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940
and mine is posted at
http://groups-beta.google.com/group/comp.lang.python/msg/7d27a53385e89924?hl=en
(I think that's the last version ;-) Mine was to preset local variables without
using the default-argument hack. This led to a currying version of that, which
lets you modify the signature of an existing function by setting a local variable
to a constant and eliminating that name from the signature, so there is one level
of call as before, not the double call level from wrapping that you will get from some
other currying recipes.

I think you can see how byte code mods work from the code, and where the byte code definitions
come from etc. If you want to generate optimized code by replacing current code generation,
you will have to look into AST stuff as Diez mentions above. The sources are all there,
and there is pure python equivalents for the python run-time compiler (which is (all?) in C I think).

Import compiler and parser etc. and poke around. You'll find interesting things ;-)

Regards,
Bengt Richter