Automatic, portable optimization of global access

[Raymond Hettinger]

FWIW, I've posted a brief, but powerful recipe for a bytecode
optimization that saves known globals as constants:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

It's not psyco, but it does run everywhere and is easy to use.

A key benefit is being able to localize variable access without adding
clutter to your code like: _random=random; _len=len.

One caveat is to run it in the builtin_only mode whenever some of the
module globals are going to be updated at runtime.


Raymond Hettinger

 

[Paul Rubin]

FWIW, I've posted a brief, but powerful recipe for a bytecode
optimization that saves known globals as constants:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

It's not psyco, but it does run everywhere and is easy to use.

This looks important and will clean up a lot of code, but what happens
if the function calls exec or eval, or assigns something into
globals(), or whatever?

 

[Raymond Hettinger]

[Raymond Hettinger]

FWIW, I've posted a brief, but powerful recipe for a bytecode
optimization that saves known globals as constants

[Paul]
This looks important and will clean up a lot of code, but what happens
if the function calls exec or eval, or assigns something into
globals(), or whatever?

Quoth the recipe:

""" Binding should be applied selectively to those functions where
speed is important and dynamic updates are not desired (i.e. the
globals do not change). In more dynamic environments, a more
conservative approach is to set builtin_only to True so that only the
builtins get optimized (this includes functions like len(), exceptions
like IndexError, and constants like True or False).
"""


Raymond

 

[Paul Rubin]

""" Binding should be applied selectively to those functions where
speed is important and dynamic updates are not desired (i.e. the
globals do not change). In more dynamic environments, a more
conservative approach is to set builtin_only to True so that only the
builtins get optimized (this includes functions like len(), exceptions
like IndexError, and constants like True or False).
"""

Well, it's possible that builtins get changed too. I've used "len" as
a variable a few times without realizing that I was clobbering a builtin.

Maybe the compiler could detect when it's ok to run the optimizer, and
automatically run it when it can.

 

[Raymond Hettinger]

[Paul Rubin]

Well, it's possible that builtins get changed too. I've used "len" as
a variable a few times without realizing that I was clobbering a builtin.

Paul, I don't know how to say this more clearly. Don't use constant
binding in places where the underlying values can change.


Raymond

 

[Paul Rubin]

Paul, I don't know how to say this more clearly. Don't use constant
binding in places where the underlying values can change.

I'm saying your optimization is important enough that it should be run
on everything, except the rare cases where it's invalid to do so, and
running it should not require manual intervention on the user's part.
I think the places where the underlying values can change can be
detected by the compiler enough of the time to usually run the optimizer.

 

[Raymond Hettinger]

[Paul Rubin]

I'm saying your optimization is important enough that it should be run
on everything, except the rare cases where it's invalid to do so ...

Thanks.

You might be interested in the latest refinement. Whole modules can
now be optimized by just adding two lines:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

I tried it out on some standard library modules. It was easy to add
and the testsuites ran flawlessly: asynchat, asyncore, calendar, csv,
difflib, dis, optparse, pickle, Queue, random, sets, sre, sre_compile,
sre_parse, textwrap, unittest, urllib2, urlparse, and warnings.


Raymond Hettinger

 

[Michael Hudson]

FWIW, I've posted a brief, but powerful recipe for a bytecode
optimization that saves known globals as constants:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

You seem to have reimplemented:

http://bytecodehacks.sourceforge.net/bch-docs/bch/module-bytecodehacks.closure.html



Indeed the first "bytecodehack" was very similar to this.

Cheers,
mwh