Performance impact of using decorators

[vinjvinj]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code


I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

 

[Alex Martelli]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code

I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

At def-execution time, presumably 6 (the two decorators w/o args, plus 2
each for those w/args); at call time, it depends what the decorators are
doing (if each adds exactly one wrapping closure, for example, there
will indeed be 5 nested calls). Unfortunately I do not know much of
today's cherrypy internals, so I don't know what each decorator is doing
internally.


Alex

 

[Steve Holden]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code


I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

Without reading your code I can't be sure, but this is sure looking like
the classic "to a man with only a hammer all problems look like a nail"
solution. I'm not going to tell you that decorators aren't the answer to
all programming problems, because you already know that in your heart

regards
Steve

 

[vinjvinj]

solution. I'm not going to tell you that decorators aren't the answer to
I was fearing that. The expose decorator is the only one that comes
with cherrypy. The other ones are mine and are of the format:

def decorator(func):
def wrapper(self, *args, **kwargs)
some code
return wrapper

I'll stick with what I'm doing currently and eventually (if the need
arises from a performance perspective) merge the three into one
decorator. I'll also need to eventually add a caching docrator.Do other
people have this problem, especially for developing web applications.
How many decorators, if any, do you use?

@expose -> cherrypy decorator
@startTransactrionAndBuildPage -> starts a db transaction, populates
the user in the session. Does some error handling. Adds header, footer
and error messages to the page.
@partOfTabUi -> besides the top level navigation, I have tab level
(with actions) for navigation on individual pages
@convert -> this converts boolean like 'True' or '0' to python True,
'231' -> int
@cache -> (not implemented, but will ad it).

I would love to hear other people's experience with using decorators
for web application building.

 

[Fredrik Lundh]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code

I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

the decorators themselves are only called when the function is defined.

what happens at runtime depends on what the decorators do (in pretty
much the same way as the output and execution time for this script

x = lambda: return "hello"
x = foo(x)
x = fie(x)
x = fum(x)
print x()

depends on what the foo, fie, and fum functions do...)

</F>

 

[Diez B. Roggisch]

@expose -> cherrypy decorator

@startTransactrionAndBuildPage -> starts a db transaction, populates
the user in the session.

I guess that is ok - transaction handling is a "classic" for decorator-like
concepts. After all, you don't want

begin()
try:
pass
commit()
finally:
if not comitted():
rollback()

all over the place.

Does some error handling. Adds header, footer
and error messages to the page.

That sounds like something for the templating engine, and _certainly_ not
for a decorator that otherwise deals with transactions.

@partOfTabUi -> besides the top level navigation, I have tab level
(with actions) for navigation on individual pages

Template I guess.

@convert -> this converts boolean like 'True' or '0' to python True,
'231' -> int

Looks ok to me.

@cache -> (not implemented, but will ad it).

I would love to hear other people's experience with using decorators
for web application building.

I used them in turbogears (which builds on cherrypy) - and I found them
useful for some aspects.

Diez

 

[Fredrik Lundh]

I was fearing that. The expose decorator is the only one that comes
with cherrypy. The other ones are mine and are of the format:

def decorator(func):
def wrapper(self, *args, **kwargs)
some code
return wrapper

what exactly made you think that Python would be able to run your
code *without* calling your function ?

</F>

 

[vinjvinj]

That sounds like something for the templating engine, and _certainly_ not
The actual code for the page layout is in a preppy template. But the
calls to the template engine are made in the
startTransactrionAndBuildPage decorator

Seemed like an overkill for the template engine.

 

[Terry Reedy]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code


I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

As Alex said, perhaps. A decorator that would not result in a runtime call
would be one that, for example, registers the function somewhere and
returns it unchanged and unwrapped. @expose and @partOfTabUI both seem
like they might do something like that.

IF the overhead becomes a problem, and if you use the same stack (or parts
of a stack) of decorators for multiple functions, then you could write a
few decorators that combine multiple actions with one call.

Terry Jan Reedy

 

[vinjvinj]

what exactly made you think that Python would be able to run your
I was hoping that when the compiler finds decorators with wrapers that
have the same signature it can some how "magically" combine them into
one function (which gets called at run time) and not have 5 nested
function calls. That is similar to what I will have to do eventually.

Given python's dynamic nature, I'm sure there are reasons why this is
not done.

 

[Diez B. Roggisch]

The actual code for the page layout is in a preppy template. But the
calls to the template engine are made in the
startTransactrionAndBuildPage decorator

Well, even if it would fit in a decorator - it certainly belongs to its
_own_ decorator.


Diez

 

[Alex Martelli]

begin()
try:
pass
commit()
finally:
if not comitted():
rollback()

Feels like a natural for 2.5's 'with' statement -- as has been the case
for 2.3 and 2.4, 2.5 won't have many language-level changes, but what
little there IS, is... wonderful!

with transaction():
...your code goes here...

is SO much better than the try/finally routine...!


Alex

 

[Alex Martelli]

I was hoping that when the compiler finds decorators with wrapers that
have the same signature it can some how "magically" combine them into
one function (which gets called at run time) and not have 5 nested
function calls. That is similar to what I will have to do eventually.

Given python's dynamic nature, I'm sure there are reasons why this is
not done.

Yep, you'll have to build new functions yourself if you need them:-(


Alex

 

[Peter Otten]

I'm building an application with cherrypy and have started using
decorators quite extensively. A lot of my exposed functions look like:

@expose
@startTransactrionAndBuildPage
@partOfTabUi(tabId)
@convert(arg1=int, arg2=str)
def do_main_page(self, arg1, arg2):
some code


I've become really fond of decorators and use them quite a lot. I've
also ready that function calls are expensive in python. In the above
example, does the interpreter call 5 different functions?

A typical function calls a few other functions already, so three extra
function calls (I suppose expose just sets an attribute) shouldn't matter
much. You shouldn't even start rewriting your code unless you have
identified do_main_page() as a performance bottleneck. In a web app,
candidates would be functions that are called hundred or thousand times per
rendered page rather than once. Does do_main_page() render a complete page?
Forget optimizing three function calls away. You will see no effect.

Peter