decorators and closures

A

Andrea Crotti

With one colleague I discovered that the decorator code is always
executed, every time I call
a nested function:

def dec(fn):
print("In decorator")
def _dec():
fn()

return _dec

def nested():
@dec
def fun():
print("here")

nested()
nested()

Will give:
In decorator
In decorator

So we were wondering, would the interpreter be able to optimize this
somehow?
I was betting it's not possible, but I'm I would like to be wrong :)
 
S

Steven D'Aprano

With one colleague I discovered that the decorator code is always
executed, every time I call a nested function:

"def" is a statement which is executed at runtime. Often people will talk
about "definition time" instead of "compile time".

Python compiles your source into byte code (compile time), then executes
the byte code. The function doesn't actually get created until the byte
code is executed, i.e. at run time. This is not as slow as it sounds,
because the function is created from pre-compiled parts.

In effect, if you have a module:

x = 23
def spam(a):
print x
print x+1
return x**3


then the body of the function is compiled into a "code object" at compile
time, and at runtime the function object itself is assembled from the
code object, name, and whatever other bits and pieces are needed. In
pseudo-code, the byte code looks like this:

bind name "x" to object 23
build a function object "spam" from code object
bind name "spam" to function object

The hard part is creating the code object, as that requires parsing the
source code of the body and generating byte code. That's done once, ahead
of time, so the actual "build a function" part is fast.

Now, if you have an ordinary nested function:

def spam(a):
def ham(b):
return a+b
return ham(a+42) # returns a numeric value

or a closure:

def spam(a):
def ham(b):
return a+b
return ham # returns a closure (function object)

the process is no different: the inner function doesn't get created until
runtime, that is, when spam gets *called*. But it gets created from parts
that were prepared earlier at compile time, and so is fast.

Add a decorator, and the basic process remains. Remember that decorator
syntax is just syntactic sugar. This:

@decorator
def spam():
pass

is exactly the same as this:

def spam():
pass

spam = decorator(spam)

which clearly has to be done at runtime, not compile time. That applies
regardless of whether the function is nested or top-level.
 
8

88888 Dihedral

With one colleague I discovered that the decorator code is always
executed, every time I call
a nested function:

def dec(fn):
print("In decorator")
def _dec():
fn()

return _dec

def nested():
@dec
def fun():
print("here")

nested()
nested()

Will give:
In decorator
In decorator

So we were wondering, would the interpreter be able to optimize this
somehow?
I was betting it's not possible, but I'm I would like to be wrong :)

I love to use decorators. I did the same things to functions and structures in c long time ago. I think that might be the dark night era in programming in the early 90's long long ago.

Cheers.
 
8

88888 Dihedral

With one colleague I discovered that the decorator code is always
executed, every time I call
a nested function:

def dec(fn):
print("In decorator")
def _dec():
fn()

return _dec

def nested():
@dec
def fun():
print("here")

nested()
nested()

Will give:
In decorator
In decorator

So we were wondering, would the interpreter be able to optimize this
somehow?
I was betting it's not possible, but I'm I would like to be wrong :)

I love to use decorators. I did the same things to functions and structures in c long time ago. I think that might be the dark night era in programming in the early 90's long long ago.

Cheers.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,774
Messages
2,569,596
Members
45,139
Latest member
JamaalCald
Top