B
Brian Sabbey
Here is a first draft of a PEP for thunks. Please let me know what you
think. If there is a positive response, I will create a real PEP.
I made a patch that implements thunks as described here. It is available
at:
http://staff.washington.edu/sabbey/py_do
Good background on thunks can be found in ref. [1].
Simple Thunks
-------------
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
A Set of Examples
=================
Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to
the function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
The return value can be captured
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8
Thunks blend into their environment
def f(thunk):
thunk(6,7)
a = 20
do x,y in f():
a = 54
print a,x,y
==> 54,6,7
Thunks can return values. Since using 'return' would leave it unclear
whether it is the thunk or the surrounding function that is returning, a
different keyword should be used. By analogy with 'for' and 'while' loops,
the 'continue' keyword is used for this purpose:
def f(thunk):
before()
t = thunk()
# t == 11
after()
do f():
continue 11
Exceptions raised in the thunk pass through the thunk's caller's frame
before returning to the frame in which the thunk is defined:
def catch_everything(thunk):
try:
thunk()
except:
pass # SomeException gets caught here
try:
do catch_everything():
raise SomeException
except:
pass # SomeException doesn't get caught here because it was
already caught
Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished:
thunk_saver = None
def f(thunk):
global thunk_saver
thunk_saver = thunk
do f():
pass
thunk_saver() # exception, thunk has expired
'break' and 'return' should probably not be allowed in thunks. One could
use exceptions to simulate these, but it would be surprising to have
exceptions occur in what would otherwise be a non-exceptional situation.
One would have to use try/finally blocks in all code that calls thunks
just to deal with normal situations. For example, using code like
def f(thunk):
thunk()
prevent_core_meltdown()
with code like
do f():
p = 1
return p
would have a different effect than using it with
do f():
return 1
This behavior is potentially a cause of bugs since these two examples
might seem identical at first glance.
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
def f(thunk):
frame = thunk.tk_frame
do f():
pass
Motivation
==========
Thunks can be used to solve most of the problems addressed by PEP 310 [2]
and PEP 288 [3].
PEP 310 deals with the abstraction of acquire/release code. Such code is
needed when one needs to acquire a resource before its use and release it
after. This often requires boilerplate, it is easy to get wrong, and
there is no visual indication that the before and after parts of the code
are related. Thunks solve these problems by allowing the acquire/release
code to be written in a single, re-usable function.
def acquire_release(thunk):
f = acquire()
try:
thunk(f)
finally:
f.release()
do t in acquire_release():
print t
More generally, thunks can be used whenever there is a repeated need for
the same code to appear before and after other code. For example,
do WaitCursor():
compute_for_a_long_time()
is more organized, easier to read and less bug-prone than the code
DoWaitCursor(1)
compute_for_a_long_time()
DoWaitCursor(-1)
PEP 288 tries to overcome some of the limitations of generators. One
limitation is that a 'yield' is not allowed in the 'try' block of a
'try'/'finally' statement.
def get_items():
f = acquire()
try:
for i in f:
yield i # syntax error
finally:
f.release()
for i in get_items():
print i
This code is not allowed because execution might never return after the
'yield' statement and therefore there is no way to ensure that the
'finally' block is executed. A prohibition on such yields lessens the
suitability of generators as a way to produce items from a resource that
needs to be closed. Of course, the generator could be wrapped in a class
that closes the resource, but this is a complication one would like to
avoid, and does not ensure that the resource will be released in a timely
manner. Thunks do not have this limitation because the thunk-accepting
function is in control-- execution cannot break out of the 'do' statement
without first passing through the thunk-accepting function.
def get_items(thunk): # <-- "thunk-accepting function"
f = acquire()
try:
for i in f:
thunk(i) # A-OK
finally:
f.release()
do i in get_items():
print i
Even though thunks can be used in some ways that generators cannot, they
are not nearly a replacement for generators. Importantly, one has no
analogue of the 'next' method of generators when using thunks:
def f():
yield 89
yield 91
g = f()
g.next() # == 89
g.next() # == 91
[1] see the "Extended Function syntax" thread,
http://mail.python.org/pipermail/python-dev/2003-February/
[2] http://www.python.org/peps/pep-0310.html
[3] http://www.python.org/peps/pep-0288.html
think. If there is a positive response, I will create a real PEP.
I made a patch that implements thunks as described here. It is available
at:
http://staff.washington.edu/sabbey/py_do
Good background on thunks can be found in ref. [1].
Simple Thunks
-------------
Thunks are, as far as this PEP is concerned, anonymous functions that
blend into their environment. They can be used in ways similar to code
blocks in Ruby or Smalltalk. One specific use of thunks is as a way to
abstract acquire/release code. Another use is as a complement to
generators.
A Set of Examples
=================
Thunk statements contain a new keyword, 'do', as in the example below. The
body of the thunk is the suite in the 'do' statement; it gets passed to
the function appearing next to 'do'. The thunk gets inserted as the first
argument to the function, reminiscent of the way 'self' is inserted as the
first argument to methods.
def f(thunk):
before()
thunk()
after()
do f():
stuff()
The above code has the same effect as:
before()
stuff()
after()
Other arguments to 'f' get placed after the thunk:
def f(thunk, a, b):
# a == 27, b == 28
before()
thunk()
after()
do f(27, 28):
stuff()
Thunks can also accept arguments:
def f(thunk):
thunk(6,7)
do x,y in f():
# x==6, y==7
stuff(x,y)
The return value can be captured
def f(thunk):
thunk()
return 8
do t=f():
# t not bound yet
stuff()
print t
==> 8
Thunks blend into their environment
def f(thunk):
thunk(6,7)
a = 20
do x,y in f():
a = 54
print a,x,y
==> 54,6,7
Thunks can return values. Since using 'return' would leave it unclear
whether it is the thunk or the surrounding function that is returning, a
different keyword should be used. By analogy with 'for' and 'while' loops,
the 'continue' keyword is used for this purpose:
def f(thunk):
before()
t = thunk()
# t == 11
after()
do f():
continue 11
Exceptions raised in the thunk pass through the thunk's caller's frame
before returning to the frame in which the thunk is defined:
def catch_everything(thunk):
try:
thunk()
except:
pass # SomeException gets caught here
try:
do catch_everything():
raise SomeException
except:
pass # SomeException doesn't get caught here because it was
already caught
Because thunks blend into their environment, a thunk cannot be used after
its surrounding 'do' statement has finished:
thunk_saver = None
def f(thunk):
global thunk_saver
thunk_saver = thunk
do f():
pass
thunk_saver() # exception, thunk has expired
'break' and 'return' should probably not be allowed in thunks. One could
use exceptions to simulate these, but it would be surprising to have
exceptions occur in what would otherwise be a non-exceptional situation.
One would have to use try/finally blocks in all code that calls thunks
just to deal with normal situations. For example, using code like
def f(thunk):
thunk()
prevent_core_meltdown()
with code like
do f():
p = 1
return p
would have a different effect than using it with
do f():
return 1
This behavior is potentially a cause of bugs since these two examples
might seem identical at first glance.
The thunk evaluates in the same frame as the function in which it was
defined. This frame is accessible:
def f(thunk):
frame = thunk.tk_frame
do f():
pass
Motivation
==========
Thunks can be used to solve most of the problems addressed by PEP 310 [2]
and PEP 288 [3].
PEP 310 deals with the abstraction of acquire/release code. Such code is
needed when one needs to acquire a resource before its use and release it
after. This often requires boilerplate, it is easy to get wrong, and
there is no visual indication that the before and after parts of the code
are related. Thunks solve these problems by allowing the acquire/release
code to be written in a single, re-usable function.
def acquire_release(thunk):
f = acquire()
try:
thunk(f)
finally:
f.release()
do t in acquire_release():
print t
More generally, thunks can be used whenever there is a repeated need for
the same code to appear before and after other code. For example,
do WaitCursor():
compute_for_a_long_time()
is more organized, easier to read and less bug-prone than the code
DoWaitCursor(1)
compute_for_a_long_time()
DoWaitCursor(-1)
PEP 288 tries to overcome some of the limitations of generators. One
limitation is that a 'yield' is not allowed in the 'try' block of a
'try'/'finally' statement.
def get_items():
f = acquire()
try:
for i in f:
yield i # syntax error
finally:
f.release()
for i in get_items():
print i
This code is not allowed because execution might never return after the
'yield' statement and therefore there is no way to ensure that the
'finally' block is executed. A prohibition on such yields lessens the
suitability of generators as a way to produce items from a resource that
needs to be closed. Of course, the generator could be wrapped in a class
that closes the resource, but this is a complication one would like to
avoid, and does not ensure that the resource will be released in a timely
manner. Thunks do not have this limitation because the thunk-accepting
function is in control-- execution cannot break out of the 'do' statement
without first passing through the thunk-accepting function.
def get_items(thunk): # <-- "thunk-accepting function"
f = acquire()
try:
for i in f:
thunk(i) # A-OK
finally:
f.release()
do i in get_items():
print i
Even though thunks can be used in some ways that generators cannot, they
are not nearly a replacement for generators. Importantly, one has no
analogue of the 'next' method of generators when using thunks:
def f():
yield 89
yield 91
g = f()
g.next() # == 89
g.next() # == 91
[1] see the "Extended Function syntax" thread,
http://mail.python.org/pipermail/python-dev/2003-February/
[2] http://www.python.org/peps/pep-0310.html
[3] http://www.python.org/peps/pep-0288.html