pre-PEP: Simple Thunks

[Serhiy Storchaka]

do f in with_file('file.txt'):
print f.read()

def with_file(filename):
f = open(filename)
yield f
f.close()

for f in with_file('file.txt'):
print f.read()

t = "no file read yet"
do f in with_file('file.txt'):
t = f.read()

t = "no file read yet"
for f in with_file('file.txt'):
t = f.read()

 

[Ron_Adam]

I also wouldn't do it that way. I don't see a class as being much better,
though. If I understand you correctly, with classes you would have
something like:

p = Pickled('pickled.txt')
p.load()
p.data.append('more data')
p.data.append('even more data')
p.dump()

The load and dump would be private to the data class object. Here's a
more complete example.

import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...

pdata = PickledData('filename')

pdata.append('more data')
pdata.append('even more data')

print pdata.L
['more data', 'even more data']

This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.

A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.

Cheers,
Ron

 

[Brian Sabbey]

The load and dump would be private to the data class object. Here's a
more complete example.

import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...

pdata = PickledData('filename')

pdata.append('more data')
pdata.append('even more data')

print pdata.L
['more data', 'even more data']

This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.

A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.

This class isn't reusable in the case that one wants to pickle something
other than an array. Every type of object that one would wish to pickle
would require its own class.

Also, this implementation behaves differently because the object is
re-pickled after every modification. This could be a problem when writing
over a network, or to a shared resource.

-Brian

 

[Ron_Adam]

The load and dump would be private to the data class object. Here's a
more complete example.

import pickle
class PickledData(object):
def __init__(self, filename):
self.filename = filename
self.L = None
try:
self._load()
except IOError:
self.L = []
def _load(self):
f = open(self.filename, 'r')
self.L = pickle.load(f)
f.close()
def _update(self):
f = open(self.filename, 'w')
pickle.dump(self.L, f)
f.close()
def append(self, record):
self.L.append(record)
self._update()
# add other methods as needed ie.. get, sort, clear, etc...

pdata = PickledData('filename')

pdata.append('more data')
pdata.append('even more data')

print pdata.L
['more data', 'even more data']

This has the same issues as with opening and closing files: losing the
'dump', having to always use try/finally if needed, accidentally
re-binding 'p', significantly more lines. Moreover, class 'Pickled' won't
be as readable as the 'pickled_file' function above since 'load' and
'dump' are separate methods that share data through 'self'.

A few more lines to create the class, but it encapsulates the data
object better. It is also reusable and extendable.

This class isn't reusable in the case that one wants to pickle something
other than an array. Every type of object that one would wish to pickle
would require its own class.

....Or in a function, or the 3 to 6 lines of pickle code someplace.
Many programs would load data when they start, and then save it when
the user requests it to be saved. So there is no one method fits all
situations. Your thunk example does handle some things better.


Here's yet another way to do it, but it has some limitations as well.

import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()

file = 'filename'
L = []

opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)

Also, this implementation behaves differently because the object is
re-pickled after every modification. This could be a problem when writing
over a network, or to a shared resource.

-Brian

In some cases writing to the file after ever modification would be
desired. A way around that would be to use a buffer of some sort. But
then again, that adds another level of complexity and you would have
to insure it's flushed at some point which get's back to the issue of
not closing a file.

Thanks for explaining how thunks works. I'm still undecided on
whether it should be built in feature or not.

I would rather have a way to store a block of code and pass it to a
function, then execute it at the desired time. That would solve both
the issue where you would use a thunk, and replace lambdas as well.
But I understand there's a lot of resistance to that because of the
potential abuse.

Cheers,
Ron

 

[Greg Ewing]

do f in with_file('file.txt'):
print f.read()

I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???

To sound right it would have to be something like

with_file('file.txt') as f do:
print f.read()

But, while that works with this particular function
name, and others of the form "with_xxx", there are
bound to be other use cases which would require
different words or word orders in order not to sound
contrived.

It's very difficult to come up with a good syntax for
this that isn't skewed towards one kind of use case.
That's probably a large part of the reason why nothing
like it has so far been seriously considered for
adoption.

 

[Steven Bethard]

I don't like this syntax. Try to read it as an English sentence:
"Do f in with file 'file.txt'". Say what???

To sound right it would have to be something like

with_file('file.txt') as f do:
print f.read()

This is still strange since f is the arguments the thunk was called
with, e.g. the current syntax is basically:

do <unpack_list> in <returnval> = <callable>(<params>):
<code>

I don't really know a more readable sequence of keywords, though someone
suggested 'with' and 'from', which might read something like:

with <unpack_list> from <callable>(<params>):
<code>

which looks okay to me, though I'm not sure that 'with' makes it clear
that this is not a normal block... I also find readability problems
when I try to stick <returnval> back in.

One of the other issues is that, with the current proposal, the thunk
can be called multiple times within a function, so the keywords have to
make sense both with a single iteration interpretation and a multiple
iteration interpretation... Makes it even harder...

STeVe

 

[Mike Meyer]

Here's yet another way to do it, but it has some limitations as well.

import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()

file = 'filename'
L = []

opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)

Um - it doesn't look like this will work. You pass L in as the "obj"
paremeter, and then it gets changed to the results of a
pickle.load. However, you call L.append in the for loop, so the data
will be appended to L, not obj. You'll lose an list elements that were
in the pickle.

<mike

 

[Ron]

Here's yet another way to do it, but it has some limitations as well.

import pickle
def pickle_it(filename, obj, commands):
try:
f = open(filename, 'r')
obj = pickle.load(f)
f.close()
except IOError:
pass
for i in commands:
i[0](i[1])
f = open(filename, 'w')
pickle.dump(obj, f)
f.close()

file = 'filename'
L = []

opps = [ (L.append,'more data'),
(L.append,'even more data') ]
pickle_it(file, L, opps)

Um - it doesn't look like this will work. You pass L in as the "obj"
paremeter, and then it gets changed to the results of a
pickle.load. However, you call L.append in the for loop, so the data
will be appended to L, not obj. You'll lose an list elements that were
in the pickle.

<mike

Hmmm, you are correct. :-/

I was trying to not use eval(). This could be made to work, but it will
get messy, which is another reason not to do it.

Cheers,
Ron