dynamic type changing

[andychambers2002]

I'm working on a "TempFile" class that stores the data in memory until
it gets larger than a specified threshold (as per PEP 42). Whilst
trying to implement it, I've come across some strange behaviour. Can
anyone explain this?

The test case at the bottom starts a TempFile at size 50 and prints its
type. It then increases the size to the threshold at which point
"self" is changed to being a TemporaryFile. It seems that the next
call correctly uses the write() method of TemporaryFile (since we don't
see "changing type" in the output). However, type(tmp) still equals
TempFile. Not only that, tmp can still access the method dummy() that
exists only in TempFile.

#!/usr/bin/env python
from StringIO import StringIO
import tempfile

class TempFile(StringIO, object):
"""A temporary file implementation that uses memory unless
either capacity is breached or fileno is requested, at which
point a real temporary file will be created and the relevant
details returned
"""
def __init__(self, buffer, capacity):
"""Creates a TempFile object containing the specified buffer.
If capacity is specified, we use a real temporary file once the

file gets larger than that size. Otherwise, the data is stored

in memory.
"""
self.capacity = capacity
if len(buffer) > capacity:
self = tempfile.TemporaryFile()
self.write(buffer)
else:
super(TempFile, self).__init__(buffer)

def dummy(self):
pass

def write(self, str):
self.seek(0, 2) # find end of file
if((self.tell() + len(str)) >= self.capacity):
print "changing type"
flo = tempfile.TemporaryFile()
flo.write(self.getvalue())
self = flo
print type(self)
else:
super(TempFile, self).write(str)


print "testing tempfile:"
tmp = TempFile("", 100)
ten_chars = "1234567890"
tmp.write(ten_chars * 5)
tmp.dummy()
print "tmp < 100: " + str(type(tmp))
tmp.write(ten_chars * 5)
tmp.dummy()
print "tmp == 100: " + str(type(tmp))
tmp.write("the last straw")
tmp.dummy()
print "tmp > 100: " + str(type(tmp))

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

I'm working on a "TempFile" class that stores the data in memory until
it gets larger than a specified threshold (as per PEP 42). Whilst
trying to implement it, I've come across some strange behaviour. Can
anyone explain this?

The test case at the bottom starts a TempFile at size 50 and prints its
type. It then increases the size to the threshold at which point
"self" is changed to being a TemporaryFile.

Changed how ?-)

It seems that the next
call correctly uses the write() method of TemporaryFile (since we don't
see "changing type" in the output). However, type(tmp) still equals
TempFile. Not only that, tmp can still access the method dummy() that
exists only in TempFile.

#!/usr/bin/env python
from StringIO import StringIO
import tempfile

class TempFile(StringIO, object):
"""A temporary file implementation that uses memory unless
either capacity is breached or fileno is requested, at which
point a real temporary file will be created and the relevant
details returned
"""
def __init__(self, buffer, capacity):
"""Creates a TempFile object containing the specified buffer.
If capacity is specified, we use a real temporary file once the

file gets larger than that size. Otherwise, the data is stored

in memory.
"""
self.capacity = capacity
if len(buffer) > capacity:
self = tempfile.TemporaryFile()

assigning to 'self' in a method doesn't impact the object itself - it
only rebinds the *local* name 'self' for the rest of the block.

If you want to change the class of an object, you must assign to
self.__class__ - but, while perfectly legal (and in fact the simplest
possible implementation of the state pattern in Python), it may be
somewhat risky.

(snip)

def write(self, str):
self.seek(0, 2) # find end of file
if((self.tell() + len(str)) >= self.capacity):
print "changing type"
flo = tempfile.TemporaryFile()
flo.write(self.getvalue())
self = flo
print type(self)

Same comment here.

(snip)

Now for a practical solution : what you want is the strategy pattern.


from StringIO import StringIO
from tempfile import TemporaryFile
import sys

class TempFile(object):
"""A temporary file implementation that uses memory unless
either capacity is breached or fileno is requested, at which
point a real temporary file will be created and the relevant
details returned
"""

_strategies = (StringIO, TemporaryFile)

def __init__(self, buffer, capacity):
"""Creates a TempFile object containing the specified buffer.

If capacity is specified, we use a real temporary file once the
file gets larger than that size. Otherwise, the data is stored
in memory.
"""
self.capacity = capacity
self._delegate = self._strategies[len(buffer) > self.capacity]()
self.write(buffer)

def write(self, value):
print >> sys.stderr, \
"about to write %d more characters" % len(value)
if isinstance(self._delegate, self._strategies[0]):
len_value = len(value)
if len_value >= self.capacity:
needs_new_strategy = True
else:
self.seek(0, 2) # find end of file
needs_new_strategy = \
self.tell() + len_value >= self.capacity

if needs_new_strategy:
print >> sys.stderr, "changing strategy"
new_delegate = self._strategies[1]()
new_delegate.write(self.getvalue())
self._delegate = new_delegate

self._delegate.write(value)


def __getattr__(self, name):
# Takes care of automatic delegation,
# customize this according to your needs.
# Hint : this will only be called if normal lookup
# failed, so to control access to any _delegate's method,
# just implement a method with same name
try:
return getattr(self._delegate, name)
except AttributeError:
# hide the delegation
e = "object '%s' has no attribute '%s'" \
% (self.__class__.__name__, name)
raise AttributeError(e)


if __name__ == "__main__":
print "testing tempfile:"
tmp = TempFile("", 100)
ten_chars = "1234567890"
tmp.write(ten_chars * 5)
print "tmp < 100: ", tmp._delegate.__class__.__name__
tmp.write(ten_chars * 5)
print "tmp == 100: " , tmp._delegate.__class__.__name__
tmp.write("the last straw")
print "tmp > 100: " , tmp._delegate.__class__.__name__

 

[andychambers2002]

I'm working on a "TempFile" class that stores the data in memory until

Changed how ?-)

Just by being assigned with a TemporaryFile object. I thought that if
you do

instance = TempFile()

that "instance" and "self" defined in the Class were the same thing so
that if you changed the class of self, the class of instance would also
change.

Thanks very much for your example. It has solved my problem and helped
me understand a new pattern at the same time.

 

[David C. Ullrich]

Just by being assigned with a TemporaryFile object. I thought that if
you do

instance = TempFile()

that "instance" and "self" defined in the Class were the same thing so
that if you changed the class of self, the class of instance would also
change.

Thanks very much for your example. It has solved my problem and helped
me understand a new pattern at the same time.

Bruno says you _can_ assign to __class__ but calls that "risky".
If you ever do feel the urge to assign a new value to some
object's __class__ it might be a good idea to first make certain
you can predict the behavior of the following:

class A:
msg = 'A'
def hmm(self):
print self.msg

class B:
msg = 'B'
def hmm(self):
print self.msg

x = A()
x.hmm()
x.__class__ = B
x.hmm()

class C:
def __init__(self):
self.msg = 'C'
def hmm(self):
print self.msg

class D:
def __init__(self):
self.msg = 'D'
def hmm(self):
print self.msg

x = C()
x.hmm()
x.__class__ = D
x.hmm()

************************

David C. Ullrich

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

Just by being assigned with a TemporaryFile object.
>
I thought that if
you do

instance = TempFile()

that "instance" and "self" defined in the Class

They are not defined "in the class".

were the same thing so
that if you changed the class of self,
the class of instance would also
change.

Yes, of course - but you didn't change the class of 'self' !-)

Python's "variable" are really just names "bound to" (referring to)
objects. Rebinding (ie: assignment) does not impact the object (well,
not directly), it just associate the name to another object. This is
totally different from changing the state of the object.

There's nothing magical about the name 'self' - FWIW, you could replace
it by any other valid python identifier. In Python, a method is just a
plain function that takes the instance as the first argument. This
function is wrapped into a method descriptor (google for python's
descriptor protocol - it's the same thing that is used for properties)
that takes care of feeding the function with the instance.

FWIW, given:
class Obj(object):
def someMethod(self):
pass

obj = Obj()

then
obj.someMethod()

is the same as
Obj.someMethod(obj)

or also:
obj.someMethod.im_func(obj)


So, inside someMethod's code, normal scoping rules apply. This means
that 'self' is a *local* name, and rebinding it only affect the local
scope. And it doesn't "change the type" of the object (previously) bound
to 'self', it really re-bind 'self' to another object (IOW: makes 'self'
a reference to another object). Just like it does in any other Python code.

As I wrote, to dynamicall change the class of an object, you must rebind
obj.__class__ to another class, ie:

class Other(object):
def __init__(self, name):
self.name = name

obj = Obj()
print type(obj)
obj.__class__ = Other
print type(obj)

Now a big warning : this is not garanteed to work seamlessly ! 'obj'
will keep it's original instance attributes, and the instance attributes
normally set up by the new class (here, 'Other') won't exist since the
class initializer won't be called.

So, while this may not be a problem if the original and new classes are
designed to be used that way (which makes a very straightforward
implementation of the state pattern), it's usually not a good idea to do
such a thing. FWIW, it's usually simpler and safer - evn if a bit less
elegant - to implement the state pattern just like I did in the example:
by using composition/delegation.

Thanks very much for your example.

It has solved my problem and helped
me understand a new pattern at the same time.

<ot>
FWIW, there's no clear, well defined boudary between State and Strategy
- the main difference depends mostly on the intention. Your use case
could also be viewed as a State pattern, with 2 states : buffer <
capacity, and buffer >= capacity. But the intention is not to know in
which state is the object - on the contrary, you're trying to hide away
the chosen implementation (StringIO or TemporayFile) - so it's really a
Strategy IMHO.
</ot>