Operator += works once, fails if called again

[Frederic Rentsch]

Hi all,

I have a class Time_Series derived from list. It lists days and
contains a dictionary of various Lists also derived from list which
contain values related to said days. (e.g. Stock quotes, volumes traded,
etc.)
I defined an operator += which works just fine, but only once. If I
repeat the operation, it fails and leaves me utterly mystified and crushed.

Craving to be uncrushed by a superior intelligence.

Frederic


---------------------------------------------------------------

A test:
TS1 | Date | Numbers |

0.00 | 1900.00.00 | 22.00 |
1.00 | 1900.01.01 | 44.00 |
2.00 | 1900.01.02 | 323.00 |
3.00 | 1900.01.03 | 62.00 |
4.00 | 1900.01.04 | 342.00 |
5.00 | 1900.01.05 | 50.00 |
6.00 | 1900.01.06 | 63.00 |
7.00 | 1900.01.07 | 77.00 |
8.00 | 1900.01.08 | -1.00 |
9.00 | 1900.01.09 | 0.00 |

Perfect!

Traceback (most recent call last):
File "<pyshell#311>", line 1, in -toplevel-
TS += TS2
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1314, in __iadd__
return self.__add__ (other)
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1273, in __add__
Sum = copy.deepcopy (self)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 320, in _reconstruct
y = callable(*args)
File "C:\PYTHON24\lib\copy_reg.py", line 92, in __newobj__
return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0


It seems to crash on the second call to deepcopy. Why? The type of the
deepcopied object is still 'Time_Series'.

Here's Time_Series.__add__ () but I don't think there's anything wrong
with that.

def __add__ (self, other):

if self [0] <= other [0]: # One or the other ...
Sum = copy.deepcopy (self)
Summand = copy.deepcopy (other)
else: # depending on which starts earlier
Sum = copy.deepcopy (other)
Summand = copy.deepcopy (self)

if Sum [0] != Summand [0]:
Summand.insert (0, Sum [0])
for list_name in Summand.Lists:
Summand.Lists [list_name].insert (0, None)
if Sum [-1] < Summand [-1]:
Sum.append (Summand [-1])
elif Sum [-1] > Summand [-1]:
Summand.append (Sum [-1])

Sum.make_continuous () # Fills in missing days and values
Summand.make_continuous ()

for list_name in Summand.Lists:
if Sum.Lists.has_key (list_name):
Sum.Lists [list_name] += Summand.Lists [list_name]
# List operators work fine

return Sum


---------------------------------------------------------------------------

 

[John Machin]

Hi all,

I have a class Time_Series derived from list. It lists days and
contains a dictionary of various Lists also derived from list which
contain values related to said days. (e.g. Stock quotes, volumes traded,
etc.)
I defined an operator += which works just fine, but only once. If I
repeat the operation, it fails and leaves me utterly mystified and crushed.

Craving to be uncrushed by a superior intelligence.

Frederic

[perfection snipped]

Perfect!

Traceback (most recent call last):
File "<pyshell#311>", line 1, in -toplevel-
TS += TS2
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1314, in __iadd__
return self.__add__ (other)
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1273, in __add__
Sum = copy.deepcopy (self)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 320, in _reconstruct
y = callable(*args)
File "C:\PYTHON24\lib\copy_reg.py", line 92, in __newobj__

Oho! What's it doing there, in copy_reg.py? Something has stashed away
a reference to copy_reg.__newobj__, and the stasher is *not* the copy
module ....

return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0

From a quick browse through the code for copy.py and copy_reg.py [Isn't

open source a wonderful thing?], I'm guessing:
(1) The "cls" is *your* class. You can confirm that with a debugger
(or by hacking in a print statement!).
(2) You may need to read this in the copy docs:
"""
Classes can use the same interfaces to control copying that they use to
control pickling. See the description of module pickle for information
on these methods. The copy module does not use the copy_reg
registration module.
"""
(3) You may need to read this in the pickle docs:
"""
New-style types can provide a __getnewargs__() method that is used for
protocol 2. Implementing this method is needed if the type establishes
some internal invariants when the instance is created, or if the memory
allocation is affected by the values passed to the __new__() method for
the type (as it is for tuples and strings). Instances of a new-style
type C are created using
obj = C.__new__(C, *args)
where args is the result of calling __getnewargs__() on the original
object; if there is no __getnewargs__(), an empty tuple is assumed.
"""
(4) You may need to debug your pickling/unpickling before you debug
your deepcopying.
(5) Look at function _copy_inst at line 134 in copy.py -- work out what
it will do with your class instance, depending on what __magic__
method(s) you have implemented.

Hope some of this helps,
John

 

[Frederic Rentsch]

Hi all,

I have a class Time_Series derived from list. It lists days and
contains a dictionary of various Lists also derived from list which
contain values related to said days. (e.g. Stock quotes, volumes traded,
etc.)
I defined an operator += which works just fine, but only once. If I
repeat the operation, it fails and leaves me utterly mystified and crushed.

Craving to be uncrushed by a superior intelligence.

Frederic

[perfection snipped]

Perfect!

TS1 += TS2 # Second call

Traceback (most recent call last):
File "<pyshell#311>", line 1, in -toplevel-
TS += TS2
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1314, in __iadd__
return self.__add__ (other)
File "c:\i\sony\fre\src\python\TIME_SERIES_7.py", line 1273, in __add__
Sum = copy.deepcopy (self)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 335, in _reconstruct
state = deepcopy(state, memo)
File "C:\PYTHON24\lib\copy.py", line 161, in deepcopy
y = copier(x, memo)
File "C:\PYTHON24\lib\copy.py", line 252, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "C:\PYTHON24\lib\copy.py", line 188, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\PYTHON24\lib\copy.py", line 320, in _reconstruct
y = callable(*args)
File "C:\PYTHON24\lib\copy_reg.py", line 92, in __newobj__

Oho! What's it doing there, in copy_reg.py? Something has stashed away
a reference to copy_reg.__newobj__, and the stasher is *not* the copy
module ....

return cls.__new__(cls, *args)
TypeError: instancemethod expected at least 2 arguments, got 0

From a quick browse through the code for copy.py and copy_reg.py [Isn't

open source a wonderful thing?], I'm guessing:
(1) The "cls" is *your* class. You can confirm that with a debugger
(or by hacking in a print statement!).
(2) You may need to read this in the copy docs:
"""
Classes can use the same interfaces to control copying that they use to
control pickling. See the description of module pickle for information
on these methods. The copy module does not use the copy_reg
registration module.
"""
(3) You may need to read this in the pickle docs:
"""
New-style types can provide a __getnewargs__() method that is used for
protocol 2. Implementing this method is needed if the type establishes
some internal invariants when the instance is created, or if the memory
allocation is affected by the values passed to the __new__() method for
the type (as it is for tuples and strings). Instances of a new-style
type C are created using
obj = C.__new__(C, *args)
where args is the result of calling __getnewargs__() on the original
object; if there is no __getnewargs__(), an empty tuple is assumed.
"""
(4) You may need to debug your pickling/unpickling before you debug
your deepcopying.
(5) Look at function _copy_inst at line 134 in copy.py -- work out what
it will do with your class instance, depending on what __magic__
method(s) you have implemented.

Hope some of this helps,
John

John,

Thank you very much for your suggestions. I followed them one by one and
in the end found the cause of the problem to be a circular reference.
Some species of contained Lists need a reference to the containing
Time_Series and that circular reference trips up deepcopy. Supposedly I
can define my own __deepcopy__, but didn't understand whose method it is
supposed to be. Time_Series.__deepcopy__ () doesn't make sens, nor does
it work. Time_Series should be the argument of the method not the owner.
Anyway, I managed without deepcopy making a new instance and loading it
as I would, if it were the first one.

Thanks again

Frederic