Can't deepcopy bytes-derived class

[Dan]

I have a simple type derived from bytes...

class atom(bytes):
pass

.... that I cannot deepcopy(). The session below demonstrates how
deepcopy() of "bytes" works fine, but deepcopy() of "atom" does not.
What's going wrong?

Thanks,
Dan.


Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit
(Intel)]
Type "help", "copyright", "credits" or "license" for more information..... pass
....Traceback (most recent call last):
File "C:\Program Files (x86)\Wing IDE 3.2\src\debug\tserver
\_sandbox.py", line 1, in <module>
# Used internally for debug sandbox under external interpreter
File "C:\Python31\Lib\copy.py", line 173, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\Python31\Lib\copy.py", line 280, in _reconstruct
y = callable(*args)
File "C:\Python31\Lib\copyreg.py", line 88, in __newobj__
return cls.__new__(cls, *args)
builtins.TypeError: string argument without an encoding

 

[Terry Reedy]

I have a simple type derived from bytes...

class atom(bytes):
pass

... that I cannot deepcopy(). The session below demonstrates how
deepcopy() of "bytes" works fine, but deepcopy() of "atom" does not.
What's going wrong?

Thanks,
Dan.


Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit
(Intel)]
Type "help", "copyright", "credits" or "license" for more information.... pass
...Traceback (most recent call last):
File "C:\Program Files (x86)\Wing IDE 3.2\src\debug\tserver
\_sandbox.py", line 1, in<module>
# Used internally for debug sandbox under external interpreter
File "C:\Python31\Lib\copy.py", line 173, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\Python31\Lib\copy.py", line 280, in _reconstruct
y = callable(*args)
File "C:\Python31\Lib\copyreg.py", line 88, in __newobj__
return cls.__new__(cls, *args)
builtins.TypeError: string argument without an encoding

You could trace through copy.py and copyreg.py to see where bytes and
atom get treated differently.

This might be a bug that should be reported on the tracker, but I do not
know. Let see what anyone else says.

 

[Ned Deily]

I have a simple type derived from bytes...

class atom(bytes):
pass

... that I cannot deepcopy(). The session below demonstrates how
deepcopy() of "bytes" works fine, but deepcopy() of "atom" does not. [...]
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit
(Intel)]
Type "help", "copyright", "credits" or "license" for more information.

import copy
class atom(bytes):

... pass
...

copy.deepcopy(b'abc') b'abc'
copy.deepcopy(atom(b'abc'))

Traceback (most recent call last):
File "C:\Program Files (x86)\Wing IDE 3.2\src\debug\tserver
\_sandbox.py", line 1, in<module>
# Used internally for debug sandbox under external interpreter
File "C:\Python31\Lib\copy.py", line 173, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "C:\Python31\Lib\copy.py", line 280, in _reconstruct
y = callable(*args)
File "C:\Python31\Lib\copyreg.py", line 88, in __newobj__
return cls.__new__(cls, *args)
builtins.TypeError: string argument without an encoding

You could trace through copy.py and copyreg.py to see where bytes and
atom get treated differently.

This might be a bug that should be reported on the tracker, but I do not
know. Let see what anyone else says.

FWIW, the example still fails with Python 3.1.3 but seems to work OK
with a recent alpha build of Python 3.2. What specific change might
have fixed it doesn't come immediately to mind.

 

[Terry Reedy]

I have a simple type derived from bytes...

class atom(bytes): pass

... that I cannot deepcopy(). The session below demonstrates
how deepcopy() of "bytes" works fine, but deepcopy() of "atom"
does not. [...]
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32
bit (Intel)] Type "help", "copyright", "credits" or "license" for
more information.
import copy class atom(bytes):
... pass ...
copy.deepcopy(b'abc')
b'abc'
copy.deepcopy(atom(b'abc'))
Traceback (most recent call last): File "C:\Program Files
(x86)\Wing IDE 3.2\src\debug\tserver \_sandbox.py", line 1,
in<module> # Used internally for debug sandbox under external
interpreter File "C:\Python31\Lib\copy.py", line 173, in
deepcopy y = _reconstruct(x, rv, 1, memo) File
"C:\Python31\Lib\copy.py", line 280, in _reconstruct y =
callable(*args) File "C:\Python31\Lib\copyreg.py", line 88, in
__newobj__ return cls.__new__(cls, *args) builtins.TypeError:
string argument without an encoding

You could trace through copy.py and copyreg.py to see where bytes
and atom get treated differently.

This might be a bug that should be reported on the tracker, but I
do not know. Let see what anyone else says.

FWIW, the example still fails with Python 3.1.3 but seems to work OK
with a recent alpha build of Python 3.2. What specific change might
have fixed it doesn't come immediately to mind.

According to difflib.unified_diff, the only difference between
Lib/copy.py from 3.1 to 3.2 is 4 lines:
+
+def _deepcopy_method(x, memo): # Copy instance methods
+ return type(x)(x.__func__, deepcopy(x.__self__, memo))
+_deepcopy_dispatch[types.MethodType] = _deepcopy_method

(and none in copyreg.py)

These were added in rev76572 as part of
http://bugs.python.org/issue1515

Guido decreed this to be a new feature rather than bugfix, so the change
went into future 2.7 and 3.2 and not existing 2.6 and 3.1.

Tto verify that these cause the change, comment out in 3.2 or add to
3.1. I have not read through the issue to understand it or why it would
affect the OP's code case.

 

[Dan]

On 12/8/2010 2:42 PM, Dan wrote:
I have a simple type derived from bytes...
class atom(bytes): pass
... that I cannot deepcopy().  The session below demonstrates
how deepcopy() of "bytes" works fine, but deepcopy() of "atom"
does not. [...]
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32
bit (Intel)] Type "help", "copyright", "credits" or "license" for
more information.
import copy class atom(bytes):
...        pass ...
copy.deepcopy(b'abc')
b'abc'
copy.deepcopy(atom(b'abc'))
Traceback (most recent call last): File "C:\Program Files
(x86)\Wing IDE 3.2\src\debug\tserver \_sandbox.py", line 1,
in<module> # Used internally for debug sandbox under external
interpreter File "C:\Python31\Lib\copy.py", line 173, in
deepcopy y = _reconstruct(x, rv, 1, memo) File
"C:\Python31\Lib\copy.py", line 280, in _reconstruct y =
callable(*args) File "C:\Python31\Lib\copyreg.py", line 88, in
__newobj__ return cls.__new__(cls, *args) builtins.TypeError:
string argument without an encoding
You could trace through copy.py and copyreg.py to see where bytes
and atom get treated differently.
This might be a bug that should be reported on the tracker, but I
do not know. Let see what anyone else says.

FWIW, the example still fails with Python 3.1.3 but seems to work OK
with a recent alpha build of Python 3.2.  What specific change might
have fixed it doesn't come immediately to mind.

According to difflib.unified_diff, the only difference between
Lib/copy.py from 3.1 to 3.2 is 4 lines:
+
+def _deepcopy_method(x, memo): # Copy instance methods
+    return type(x)(x.__func__, deepcopy(x.__self__, memo))
+_deepcopy_dispatch[types.MethodType] = _deepcopy_method

(and none in copyreg.py)

These were added in rev76572 as part ofhttp://bugs.python.org/issue1515

Guido decreed this to be a new feature rather than bugfix, so the change
went into future 2.7 and 3.2 and not existing 2.6 and 3.1.

Tto verify that these cause the change, comment out in 3.2 or add to
3.1. I have not read through the issue to understand it or why it would
affect the OP's code case.

Thank you everyone. Knowing that it works in 3.2 is good enough for
me.

I tested the 4 line change anyway...removing it from 3.2 had no
detrimental effect, adding it to 3.1 had no effect. It must be
something else.

Regards,
Dan.

 

[Terry Reedy]

I tested the 4 line change anyway...removing it from 3.2 had no
detrimental effect, adding it to 3.1 had no effect. It must be
something else.

Then there must have been a change in the bytes object, which is deeper
than I want to go. 3.2 has a LOT of little improvements. When it comes
out, just about all 3.1 users should upgrade unless one has production
code disabled by a bugfix.