Method returning new instance of class?

[Arthur]

A bit inspired by the decorator discussions, I'm trying to tackle something
I had been avoiding.

Essentially I am trying to create a non-destructive tranformation of an
instance of a class - is one way of putting it.

The way I am currently conceptualizing a solution, what I need is a method
of the class that returns a new instance of the class.

I'm sure this is not new territory.

Suggestions appreciated.

Art

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Essentially I am trying to create a non-destructive tranformation of an
instance of a class - is one way of putting it.

The way I am currently conceptualizing a solution, what I need is a method
of the class that returns a new instance of the class.

So you want a copy of the object. I'd use copy.copy for this, perhaps
copy.deepcopy.

Regards,
Martin

 

[Paul Rubin]

The way I am currently conceptualizing a solution, what I need is a method
of the class that returns a new instance of the class.

class foo:
def bar(self):
return foo()

What's the problem?

 

[Arthur]

So you want a copy of the object. I'd use copy.copy for this, perhaps
copy.deepcopy.

That was my first instinct. And perhaps my problem is in here somewhere.

The app is graphical, and I use a Python extensions in C++ using the Boost
library (vpython, new version). My class instance has an attribute which is
a vpython object. Copy.copy doesn't get me where I need to be because my new
instance gets a reference to the same vpython object, and changes to it are
reflected in the original instance. Copy.deepcopy doesn't work for more
obscure reasons. I get an error message generating up from vpython when I
try to change an attribute of the object on the new instance - though I am
interacting with it in the same manner that works fine when performed on the
original instance.

But do you see any reason why this might be?

If it sounds totally illogical, I'll go back and check myself - because of
course the actual sitruation is a bit more complicated than what I am
describing, and I guess it is possible I am falling off the ledge somewhere
else.

Art

 

[Arthur]

class foo:
def bar(self):
return foo()

What's the problem?

As I responded to Martin, what I am looking for - though I didn't describe
it well - is a copy of the original instance (with, for example, same
version of attributes generated with reference to its original arguments).

And I am trying to work around something - or diagnose something I am
running into - using copy.deepcopy.

Is this the time to play with __dict__?

Art

 

[Arthur]

Is this the time to play with __dict__?

Art

I'm afraid -

That didn't come out very elegant;ly, did it

Art

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

The app is graphical, and I use a Python extensions in C++ using the Boost
library (vpython, new version). My class instance has an attribute which is
a vpython object. Copy.copy doesn't get me where I need to be because my new
instance gets a reference to the same vpython object, and changes to it are
reflected in the original instance. [...]

But do you see any reason why this might be?

Certainly. copy.copy expects that each object follows a certain protocol
for copying. Copying of certain types (including all classic classes)
is build into copy.py. For newstyle classes and all other types, copying
procedures must be registered with copy_reg. If a type is not registered
with copy_reg, as a last fall back, the __reduce_ex__ and __reduce__
functions are invoked for the type. If not specifically overridden, they
always return the original object.

Regards,
Martin

 

[Jp Calderone]

=



That was my first instinct. And perhaps my problem is in here somewhere.
=

The app is graphical, and I use a Python extensions in C++ using the Boost
library (vpython, new version). My class instance has an attribute which= is
a vpython object. Copy.copy doesn't get me where I need to be because my = new
instance gets a reference to the same vpython object, and changes to it a= re
reflected in the original instance. Copy.deepcopy doesn't work for more
obscure reasons. I get an error message generating up from vpython when I
try to change an attribute of the object on the new instance - though I = am
interacting with it in the same manner that works fine when performed on = the
original instance.
=

But do you see any reason why this might be?
=

If it sounds totally illogical, I'll go back and check myself - because of
course the actual sitruation is a bit more complicated than what I am
describing, and I guess it is possible I am falling off the ledge somewhe= re
else.

It sounds terribly logical. Copying objects is extremely difficult =

to do correctly. There is no way to do it generically and correctly. =

Make the copy too shallow, and you get unwanted shared state. Make the =

copy too deep and the new object is useless. How do you decide how deep =

is deep enough? You can't, at least not without intimate knowledge of =

the object you're dealing with.

Once you figure out exactly how deep you need to make your copy, you =

can tell copy.deepcopy() how to behave with the copy_reg module. Of =

course, since this knowledge is quite closely tied to VPython, it should =

really be *part* of the VPython package, so that changes to the =

internals of VPython can be made right alongside changes to the code =

which knows how to copy its objects.

Jp

 

[Arthur]

= =

It sounds terribly logical. Copying objects is extremely difficult =

to do correctly. There is no way to do it generically and correctly. =

Make the copy too shallow, and you get unwanted shared state. Make the =

copy too deep and the new object is useless. How do you decide how deep =

is deep enough? You can't, at least not without intimate knowledge of =

the object you're dealing with.

Once you figure out exactly how deep you need to make your copy, you =

can tell copy.deepcopy() how to behave with the copy_reg module. Of =

course, since this knowledge is quite closely tied to VPython, it should =

really be *part* of the VPython package, so that changes to the =

internals of VPython can be made right alongside changes to the code =

which knows how to copy its objects.

Well a quick look at copy_reg and I backed away. Particularly since I am
accessing the internet from a dial-up at the moment and am not in a position
to do the research necessary to get a handle on it. The module docs are
much too sketchy here for someone starting with this from where I am
starting. And as you say, there looked like I might need to get at VPython
internals to do it correctly.

However I did realize, in reading the docs, that overriding __copy__ might
be all I needed to do. Really what I was looking for was a copy of the
original instance with a fresh VPython object on which to operate.
Conceptually I am doing a non-destructive transformation of the instance,
with the transformed object the same, exscept at a transformed location.
Something like this seemed to work:

def __copy__(self):
newcopy=copy.deepcopy(self)
newcopy.vpythonobject=<intialize new vpython object here>
return newcopy

But before getting too far into testing whether this accomplished the copy I
was looking for (perhaps the success was more apparent than real, and the
advice to resort to copy_reg is more unavoidable than I understand) I
realized that I had misconceived something more fundamental. That the
normal instance creation process involves registerting the new instance with
the app in various ways, and that a copy won't accomplish this.

That is until wrote the last paragraph, and realized that I might be able to
throw the registration routines into the __copy__ method.

Hmmm.

Let's try.

Art

 

[Arthur]

Something like this seemed to work:

def __copy__(self):
newcopy=copy.deepcopy(self)
newcopy.vpythonobject=<intialize new vpython object here>
return newcopy

But before getting too far into testing whether this accomplished the copy I
was looking for (perhaps the success was more apparent than real, and the
advice to resort to copy_reg is more unavoidable than I understand) I
realized that I had misconceived something more fundamental. That the
normal instance creation process involves registerting the new instance with
the app in various ways, and that a copy won't accomplish this.

That is until wrote the last paragraph, and realized that I might be able to
throw the registration routines into the __copy__ method.

Hmmm.

Let's try.

With the further realization that I wasn't buying myself anything in
particular by overriding __copy__ at this stage, but that some function
starting with a deepcopy and then reassigning some of the attributes derived
from VPython (which luckily are the attributes that I want as fresh for what
I am trying to do), then running through the "registration" routines
normally done on __init__, then returning the manipulated deepcopy - I seem
to be home.

Now on to the functionality I had in mind, given this capability.

Which should be fun.

Art

 

[Alex Martelli]

A bit inspired by the decorator discussions, I'm trying to tackle something
I had been avoiding.

Essentially I am trying to create a non-destructive tranformation of an
instance of a class - is one way of putting it.

The way I am currently conceptualizing a solution, what I need is a method
of the class that returns a new instance of the class.

theclass.__new__(theclass, *args, **kwds)

is one way to return a new instance of class theclass, which will
typically be empty (unless theclass's instances are immutable, since, in
that case, __new__ must do the initializaiton work... because it would
be too late by __init__ time).

Not clear if it's what you need, but it seems to be what you ask for.


Alex

 

[Arthur]

The app is graphical, and I use a Python extensions in C++ using the Boost
library (vpython, new version). My class instance has an attribute which is
a vpython object. Copy.copy doesn't get me where I need to be because my new
instance gets a reference to the same vpython object, and changes to it are
reflected in the original instance. [...]

But do you see any reason why this might be?

Certainly. copy.copy expects that each object follows a certain protocol
for copying. Copying of certain types (including all classic classes)
is build into copy.py. For newstyle classes and all other types, copying
procedures must be registered with copy_reg. If a type is not registered
with copy_reg, as a last fall back, the __reduce_ex__ and __reduce__
functions are invoked for the type. If not specifically overridden, they
always return the original object.

Back at a high bandwidth connection I decided to try to do some
research to try to reasonably follow what you are telling - despite
the fact that the immediate problem I had been trying to solve seems
to be solved. By using deepcopy(), than overwriting the offending
attributes with fresh instantations.

But both Martin and JP bring copy_reg into the equation in solving
copy issues "by the book".

But the book at:

3.18 copy -- Shallow and deep copy operations (from the 2.3 docs)

says,
on one hand:

"""Classes can use the same interfaces to control copying that they
use to control pickling"""

and on the other:

"""The copy module does not use the copy_reg registration module."""

This is actually pretty esoteric stuff for someone who has not delved
into these mysteries before, so I am a bit lost.

Is it that copy_reg comes into play in defining a custom method for
copying, that is called as a regular method, and not via the copy
module?

Does anyone have a reference for copy_reg used in the specific context
of copying, rather than pickling?

Art

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

"""The copy module does not use the copy_reg registration module."""

This is actually pretty esoteric stuff for someone who has not delved
into these mysteries before, so I am a bit lost.

It appears that this documentation is incorrect. Just look at the
source of copy.py and see for yourself:

from copy_reg import dispatch_table
....

def copy(x):
...
reductor = dispatch_table.get(cls)
...

Is it that copy_reg comes into play in defining a custom method for
copying, that is called as a regular method, and not via the copy
module?

It is a customization of the copy module.

Does anyone have a reference for copy_reg used in the specific context
of copying, rather than pickling?

Sure: Assume you want to copy file objects, and that this should create
a file object for the same file name and mode, but starting from the
beginning. You do

def copy_file(f):
return file, (f.name, f.mode)
copy_reg.pickle(file, copy_file, file)

Then you can do
'root:x:0:0:root:/root:/bin/bash\n'

Regards,
Martin

 

[Alex Martelli]

It appears that this documentation is incorrect. Just look at the
source of copy.py and see for yourself:

You're right, of course, but it's hard to reconcile that with the recent
diktat on py-dev that what's in the source doesn't matter -- only what's
documented to work matters (it's in Brett Cannon's python-dev summary of
3 weeks ago, I can find the links from there to the archives if you
wish). Given that peculiar decision, the fact that copy DOES use
copy_reg, in total violation of the docs, looks like it could be taken
away anytime, as a bugfix...

If we want to keep advising people to check the sources, then that
particular python-dev decision must then be reversed; otherwise we
should stop advising using the sources as the ultimate reference... I
don't think we can have it both ways!


Alex

 

[Arthur]

It appears that this documentation is incorrect.

I knew that that was one of the possibilities. On the other hand I
wouldn't have bet much on it. An equally live possiblity was that I
was misunderstanding correct documentation - considering this is all
a bit foggy to me at the moment.

Thanks for making that much clear.

Just look at the
source of copy.py and see for yourself:

from copy_reg import dispatch_table
...

def copy(x):
...
reductor = dispatch_table.get(cls)
...


It is a customization of the copy module.


Sure: Assume you want to copy file objects, and that this should create
a file object for the same file name and mode, but starting from the
beginning. You do

def copy_file(f):
return file, (f.name, f.mode)
copy_reg.pickle(file, copy_file, file)

Then you can do

'root:x:0:0:root:/root:/bin/bash\n'

And thanks for that. Which among other things seems to tell me that I
can in fact follow the documentation, and what else I can find, on
copy_reg and pickling pretty much verbatim, when focusing on copying
as my end result. Wasn't sure, among other things, whether the same
copy_reg.pickle(xx) syntax applied to copying application. Aparently
yes.

Art