Hard times with packages and instances

[Kay Schluehr]

Hi people,

I wonder why the isinstance() function is sensitive about the import
path i.e. the result depends not only on the class and the instance but
also on how a class is imported?

Example:

MyPackage/ Top-level package
__init__.py Initialize package
__me__.py Module used for setting Python-path
A.py Use objects of ForeignPackage and
subpackages
ForeignPackage/ Don't touch!
__init__.py Initialize package
B.py Defines class B1
MySubPackage/ Subpackage
__init__.py Initialize subpackage
C.py Defines instance checker for B1 instances


Provide some implementations:

_____________________________________________________________________________

# MyPackage.__init__.py

import __me__
__me__.setmypath()


# MyPackage.__me__.py

def setmypath():
import __me__
import inspect
import sys
absfile = inspect.getabsfile(__me__)
abspath =
"/".join(absfile.split("/")[:-1]+absfile.split("\\")[:-1])
sys.path.append(abspath)


# MyPackage.A.py

from MySubPackage.C import*
from ForeignPackage.B import*

b1 = B1()
checkInstance(b1)
#print b1.__class__


# MyPackage.ForeignPackage.__init__.py
# left empty

# MyPackage.ForeignPackage.B.py

class B1(object):
def __init__(self)ass


# MyPackage.MySubPackage.C.py

import MyPackage
import ForeignPackage.B as B
#import MyPackage.ForeignPackage.B as B

def checkInstance(inst):
if isinstance(inst,B.B1):
print "class is %s"%inst.__class__
else:
raise TypeError,"Instance of unknown class '%s'
found"%inst.__class__

____________________________________________________________________________


No run A.py :

A will call checkInstance with a B1 instance and prints

"class is <class 'MySubPackage.B.B1'>"

as expected.

But if one comments in the line

import MyPackage.ForeignPackage.B as B

in module C.py and calls A.py one gets:

Traceback (most recent call last):
....
TypeError: Instance of unknown class '<class 'ForeignPackage.B.B1'>'
found

The class which was expected by the checkInstance() function is

<class 'MyPackage.ForeignPackage.B.B1'>


I'm curios if someone could explain me the difference between this two
class objects?

Regards,
Kay

 

[Fredrik Lundh]

I wonder why the isinstance() function is sensitive about the import
path i.e. the result depends not only on the class and the instance but
also on how a class is imported?

isinstance uses class object identity.

if you manage to import the same thing multiple times, you'll have
multiple class objects representing the same source code, and is-
instance won't work properly.

Example:

MyPackage/ Top-level package
__init__.py Initialize package
__me__.py Module used for setting Python-path
A.py Use objects of ForeignPackage and
subpackages
ForeignPackage/ Don't touch!
__init__.py Initialize package
B.py Defines class B1
MySubPackage/ Subpackage
__init__.py Initialize subpackage
C.py Defines instance checker for B1 instances

in my newsreader, it looks like the C.py module is defined some-
where inbetween MyPackage and MyPackage/ForeignPackage.
what file system are you using? ;-)

(I don't have the energy to decipher your convoluted import
and path-manipulation code; but my intuition tells me that if
you flatten the hierarchy, put MyPackage and ForeignPackage
at the same level, and stop messing with the path inside the
__init__ files, your problems will just disappear).

</F>

 

[Kay Schluehr]

isinstance uses class object identity.

if you manage to import the same thing multiple times, you'll have
multiple class objects representing the same source code, and is-
instance won't work properly.

Importing a class/module multiple times does not cause the problem.
Writing

import ForeignPackage.B as B1
import ForeignPackage.B as B2

in C.py does not destroy the identity B1 == B2. Perhaps it is recovered
from sys.modules["ForeignPackage.B"]? It is not actually isinstance()
that depends on path information but the module recovery. The importer
seems to disambiguate to greedy.

in my newsreader, it looks like the C.py module is defined some-
where inbetween MyPackage and MyPackage/ForeignPackage.
what file system are you using? ;-)

This is surprisingly irrelevant. I used __me__.py for several projects
under Win2K without harm. Python seems to be more intelligent in path
handling than Windows.

(I don't have the energy to decipher your convoluted import
and path-manipulation code;
but my intuition tells me that if
you flatten the hierarchy, put MyPackage and ForeignPackage
at the same level, and stop messing with the path inside the
__init__ files, your problems will just disappear).

Flattening the package structure would solve the problem as well as
destroying all file-system based packages and abandon them from Python.
A bad joke? Yes, but managing all code using a database instaed of an
OS dependent filesys would make sense to me.

Ciao,
Kay

 

[Fredrik Lundh]

Importing a class/module multiple times does not cause the problem.
Writing

import ForeignPackage.B as B1
import ForeignPackage.B as B2

that doesn't import it more than once; Python uses a cache to hold
modules, but the cache only works if Python can be sure that the
modules are the same thing, based on the information in the *import*
statement. in this case, "ForeignPackage.B" is clearly the same thing
as "ForeignPackage.B".

if you change the path around (in your case, by adding MyPackage to
the path, rather than relying on path-relative import), you'll introduce
multiple ways to import the same module, and the cache won't work
properly. and when the cache doesn't work, isinstance breaks down
for the reason I stated.

Flattening the package structure would solve the problem as well as
destroying all file-system based packages and abandon them from Python.

well, packages for just fine for me. if you cannot get them to work,
maybe you should stop doing things that don't work.

</F>

 

[Kay Schluehr]

that doesn't import it more than once; Python uses a cache to hold
modules, but the cache only works if Python can be sure that the
modules are the same thing, based on the information in the *import*
statement. in this case, "ForeignPackage.B" is clearly the same thing
as "ForeignPackage.B".

Yes, that's what I did expect.

if you change the path around (in your case, by adding MyPackage to
the path, rather than relying on path-relative import), you'll introduce
multiple ways to import the same module, and the cache won't work
properly. and when the cache doesn't work, isinstance breaks down
for the reason I stated.

Exactly. "ForeignPackage.B" and "MyPackage.ForeignPackage.B" are two
paths pointing to the same module but it will be imported twice because
the cache does not lookup a strong name ( e.g. a hash of the module )
but the import path. This may also be the reason why it is not possible
to import a module in an interactive session, than edit it and finally
re-import it into the same running session. The hash value would have
been changed but the path remains the same.

Python.

well, packages for just fine for me.

In some cases they do, in other they don't. The __me__.py module comes
from working with ClearCase where the Python package is not installed
by an installation routine relative to an existing Python path. The
package has to recognize it's own position somewhere relative to the
virtual directory which can be arbitrary. Installing the package under
site-packages/ is a no go.

if you cannot get them to work,
maybe you should stop doing things that don't work.

For shure, but people tend to do workarounds. So I did.

Regards,
Kay