seems like a bug in isinstance()

[dmitrey]

hi all,

suppose I've created a class Point in file .../openopt/kernel/Point.py

Consider the code in file .../somewhere/file1.py
from openopt.kernel.Point import Point
p = Point()

now let's pass p into a func from .../openopt/kernel/file2.py and
check
from Point import Point
isinstance(p, Point)

So, it returns False!

p is <Point.Point instance at 0x30801b8>, while Point is <class
openopt.kernel.Point.Point at 0x2048e20>

I have Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
D.

 

[Chris Rebert]

hi all,

suppose I've created a class Point in file .../openopt/kernel/Point.py

Consider the code in file .../somewhere/file1.py
from openopt.kernel.Point import Point
p = Point()

now let's pass p into a func from .../openopt/kernel/file2.py and
check
from Point import Point
isinstance(p, Point)

So, it returns False!

p is <Point.Point instance at 0x30801b8>, while Point is <class
openopt.kernel.Point.Point at 0x2048e20>

[Subject: seems like a bug in isinstance()]

This is due to a peculiarity of how (C)Python's import machinery
works; isinstance() is working just fine.
(And if you ever think you've found a bug in Python's built-ins, odds
are you haven't. Python has been around too long, someone ought to
have encountered it earlier, statistically speaking.)

Note how the class is referred to as both Point.Point and
openopt.kernel.Point.Point. This is because you did `from Point import
....` in file2.py, whereas in file1.py you did `from
openopt.kernel.Point import ...`. These 2 different ways of referring
to the same module are sufficient to "trick"/"outsmart" (C)Python and
cause it to import the same module twice as 2 separate instances (i.e.
it gets re-executed). Why the import machinery isn't smarter in this
situation, I don't recall.

The output of this should be illuminating:
print(Point, type(p), type(p) is Point, id(Point), id(type(p)))
As this demonstrates, you're dealing with 2 separate definitions of
the same Point class.

Solution: Avoid the implicitly-relative `from Point import ...` style
of import; always use the absolute `from openopt.kernel.Point import
....` style instead. Subsequent imports will thus reference the
already-previously-imported instance of a module rather than importing
a copy of it from scratch again.

Cheers,
Chris

 

[dmitrey]

hi all,

suppose I've created a class Point in file .../openopt/kernel/Point.py

Consider the code in file .../somewhere/file1.py
from openopt.kernel.Point import Point
p = Point()

now let's pass p into a func from .../openopt/kernel/file2.py and
check
from Point import Point
isinstance(p, Point)

So, it returns False!

p is <Point.Point instance at 0x30801b8>, while Point is <class
openopt.kernel.Point.Point at 0x2048e20>

[Subject: seems like a bug in isinstance()]

This is due to a peculiarity of how (C)Python's import machinery
works; isinstance() is working just fine.
(And if you ever think you've found a bug in Python's built-ins, odds
are you haven't. Python has been around too long, someone ought to
have encountered it earlier, statistically speaking.)

Note how the class is referred to as both Point.Point and
openopt.kernel.Point.Point. This is because you did `from Point import
...` in file2.py, whereas in file1.py you did `from
openopt.kernel.Point import ...`. These 2 different ways of referring
to the same module are sufficient to "trick"/"outsmart" (C)Python and
cause it to import the same module twice as 2 separate instances (i.e.
it gets re-executed). Why the import machinery isn't smarter in this
situation, I don't recall.

The output of this should be illuminating:
print(Point, type(p), type(p) is Point, id(Point), id(type(p)))
As this demonstrates, you're dealing with 2 separate definitions of
the same Point class.

Solution: Avoid the implicitly-relative `from Point import ...` style
of import; always use the absolute `from openopt.kernel.Point import
...` style instead. Subsequent imports will thus reference the
already-previously-imported instance of a module rather than importing
a copy of it from scratch again.

Cheers,
Chris
--http://rebertia.com

Thanks Cris, however, I had understood reason of the bug and mere
informed Python developers of the bug to fix it.
(<class openopt.kernel.Point.Point at 0x2a29d50>, <type 'instance'>,
False, 44211536, 8585344)

The proposed solution of using `from openopt.kernel.Point import ... '
everywhere is already performed but is not nice for me. I have lots of
places like that in my code; also, when I import something from
openopt it performs recursive import of many files including that one
where Point is defined, thus I have cycled imports (well, it somehow
works, but is unstable and may lead to future bugs). Also, writing
"from Point import Point" is much simpler than using each time the
long string "from name1.name2.Point import Point". I think it's Python
developers who have to fix the issue, not users. I have 5 years of
intensive Python experience yet it took same time to me to locate the
bug, because my algorithm got "False" from isinstance() and went
absolutely different thread from "if isinstance(): ... else: ...".
This bug could be encountered very seldom under rare circumstances and
thus be quite difficult to be located, especially for Python newbies
unawared of this one.

Regards, D.

 

[Ian Kelly]

Thanks Cris, however, I had understood reason of the bug and mere
informed Python developers of the bug to fix it.

No you haven't. Few if any Python developers make a habit of reading
this newsgroup. To actually report the issue so that it might get
fixed, you need to add it to the issue tracker at
http://bugs.python.org

Anyway, I played around with this a little and was unable to reproduce
it, so I suspect the issue is actually a bit more complicated than
what Chris is describing.

 

[Gregory Ewing]

This is because you did `from Point import
...` in file2.py, whereas in file1.py you did `from
openopt.kernel.Point import ...`. These 2 different ways of referring
to the same module are sufficient to "trick"/"outsmart" (C)Python and
cause it to import the same module twice

That can't be the whole story, because those two ways of
referring to the module *should* have returned the same
module object, even though one is absolute and the other
relative.

I suspect what's happened is that somehow sys.path contains
both the directory containing .../openopt *and* the directory
..../openopt/kernel, and the second import is picking up
Point.py a second time as though it were a top-level module.

This could have happened by starting an interactive session
while cd'ed to .../openopt/kernel, or by launching a .py
file from there as a main script.

The solution is to make sure that sys.path only contains the
top level directories of the package hierarchy, and doesn't
also contain any of their subdirectories.

Always using absolute imports may be a good idea stylistically,
but in this case it will only mask the symptoms of whatever
is really wrong, and further problems could result from the
same cause.

 

[dmitrey]

That can't be the whole story, because those two ways of
referring to the module *should* have returned the same
module object, even though one is absolute and the other
relative.

I suspect what's happened is that somehow sys.path contains
both the directory containing .../openopt *and* the directory
.../openopt/kernel, and the second import is picking up
Point.py a second time as though it were a top-level module.

This could have happened by starting an interactive session
while cd'ed to .../openopt/kernel, or by launching a .py
file from there as a main script.

The solution is to make sure that sys.path only contains the
top level directories of the package hierarchy, and doesn't
also contain any of their subdirectories.

Always using absolute imports may be a good idea stylistically,
but in this case it will only mask the symptoms of whatever
is really wrong, and further problems could result from the
same cause.

I suspect what's happened is that somehow sys.path contains

both the directory containing .../openopt *and* the directory
..../openopt/kernel
Yes, you are right. But I have to do it due to another issue I haven't
solved yet: Python3 imports don't see files from same directory
http://groups.google.com/group/comp.lang.python/browse_thread/thread/9470dbdacc138709#

Regards, D.

 

[Gregory Ewing]

Yes, you are right. But I have to do it due to another issue I haven't
solved yet: Python3 imports don't see files from same directory
http://groups.google.com/group/comp.lang.python/browse_thread/thread/9470dbdacc138709#

That's because the syntax for relative imports has changed
in Python 3. See the recent comment on your posting there
for details.