Finding and loading subclasses dynamically. issubclass(x,base_plugin.Plugin) fails.

[Osmo Maatta]

Hello,

Sub class test fails.
======================
I have a program that needs to load plugin-classes during runtime.

The program has these subdirectories (modules).

$ tree
..
`-- test.py
|
|-- plugins
| |-- base_plugin.py
| |-- base_plugin.pyc
| |-- __init__.py
| `-- oca
| |-- __init__.py
| |-- open_clipart.py


The plugins (sub directory) contains one or more plugin modules, in this
test-case there is only one; oca/open_clipart.py.

The plugins/base_plugin.py (contains class Plugin()) is a base class of
all plugins.

I want to list and load all plugin-classes (that inherit from
base_plugin.Plugin). I have a Python code that successfully browses the
plugins, but the the test issubclass(x, base_plugin.Plugin) fails.

Please see this Python code:
http://futuredesktop.com/tmp/test6.tar.gz

Why the issubclass(entry, cls) test fails?

if issubclass(entry, cls):
.....print "Found a subclass: " + key
.....subclasses.append(entry)

I can see that the class names are right, even objects of these classes
are ok.

My OS is Ubuntu 10.04.
$ python --version
Python 2.6.5

The actual, final product will be this
http://www.futuredesktop.com/clipart-applet/clipart-applet.ogv
It is a ClipArt query-engine and browser.
It can translate queries from "any" language to english ;-)

Most kindly
Osmo Antero Maatta
Grønland, Oslo

 

[Arnaud Delobelle]

Hello,

Sub class test fails.
======================
I have a program that needs to load plugin-classes during runtime.

The program has these subdirectories (modules).

$ tree
.
`-- test.py
|
|-- plugins
| |-- base_plugin.py
| |-- base_plugin.pyc
| |-- __init__.py
| `-- oca
| |-- __init__.py
| |-- open_clipart.py


The plugins (sub directory) contains one or more plugin modules, in this
test-case there is only one; oca/open_clipart.py.

The plugins/base_plugin.py (contains class Plugin()) is a base class of
all plugins.

I want to list and load all plugin-classes (that inherit from
base_plugin.Plugin). I have a Python code that successfully browses the
plugins, but the the test issubclass(x, base_plugin.Plugin) fails.

Please see this Python code:
http://futuredesktop.com/tmp/test6.tar.gz

Why the issubclass(entry, cls) test fails?

if issubclass(entry, cls):
....print "Found a subclass: " + key
....subclasses.append(entry)

I can see that the class names are right, even objects of these classes
are ok.

My OS is Ubuntu 10.04.
$ python --version
Python 2.6.5

The actual, final product will be this
http://www.futuredesktop.com/clipart-applet/clipart-applet.ogv
It is a ClipArt query-engine and browser.
It can translate queries from "any" language to english ;-)

Most kindly
Osmo Antero Maatta
Grønland, Oslo

Hi,

I haven't looked closely at your problem, but are you aware of the
__subclasses__ class method? E.g.

This might help you.

 

[Peter Otten]

Hello,

Sub class test fails.
======================
I have a program that needs to load plugin-classes during runtime.

The program has these subdirectories (modules).

$ tree
.
`-- test.py
|
|-- plugins
| |-- base_plugin.py
| |-- base_plugin.pyc
| |-- __init__.py
| `-- oca
| |-- __init__.py
| |-- open_clipart.py


The plugins (sub directory) contains one or more plugin modules, in this
test-case there is only one; oca/open_clipart.py.

The plugins/base_plugin.py (contains class Plugin()) is a base class of
all plugins.

I want to list and load all plugin-classes (that inherit from
base_plugin.Plugin). I have a Python code that successfully browses the
plugins, but the the test issubclass(x, base_plugin.Plugin) fails.

Please see this Python code:
http://futuredesktop.com/tmp/test6.tar.gz

Why the issubclass(entry, cls) test fails?

base_plugin.py is imported twice, once as plugins.base_plugin and a second
time as base_plugin. You can see that when you add a bit more debug code
into your test.py:


--- a/test.py Sun Aug 29 12:57:25 2010 +0200
+++ b/test.py Sun Aug 29 12:58:25 2010 +0200
@@ -24,6 +24,8 @@
#print "entry's class<%s> -- cls's class=<%s>" %
(obj1.get_name(), obj2.get_name(), )

print "Check %s against %s" % (entry, cls, )
+ print entry.__bases__
+ print cls

if issubclass(entry, cls):
print "Found a subclass: " + key

$ python test.py
Importing plugins.base_plugin
Importing plugins.oca.open_clipart
Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
'plugins.base_plugin.Plugin'>
(<class 'base_plugin.Plugin'>,)
<class 'plugins.base_plugin.Plugin'>
Got subclasses= []

The solution is to remove all sys.path gymnastics and to adapt your import
statements accordingly:

$ hg diff
diff -r 9fe6129ba8fc plugins/oca/open_clipart.py
--- a/plugins/oca/open_clipart.py Sun Aug 29 12:51:51 2010 +0200
+++ b/plugins/oca/open_clipart.py Sun Aug 29 13:02:55 2010 +0200
@@ -2,9 +2,7 @@
import os
import threading, thread

-sys.path.insert(0, '..')
-import base_plugin
-#sys.path.insert(0, '../..')
+from .. import base_plugin

# ------------------------------------------
# class OpenClipArt
diff -r 9fe6129ba8fc test.py
--- a/test.py Sun Aug 29 12:51:51 2010 +0200
+++ b/test.py Sun Aug 29 13:02:55 2010 +0200
@@ -1,7 +1,6 @@
import os, sys
import inspect

-sys.path.insert(0, "plugins")

# Thanks to http://www.luckydonkey.com/2008/01/02/python-style-plugins-
made-easy/
def find_subclasses(path, cls):


$ python test.py
Importing plugins.base_plugin
Importing plugins.oca.open_clipart
Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
'plugins.base_plugin.Plugin'>
Found a subclass: OpenClipArt
Got subclasses= [<class 'plugins.oca.open_clipart.OpenClipArt'>]

You should also remove the __init__.py from the folder containing test.py
which is just begging for the same problem when you import your plugins as
test.plugins.whatever.

Peter

 

[Osmo Maatta]

Re-hi and thank you.
That solved my problem.

I can now see that the base_plugin.Plugin is loaded several times.
The numeric id(the_class) is not the same in all places.

Anyway, I thought that a class is always the same if it has been loaded
from the same module (in Linux/Unix; from the same file and inode). So
this experience was a bit confusing.

I need to learn more about how to set the search path (sys.path) right.
More Python!

Thanks
Osmo Antero
Grønland, Oslo

Hello,

Sub class test fails.
======================
I have a program that needs to load plugin-classes during runtime.

The program has these subdirectories (modules).

$ tree
.
`-- test.py
|
|-- plugins
| |-- base_plugin.py
| |-- base_plugin.pyc
| |-- __init__.py
| `-- oca
| |-- __init__.py
| |-- open_clipart.py


The plugins (sub directory) contains one or more plugin modules, in this
test-case there is only one; oca/open_clipart.py.

The plugins/base_plugin.py (contains class Plugin()) is a base class of
all plugins.

I want to list and load all plugin-classes (that inherit from
base_plugin.Plugin). I have a Python code that successfully browses the
plugins, but the the test issubclass(x, base_plugin.Plugin) fails.

Please see this Python code:
http://futuredesktop.com/tmp/test6.tar.gz

Why the issubclass(entry, cls) test fails?

base_plugin.py is imported twice, once as plugins.base_plugin and a second
time as base_plugin. You can see that when you add a bit more debug code
into your test.py:


--- a/test.py Sun Aug 29 12:57:25 2010 +0200
+++ b/test.py Sun Aug 29 12:58:25 2010 +0200
@@ -24,6 +24,8 @@
#print "entry's class<%s> -- cls's class=<%s>" %
(obj1.get_name(), obj2.get_name(), )

print "Check %s against %s" % (entry, cls, )
+ print entry.__bases__
+ print cls

if issubclass(entry, cls):
print "Found a subclass: " + key

$ python test.py
Importing plugins.base_plugin
Importing plugins.oca.open_clipart
Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
'plugins.base_plugin.Plugin'>
(<class 'base_plugin.Plugin'>,)
<class 'plugins.base_plugin.Plugin'>
Got subclasses= []

The solution is to remove all sys.path gymnastics and to adapt your import
statements accordingly:

$ hg diff
diff -r 9fe6129ba8fc plugins/oca/open_clipart.py
--- a/plugins/oca/open_clipart.py Sun Aug 29 12:51:51 2010 +0200
+++ b/plugins/oca/open_clipart.py Sun Aug 29 13:02:55 2010 +0200
@@ -2,9 +2,7 @@
import os
import threading, thread

-sys.path.insert(0, '..')
-import base_plugin
-#sys.path.insert(0, '../..')
+from .. import base_plugin

# ------------------------------------------
# class OpenClipArt
diff -r 9fe6129ba8fc test.py
--- a/test.py Sun Aug 29 12:51:51 2010 +0200
+++ b/test.py Sun Aug 29 13:02:55 2010 +0200
@@ -1,7 +1,6 @@
import os, sys
import inspect

-sys.path.insert(0, "plugins")

# Thanks to http://www.luckydonkey.com/2008/01/02/python-style-plugins-
made-easy/
def find_subclasses(path, cls):


$ python test.py
Importing plugins.base_plugin
Importing plugins.oca.open_clipart
Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
'plugins.base_plugin.Plugin'>
Found a subclass: OpenClipArt
Got subclasses= [<class 'plugins.oca.open_clipart.OpenClipArt'>]

You should also remove the __init__.py from the folder containing test.py
which is just begging for the same problem when you import your plugins as
test.plugins.whatever.

Peter