Finding and loading subclasses dynamically. issubclass(x,base_plugin.Plugin) fails.

Discussion in 'Python' started by Osmo Maatta, Aug 29, 2010.

  1. Osmo Maatta

    Osmo Maatta Guest

    Hello,

    Sub class test fails.
    ======================
    I have a program that needs to load plugin-classes during runtime.

    The program has these subdirectories (modules).

    $ tree
    ..
    `-- test.py
    |
    |-- plugins
    | |-- base_plugin.py
    | |-- base_plugin.pyc
    | |-- __init__.py
    | `-- oca
    | |-- __init__.py
    | |-- open_clipart.py


    The plugins (sub directory) contains one or more plugin modules, in this
    test-case there is only one; oca/open_clipart.py.

    The plugins/base_plugin.py (contains class Plugin()) is a base class of
    all plugins.

    I want to list and load all plugin-classes (that inherit from
    base_plugin.Plugin). I have a Python code that successfully browses the
    plugins, but the the test issubclass(x, base_plugin.Plugin) fails.

    Please see this Python code:
    http://futuredesktop.com/tmp/test6.tar.gz

    Why the issubclass(entry, cls) test fails?

    if issubclass(entry, cls):
    .....print "Found a subclass: " + key
    .....subclasses.append(entry)

    I can see that the class names are right, even objects of these classes
    are ok.

    My OS is Ubuntu 10.04.
    $ python --version
    Python 2.6.5

    The actual, final product will be this
    http://www.futuredesktop.com/clipart-applet/clipart-applet.ogv
    It is a ClipArt query-engine and browser.
    It can translate queries from "any" language to english ;-)

    Most kindly
    Osmo Antero Maatta
    Grønland, Oslo
     
    Osmo Maatta, Aug 29, 2010
    #1
    1. Advertising

  2. Re: Finding and loading subclasses dynamically. issubclass(x, base_plugin.Plugin) fails.

    Osmo Maatta <> writes:

    > Hello,
    >
    > Sub class test fails.
    > ======================
    > I have a program that needs to load plugin-classes during runtime.
    >
    > The program has these subdirectories (modules).
    >
    > $ tree
    > .
    > `-- test.py
    > |
    > |-- plugins
    > | |-- base_plugin.py
    > | |-- base_plugin.pyc
    > | |-- __init__.py
    > | `-- oca
    > | |-- __init__.py
    > | |-- open_clipart.py
    >
    >
    > The plugins (sub directory) contains one or more plugin modules, in this
    > test-case there is only one; oca/open_clipart.py.
    >
    > The plugins/base_plugin.py (contains class Plugin()) is a base class of
    > all plugins.
    >
    > I want to list and load all plugin-classes (that inherit from
    > base_plugin.Plugin). I have a Python code that successfully browses the
    > plugins, but the the test issubclass(x, base_plugin.Plugin) fails.
    >
    > Please see this Python code:
    > http://futuredesktop.com/tmp/test6.tar.gz
    >
    > Why the issubclass(entry, cls) test fails?
    >
    > if issubclass(entry, cls):
    > ....print "Found a subclass: " + key
    > ....subclasses.append(entry)
    >
    > I can see that the class names are right, even objects of these classes
    > are ok.
    >
    > My OS is Ubuntu 10.04.
    > $ python --version
    > Python 2.6.5
    >
    > The actual, final product will be this
    > http://www.futuredesktop.com/clipart-applet/clipart-applet.ogv
    > It is a ClipArt query-engine and browser.
    > It can translate queries from "any" language to english ;-)
    >
    > Most kindly
    > Osmo Antero Maatta
    > Grønland, Oslo


    Hi,

    I haven't looked closely at your problem, but are you aware of the
    __subclasses__ class method? E.g.

    >>> class A(object): pass

    ....
    >>> class B(A): pass

    ....
    >>> class C(A): pass

    ....
    >>> A.__subclasses__()

    [<class '__main__.B'>, <class '__main__.C'>]
    >>>


    This might help you.

    --
    Arnaud
     
    Arnaud Delobelle, Aug 29, 2010
    #2
    1. Advertising

  3. Osmo Maatta

    Peter Otten Guest

    Re: Finding and loading subclasses dynamically. issubclass(x, base_plugin.Plugin) fails.

    Osmo Maatta wrote:

    > Hello,
    >
    > Sub class test fails.
    > ======================
    > I have a program that needs to load plugin-classes during runtime.
    >
    > The program has these subdirectories (modules).
    >
    > $ tree
    > .
    > `-- test.py
    > |
    > |-- plugins
    > | |-- base_plugin.py
    > | |-- base_plugin.pyc
    > | |-- __init__.py
    > | `-- oca
    > | |-- __init__.py
    > | |-- open_clipart.py
    >
    >
    > The plugins (sub directory) contains one or more plugin modules, in this
    > test-case there is only one; oca/open_clipart.py.
    >
    > The plugins/base_plugin.py (contains class Plugin()) is a base class of
    > all plugins.
    >
    > I want to list and load all plugin-classes (that inherit from
    > base_plugin.Plugin). I have a Python code that successfully browses the
    > plugins, but the the test issubclass(x, base_plugin.Plugin) fails.
    >
    > Please see this Python code:
    > http://futuredesktop.com/tmp/test6.tar.gz
    >
    > Why the issubclass(entry, cls) test fails?


    base_plugin.py is imported twice, once as plugins.base_plugin and a second
    time as base_plugin. You can see that when you add a bit more debug code
    into your test.py:


    --- a/test.py Sun Aug 29 12:57:25 2010 +0200
    +++ b/test.py Sun Aug 29 12:58:25 2010 +0200
    @@ -24,6 +24,8 @@
    #print "entry's class<%s> -- cls's class=<%s>" %
    (obj1.get_name(), obj2.get_name(), )

    print "Check %s against %s" % (entry, cls, )
    + print entry.__bases__
    + print cls

    if issubclass(entry, cls):
    print "Found a subclass: " + key

    $ python test.py
    Importing plugins.base_plugin
    Importing plugins.oca.open_clipart
    Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
    'plugins.base_plugin.Plugin'>
    (<class 'base_plugin.Plugin'>,)
    <class 'plugins.base_plugin.Plugin'>
    Got subclasses= []

    The solution is to remove all sys.path gymnastics and to adapt your import
    statements accordingly:

    $ hg diff
    diff -r 9fe6129ba8fc plugins/oca/open_clipart.py
    --- a/plugins/oca/open_clipart.py Sun Aug 29 12:51:51 2010 +0200
    +++ b/plugins/oca/open_clipart.py Sun Aug 29 13:02:55 2010 +0200
    @@ -2,9 +2,7 @@
    import os
    import threading, thread

    -sys.path.insert(0, '..')
    -import base_plugin
    -#sys.path.insert(0, '../..')
    +from .. import base_plugin

    # ------------------------------------------
    # class OpenClipArt
    diff -r 9fe6129ba8fc test.py
    --- a/test.py Sun Aug 29 12:51:51 2010 +0200
    +++ b/test.py Sun Aug 29 13:02:55 2010 +0200
    @@ -1,7 +1,6 @@
    import os, sys
    import inspect

    -sys.path.insert(0, "plugins")

    # Thanks to http://www.luckydonkey.com/2008/01/02/python-style-plugins-
    made-easy/
    def find_subclasses(path, cls):


    $ python test.py
    Importing plugins.base_plugin
    Importing plugins.oca.open_clipart
    Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
    'plugins.base_plugin.Plugin'>
    Found a subclass: OpenClipArt
    Got subclasses= [<class 'plugins.oca.open_clipart.OpenClipArt'>]

    You should also remove the __init__.py from the folder containing test.py
    which is just begging for the same problem when you import your plugins as
    test.plugins.whatever.

    Peter
     
    Peter Otten, Aug 29, 2010
    #3
  4. Osmo Maatta

    Osmo Maatta Guest

    Re-hi and thank you.
    That solved my problem.

    I can now see that the base_plugin.Plugin is loaded several times.
    The numeric id(the_class) is not the same in all places.

    Anyway, I thought that a class is always the same if it has been loaded
    from the same module (in Linux/Unix; from the same file and inode). So
    this experience was a bit confusing.

    I need to learn more about how to set the search path (sys.path) right.
    More Python!

    Thanks
    Osmo Antero
    Grønland, Oslo


    On 08/29/2010 01:06 PM, Peter Otten wrote:
    > Osmo Maatta wrote:
    >
    >> Hello,
    >>
    >> Sub class test fails.
    >> ======================
    >> I have a program that needs to load plugin-classes during runtime.
    >>
    >> The program has these subdirectories (modules).
    >>
    >> $ tree
    >> .
    >> `-- test.py
    >> |
    >> |-- plugins
    >> | |-- base_plugin.py
    >> | |-- base_plugin.pyc
    >> | |-- __init__.py
    >> | `-- oca
    >> | |-- __init__.py
    >> | |-- open_clipart.py
    >>
    >>
    >> The plugins (sub directory) contains one or more plugin modules, in this
    >> test-case there is only one; oca/open_clipart.py.
    >>
    >> The plugins/base_plugin.py (contains class Plugin()) is a base class of
    >> all plugins.
    >>
    >> I want to list and load all plugin-classes (that inherit from
    >> base_plugin.Plugin). I have a Python code that successfully browses the
    >> plugins, but the the test issubclass(x, base_plugin.Plugin) fails.
    >>
    >> Please see this Python code:
    >> http://futuredesktop.com/tmp/test6.tar.gz
    >>
    >> Why the issubclass(entry, cls) test fails?

    >
    > base_plugin.py is imported twice, once as plugins.base_plugin and a second
    > time as base_plugin. You can see that when you add a bit more debug code
    > into your test.py:
    >
    >
    > --- a/test.py Sun Aug 29 12:57:25 2010 +0200
    > +++ b/test.py Sun Aug 29 12:58:25 2010 +0200
    > @@ -24,6 +24,8 @@
    > #print "entry's class<%s> -- cls's class=<%s>" %
    > (obj1.get_name(), obj2.get_name(), )
    >
    > print "Check %s against %s" % (entry, cls, )
    > + print entry.__bases__
    > + print cls
    >
    > if issubclass(entry, cls):
    > print "Found a subclass: " + key
    >
    > $ python test.py
    > Importing plugins.base_plugin
    > Importing plugins.oca.open_clipart
    > Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
    > 'plugins.base_plugin.Plugin'>
    > (<class 'base_plugin.Plugin'>,)
    > <class 'plugins.base_plugin.Plugin'>
    > Got subclasses= []
    >
    > The solution is to remove all sys.path gymnastics and to adapt your import
    > statements accordingly:
    >
    > $ hg diff
    > diff -r 9fe6129ba8fc plugins/oca/open_clipart.py
    > --- a/plugins/oca/open_clipart.py Sun Aug 29 12:51:51 2010 +0200
    > +++ b/plugins/oca/open_clipart.py Sun Aug 29 13:02:55 2010 +0200
    > @@ -2,9 +2,7 @@
    > import os
    > import threading, thread
    >
    > -sys.path.insert(0, '..')
    > -import base_plugin
    > -#sys.path.insert(0, '../..')
    > +from .. import base_plugin
    >
    > # ------------------------------------------
    > # class OpenClipArt
    > diff -r 9fe6129ba8fc test.py
    > --- a/test.py Sun Aug 29 12:51:51 2010 +0200
    > +++ b/test.py Sun Aug 29 13:02:55 2010 +0200
    > @@ -1,7 +1,6 @@
    > import os, sys
    > import inspect
    >
    > -sys.path.insert(0, "plugins")
    >
    > # Thanks to http://www.luckydonkey.com/2008/01/02/python-style-plugins-
    > made-easy/
    > def find_subclasses(path, cls):
    >
    >
    > $ python test.py
    > Importing plugins.base_plugin
    > Importing plugins.oca.open_clipart
    > Check <class 'plugins.oca.open_clipart.OpenClipArt'> against <class
    > 'plugins.base_plugin.Plugin'>
    > Found a subclass: OpenClipArt
    > Got subclasses= [<class 'plugins.oca.open_clipart.OpenClipArt'>]
    >
    > You should also remove the __init__.py from the folder containing test.py
    > which is just begging for the same problem when you import your plugins as
    > test.plugins.whatever.
    >
    > Peter
    >
     
    Osmo Maatta, Aug 29, 2010
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris Green

    when does issubclass fail?

    Chris Green, Apr 16, 2004, in forum: Python
    Replies:
    1
    Views:
    304
    =?ISO-8859-1?Q?Walter_D=F6rwald?=
    Apr 20, 2004
  2. Nathan Bullock

    How do you make issubclass work

    Nathan Bullock, Sep 11, 2004, in forum: Python
    Replies:
    4
    Views:
    640
    Alex Martelli
    Sep 11, 2004
  3. kj

    issubclass(dict, Mapping)

    kj, Dec 22, 2010, in forum: Python
    Replies:
    10
    Views:
    545
  4. kj
    Replies:
    5
    Views:
    418
  5. Replies:
    1
    Views:
    158
    Steven D'Aprano
    May 31, 2012
Loading...

Share This Page