plugin development best practices

[Flavio]

Hi,

Nowadays the addition of functionality to programs by means of
plugins is very frequent.

I want to know the opinions of experienced Python developers about the
best practices when it comes to developing a plugin system for a
Python package.

Should plugins be modules in a separate package?
Should there be a registry of available plugins? how would such a
registry be implemented? etc.

thanks,

Flávio

 

[Diez B. Roggisch]

Hi,

Nowadays the addition of functionality to programs by means of
plugins is very frequent.

I want to know the opinions of experienced Python developers about the
best practices when it comes to developing a plugin system for a
Python package.

Should plugins be modules in a separate package?
Should there be a registry of available plugins? how would such a
registry be implemented? etc.

There have been a gazillion discussions about this on this newsgroup/mailing
list. Searching the archives will get you to them.

The dynamic nature of python makes a whole range of options available.
Depending on what you want to do (how are the plugins made available and so
on), one or the other might be preferable.

One relatively current development is the usage of setuptools "entry-points"
feature, which is precisely made for discovering installed plugins:

http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-of-services-and-plugins


Diez

 

[Flavio]

There have been a gazillion discussions about this on this newsgroup/mailing
list. Searching the archives will get you to them.

The dynamic nature of python makes a whole range of options available.
Depending on what you want to do (how are the plugins made available and so
on), one or the other might be preferable.

One relatively current development is the usage of setuptools "entry-points"
feature, which is precisely made for discovering installed plugins:

http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-...

Diez

Thanks Diez,

I will search the archives. I am aware of the setuptools feature It is
quite nice. But the documentation is not very clear on how to define
"entry point groups" on the importing end, i.e. how to prepare a an
application to accept plugins it doesn't even now exist.

Flavio

 

[Diez B. Roggisch]

I will search the archives. I am aware of the setuptools feature It is
quite nice. But the documentation is not very clear on how to define
"entry point groups" on the importing end, i.e. how to prepare a an
application to accept plugins it doesn't even now exist.

The entry-points are just a method of discovery. Writing a plugin by
itself - if that is what you are talking about here - is usually done by
exposing a certain interface, and invoking some registration
functionality - after the plugin is discovered, that is.


Diez

 

[Flavio]

Best practice may be to not develop a new plugin system. There are quite a
few available already. Most likely, at least one of them is suitable for your
application.

Here are a couple starting points:

http://twistedmatrix.com/projects/core/documentation/howto/plugin.html

http://peak.telecommunity.com/DevCenter/setuptools#dynamic-discovery-...

Jean-Paul

The plugin system provided by twisted looks interesting (though not as
simple as it could be, IMHO). Its main problem is that it introduces
two dependencies : Twisted plugin and ZopeInterface. I think a
functional plugin system could(and should) be done using only the
standard library.

Flávio

 

[Flavio]

Hi,

Nowadays the addition of functionality to programs by means of
plugins is very frequent.

I want to know the opinions of experienced Python developers about the
best practices when it comes to developing apluginsystemfor a
Python package.

Should plugins be modules in a separate package?
Should there be a registry of available plugins? how would such a
registry be implemented? etc.

thanks,

Flávio

let me extend my original question with an example:

Simple plugin system proposal:

have a package (directory with __init__.py) called plugins where the
actual plugins are modules in this directory.

When the main script imports the plugins package, all plugin modules
would be available as plugins.pluginA, plugins.pluginB , etc.

A registry of available plugins would be available as a simple
dir(plugins).

code in the main script than wished to use a given plugin, would only
have to look in the registry before calling any code from a given
plugin.

What is wrong/missing with this simple framework?

I'd appreciate any comments.

Flávio

 

[Diez B. Roggisch]

Simple plugin system proposal:

have a package (directory with __init__.py) called plugins where the
actual plugins are modules in this directory.

When the main script imports the plugins package, all plugin modules
would be available as plugins.pluginA, plugins.pluginB , etc.

A registry of available plugins would be available as a simple
dir(plugins).

code in the main script than wished to use a given plugin, would only
have to look in the registry before calling any code from a given
plugin.

What is wrong/missing with this simple framework?

Nothing wrong. It's just one way of doing it. But it requires you to have
all plugins being part of one module, in one location. Depending on what
you want to do, this won't suffice. For example if your app is installed in
a system path you aren't supposed to write to - how do you install your
individual plugin? easy_install allows you to install to a folder that is
contained in the PYTHONPATH, and then you can discover entrypoints.

But please, do as we suggested: read the past discussions.

Depending on what _you_ actually want to accomplish, you're proposal is
enough. But don't expect it to become the "one plugin system to rule them
all"-solution.

diez

 

[Flavio]

Nothing wrong. It's just one way of doing it. But it requires you to have
all plugins being part of one module, in one location. Depending on what
you want to do, this won't suffice. For example if your app is installed in
a system path you aren't supposed to write to - how do you install your
individual plugin? easy_install allows you to install to a folder that is
contained in the PYTHONPATH, and then you can discover entrypoints.

But please, do as we suggested: read the past discussions.

Depending on what _you_ actually want to accomplish, you're proposal is
enough. But don't expect it to become the "one plugin system to rule them
all"-solution.

diez

Oh, I have read all the links that have been suggested, but I am
looking for the simplest possible solution.

I have no intention to create the "next Plugin system" I am just
trying to solve my own problem and in the process leave a record of a
fruitfull discussion about plugin systems.

BTW I have yet to check TRAC's plugin system.

 

[Flavio]

Oh, I have read all the links that have been suggested, but I am
looking for the simplest possible solution.

I have no intention to create the "next Plugin system" I am just
trying to solve my own problem and in the process leave a record of a
fruitfull discussion about plugin systems.

BTW I have yet to check TRAC's plugin system.

I have look at Trac's component based pluging system and I liked it
very much. If external dependencies is not an issue I believe this is
the best solution.
http://trac.edgewall.org/wiki/TracPlugins