Plugin architectures

[Ross]

What are the best strategies for creating a Java application with a
plugin architecture.

My current plan is to have a directory of plugins, to read the files
manually, use ClassLoader.defineClass to create the class file,
Class.newInstance() to make one, and of course a number of interfaces
that plugins can implement, one for each plugin type.

Are there any better strategies that people would recommend? I don't
want to add the plugin directory to the classpath, as that would make
installation of the software more tricky than just putting a .jar file
somewhere and double clicking on it.

 

[markspace]

What are the best strategies for creating a Java application with a
plugin architecture.

My current plan is to have a directory of plugins, to read the files
manually, use ClassLoader.defineClass to create the class file,
Class.newInstance() to make one, and of course a number of interfaces
that plugins can implement, one for each plugin type.

Are there any better strategies that people would recommend? I don't
want to add the plugin directory to the classpath, as that would make
installation of the software more tricky than just putting a .jar file
somewhere and double clicking on it.

If you are using a Jar file, it wouldn't look at the class path anyway.
You have to set the Class-path property in the manifest file of the
Jar file to change the class path of a Jar.

If you add "." to the class path on the manifest, then the regular old
class loader will look in the Jar's home directory for resources, and
classes. If you add "plugin" (I don't think a / is required, although
it might be), the class loader will also look in a directory of the same
name in the home directory of the Jar file.

You may have to fuss around a bit to get Jars loaded instead of loose
classes, and to devise a good naming scheme that works for you and isn't
too hard on your clients. I haven't actually implemented such a thing
so I can't really offer any advice.

 

[Ross]

If you are using a Jar file, it wouldn't look at the class path anyway.
You have to set the Class-path property in the manifest file of the
Jar file to change the class path of a Jar.

If you add "." to the class path on the manifest, then the regular old
class loader will look in the Jar's home directory for resources, and
classes. If you add "plugin" (I don't think a / is required, although
it might be), the class loader will also look in a directory of the same
name in the home directory of the Jar file.

You may have to fuss around a bit to get Jars loaded instead of loose
classes, and to devise a good naming scheme that works for you and isn't
too hard on your clients. I haven't actually implemented such a thing
so I can't really offer any advice.

That's an interesting way of doing it. I think I'm going to try both
and then choose the best solution. I've never set a classpath in a
manifest file, I really should do. There's a number of jar related
functions I've never gotten around to trying, e.g. storing images and
other resources in jar files.

 

[John B. Matthews]

What are the best strategies for creating a Java application with a
plugin architecture.

My current plan is to have a directory of plugins, to read the files
manually, use ClassLoader.defineClass to create the class file,
Class.newInstance() to make one, and of course a number of interfaces
that plugins can implement, one for each plugin type.

Are there any better strategies that people would recommend? I don't
want to add the plugin directory to the classpath, as that would make
installation of the software more tricky than just putting a .jar
file somewhere and double clicking on it.

IIRC, this fractal project has a similar plugin architecture that
modifies templates written in Java and compiles them _in_situ_:

<http://sourceforge.net/projects/benojt/>

Arachnophilia invokes fixed-signature methods by name, apparently using
introspection:

<http://vps.arachnoid.com/arachnophilia/Documentation/
JavaCustomClassInterface.html>

Here's a platform-specific example:

<http://home.roadrunner.com/~jbmatthews/java/macarach.html>

 

[Ross]

IIRC, this fractal project has a similar plugin architecture that
modifies templates written in Java and compiles them _in_situ_:

<http://sourceforge.net/projects/benojt/>

Arachnophilia invokes fixed-signature methods by name, apparently using
introspection:

<http://vps.arachnoid.com/arachnophilia/Documentation/
JavaCustomClassInterface.html>

Here's a platform-specific example:

<http://home.roadrunner.com/~jbmatthews/java/macarach.html>

Those are both very interesting technological solutions. E.g
Arachnophilia allows objects to be treated as satisfying an interface,
even if it formally doesn't. I've done something similar in
writing a compatibility layer for Max/MSP Java objects that route
messages to matching methods. (Using Introspection).

But I'm not sure I need to be quite that sophisticated in my current
application. Either loading the classes as bytes as in my first
solution, or using a .jar file relative class paths and Class.forName
() are simpler, and suitable, solutions.

 

[Dave Searles]

If you are using a Jar file, it wouldn't look at the class path anyway.
You have to set the Class-path property in the manifest file of the Jar
file to change the class path of a Jar.

If you add "." to the class path on the manifest, then the regular old
class loader will look in the Jar's home directory for resources, and
classes. If you add "plugin" (I don't think a / is required, although
it might be), the class loader will also look in a directory of the same
name in the home directory of the Jar file.

You may have to fuss around a bit to get Jars loaded instead of loose
classes, and to devise a good naming scheme that works for you and isn't
too hard on your clients. I haven't actually implemented such a thing
so I can't really offer any advice.

I'd dispense with the reflection and newInstance crap too. Instead, each
plugin would come as .class files and a .obj file. Plugin enumeration
consists of finding all the .obj files in ./plugins, then reading them
as serialized Java objects and casting to Plugin. The Plugin
class/interface defines methods that allow querying the plugin, and
appropriate other methods. For example a codec might have a code and a
decode method, and an added document type document factory methods to
produce document objects of some class or another. If a plugin is
installed incorrectly, either it doesn't show up (.obj file missing) or
deserializing the .obj doesn't work (some .class is missing or broken
and ObjectInputStream throws ClassNotFoundException or
InvalidClassException, or the .obj is broken and one gets
StreamCorruptedException or plain old IOException).

The plugins of course can have nice installers, or just come as zips to
be unzipped into the plugins directory.

The closest this comes to reflection is the use of serialization and a
cast. It makes for much nicer code that just does I/O and then has
factory/etc. objects in a list somewhere.

 

[Tom Anderson]

What are the best strategies for creating a Java application with a
plugin architecture.

The officially supported one.

One! You define an interface for your plugins.

Two! You package plugin implementations in JAR files, using the Service
Provider layout:

http://java.sun.com/javase/6/docs/technotes/guides/jar/jar.html#Service Provider

Three! You put JARs on the classpath. Under java 6, the best way to do
this, IMHO, is with a wildcarded relative path in the JAR's Class-Path
attribute; something like:

Class-Path: plugins/*

That will add all JARs in the plugins directory adjacent to your app JAR
to the classpath.

On pre-6 javas, there are no wildcards, so the only way to add multiple
JARs is by explicitly listing them in the Class-Path, which is lame, or
putting them in the JRE's ext directory, which is lame, or by somehow
merging JARs at runtime, which is lame. Or some classloader trickery,
which is not lame, but is voodoo. You pays your money and you takes your
choice.

Four! You look the plugins up using java.util.ServiceLoader:

http://java.sun.com/javase/6/docs/api/java/util/ServiceLoader.html

Unless you're on a pre-6 java, in which case you use the celebrated but
undocumented sun.misc.Service (which has been there since 1.3) - consult
javap, google, and your pineal gland for details.

Or the ServiceRegistry from ImageIO (which has been there since 1.4), even
though you're not doing image IO:

http://java.sun.com/javase/6/docs/api/javax/imageio/spi/ServiceRegistry.html

tom

 

[markspace]

That's an interesting way of doing it. I think I'm going to try both
and then choose the best solution. I've never set a classpath in a
manifest file, I really should do. There's a number of jar related
functions I've never gotten around to trying, e.g. storing images and
other resources in jar files.

If you make a short test program like this

public class Main {

public static void main( String... args ) {

URL url = Main.class.getResource( "/test" );
System.out.println( url );
url = Main.class.getResource( "/testdir/test2" );
System.out.println( url );
}
}

You can play around with the Class-path property and make files on the
file system to see what it finds and where. Handy to test things out
quickly.

 

[Arne Vajhøj]

What are the best strategies for creating a Java application with a
plugin architecture.

My current plan is to have a directory of plugins, to read the files
manually, use ClassLoader.defineClass to create the class file,
Class.newInstance() to make one, and of course a number of interfaces
that plugins can implement, one for each plugin type.

Are there any better strategies that people would recommend? I don't
want to add the plugin directory to the classpath, as that would make
installation of the software more tricky than just putting a .jar file
somewhere and double clicking on it.

If the application is for developers then: create an interface, let
all plugins implement that interface and load using Spring.

For end-users you will need some GUI stuff and then your code
can do everything necessary.

Arne

 

[Arne Vajhøj]

The Eclipse IDE is written in Java and uses plugins quite heavily.
The Eclipse developers have found it wise to use the OSGi framework
for that. OSGi is a published specification for some services;
several implementations exist. The one which Eclipse uses is called
Equinox and is open-source. See: http://www.eclipse.org/equinox/

It is quite plausible that OSGi does much more than what you need. The
main non-trivial thing that OSGi does is that it handles live updates of
code (i.e. upgrading a plugin without stopping the application). Not
every application really needs that...

(I have not used OSGi extensively myself; I am just reporting that
Eclipse uses it and its developers appear to be happy with it.)

But it only works well with the GUI update/install feature.

Unpacking something in the plugin directory usually goes wrong.

Arne

 

[Ross]

Can a java program temporary add directories and .jar files to its own
classpath. Surely there's some method in ClassLoader for this?
Hmmm..... can't see one. Anybody?

 

[Eric Sosman]

Can a java program temporary add directories and .jar files to its own
classpath. Surely there's some method in ClassLoader for this?
Hmmm..... can't see one. Anybody?

"The classpath" is just a list of places where some of
the default ClassLoaders hunt for classes. If you want to
look in unusual places, write a ClassLoader that searches them.
(Note that you can delegate most of the heavy lifting.)

Or have I misunderstood your question?

 

[Ross]

"The classpath" is just a list of places where some of
the default ClassLoaders hunt for classes. If you want to
look in unusual places, write a ClassLoader that searches them.
(Note that you can delegate most of the heavy lifting.)

Or have I misunderstood your question?

I'm sure you've understood my question perfectly. I was thinking that
writing a custom classloader would be a potential solution.

Though, for my current need, I'm probably going to make it simpler
than this.

 

[markspace]

I'm sure you've understood my question perfectly. I was thinking that
writing a custom classloader would be a potential solution.

Though, for my current need, I'm probably going to make it simpler
than this.

Certain types of existing class loaders allow you to add to its search
path. For example,

ClassLoader cl = MyClass.class.getClassLoader();
if( cl instanceof URLClassLoader ) {
((URLClassLoader)cl).addURL( urlPath );
}

would do it if your current class loader happens to be a URLClassloader.
I have no idea what the normal default type of a class loader is though.

 

[markspace]

Certain types of existing class loaders allow you to add to its search
path. For example,

ClassLoader cl = MyClass.class.getClassLoader();
if( cl instanceof URLClassLoader ) {
((URLClassLoader)cl).addURL( urlPath );
}

would do it if your current class loader happens to be a URLClassloader.
I have no idea what the normal default type of a class loader is though.

A quick test reveals that there is some likelihood that you already have
a URLClassLoader:

<code>
public class ClassLoaderTest {

public static void main( String[] args )
{
ClassLoader cl = ClassLoaderTest.class.getClassLoader();
System.out.println( cl );
if( cl instanceof URLClassLoader ) {
System.out.println( " is a URLClassLoader" );
}
}
}
</code>
<output>
run:
sun.misc.Launcher$AppClassLoader@fabe9
is a URLClassLoader
BUILD SUCCESSFUL (total time: 1 second)
</output>


A quickie solution might be to count on having a URLClassLoader, then
just disable the plugin technology if for some reason you don't. Once
you get feedback from the field, you'll have an idea whether it would be
worth it to develop something more complex, or if the default program
behavior is good enough.

Start small, build up. Don't write code until you have to. Laziness is
a virtue. Gold-plating is an anti-pattern, and many of these
suggestions on this thread strike me as gold-plating.

 

[Dagon]

A quickie solution might be to count on having a URLClassLoader, then
just disable the plugin technology if for some reason you don't.

If the plugin system is an important product feature, it's not _THAT_ much
extra work to have your main class instantiate a known classloader under your
control and use that to load your application. Or to have your addon system
use a new classloader that has _ONLY_ the addons in it's path. Addons
implement an Interface defined in the app classloader, so the addon loader can
instantiate objects and cast them to something usable by main app code.

You'll have to do this (and your classloader will need to be customized,
likely as an extension of URLClassloader) as soon as you need additional
features like not locking addon jars on dumb OSs, or class reloading, or addon
validation before load, or other fancy things.

Start small, build up. Don't write code until you have to. Laziness is
a virtue. Gold-plating is an anti-pattern, and many of these
suggestions on this thread strike me as gold-plating.

Agreed. Get the base working early and work from there. Some of the
gold-plating will be structurally needed, though, so don't do too much work
avoiding it. Balance brings peace

 

[Arne Vajhøj]

Can a java program temporary add directories and .jar files to its own
classpath. Surely there's some method in ClassLoader for this?

You can create a new URLClassLoader.

But the most common is that the jar files are those in
classpath at startup and adding requires restarting.

Arne