Confused about class loaders

[Matthew Keene]

I'm trying to dynamically add some libraries to the application class
path by using a URLClassLoader, but it doesn't seem to be behaving the
way that I want it to. I have to classes as follows:

import java.lang.reflect.* ;
import java.io.* ;
import java.net.* ;

public class ClassLoaderTestParent {

public ClassLoaderTestParent() {
}

public static void main(String[] args) throws Exception {

File f = new
File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
URL u = f.toURL() ;
URL[] urls = {u} ;
URLClassLoader ucl = new URLClassLoader(urls) ;
Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
// Also tried using forName, as below
//Class myClass = Class.forName("ClassLoaderTestChild",true,ucl) ;
Object myInstance = myClass.newInstance() ;
Class[] methodArgClass = {} ;
Method myMethod = myClass.getMethod("childMethod",methodArgClass) ;
Object[] methodArgVals = {} ;
System.out.println("About to invoke method on dynamically loaded
class") ;
Object result = myMethod.invoke(myInstance,methodArgVals) ;
}
}

This class will create a URL class loader with the specified jar file
added to the classpath, then load the desired class, instantiate it
and then dynamically invoke a method on the instance. The class which
is invoked is shown below

import gnu.regexp.* ;

public class ClassLoaderTestChild {

public ClassLoaderTestChild() {
}

public String childMethod() throws Exception {

System.out.println("Inside child method") ;
System.out.println("Class path is " +
System.getProperty("java.class.path")) ;
RE testRE = new RE("abcde") ;
return "Hello" ;

}
}

When I run this, I expect that the GNU regexp libraries should be
available to the child class, as I added them to the class loader in
the parent. Instead, I get a NoClassDefFoundError, as shown below

H:\Data\Java>java -classpath . ClassLoaderTestParent
About to invoke method on dynamically loaded class
Inside child method
Class path is .
Exception in thread "main"
java.lang.reflect.InvocationTargetException:
java.lang.NoClassDefFoundError: gnu/regexp/RE
at ClassLoaderTestChild.childMethod(ClassLoaderTestChild.java:12)
at java.lang.reflect.Method.invoke(Native Method)
at ClassLoaderTestParent.main(ClassLoaderTestParent.java:24)

Can anybody explain this behaviour, and even better, tell me what I
would have to do to get the behaviour I'm after (ie to have the
dynamically added class libraries available to the loaded class) ?

TIA

Matthew

 

[John C. Bollinger]

I'm trying to dynamically add some libraries to the application class
path by using a URLClassLoader, but it doesn't seem to be behaving the
way that I want it to. I have to classes as follows:

import java.lang.reflect.* ;
import java.io.* ;
import java.net.* ;

public class ClassLoaderTestParent {

public ClassLoaderTestParent() {
}

public static void main(String[] args) throws Exception {

File f = new
File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
URL u = f.toURL() ;
URL[] urls = {u} ;
URLClassLoader ucl = new URLClassLoader(urls) ;
Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
[...]

}

This class will create a URL class loader with the specified jar file
added to the classpath, then load the desired class, instantiate it
and then dynamically invoke a method on the instance. The class which
is invoked is shown below
[...]

When I run this, I expect that the GNU regexp libraries should be
available to the child class, as I added them to the class loader in
the parent. Instead, I get a NoClassDefFoundError, as shown below
[...]

Can anybody explain this behaviour, and even better, tell me what I
would have to do to get the behaviour I'm after (ie to have the
dynamically added class libraries available to the loaded class) ?

The class ClassLoaderTestChild is available in the application's class
path. It is therefore loaded by the system ClassLoader (to which the
URLClassLoader delegates) instead of by the URLClassLoader itself. As a
result, when the ClassLoaderTestChild instance requires the RE class the
system uses the system ClassLoader (the ClassLoader of
ClassLoaderTestChild) to attempt to load it.

Are you trying to support multiple regex packages? Otherwise why not
just put the gnu-regex jar in the system classpath?


John Bollinger
(e-mail address removed)

 

[Matthew Keene]

The class ClassLoaderTestChild is available in the application's class
path. It is therefore loaded by the system ClassLoader (to which the
URLClassLoader delegates) instead of by the URLClassLoader itself. As a
result, when the ClassLoaderTestChild instance requires the RE class the
system uses the system ClassLoader (the ClassLoader of
ClassLoaderTestChild) to attempt to load it.

Right, I think I understand. So does this mean that if I create the
URLClassLoader with a null parent that the system will be forced to use
my class loader and will therefore inherit the classpath associated with
my class loader (in this case I would probably add the libraries from
the original class path into the new class loader). Is there any reason
I'm not aware of that this would be a bad idea ?

Are you trying to support multiple regex packages? Otherwise why not
just put the gnu-regex jar in the system classpath?

I'm trying to write a framework that will allow for custom functionality
to be plugged in at various points of the processing. Because I don't
know from the outset what the requirements of these plugins will be (and
because I don't directly have control of the system classpath - don't
ask) I'm trying to specify additional libraries in the config file which
defines the plugin and load them dynamically when I load the plugin.
Regex was just a simple example, in fact the most common example will be
JDBC drivers.

John Bollinger
(e-mail address removed)

Thanks very much for your help so far.

 

[Matthew Keene]

Right, I think I understand. So does this mean that if I create the

URLClassLoader with a null parent that the system will be forced to use
my class loader and will therefore inherit the classpath associated with
my class loader (in this case I would probably add the libraries from
the original class path into the new class loader). Is there any reason
I'm not aware of that this would be a bad idea ?

...and I guess the other thing that I could do would be just to make
sure that the child class is not loadable by the system class loader,
although this might be a bit of a pain.

 

[Matthew Keene]

So here's what I ended up doing, which seems to work, if anybody can
tell my why this is A Bad Idea then please let me know. I've tried to
read up about overriding the default class loader delegation hierarchy
and people mention possible security problems, which doesn't really
seem to be a big issue for us.
If people have enough access to the system that they can recompile and
replace jar files in production then there are easier and more
destructive things that they could do if they were feeling malicious.

(Apologies for the line wrapping - damn you, Google, damn you)

import java.lang.reflect.* ;
import java.io.* ;
import java.util.* ;
import java.net.* ;

public class ClassLoaderTestParent {

public ClassLoaderTestParent() {
}

public static void main(String[] args) throws Exception {
Vector v = new Vector() ;
// Create a list of the libraries in the current path to add
into the new class loader
StringTokenizer st = new
StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
while (st.hasMoreTokens()) {
String systemLibrary = st.nextToken() ;
if (systemLibrary.equals("."))
systemLibrary = System.getProperty("user.dir") ;

System.out.println("Adding system library " +
systemLibrary + " to new classloader");
v.add(new File(systemLibrary).toURL()) ;
}
// Add the desired extras
File f = new
File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
URL u = f.toURL() ;
v.add(u) ;
URL[] urls = (URL[])v.toArray(new URL[0]) ;
// Create the new class loader with no parent to ensure that
this class loader is actually used
URLClassLoader ucl = new URLClassLoader(urls,null) ;
Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
Object myInstance = myClass.newInstance() ;
Class[] methodArgClass = {} ;
Method myMethod = myClass.getMethod("childMethod",methodArgClass)
;
Object[] methodArgVals = {} ;
System.out.println("About to invoke method on dynamically loaded
class") ;
Object result = myMethod.invoke(myInstance,methodArgVals) ;
}
}

 

[winslave]

There are issues around custom class loaders.
I've been able to create a pretty stable and usable plugin architecture
by doing a "bootstap" class. That is one class to read in all the plugin
config files and setup the class path then start a new java process with
the dynamic classpath to run the application's main class. This works
fine and doesn't have the usual forking performance problem because as
soon as you start the main class the original process dies and releases
system resources.

As long as you don't run into class def not found exceptions in your
custom loader go with it. But it seems like it could be a lot of unnecessary
headaches at some point. Maybe not.

Here a good article on class loaders if you haven't already seen it
http://www.javaworld.com/javaworld/jw-10-1996/jw-10-indepth.html

So here's what I ended up doing, which seems to work, if anybody can
tell my why this is A Bad Idea then please let me know. I've tried to
read up about overriding the default class loader delegation hierarchy
and people mention possible security problems, which doesn't really
seem to be a big issue for us.
If people have enough access to the system that they can recompile and
replace jar files in production then there are easier and more
destructive things that they could do if they were feeling malicious.

(Apologies for the line wrapping - damn you, Google, damn you)

import java.lang.reflect.* ;
import java.io.* ;
import java.util.* ;
import java.net.* ;

public class ClassLoaderTestParent {

public ClassLoaderTestParent() {
}

public static void main(String[] args) throws Exception {
Vector v = new Vector() ;
// Create a list of the libraries in the current path to add
into the new class loader
StringTokenizer st = new
StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
while (st.hasMoreTokens()) {
String systemLibrary = st.nextToken() ;
if (systemLibrary.equals("."))
systemLibrary = System.getProperty("user.dir") ;

System.out.println("Adding system library " +
systemLibrary + " to new classloader");
v.add(new File(systemLibrary).toURL()) ;
}
// Add the desired extras
File f = new
File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
URL u = f.toURL() ;
v.add(u) ;
URL[] urls = (URL[])v.toArray(new URL[0]) ;
// Create the new class loader with no parent to ensure that
this class loader is actually used
URLClassLoader ucl = new URLClassLoader(urls,null) ;
Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
Object myInstance = myClass.newInstance() ;
Class[] methodArgClass = {} ;
Method myMethod = myClass.getMethod("childMethod",methodArgClass)
;
Object[] methodArgVals = {} ;
System.out.println("About to invoke method on dynamically loaded
class") ;
Object result = myMethod.invoke(myInstance,methodArgVals) ;
}
}

 

[John C. Bollinger]

John C. Bollinger wrote:



Right, I think I understand. So does this mean that if I create the
URLClassLoader with a null parent that the system will be forced to use
my class loader and will therefore inherit the classpath associated with
my class loader (in this case I would probably add the libraries from
the original class path into the new class loader).

Not entirely. Chances are that the new ClassLoader will then have the
system's bootstrap ClassLoader for a parent. That's good, because
otherwise the classes it loaded couldn't use any of the Java platform API.

Is there any reason
I'm not aware of that this would be a bad idea ?

It's not very different from what you were doing before, and it's more
work for you now, and more complicated for future maintainers. All you
really need to do is split out the classes you want to load with that
ClassLoader into jars that are not accessible from the system classpath
(but are accessible via the new ClassLoader). Define interfaces or
superclasses in the system classpath by which to type references to
these dynamically loaded classes, if it should be necessary to hold
references to them at all.

I'm trying to write a framework that will allow for custom functionality
to be plugged in at various points of the processing. Because I don't
know from the outset what the requirements of these plugins will be (and
because I don't directly have control of the system classpath - don't
ask) I'm trying to specify additional libraries in the config file which
defines the plugin and load them dynamically when I load the plugin.
Regex was just a simple example, in fact the most common example will be
JDBC drivers.

JDBC drivers are an odd choice for this, because the architecture is
designed to not require such tricks. Generally you can put all the
drivers you want into the system classpath, and just
Class.forName("com.dbcompany.jdbc.WhizzBangJDBCDriver")
early on to make the driver of your choice load. You can load multiple
drivers without trouble as long as they all use different DB URL
schemes. The only issue there is if you are supporting different
drivers with identical class names for incompatible classes.


John Bollinger
(e-mail address removed)

 

[John C. Bollinger]

So here's what I ended up doing, which seems to work, if anybody can
tell my why this is A Bad Idea then please let me know. I've tried to

It's ugly and probably unnecessary. It may present problems if you pass
objects from instances of classes created by the system ClassLoader to
instances of classes created by your user-defined ClassLoader. It may
also present problems if you rely on static data in classes not
accessible from the bootstrap ClassLoader (and you may so rely without
knowing it if you are using third-party classes). See also below.

read up about overriding the default class loader delegation hierarchy
and people mention possible security problems, which doesn't really
seem to be a big issue for us.
If people have enough access to the system that they can recompile and
replace jar files in production then there are easier and more
destructive things that they could do if they were feeling malicious.

Right. No point in closing a security pinhole when it's necessarilly
standing next to a gaping hole.

(Apologies for the line wrapping - damn you, Google, damn you)

import java.lang.reflect.* ;
import java.io.* ;
import java.util.* ;
import java.net.* ;

public class ClassLoaderTestParent {

public ClassLoaderTestParent() {
}

public static void main(String[] args) throws Exception {
Vector v = new Vector() ;
// Create a list of the libraries in the current path to add
into the new class loader
StringTokenizer st = new
StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
while (st.hasMoreTokens()) {
String systemLibrary = st.nextToken() ;
if (systemLibrary.equals("."))
systemLibrary = System.getProperty("user.dir") ;

System.out.println("Adding system library " +
systemLibrary + " to new classloader");
v.add(new File(systemLibrary).toURL()) ;
}
// Add the desired extras
File f = new
File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
URL u = f.toURL() ;
v.add(u) ;
URL[] urls = (URL[])v.toArray(new URL[0]) ;
// Create the new class loader with no parent to ensure that
this class loader is actually used
URLClassLoader ucl = new URLClassLoader(urls,null) ;
Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
Object myInstance = myClass.newInstance() ;
Class[] methodArgClass = {} ;
Method myMethod = myClass.getMethod("childMethod",methodArgClass)
;
Object[] methodArgVals = {} ;
System.out.println("About to invoke method on dynamically loaded
class") ;
Object result = myMethod.invoke(myInstance,methodArgVals) ;
}
}

Consider how you're going to _use_ whatever plugin you load. In the
example, you reflectively invoke methods on your ClassLoaderTestChild
instance. That rather defeats the purpose, as not only have you encoded
the plugin's API into the application, you have done it in a hard to
read, plugin-specific, and difficult to maintain way. You would need to
do the same for every plugin you want to use.

There are two conceivable use cases, not mutually exclusive:
(1) You want to support optional functionality
(2) You want to permit interchangeable plugin implementations

For case (1) the easiest thing to do is to dynamically build the system
classpath in your application's launcher -- i.e. before starting Java --
and then detect which optional packages are available at runtime and
disable access to any others. This does not require a user-defined
ClassLoader.

For case (2) to be feasible at any level you have to abstract the
functionality to be provided by each kind of plugin. That maps very
nicely onto definition of a Java interface, and you can then put that
interface into the system classpath and refer to directly in your
application code. For each plugin implementation you then need an
adapter class to match the implementation's API to the interface
definition -- this should not be in the system classpath; it can be in
the same directory / jar as the implementation itself, but does not need
to be, as long as both are accessible by the same user-defined
ClassLoader. (The user-defined ClassLoader can and probably should
delegate to the system ClassLoader.) You use an appropriate
user-defined ClassLoader to load the adapter class, then you instantiate
the class and assign it to a variable of the appropriate interface type,
and off you go.

Interchangeable optional packages (case 1 + 2) is a pretty
straightforward combination of the techniques described for the
individual cases.


John Bollinger
(e-mail address removed)