Confused about class loaders

Discussion in 'Java' started by Matthew Keene, Apr 2, 2004.

  1. I'm trying to dynamically add some libraries to the application class
    path by using a URLClassLoader, but it doesn't seem to be behaving the
    way that I want it to. I have to classes as follows:

    import java.lang.reflect.* ;
    import java.io.* ;
    import java.net.* ;

    public class ClassLoaderTestParent {

    public ClassLoaderTestParent() {
    }

    public static void main(String[] args) throws Exception {

    File f = new
    File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
    URL u = f.toURL() ;
    URL[] urls = {u} ;
    URLClassLoader ucl = new URLClassLoader(urls) ;
    Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
    // Also tried using forName, as below
    //Class myClass = Class.forName("ClassLoaderTestChild",true,ucl) ;
    Object myInstance = myClass.newInstance() ;
    Class[] methodArgClass = {} ;
    Method myMethod = myClass.getMethod("childMethod",methodArgClass) ;
    Object[] methodArgVals = {} ;
    System.out.println("About to invoke method on dynamically loaded
    class") ;
    Object result = myMethod.invoke(myInstance,methodArgVals) ;
    }
    }

    This class will create a URL class loader with the specified jar file
    added to the classpath, then load the desired class, instantiate it
    and then dynamically invoke a method on the instance. The class which
    is invoked is shown below

    import gnu.regexp.* ;

    public class ClassLoaderTestChild {

    public ClassLoaderTestChild() {
    }

    public String childMethod() throws Exception {

    System.out.println("Inside child method") ;
    System.out.println("Class path is " +
    System.getProperty("java.class.path")) ;
    RE testRE = new RE("abcde") ;
    return "Hello" ;

    }
    }

    When I run this, I expect that the GNU regexp libraries should be
    available to the child class, as I added them to the class loader in
    the parent. Instead, I get a NoClassDefFoundError, as shown below

    H:\Data\Java>java -classpath . ClassLoaderTestParent
    About to invoke method on dynamically loaded class
    Inside child method
    Class path is .
    Exception in thread "main"
    java.lang.reflect.InvocationTargetException:
    java.lang.NoClassDefFoundError: gnu/regexp/RE
    at ClassLoaderTestChild.childMethod(ClassLoaderTestChild.java:12)
    at java.lang.reflect.Method.invoke(Native Method)
    at ClassLoaderTestParent.main(ClassLoaderTestParent.java:24)

    Can anybody explain this behaviour, and even better, tell me what I
    would have to do to get the behaviour I'm after (ie to have the
    dynamically added class libraries available to the loaded class) ?

    TIA

    Matthew
    Matthew Keene, Apr 2, 2004
    #1
    1. Advertising

  2. Matthew Keene wrote:

    > I'm trying to dynamically add some libraries to the application class
    > path by using a URLClassLoader, but it doesn't seem to be behaving the
    > way that I want it to. I have to classes as follows:
    >
    > import java.lang.reflect.* ;
    > import java.io.* ;
    > import java.net.* ;
    >
    > public class ClassLoaderTestParent {
    >
    > public ClassLoaderTestParent() {
    > }
    >
    > public static void main(String[] args) throws Exception {
    >
    > File f = new
    > File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
    > URL u = f.toURL() ;
    > URL[] urls = {u} ;
    > URLClassLoader ucl = new URLClassLoader(urls) ;
    > Class myClass = ucl.loadClass("ClassLoaderTestChild") ;


    [...]

    > }
    >
    > This class will create a URL class loader with the specified jar file
    > added to the classpath, then load the desired class, instantiate it
    > and then dynamically invoke a method on the instance. The class which
    > is invoked is shown below


    [...]

    > When I run this, I expect that the GNU regexp libraries should be
    > available to the child class, as I added them to the class loader in
    > the parent. Instead, I get a NoClassDefFoundError, as shown below


    [...]

    > Can anybody explain this behaviour, and even better, tell me what I
    > would have to do to get the behaviour I'm after (ie to have the
    > dynamically added class libraries available to the loaded class) ?


    The class ClassLoaderTestChild is available in the application's class
    path. It is therefore loaded by the system ClassLoader (to which the
    URLClassLoader delegates) instead of by the URLClassLoader itself. As a
    result, when the ClassLoaderTestChild instance requires the RE class the
    system uses the system ClassLoader (the ClassLoader of
    ClassLoaderTestChild) to attempt to load it.

    Are you trying to support multiple regex packages? Otherwise why not
    just put the gnu-regex jar in the system classpath?


    John Bollinger
    John C. Bollinger, Apr 2, 2004
    #2
    1. Advertising

  3. John C. Bollinger wrote:

    >
    > The class ClassLoaderTestChild is available in the application's class
    > path. It is therefore loaded by the system ClassLoader (to which the
    > URLClassLoader delegates) instead of by the URLClassLoader itself. As a
    > result, when the ClassLoaderTestChild instance requires the RE class the
    > system uses the system ClassLoader (the ClassLoader of
    > ClassLoaderTestChild) to attempt to load it.
    >

    Right, I think I understand. So does this mean that if I create the
    URLClassLoader with a null parent that the system will be forced to use
    my class loader and will therefore inherit the classpath associated with
    my class loader (in this case I would probably add the libraries from
    the original class path into the new class loader). Is there any reason
    I'm not aware of that this would be a bad idea ?

    > Are you trying to support multiple regex packages? Otherwise why not
    > just put the gnu-regex jar in the system classpath?
    >

    I'm trying to write a framework that will allow for custom functionality
    to be plugged in at various points of the processing. Because I don't
    know from the outset what the requirements of these plugins will be (and
    because I don't directly have control of the system classpath - don't
    ask) I'm trying to specify additional libraries in the config file which
    defines the plugin and load them dynamically when I load the plugin.
    Regex was just a simple example, in fact the most common example will be
    JDBC drivers.

    >
    > John Bollinger
    >
    >


    Thanks very much for your help so far.
    Matthew Keene, Apr 3, 2004
    #3
  4. > Right, I think I understand. So does this mean that if I create the
    > URLClassLoader with a null parent that the system will be forced to use
    > my class loader and will therefore inherit the classpath associated with
    > my class loader (in this case I would probably add the libraries from
    > the original class path into the new class loader). Is there any reason
    > I'm not aware of that this would be a bad idea ?
    >

    ...and I guess the other thing that I could do would be just to make
    sure that the child class is not loadable by the system class loader,
    although this might be a bit of a pain.
    Matthew Keene, Apr 3, 2004
    #4
  5. So here's what I ended up doing, which seems to work, if anybody can
    tell my why this is A Bad Idea then please let me know. I've tried to
    read up about overriding the default class loader delegation hierarchy
    and people mention possible security problems, which doesn't really
    seem to be a big issue for us.
    If people have enough access to the system that they can recompile and
    replace jar files in production then there are easier and more
    destructive things that they could do if they were feeling malicious.

    (Apologies for the line wrapping - damn you, Google, damn you)

    import java.lang.reflect.* ;
    import java.io.* ;
    import java.util.* ;
    import java.net.* ;

    public class ClassLoaderTestParent {

    public ClassLoaderTestParent() {
    }

    public static void main(String[] args) throws Exception {
    Vector v = new Vector() ;
    // Create a list of the libraries in the current path to add
    into the new class loader
    StringTokenizer st = new
    StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
    while (st.hasMoreTokens()) {
    String systemLibrary = st.nextToken() ;
    if (systemLibrary.equals("."))
    systemLibrary = System.getProperty("user.dir") ;

    System.out.println("Adding system library " +
    systemLibrary + " to new classloader");
    v.add(new File(systemLibrary).toURL()) ;
    }
    // Add the desired extras
    File f = new
    File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
    URL u = f.toURL() ;
    v.add(u) ;
    URL[] urls = (URL[])v.toArray(new URL[0]) ;
    // Create the new class loader with no parent to ensure that
    this class loader is actually used
    URLClassLoader ucl = new URLClassLoader(urls,null) ;
    Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
    Object myInstance = myClass.newInstance() ;
    Class[] methodArgClass = {} ;
    Method myMethod = myClass.getMethod("childMethod",methodArgClass)
    ;
    Object[] methodArgVals = {} ;
    System.out.println("About to invoke method on dynamically loaded
    class") ;
    Object result = myMethod.invoke(myInstance,methodArgVals) ;
    }
    }
    Matthew Keene, Apr 5, 2004
    #5
  6. Matthew Keene

    winslave Guest

    There are issues around custom class loaders.
    I've been able to create a pretty stable and usable plugin architecture
    by doing a "bootstap" class. That is one class to read in all the plugin
    config files and setup the class path then start a new java process with
    the dynamic classpath to run the application's main class. This works
    fine and doesn't have the usual forking performance problem because as
    soon as you start the main class the original process dies and releases
    system resources.

    As long as you don't run into class def not found exceptions in your
    custom loader go with it. But it seems like it could be a lot of unnecessary
    headaches at some point. Maybe not.

    Here a good article on class loaders if you haven't already seen it
    http://www.javaworld.com/javaworld/jw-10-1996/jw-10-indepth.html

    (Matthew Keene) wrote in message news:<>...
    > So here's what I ended up doing, which seems to work, if anybody can
    > tell my why this is A Bad Idea then please let me know. I've tried to
    > read up about overriding the default class loader delegation hierarchy
    > and people mention possible security problems, which doesn't really
    > seem to be a big issue for us.
    > If people have enough access to the system that they can recompile and
    > replace jar files in production then there are easier and more
    > destructive things that they could do if they were feeling malicious.
    >
    > (Apologies for the line wrapping - damn you, Google, damn you)
    >
    > import java.lang.reflect.* ;
    > import java.io.* ;
    > import java.util.* ;
    > import java.net.* ;
    >
    > public class ClassLoaderTestParent {
    >
    > public ClassLoaderTestParent() {
    > }
    >
    > public static void main(String[] args) throws Exception {
    > Vector v = new Vector() ;
    > // Create a list of the libraries in the current path to add
    > into the new class loader
    > StringTokenizer st = new
    > StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
    > while (st.hasMoreTokens()) {
    > String systemLibrary = st.nextToken() ;
    > if (systemLibrary.equals("."))
    > systemLibrary = System.getProperty("user.dir") ;
    >
    > System.out.println("Adding system library " +
    > systemLibrary + " to new classloader");
    > v.add(new File(systemLibrary).toURL()) ;
    > }
    > // Add the desired extras
    > File f = new
    > File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
    > URL u = f.toURL() ;
    > v.add(u) ;
    > URL[] urls = (URL[])v.toArray(new URL[0]) ;
    > // Create the new class loader with no parent to ensure that
    > this class loader is actually used
    > URLClassLoader ucl = new URLClassLoader(urls,null) ;
    > Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
    > Object myInstance = myClass.newInstance() ;
    > Class[] methodArgClass = {} ;
    > Method myMethod = myClass.getMethod("childMethod",methodArgClass)
    > ;
    > Object[] methodArgVals = {} ;
    > System.out.println("About to invoke method on dynamically loaded
    > class") ;
    > Object result = myMethod.invoke(myInstance,methodArgVals) ;
    > }
    > }
    winslave, Apr 6, 2004
    #6
  7. Matthew Keene wrote:

    > John C. Bollinger wrote:
    >
    >
    >>The class ClassLoaderTestChild is available in the application's class
    >>path. It is therefore loaded by the system ClassLoader (to which the
    >>URLClassLoader delegates) instead of by the URLClassLoader itself. As a
    >>result, when the ClassLoaderTestChild instance requires the RE class the
    >>system uses the system ClassLoader (the ClassLoader of
    >>ClassLoaderTestChild) to attempt to load it.
    >>

    >
    > Right, I think I understand. So does this mean that if I create the
    > URLClassLoader with a null parent that the system will be forced to use
    > my class loader and will therefore inherit the classpath associated with
    > my class loader (in this case I would probably add the libraries from
    > the original class path into the new class loader).


    Not entirely. Chances are that the new ClassLoader will then have the
    system's bootstrap ClassLoader for a parent. That's good, because
    otherwise the classes it loaded couldn't use any of the Java platform API.

    > Is there any reason
    > I'm not aware of that this would be a bad idea ?


    It's not very different from what you were doing before, and it's more
    work for you now, and more complicated for future maintainers. All you
    really need to do is split out the classes you want to load with that
    ClassLoader into jars that are not accessible from the system classpath
    (but are accessible via the new ClassLoader). Define interfaces or
    superclasses in the system classpath by which to type references to
    these dynamically loaded classes, if it should be necessary to hold
    references to them at all.

    >>Are you trying to support multiple regex packages? Otherwise why not
    >>just put the gnu-regex jar in the system classpath?
    >>

    >
    > I'm trying to write a framework that will allow for custom functionality
    > to be plugged in at various points of the processing. Because I don't
    > know from the outset what the requirements of these plugins will be (and
    > because I don't directly have control of the system classpath - don't
    > ask) I'm trying to specify additional libraries in the config file which
    > defines the plugin and load them dynamically when I load the plugin.
    > Regex was just a simple example, in fact the most common example will be
    > JDBC drivers.


    JDBC drivers are an odd choice for this, because the architecture is
    designed to not require such tricks. Generally you can put all the
    drivers you want into the system classpath, and just
    Class.forName("com.dbcompany.jdbc.WhizzBangJDBCDriver")
    early on to make the driver of your choice load. You can load multiple
    drivers without trouble as long as they all use different DB URL
    schemes. The only issue there is if you are supporting different
    drivers with identical class names for incompatible classes.


    John Bollinger
    John C. Bollinger, Apr 7, 2004
    #7
  8. Matthew Keene wrote:

    > So here's what I ended up doing, which seems to work, if anybody can
    > tell my why this is A Bad Idea then please let me know. I've tried to


    It's ugly and probably unnecessary. It may present problems if you pass
    objects from instances of classes created by the system ClassLoader to
    instances of classes created by your user-defined ClassLoader. It may
    also present problems if you rely on static data in classes not
    accessible from the bootstrap ClassLoader (and you may so rely without
    knowing it if you are using third-party classes). See also below.

    > read up about overriding the default class loader delegation hierarchy
    > and people mention possible security problems, which doesn't really
    > seem to be a big issue for us.
    > If people have enough access to the system that they can recompile and
    > replace jar files in production then there are easier and more
    > destructive things that they could do if they were feeling malicious.


    Right. No point in closing a security pinhole when it's necessarilly
    standing next to a gaping hole.

    > (Apologies for the line wrapping - damn you, Google, damn you)
    >
    > import java.lang.reflect.* ;
    > import java.io.* ;
    > import java.util.* ;
    > import java.net.* ;
    >
    > public class ClassLoaderTestParent {
    >
    > public ClassLoaderTestParent() {
    > }
    >
    > public static void main(String[] args) throws Exception {
    > Vector v = new Vector() ;
    > // Create a list of the libraries in the current path to add
    > into the new class loader
    > StringTokenizer st = new
    > StringTokenizer(System.getProperty("java.class.path"),System.getProperty("path.separator"));
    > while (st.hasMoreTokens()) {
    > String systemLibrary = st.nextToken() ;
    > if (systemLibrary.equals("."))
    > systemLibrary = System.getProperty("user.dir") ;
    >
    > System.out.println("Adding system library " +
    > systemLibrary + " to new classloader");
    > v.add(new File(systemLibrary).toURL()) ;
    > }
    > // Add the desired extras
    > File f = new
    > File("H:/Data/Java/ExceptionReportEngine/gnu-regexp-1.1.4.jar") ;
    > URL u = f.toURL() ;
    > v.add(u) ;
    > URL[] urls = (URL[])v.toArray(new URL[0]) ;
    > // Create the new class loader with no parent to ensure that
    > this class loader is actually used
    > URLClassLoader ucl = new URLClassLoader(urls,null) ;
    > Class myClass = ucl.loadClass("ClassLoaderTestChild") ;
    > Object myInstance = myClass.newInstance() ;
    > Class[] methodArgClass = {} ;
    > Method myMethod = myClass.getMethod("childMethod",methodArgClass)
    > ;
    > Object[] methodArgVals = {} ;
    > System.out.println("About to invoke method on dynamically loaded
    > class") ;
    > Object result = myMethod.invoke(myInstance,methodArgVals) ;
    > }
    > }


    Consider how you're going to _use_ whatever plugin you load. In the
    example, you reflectively invoke methods on your ClassLoaderTestChild
    instance. That rather defeats the purpose, as not only have you encoded
    the plugin's API into the application, you have done it in a hard to
    read, plugin-specific, and difficult to maintain way. You would need to
    do the same for every plugin you want to use.

    There are two conceivable use cases, not mutually exclusive:
    (1) You want to support optional functionality
    (2) You want to permit interchangeable plugin implementations

    For case (1) the easiest thing to do is to dynamically build the system
    classpath in your application's launcher -- i.e. before starting Java --
    and then detect which optional packages are available at runtime and
    disable access to any others. This does not require a user-defined
    ClassLoader.

    For case (2) to be feasible at any level you have to abstract the
    functionality to be provided by each kind of plugin. That maps very
    nicely onto definition of a Java interface, and you can then put that
    interface into the system classpath and refer to directly in your
    application code. For each plugin implementation you then need an
    adapter class to match the implementation's API to the interface
    definition -- this should not be in the system classpath; it can be in
    the same directory / jar as the implementation itself, but does not need
    to be, as long as both are accessible by the same user-defined
    ClassLoader. (The user-defined ClassLoader can and probably should
    delegate to the system ClassLoader.) You use an appropriate
    user-defined ClassLoader to load the adapter class, then you instantiate
    the class and assign it to a variable of the appropriate interface type,
    and off you go.

    Interchangeable optional packages (case 1 + 2) is a pretty
    straightforward combination of the techniques described for the
    individual cases.


    John Bollinger
    John C. Bollinger, Apr 7, 2004
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave Rudolf

    Custom Class Loaders

    Dave Rudolf, Oct 20, 2003, in forum: Java
    Replies:
    5
    Views:
    1,572
    Lee Fesperman
    Oct 21, 2003
  2. P. Flavin
    Replies:
    0
    Views:
    496
    P. Flavin
    Nov 3, 2003
  3. brian
    Replies:
    1
    Views:
    1,100
    John C. Bollinger
    Jul 1, 2004
  4. Roedy Green

    Custom Class Loaders

    Roedy Green, Sep 21, 2005, in forum: Java
    Replies:
    10
    Views:
    726
    Thomas Hawtin
    Sep 21, 2005
  5. Chris
    Replies:
    11
    Views:
    692
    Arne Vajhøj
    Mar 30, 2008
Loading...

Share This Page