Using mandatory libraries (custom class loading vs. expanding libraries)

[Karsten Wutzke]

Hi all!

I was wondering what would be the "best" approach to achieve class
loading flexibility in the case of

a) Running the application/applet via the local class files from the
development tree
b) Running the application/applet via a *single* JAR file?

First have a look at the dev tree

<root>
+- bin
+- images
+- lang
+- lib
| +- bcel.jar
| +- forms.jar
| +- ...
+- src
|- ....

Considering the cases:

a) When launching from local classes in dev tree, all ./lib/*.jar
files can be put on the classpath. Class loading can be performed by
the system class loader (desired).

b) When launching the single JAR file, there are basically two
approaches (affects the packaged JAR)

1. Write a custom class loader that somehow finds JAR's inside the
JAR's and loads classes from there. Disadvantage here is, that I have
to either set the so-called "libraries class loader" for any class
using a library class (to find those classes) OR I have to set my
library class loader to load *ALL* app classes system-wide (e.g. load
my main Controller class via the custom class loader, so subsequent
class loading is deferred to the custom class loader). The advantage
is, I can leave the JAR's as they are, where they are and package the
files.

2. Put *extracted* 3rd party libraries into the distribution JAR file
so that all classes run concurrently (next to the application specfic
code). All classes are always found by the system class loader and
avoids a custom class loader (and class loader headaches, which I
currently have).

The question here is: Is it legit to put extracted 3rd party JAR's
into my packaged JAR, libraries like BCEL or JGoodies for example?

Can anyone make some recommendations on what would be probably be the
best overall approach?

Karsten

 

[Andrew Thompson]

Karsten Wutzke wrote:

The best approach is to ..
- leave external libraries in exactly the same jars
they came in.
- jar the classes of the application and (ideally)
specify a main, and as I understand, the other
'dependant' jars in the manifest.
- distribute the lot of them. Usually installers
can handle multiple jars, and web start is
optimised for it.

I generally use ant, and compact all that
into a handful of lines in the build task.
Though admittedly, my builds are usually
for web start deployment - where manifest
files are largely redundant.

 

[Karsten Wutzke]

Karsten Wutzke wrote:

The best approach is to ..
- leave external libraries in exactly the same jars
they came in.
- jar the classes of the application and (ideally)
specify a main, and as I understand, the other
'dependant' jars in the manifest.

I've had no luck achieving *anything* with handling the 3rd party
libraries via the Class-Path manifest entry. How does the manifest
line Class-Path look like?

I keep creating the line (via Ant)

Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

to include the libs on the Class-Path when launched via JAR.


The final MANIFEST.MF is:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.0
Created-By: 1.5.0_11-b03 (Sun Microsystems Inc.)
Main-Class: am.Main
Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

created by the Ant task:

<target name="create-manifest" depends="init" description="Creates
the distribution and deployment manifest file">
<manifest file="${manifest.file}">
<attribute name="Main-Class" value="${main.class.name}"/>
<attribute name="Class-Path" value="lib/bcel-5.2.jar lib/
forms-1.1.0.jar"/>
</manifest>
</target>

To me, this looks all perfect, the JAR contains all the libs, but as I
mentioned above, I am constantly unable to get the Class-Path manifest
entry right to find sub lib classes automatically (by the system class
loader).

What am I doing wrong?

I keep reading

http://java.sun.com/docs/books/tutorial/deployment/jar/downman.html
http://mindprod.com/jgloss/jar.html

but "it" refuses find any of the classes of the respective libs. ->
NoClassDefFoundError /org/apache/bcel/generic/Type

Please help, I probably don't have to write a custom class loader to
just load standard library class files....

Karsten

 

[Thomas Fritsch]

I've had no luck achieving *anything* with handling the 3rd party
libraries via the Class-Path manifest entry. How does the manifest
line Class-Path look like?

I keep creating the line (via Ant)

Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

to include the libs on the Class-Path when launched via JAR.


The final MANIFEST.MF is:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.0
Created-By: 1.5.0_11-b03 (Sun Microsystems Inc.)
Main-Class: am.Main
Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

These relative URLs in your Class-Path mean that you have to arrange
your jar files in the following directory structure:
yourMain.jar
lib/
bcel-5.2.jar
forms-1.1.0.jar
Is that the case in your environment?

 

[Karsten Wutzke]

These relative URLs in your Class-Path mean that you have to arrange
your jar files in the following directory structure:
yourMain.jar
lib/
bcel-5.2.jar
forms-1.1.0.jar
Is that the case in your environment?

Exactly! This is the structure in my local dev tree. I then create my
distribution JAR file by putting the libs *inside* the JAR, here
yourMain.jar under the lib dir.

This doesn't work, as I've just encountered the sentence:

http://java.sun.com/docs/books/tutorial/deployment/jar/downman.html

First note:
"The Class-Path header points to classes or JAR files on the local
network, *** not JAR files within the JAR file *** or classes
accessible over internet protocols. *** To load classes in JAR files
within a JAR file into the class path, you must write custom code to
load those classes. For example, if MyJar.jar contains another JAR
file called MyUtils.jar, you cannot use the Class-Path header in
MyJar.jar's manifest to load classes in MyUtils.jar into the class
path."

(If this *really* is the case I know I have to write my custom class
loader... cause I do have JARs inside my app JAR)

So consequently, launching yourMain.jar with the libs inside *can't*
work, no matter what the manifest Class-Path contains.

OK I agree, buuuut...

Every since I create and launch application JARs I put them into my
*root* development dir, where the lib dir is located... So, even if I
create a ("wrong") JAR containing the lib dir with "non-classpath-
recognized" JARs inside and the manifest still contains the entry
"Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar", the libraries
should be found *on the local subdir lib* when launching the JAR at
all times *anyway*!

The libs inside the JAR are ignored for anything anyway, the Class-
Path lists those entries local to the JAR (System property
"user.dir"?) which are there. If those refer to relative files
"outside" of the launched JAR (user.dir?), I wonder why these lib JARs
were and are *never* found and put on the class path!

Another thing:

When launching the jar (anywhere) I print the Java System property
"java.class.path". It CONSTANTLY only has exactly ONE entry, namely
the name of the launched JAR file. This means, the manifest, no matter
what is written in the Class-Path: ... line has absolutely no effect.

Why aren't the local ./lib/*.jar files found as specified by the
manifest Class-Path entries though they exist?

Karsten

 

[Owen Jacobson]

....

Another thing:

When launching the jar (anywhere) I print the Java System property
"java.class.path". It CONSTANTLY only has exactly ONE entry, namely
the name of the launched JAR file. This means, the manifest, no matter
what is written in the Class-Path: ... line has absolutely no effect.

Why aren't the local ./lib/*.jar files found as specified by the
manifest Class-Path entries though they exist?

How are you launching the program?

$ java -cp yourMain.jar main.class.Name
or
$ java -jar yourMain.jar

I believe (possibly incorrectly, I haven't had classpath problems in a
long time) that only the latter will use the Class-Path: entries from
the yourMain.jar manifest. I know for a fact that the latter will
ignore other classpath entries (eg. from the CLASSPATH env var, or the
command line).

 

[Roedy Green]

Can anyone make some recommendations on what would be probably be the
best overall approach?

Do an experiment. The tendency today is ALWAYS to use jars.

 

[Karsten Wutzke]

How are you launching the program?

$ java -cp yourMain.jar main.class.Name
or
$ java -jar yourMain.jar

I believe (possibly incorrectly, I haven't had classpath problems in a
long time) that only the latter will use the Class-Path: entries from
the yourMain.jar manifest. I know for a fact that the latter will
ignore other classpath entries (eg. from the CLASSPATH env var, or the
command line).

"ant run-dist"

But I try it frequently with "java -jar myapp.jar" as well. No
difference. I have no special setup, and I don't use any command line
arguments. The thing is escpecially strange in that I use two
completely different computers to try what I described, both behave in
the same way irgnoring the manifest Class-Path: ... I checked the
created JAR so many times...

Here's what it looks like (I also extracted it):

<root>
+- am
+- appspecificpackage1
+- appspecificpackage2
+- appspecificpackageN
+- Main.class
+- api
+- ownapipackage1
+- ownapipackage2
+- ownapipackage3
+- images
+- imageA.jpg
+- ...
+- languages
+- LanguagePack_en_US.properties
+- LanguagePack_en_DE.properties
+- ...
+- META-INF
+- INDEX.LIST
+- MANIFEST.MF

Again, the copied MANIFEST.MF from a few mins ago:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.5.4
Created-By: 1.5.0_09-b03 (Sun Microsystems Inc.)
Main-Class: am.Main
Built-By: Karsten Wutzke
Built-On: 2007-06-29 07:57:50
Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

I'd really like to debug this, as I have absolutely no idea what could
cause all this.

BTW: Putting the lib/bcel-5.2.jar lib/forms-1.1.0.jar JARs on the
classpath and simply run via the not JARred classes works PERFECTLY!

Here's the Ant task that creates my JAR along with the manifest task:

<target name="create-manifest" depends="init">
<manifest file="${manifest.file}">
<attribute name="Main-Class" value="${main.class.name}"/>
<attribute name="Built-By" value="${author.name}"/>
<attribute name="Built-On" value="${datetime.iso}"/>
<attribute name="Class-Path" value="lib/bcel-5.2.jar lib/
forms-1.1.0.jar"/>
</manifest>
</target>

<target name="dist" depends="compile,create-manifest">
<jar destfile="${dist.file}" basedir="${bin.dir}" index="true"
manifest="${manifest.file}">
<fileset refid="fileset.images"/>
<fileset refid="fileset.languages"/>
</jar>
</target>

Well nothing special at all. In fact I use a good Ant book, so I'm
basically on the right way here. Ah BTW, init task creates an ISO
timestamp and some dirs, compile compiles the classes, again, nothing
special here. The properties are all set, I checked 1000 times.

I'm out of ideas...

Karsten

 

[Andrew Thompson]

Karsten Wutzke wrote:
...

...irgnoring the manifest Class-Path: ... I checked the
created JAR so many times...

There was an odd aspect of manifest files, in that
they had to end in a new (blank) line. Though I..
- thought it had been fixed long ago.
- would expect an ant task that creates manifests
to add the newline.

OTOH...
..

Again, the copied MANIFEST.MF from a few mins ago:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.5.4
Created-By: 1.5.0_09-b03 (Sun Microsystems Inc.)
Main-Class: am.Main
Built-By: Karsten Wutzke
Built-On: 2007-06-29 07:57:50
Class-Path: lib/bcel-5.2.jar lib/forms-1.1.0.jar

..it is not entirely clear where this file ends.

(And in case I did not mention before, I am no
expert on manifest files. Not having to write
them anymore is one the benefits of web start,
as I see it!)

Had you ruled out web start deployment?
It seems even if the jars required signing (using
ant), it would be easier to avhieve than get the
manifest working correctly. ;-)

 

[Karsten Wutzke]

Karsten Wutzke wrote:

..


There was an odd aspect of manifest files, in that
they had to end in a new (blank) line. Though I..
- thought it had been fixed long ago.
- would expect an ant task that creates manifests
to add the newline.

I know of this critter... I read Roedy's "tuto" thoroughly and even
after having readit many times, I couldn't find anything I'm missing.
The only hope now would be to have other people try my (stripped down)
setup.

OTOH...
.



.it is not entirely clear where this file ends.

There are two more lines... I just checked, there are two 0D0A at the
end, meaning two \n.

(And in case I did not mention before, I am no
expert on manifest files. Not having to write
them anymore is one the benefits of web start,
as I see it!)

Had you ruled out web start deployment?
It seems even if the jars required signing (using
ant), it would be easier to avhieve than get the
manifest working correctly. ;-)

Well I intend to use WebStart sometime in the future. But thats
another story. For now, I'm only concerned about a setup that finds my
library classes, such as BCEL, JGoodies, Apache Commons... etc. This
is not even something special I try, but it pretty much turns into a
nightmare...

Karsten

 

[Karsten Wutzke]

snip

Here's the Ant task that creates my JAR along with the manifest task:

<target name="dist" depends="compile,create-manifest">
<jar destfile="${dist.file}" basedir="${bin.dir}" index="true"
manifest="${manifest.file}">
<fileset refid="fileset.images"/>
<fileset refid="fileset.languages"/>
</jar>
</target>

Well nothing special at all. In fact I use a good Ant book, so I'm
basically on the right way here.

Well, I just tried something as a "last bastion"......

And now, by magic, the manifest setup I HAD *WORKS*!!

But how?

Easy... but extremely hard to find... as far as I remember, the author
of my smart Ant book stated it was good practice to use an index file
in the <jar> task... so I did and never worried.

<jar destfile="${dist.file}" basedir="${bin.dir}" index="true"
manifest="${manifest.file}">

Then I removed the index="true"... to my astonishment this started up
the app with the system class loader finding all library classes as
specific in the manifest! This took me soo many headaches! For such a
banal thing...

I suppose since there are so many Ant users out there and not being
able to get a manifest with lib/libraries setup because of this index
file in the JAR I would consider this a bug on someone's end. I just
can't say who to blame...

Is it Sun?

How do the index file and the manifest file relate to each other? And
why (the "%$§!) does the index file block the manifest's Class-
Path: ... library entries????

Anyway, I just hope other developers find this posting as well... it's
a real relief! (thank god it's almost weekend)

Karsten

 

[Thomas Fritsch]

There was an odd aspect of manifest files, in that
they had to end in a new (blank) line. Though I..
- thought it had been fixed long ago.

It isn't, and probably never will :-(

 

[Roedy Green]

I know of this critter... I read Roedy's "tuto" thoroughly and even
after having readit many times, I couldn't find anything I'm missing.
The only hope now would be to have other people try my (stripped down)
setup.

let's see your ant script. You might post it on a website or get
someone (e.g. me) to post in on a website for you so others can have a
look at it.

 

[Roedy Green]

I know of this critter... I read Roedy's "tuto" thoroughly and even
after having readit many times, I couldn't find anything I'm missing.
The only hope now would be to have other people try my (stripped down)
setup.

Lets have a peek at the result. Look inside the jar for a file called
manifest.mf using Winzip or similar archiver.

it should look something like this:

Manifest-Version: 1.0
Created-By: Jakarta Ant 1.7.0 (December 13 2006)
Main-Class: com.mindprod.canadiantax.CanadianTax

Name: com/mindprod/common13/HybridJ$1.class
Last-Modified: Sat, 16 Jun 2007 01:00:20 PDT
Content-Location: E:\com\mindprod\common13\HybridJ$1.class

 

[Karsten Wutzke]

Do an experiment. The tendency today is ALWAYS to use jars.

I decided to do one of my last postings on that topis here.

The problem is with Ant and using index="true" with the <jar> task. An
index list is only created in the JAR for the JAR created, it
basically ignores all entries in the manifest Class-Path.

There are *two* solutions:

1. Run "jar i myapp.jar" *after* having created the (not yet working)
JAR. This will magically create the INDEX.LIST *with* the libs used. I
suppose the MANIFEST.MF Class-Path: ... get resolved...


2. Ant 1.6.2. adds the <indexjars> tag to the <jar> tag to denote a
list of JARs for which an index should be created.

http://ant.apache.org/manual/CoreTasks/jar.html#indexjars

To read a little more:

http://mail-archives.apache.org/mod_mbox/ant-dev/200404.mbox/<[email protected]>

As you can see, this is not really new, just pretty much unknown (and
not in the Ant book I mentioned).

To see it in action:

<!-- create a temporary manifest file -->
<target name="create-manifest" depends="init" description="Creates the
temp manifest file">
<manifest file="${manifest.file}">
<attribute name="Class-Path" value="lib/bcel-5.2.jar lib/
forms-1.1.0.jar"/>
<attribute name="Main-Class" value="${main.class.name}"/>
<attribute name="Built-By" value="${author.name}"/>
<attribute name="Built-On" value="${datetime.iso}"/>
</manifest>
</target>

<!-- create a distribution JAR file from the compiled classes -->
<target name="dist" depends="compile,create-manifest"
description="Creates the distribution JAR file">
<jar destfile="${dist.file}" basedir="${bin.dir}" index="true"
manifest="${manifest.file}">
<fileset refid="fileset.images"/>
<fileset refid="fileset.languages"/>
<indexjars>
<fileset dir="${lib.dir}"/>
<!--fileset refid="fileset.libs"/-->
</indexjars>
</jar>
</target>

@Roedy:

Maybe you should include this problem into your JAR glossary,
somewhere under the Class-Path: ... , 2. option...

HTH not only me
Karsten

 

[Karsten Wutzke]

let's see your ant script. You might post it on a website or get
someone (e.g. me) to post in on a website for you so others can have a
look at it.

There was nothing terribly wrong with my Ant build file, other than
index="true" doesn't produce a correct index file. To create a correct
INDEX.LIST refer to the post in the same thread (as a final note):

http://groups.google.de/group/comp..../aad4799913d9165d?hl=de&#doc_a24dbada87498bd3

Karsten

 

[Roedy Green]

Then I removed the index="true"... to my astonishment this started up
the app with the system class loader finding all library classes as
specific in the manifest! This took me soo many headaches! For such a
banal thing

does this EVER work, or is there something odd about your situation
that is making it fail?

 

[Karsten Wutzke]

On 28 Jun., 22:30, Roedy Green <[email protected]>

To read a little more:

http://mail-archives.apache.org/mod_mbox/ant-dev/200404.mbox/<[email protected]>

*gulp*

Just enter "indexjars" in google...

Karsten

 

[Karsten Wutzke]

does this EVER work, or is there something odd about your situation
that is making it fail?

This? What exactly did you mean that could work?

There's absolutely nothing special about my app setup really. I juts
followed a few big steps in the book "Java Development with Ant", 1st
Ed. is from 2003, the 2007 "Ant in Action" is just about to be
published.

The author makes some good points about using an index file
(performance etc.) and concludes the paragraph with the sentence:

"If one line in the build file may deliver a speedup on network
application loading, why not use it?"

I had to grin when I read it... um... not really... :-/

Anyway, don't use index="true" alone when using a manifest that has
Class-Path entries... Ant users should upgrade to at least version
1.6.2. (guess what I'll be doing tonight).

Karsten

 

[Roedy Green]

Ant users should upgrade to at least version
1.6.2. (guess what I'll be doing tonight).

I am using 1.7.0. Please let us know if they have fixed it.