PermGen, Class Unloading and Garbage Collection once again


T

Tassilo Horn

Hi all,

two weeks ago (message id <[email protected]>) I asked
about the possibility to unload classes that I'm sure that they won't be
used anymore to save permanent generation space.

The context: in a model transformation framework new schemas (basically
collections of classes) are generated by a code generator and compiled
in memory. If a transformation works in many steps, the intermediate
schemas can be forgotten at some point in time, because no instances of
those classes exist anymore.

Ok, with all your help, insights and the suggested profiling tools, I've
changed the implementation to remove all hindrances for class unloading
I could find.

- Each schema now uses its one and only own custom ClassLoader (named
M1ClassManager), which is implemented as a "weak singleton",
e.g. M1ClassManager has a static Map from some identifier string to
a WeakReference<M1ClassManager>. Additionally, each schema
instance references its M1ClassManager strongly.

- There were other singletons, which are all removed or converted to
"weak singletons" now.

- There were some caches, which are cleared now.

Then I run some sample test case and attach to the process using
jvisualvm (which is basically as good as JProfiler, but free and comes
with the JDK!). When my test case finishes, I keep the JVM running with
some "while(true) Thread.sleep(3000);" loop and take a HeapDump. I can
see, that none of the generated classes of a given schema have any
instances anymore.

But why aren't they unloaded then?

Maybe there are references to the java.lang.Class objects of the
generated classes left. jvisualvm says, that there are only 9
java.lang.Class objects, but 12857 instances of java.lang.Class[].
About 5,600 classes are compiled in memory, so this number looks
plausible. But inspecting the instances and their referents gives no
real help. For example, a typical "References" jvisualvm tree for some
Class[] with entries from a schema looks like that:

References
-------------------------------+--------------+------------------
Field | Type | Value
-------------------------------+--------------+------------------
[]this Class[] #9002(3 items)
`-> parameterTypes Method #6631
`- [] [764] Method[] #59 (2,850 items)
`- referent SoftReference #2431
`- <no reference> <none> <none>

Does anyone have a clue what that means? Why are they referenced when
the SoftReference referencing them is not referenced?

Another observation is that there are three M1ClassManager instances.
That's correct, although I'd like to see them collected at that point in
time, as three schemas are compiled in memory. Then I tried getting
some PermGen infos using the command:

jmap -permstat <pid of java process>

This shows me, that the liveness of all three M1ClassManager instances
is "dead" (what does that mean exactly?), they loaded about 1,850
classes each (that's the number of classes defined in the schemas, so
it's correct), and that their parent class loader is a
sun/misc/Launcher$AppClassLoader that is also dead. The parent of that
finally is a sun/misc/Launcher$ExtClassLoader (also dead) with no parent
class loader. The only live class loader is some <bootstrap> loader
with <internal> type, which has loaded 1507 classes.

Any hints what I could try? My impression is, that there are references
to the Class objects for the generated classes, but manually checking
the referents of all of them isn't feasible.

If anyone wants to have a look at the heap dump (2 hprof files, 6.8
Megs), you can get it from:

http://www.uni-koblenz.de/~horn/heapdump.tar.bz2

The names of the generated classes all start with "eu.redseeds.scl_v".

Ah, I also tried the JVM options -XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled, but that didn't help either.

Thanks a lot for any pointers!

Bye,
Tassilo
 
Ad

Advertisements

T

Tassilo Horn

Some more details:
Another observation is that there are three M1ClassManager instances.

Those custom ClassLoaders are referenced two times. One is the static
WeakReference in the M1ClassManager class itself. That one is ok,
because its releasing only blocked because of the other strong
reference.

The other problemantic reference is:

this
`- ProtectionDomain::classloader
+- M1ClassManager::defaultDomain (loop to this) ###
`- HashMap$Entry::key
`- HashMap$Entry[]::[6]
`- HashMap::table
`- HashSet::map
`- M1ClassManager::domains (loop to this) ###

The Fields defaultDomain (ProtectionDomain) and domains (HashSet) in
M1ClassManager are inherited from ClassLoader and private, so I don't
have control over them, but I guess I shouldn't fiddle in those
internals anyway.

Ok, so the question is: Why does this ProtectionDomain hold a reference
to my class loader?

Bye,
Tassilo
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top