How to use JTree with very large data?

[Sorin Gherman]

Hi,
I have the following problem: I have this large amount of data, stored
internally in some custom data structures (hierarchical ones, given
the nature of the data - I am using some custom, tree-like structurs
for this).
I have to show the different relations within this data using several
JTrees - one per "viewpoint". (The good news is that I only have to
show one type of view at a time.)

The first implementation I came up with is just creating a tree model
(using DefaultMutableTreeNodes) for each view, and displaying it using
JTree.
Problem: memory consumption is very high (due to the duplication of
the tree structures in memory, and the fact that the user may toggle
between views fast, thus trees are created and thrown away - again,
memory is an issue so I cannot affort to store them).

I read about the possibility to have my own internal data implement
the TreeNode interface, and have the JTree uses the internal data
directly - thus avoiding the creation of extra trees.
Problem: the implementation gets very complicated very fast due to the
different views. I can still live with that, but then I realized that
due to TreeNode.getParent() method, I would still have to duplicate,
in memory, for each view, at least the "parent" references. (This is
due to the fact that most of the internal nodes in my data structures
have different parents in different views...). Which means it doesn't
save that much memory after all... (well, it still saves the child
references and the node duplication).

So I am wondering: is there any well-know "cure" for these type of
problems?

Thanks in advance,
Sorin Gherman

 

[Babu Kalakrishnan]

Hi,
I have the following problem: I have this large amount of data, stored
internally in some custom data structures (hierarchical ones, given
the nature of the data - I am using some custom, tree-like structurs
for this).
I have to show the different relations within this data using several
JTrees - one per "viewpoint". (The good news is that I only have to
show one type of view at a time.)

The first implementation I came up with is just creating a tree model
(using DefaultMutableTreeNodes) for each view, and displaying it using
JTree.
Problem: memory consumption is very high (due to the duplication of
the tree structures in memory, and the fact that the user may toggle
between views fast, thus trees are created and thrown away - again,
memory is an issue so I cannot affort to store them).

I read about the possibility to have my own internal data implement
the TreeNode interface, and have the JTree uses the internal data
directly - thus avoiding the creation of extra trees.
Problem: the implementation gets very complicated very fast due to the
different views. I can still live with that, but then I realized that
due to TreeNode.getParent() method, I would still have to duplicate,
in memory, for each view, at least the "parent" references. (This is
due to the fact that most of the internal nodes in my data structures
have different parents in different views...). Which means it doesn't
save that much memory after all... (well, it still saves the child
references and the node duplication).

The TreeModel interface never assumes that each node implements the TreeNode
interface, they can be objects of any type, so strictly speaking, a getParent()
method never needs to be implemented.

All you require in the model is the definition of a root object, and the ability
to provide a list of children for a given node - I would assume that your data
structure would allow at least this to be available somewhere (or at least
computable on the fly) since you say the structure is tree-like in nature.

Yes - implementing the event releated methods can be a little bit tricky, but if
you look carefully, all the related classes (such as TreePath) also require only
objects, and as long as you can define the tree from top down, you should be
able to create the appropriate events. (Of course it would be very much easier
if the tree weren't mutable at all).

BK

 

[ak]

I have the following problem: I have this large amount of data, stored

internally in some custom data structures (hierarchical ones, given
the nature of the data - I am using some custom, tree-like structurs
for this).
I have to show the different relations within this data using several
JTrees - one per "viewpoint". (The good news is that I only have to
show one type of view at a time.)

TreeModel is just an interface.
Make your custom data structure implement TreeModel.

 

[Will Hartung]

Hi,
I have the following problem: I have this large amount of data, stored
internally in some custom data structures (hierarchical ones, given
the nature of the data - I am using some custom, tree-like structurs
for this).
I have to show the different relations within this data using several
JTrees - one per "viewpoint". (The good news is that I only have to
show one type of view at a time.)

All I did was simply implement my own TreeModel.

The TreeModel was simply an interface to the real data, and didn't need to
consume any more memory that the underlying structure.

For my data it was simple to do (I even had cycles in my data), and very
fast when all said and done.

I loaded up over 1.5 million nodes in that tree, and once loaded, the JTree
could have cared less whether it was 1 or 1 million items, the performance
was the same -- snappy.

You can try implementing a custom TreeModel for each of your different
views, but all on top of the same base data, just navigating it differently.

Regards,

Will Hartung
([email protected])

 

[Rogan Dawes]

Hi,
I have the following problem: I have this large amount of data, stored
internally in some custom data structures (hierarchical ones, given
the nature of the data - I am using some custom, tree-like structurs
for this).
I have to show the different relations within this data using several
JTrees - one per "viewpoint". (The good news is that I only have to
show one type of view at a time.)

The first implementation I came up with is just creating a tree model
(using DefaultMutableTreeNodes) for each view, and displaying it using
JTree.
Problem: memory consumption is very high (due to the duplication of
the tree structures in memory, and the fact that the user may toggle
between views fast, thus trees are created and thrown away - again,
memory is an issue so I cannot affort to store them).

I read about the possibility to have my own internal data implement
the TreeNode interface, and have the JTree uses the internal data
directly - thus avoiding the creation of extra trees.
Problem: the implementation gets very complicated very fast due to the
different views. I can still live with that, but then I realized that
due to TreeNode.getParent() method, I would still have to duplicate,
in memory, for each view, at least the "parent" references. (This is
due to the fact that most of the internal nodes in my data structures
have different parents in different views...). Which means it doesn't
save that much memory after all... (well, it still saves the child
references and the node duplication).

So I am wondering: is there any well-know "cure" for these type of
problems?

Thanks in advance,
Sorin Gherman

All the previous replies have suggested using a TreeModel, this is just
a suggestion to derive your implementation of TreeModel from
AbstractTreeModel
(http://www.chka.de/swing/tree/AbstractTreeModel.java), which is a nice
base implementation that you can derive from.

http://www.chka.de/swing/tree/ has some nice explanations of various
swing models.

This link has some examples of working with trees and treemodels:

http://java.sun.com/products/jfc/tsc/articles/jtree/

I would not use the AbstractTreeModel that they provide, as it simply
delegates to TreeModelSupport, which just seems klunky to me.

Rogan

 

[Sorin Gherman]

All the previous replies have suggested using a TreeModel, this is just
a suggestion to derive your implementation of TreeModel from
AbstractTreeModel
(http://www.chka.de/swing/tree/AbstractTreeModel.java), which is a nice
base implementation that you can derive from.

http://www.chka.de/swing/tree/ has some nice explanations of various
swing models.

This link has some examples of working with trees and treemodels:

http://java.sun.com/products/jfc/tsc/articles/jtree/

I would not use the AbstractTreeModel that they provide, as it simply
delegates to TreeModelSupport, which just seems klunky to me.

Rogan

Thanks a lot everybody for your answers!
So, the consensus seems to be that the way to go in order to
avoid data duplication is to implement TreeModel (either directly, or
through one particular AbstractTreeModel implementation).
One problem I see with this is related to a feature request I
didn't mention (just realized it's a problem after trying to implement
TreeModel): the user is supposed to be able to sort the tree levels in
any of the views (by different criteria, dependent on the node
information).
Sorting the underlying data structures is a solution only for the
"natural view" (the one that happen to match the data structure 1:1),
but for a view that's dynamically generated (by skipping one
hierarchical level, for instance) this is not an option.
Is it still possible to use the "implementing the TreeModel"
solution and still support tree sorting (AND not use extra memory? I
understand that I can make my TreeModel implementations for getChild
and getChildIndex aware of the sort order required, but this either
seems to me to:
1. complicate the implementation a lot and make it slower (I would
also have to implement my own sorting too, since the nodes of some
tree levels are not together in the same collection in the underlying
data structure)
or
2. eat up memory, in case I store each tree level references in some
collections and sort them

Any thoughts on this one?

Thanks in advance,
Sorin Gherman

 

[Rogan Dawes]

Thanks a lot everybody for your answers!
So, the consensus seems to be that the way to go in order to
avoid data duplication is to implement TreeModel (either directly, or
through one particular AbstractTreeModel implementation).
One problem I see with this is related to a feature request I
didn't mention (just realized it's a problem after trying to implement
TreeModel): the user is supposed to be able to sort the tree levels in
any of the views (by different criteria, dependent on the node
information).
Sorting the underlying data structures is a solution only for the
"natural view" (the one that happen to match the data structure 1:1),
but for a view that's dynamically generated (by skipping one
hierarchical level, for instance) this is not an option.
Is it still possible to use the "implementing the TreeModel"
solution and still support tree sorting (AND not use extra memory? I
understand that I can make my TreeModel implementations for getChild
and getChildIndex aware of the sort order required, but this either
seems to me to:
1. complicate the implementation a lot and make it slower (I would
also have to implement my own sorting too, since the nodes of some
tree levels are not together in the same collection in the underlying
data structure)
or
2. eat up memory, in case I store each tree level references in some
collections and sort them

Any thoughts on this one?

Thanks in advance,
Sorin Gherman

Hi Sorin,

In that case, you would probably want to use an "adapter pattern" to
perform the sorting for you.

For an article discussing this technique, see
http://www-106.ibm.com/developerworks/java/library/j-filters/

You would obviously want to do this for a TreeModel, not a ListModel,
but the principles should still apply.

The important thing is that you are not duplicating your data, you are
simply adding the minimum information required to describe your
differences from the underlying data model.

Regards,

Rogan