An idea

R

Ross Bamford

Hi,

Last night I was working on some ideas for my current project, but got
sidetracked while playing with the object persistence and came up with
an idea for a more general persistence system. I've not really given it
a *lot* of thought yet but would be interested in anyone's thoughts...

It's much the same as Serializable, but it doesn't require that objects
implement that (or any other) interface. Instead it simply maps instance
fields (whether public, protected, private, or package) directly to
database fields. All fields are stored, unless they're marked
'transient'. By default, the framework maps to database fields based on
the field's name, but this can be overriden by annotating that field
with a custom annotation (@serializeTo("fieldName") for example).

public Object loadObject(String id);
public void storeObject(String id, Object o);

When asked to store an object, it's a simple thing to loop through the
class and subclasses building an array of all instance vars with that
annotation. setAccessible on that array (obviously would need permission
but then it's a persistence system!), and get their values. You can then
store these however you like - Files, JDBC, whatever. Obviously you need
to keep members with their appropriate (super)class because they have to
be set again at load (and may be private).

When loading an object, you'd first need to load a header field that
graphs the hierarchy of the stored class, much like current
Serialisation. You can run through the serialVersionUIDs (by convention)
to check you've got the same classes, and then find all it's annotated
members in the same way and load 'em.

The persistence framework could allow mappings to be set up, e.g.
SomeClassName.data -> dataField to override the mapping of members to
fields, or a custom data format could be used.

So the programmer could make simple classes:

public class DataClass
{
private @serializeTo("data") String dataField; /* to: 'data' */
private int boolField; /* to: 'boolField' */
private transient AnotherClass anotherObject; /* Not stored */
}

and just throw instances at the storage mechanism.

So far I've thought of these problems:

Not-quite-backward compatibility, although it could treat existing
Serializable objects specially if required. It *would* respect those
classes stipulations about what *not* to store, however. It could invoke
their read/writeObject methods on it's own stream, and write that data
to support their custom stuff maybe.

+ Non-primitive fields would need some thought.

+ General backward compatibility would cause lots of issues.

I'm not sure what the benefits are, transparent storage of arbitrary
objects is attractive but not that far off right now. It's just that
something is telling me there's more in this ...

Cheers,
Ross
 
D

Daniel Dyer

Hi,

Last night I was working on some ideas for my current project, but got
sidetracked while playing with the object persistence and came up with
an idea for a more general persistence system. I've not really given it
a *lot* of thought yet but would be interested in anyone's thoughts...

What advantages does your suggestion have over Hibernate
(http://www.hibernate.org), OJB (http://db.apache.org/ojb/), Castor JDO
(http://www.castor.org) or any of the many other object-relational mapping
tools?

Dan.
 
C

Chris Smith

Ross Bamford said:
Hi,

Last night I was working on some ideas for my current project, but got
sidetracked while playing with the object persistence and came up with
an idea for a more general persistence system. I've not really given it
a *lot* of thought yet but would be interested in anyone's thoughts...

You're ignoring the hard problems. For example:

1. How do you map associations between objects? By this I mean, for
example: how do you represent that an object of class A contains a map
with keys of class B and values of class C?

2. How do you persist more data than there exists primary memory to
hold? How do you allow an application to get back an interesting subset
of that data in order to perform some operation with it?

3. Is your persistence layer designed to work in a transactional system?
If so, what is the relationship between your objects and transactions?

4. In the case of a relational database, how do you get your software to
match an existing relational schema? This isn't strictly required, but
without that, you're greatly limiting the scope in which your
persistence technology is useful.

4a. What do you do with a database table that has fifty-seven fields, if
you want those fields to map to several different objects?

4b. How do you create an object whose fields are scattered through
several database tables with one-to-one relationships (or even are read-
only and represent aggregate functions in the database, or some such
thing)?

There are persistence technologies that do all of these things, and many
of them look surprisingly like yours (see EJB 3.0's annotations, for
example). However, they have all gone far beyond where you are, and
dealt with difficult problems as well.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation
 
D

Daniel Dyer

I don't know. That's the point of discussing it.

Ross

OK, I haven't given it much thought either but here's one thing on the
minus side:

It's not non-invasive. The source files for the classes you want to
persist are coupled to the persistence mechanism. With Hibernate and
Castor at least, the mapping is separated out from the model.

Dan.
 
R

Ross Bamford

Quick point: I think I've failed to make clear that I've no intention of
developing this - it was just an abstract idea ... I'm far too busy with
what I am already working on :)
You're ignoring the hard problems. For example:
1. How do you map associations between objects? By this I mean, for
example: how do you represent that an object of class A contains a map
with keys of class B and values of class C?

Yes, I'd not considered this factor in the crossover from a custom
format to database schema. It would be a nightmare.
2. How do you persist more data than there exists primary memory to
hold? How do you allow an application to get back an interesting subset
of that data in order to perform some operation with it?

It's not about persisting data - the idea was more something that one
could write a simple class and just store, without consideration of
field mapping or whatever.
3. Is your persistence layer designed to work in a transactional system?
If so, what is the relationship between your objects and transactions?

Not considered.
4. In the case of a relational database, how do you get your software to
match an existing relational schema? This isn't strictly required, but
without that, you're greatly limiting the scope in which your
persistence technology is useful.

That is something that wasn't part of the original idea, simply because
in the system I'm working on (which has a very specific requirement to
store specific classes for a certain use) it isn't an issue. Again, it
depends I guess on what it's aimed at.

This was the point - for more advanced uses you would obviously use
Hibernate or similar - this was more a lightweight thing that you could
use with e.g:

StoreManager m = StoreManager.getManager("myStore",mappings);
m.store(someObject);

Without thinking into how the record-matching would work (the easy
solution is via equals but it's difficult to keep any requirement-free
that way).
4a. What do you do with a database table that has fifty-seven fields, if
you want those fields to map to several different objects?

Originally I'd thought about mapping based on the instance field's name,
whereby the container is configured via a Map, mapping names like
'AnObject.fieldName' to database fields. If no mapping for the specific
name is found, it would try 'fieldName' (for example) before giving up.

This could be overriden with a parameter to the annotation perhaps (not
sure about giving the object that control though).
4b. How do you create an object whose fields are scattered through
several database tables with one-to-one relationships (or even are read-
only and represent aggregate functions in the database, or some such
thing)?

Not possible.
There are persistence technologies that do all of these things, and many
of them look surprisingly like yours (see EJB 3.0's annotations, for
example). However, they have all gone far beyond where you are, and
dealt with difficult problems as well.

I am not surprised, seeing as this idea has been in development for all
of ten hours ;)

In seriousness, I am aware that there are many existing systems (I use
Hibernate extensively myself). What this was about was an altogether
more lightweight solution for simple storage of objects, and as I say I
only posted out of interest anyway. This kind of thing often pops into
my head and it's nice for a change to throw it around a bit ...

Thanks for your input,
Ross
 
R

Ross Bamford

OK, I haven't given it much thought either but here's one thing on the
minus side:

It's not non-invasive. The source files for the classes you want to
persist are coupled to the persistence mechanism. With Hibernate and
Castor at least, the mapping is separated out from the model.

Dan.

No I agree, it didn't feel right to me either to give the object the
opportunity to affect it's mappings.

I suppose it would be more mapping based on field's name
(AnObject.aField), which the container is configured to map to given
database fields. It could try just 'aField' if that's not found.

This is still broken though, and now the container needs foreknowledge
of the classes that'll be stored, or allowed to map completely different
fields to the same database fields.

Perhaps something like name mangling ... ? (<g>)

Ross
 
E

enrique

I guess this is something that could be handy in a server farm where
individual requests could go anywhere, but you needed to maintain state
across the session. I've seen something like this in a past project I
was involved in.

The thing about it is the cost of the database operations, compared to
the cost of serialization of objects. So I guess it would depend on
the application of this strategy to determine whether or not it's a
good idea.
 
C

CodeFutures

It's just that
something is telling me there's more in this ...


It was very interesting to read your thought processes as you stumbled
onto the vast area of Java persistence. You've taken a good logical
approach ... however, there's really no point in reinventing
everything. You should leverage what has been done before. Since
you're keen on OSS, I'd guess that you're going to end up looking at
Hibernate pretty soon.



Choosing a Java persistence strategy:

http://www.codefutures.com/weblog/andygrove/archives/2005/01/choosing_a_java.html




Java persistence technology comparison:

http://www.codefutures.com/weblog/corporate/archives/2005/02/data_persistenc.html


Regards
PJ

PJ Murray
CodeFutures Software
 
R

Ross Bamford

It was very interesting to read your thought processes as you stumbled
onto the vast area of Java persistence. You've taken a good logical
approach ... however, there's really no point in reinventing
everything. You should leverage what has been done before. Since
you're keen on OSS, I'd guess that you're going to end up looking at
Hibernate pretty soon.

I've been using Hibernate for quite some time, and in fact use it
currently as the backing store for my project. It was while working on
some integration issues that this silly idea popped into my head and I
thought I'd see if there was mileage in it.

Not to develop it, you understand, but as a purely theoretical "Bounce
this around" kind of thing (I'm deprived of communication 8,( )

I 'stumbled' onto Java Persistence before Java was called Java.

Ross
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,766
Messages
2,569,569
Members
45,042
Latest member
icassiem

Latest Threads

Top