An idea

Discussion in 'Java' started by Ross Bamford, May 13, 2005.

  1. Ross Bamford

    Ross Bamford Guest

    Hi,

    Last night I was working on some ideas for my current project, but got
    sidetracked while playing with the object persistence and came up with
    an idea for a more general persistence system. I've not really given it
    a *lot* of thought yet but would be interested in anyone's thoughts...

    It's much the same as Serializable, but it doesn't require that objects
    implement that (or any other) interface. Instead it simply maps instance
    fields (whether public, protected, private, or package) directly to
    database fields. All fields are stored, unless they're marked
    'transient'. By default, the framework maps to database fields based on
    the field's name, but this can be overriden by annotating that field
    with a custom annotation (@serializeTo("fieldName") for example).

    public Object loadObject(String id);
    public void storeObject(String id, Object o);

    When asked to store an object, it's a simple thing to loop through the
    class and subclasses building an array of all instance vars with that
    annotation. setAccessible on that array (obviously would need permission
    but then it's a persistence system!), and get their values. You can then
    store these however you like - Files, JDBC, whatever. Obviously you need
    to keep members with their appropriate (super)class because they have to
    be set again at load (and may be private).

    When loading an object, you'd first need to load a header field that
    graphs the hierarchy of the stored class, much like current
    Serialisation. You can run through the serialVersionUIDs (by convention)
    to check you've got the same classes, and then find all it's annotated
    members in the same way and load 'em.

    The persistence framework could allow mappings to be set up, e.g.
    SomeClassName.data -> dataField to override the mapping of members to
    fields, or a custom data format could be used.

    So the programmer could make simple classes:

    public class DataClass
    {
    private @serializeTo("data") String dataField; /* to: 'data' */
    private int boolField; /* to: 'boolField' */
    private transient AnotherClass anotherObject; /* Not stored */
    }

    and just throw instances at the storage mechanism.

    So far I've thought of these problems:

    Not-quite-backward compatibility, although it could treat existing
    Serializable objects specially if required. It *would* respect those
    classes stipulations about what *not* to store, however. It could invoke
    their read/writeObject methods on it's own stream, and write that data
    to support their custom stuff maybe.

    + Non-primitive fields would need some thought.

    + General backward compatibility would cause lots of issues.

    I'm not sure what the benefits are, transparent storage of arbitrary
    objects is attractive but not that far off right now. It's just that
    something is telling me there's more in this ...

    Cheers,
    Ross

    --
    [Ross A. Bamford] [ross AT the.website.domain]
    Roscopeco Open Tech ++ Open Source + Java + Apache + CMF
    http://www.roscopec0.f9.co.uk/ + in
     
    Ross Bamford, May 13, 2005
    #1
    1. Advertising

  2. Ross Bamford

    Daniel Dyer Guest

    On Fri, 13 May 2005 13:27:14 +0100, Ross Bamford <> wrote:

    > Hi,
    >
    > Last night I was working on some ideas for my current project, but got
    > sidetracked while playing with the object persistence and came up with
    > an idea for a more general persistence system. I've not really given it
    > a *lot* of thought yet but would be interested in anyone's thoughts...


    What advantages does your suggestion have over Hibernate
    (http://www.hibernate.org), OJB (http://db.apache.org/ojb/), Castor JDO
    (http://www.castor.org) or any of the many other object-relational mapping
    tools?

    Dan.

    --
    Daniel Dyer
    http://www.footballpredictions.net
     
    Daniel Dyer, May 13, 2005
    #2
    1. Advertising

  3. Ross Bamford

    Ross Bamford Guest

    On Fri, 2005-05-13 at 13:46 +0100, Daniel Dyer wrote:
    > On Fri, 13 May 2005 13:27:14 +0100, Ross Bamford <> wrote:
    >
    > > Hi,
    > >
    > > Last night I was working on some ideas for my current project, but got
    > > sidetracked while playing with the object persistence and came up with
    > > an idea for a more general persistence system. I've not really given it
    > > a *lot* of thought yet but would be interested in anyone's thoughts...

    >
    > What advantages does your suggestion have over Hibernate
    > (http://www.hibernate.org), OJB (http://db.apache.org/ojb/), Castor JDO
    > (http://www.castor.org) or any of the many other object-relational mapping
    > tools?


    I don't know. That's the point of discussing it.

    Ross

    --
    [Ross A. Bamford] [ross AT the.website.domain]
    Roscopeco Open Tech ++ Open Source + Java + Apache + CMF
    http://www.roscopec0.f9.co.uk/ + in
     
    Ross Bamford, May 13, 2005
    #3
  4. Ross Bamford

    Chris Smith Guest

    Ross Bamford <> wrote:
    > Hi,
    >
    > Last night I was working on some ideas for my current project, but got
    > sidetracked while playing with the object persistence and came up with
    > an idea for a more general persistence system. I've not really given it
    > a *lot* of thought yet but would be interested in anyone's thoughts...


    You're ignoring the hard problems. For example:

    1. How do you map associations between objects? By this I mean, for
    example: how do you represent that an object of class A contains a map
    with keys of class B and values of class C?

    2. How do you persist more data than there exists primary memory to
    hold? How do you allow an application to get back an interesting subset
    of that data in order to perform some operation with it?

    3. Is your persistence layer designed to work in a transactional system?
    If so, what is the relationship between your objects and transactions?

    4. In the case of a relational database, how do you get your software to
    match an existing relational schema? This isn't strictly required, but
    without that, you're greatly limiting the scope in which your
    persistence technology is useful.

    4a. What do you do with a database table that has fifty-seven fields, if
    you want those fields to map to several different objects?

    4b. How do you create an object whose fields are scattered through
    several database tables with one-to-one relationships (or even are read-
    only and represent aggregate functions in the database, or some such
    thing)?

    There are persistence technologies that do all of these things, and many
    of them look surprisingly like yours (see EJB 3.0's annotations, for
    example). However, they have all gone far beyond where you are, and
    dealt with difficult problems as well.

    --
    www.designacourse.com
    The Easiest Way To Train Anyone... Anywhere.

    Chris Smith - Lead Software Developer/Technical Trainer
    MindIQ Corporation
     
    Chris Smith, May 13, 2005
    #4
  5. Ross Bamford

    Daniel Dyer Guest

    On Fri, 13 May 2005 13:55:33 +0100, Ross Bamford <> wrote:

    > On Fri, 2005-05-13 at 13:46 +0100, Daniel Dyer wrote:
    >> On Fri, 13 May 2005 13:27:14 +0100, Ross Bamford <>
    >> wrote:
    >>
    >> > Hi,
    >> >
    >> > Last night I was working on some ideas for my current project, but got
    >> > sidetracked while playing with the object persistence and came up with
    >> > an idea for a more general persistence system. I've not really given

    >> it
    >> > a *lot* of thought yet but would be interested in anyone's thoughts...

    >>
    >> What advantages does your suggestion have over Hibernate
    >> (http://www.hibernate.org), OJB (http://db.apache.org/ojb/), Castor JDO
    >> (http://www.castor.org) or any of the many other object-relational
    >> mapping
    >> tools?

    >
    > I don't know. That's the point of discussing it.
    >
    > Ross


    OK, I haven't given it much thought either but here's one thing on the
    minus side:

    It's not non-invasive. The source files for the classes you want to
    persist are coupled to the persistence mechanism. With Hibernate and
    Castor at least, the mapping is separated out from the model.

    Dan.

    --
    Daniel Dyer
    http://www.footballpredictions.net
     
    Daniel Dyer, May 13, 2005
    #5
  6. Ross Bamford

    Ross Bamford Guest

    On Fri, 2005-05-13 at 07:17 -0600, Chris Smith wrote:
    > Ross Bamford <> wrote:
    > > Hi,
    > >
    > > Last night I was working on some ideas for my current project, but got
    > > sidetracked while playing with the object persistence and came up with
    > > an idea for a more general persistence system. I've not really given it
    > > a *lot* of thought yet but would be interested in anyone's thoughts...

    >


    Quick point: I think I've failed to make clear that I've no intention of
    developing this - it was just an abstract idea ... I'm far too busy with
    what I am already working on :)

    >
    > You're ignoring the hard problems. For example:


    > 1. How do you map associations between objects? By this I mean, for
    > example: how do you represent that an object of class A contains a map
    > with keys of class B and values of class C?
    >


    Yes, I'd not considered this factor in the crossover from a custom
    format to database schema. It would be a nightmare.

    > 2. How do you persist more data than there exists primary memory to
    > hold? How do you allow an application to get back an interesting subset
    > of that data in order to perform some operation with it?


    It's not about persisting data - the idea was more something that one
    could write a simple class and just store, without consideration of
    field mapping or whatever.

    >
    > 3. Is your persistence layer designed to work in a transactional system?
    > If so, what is the relationship between your objects and transactions?


    Not considered.

    >
    > 4. In the case of a relational database, how do you get your software to
    > match an existing relational schema? This isn't strictly required, but
    > without that, you're greatly limiting the scope in which your
    > persistence technology is useful.
    >


    That is something that wasn't part of the original idea, simply because
    in the system I'm working on (which has a very specific requirement to
    store specific classes for a certain use) it isn't an issue. Again, it
    depends I guess on what it's aimed at.

    This was the point - for more advanced uses you would obviously use
    Hibernate or similar - this was more a lightweight thing that you could
    use with e.g:

    StoreManager m = StoreManager.getManager("myStore",mappings);
    m.store(someObject);

    Without thinking into how the record-matching would work (the easy
    solution is via equals but it's difficult to keep any requirement-free
    that way).

    > 4a. What do you do with a database table that has fifty-seven fields, if
    > you want those fields to map to several different objects?


    Originally I'd thought about mapping based on the instance field's name,
    whereby the container is configured via a Map, mapping names like
    'AnObject.fieldName' to database fields. If no mapping for the specific
    name is found, it would try 'fieldName' (for example) before giving up.

    This could be overriden with a parameter to the annotation perhaps (not
    sure about giving the object that control though).

    >
    > 4b. How do you create an object whose fields are scattered through
    > several database tables with one-to-one relationships (or even are read-
    > only and represent aggregate functions in the database, or some such
    > thing)?


    Not possible.

    >
    > There are persistence technologies that do all of these things, and many
    > of them look surprisingly like yours (see EJB 3.0's annotations, for
    > example). However, they have all gone far beyond where you are, and
    > dealt with difficult problems as well.
    >


    I am not surprised, seeing as this idea has been in development for all
    of ten hours ;)

    In seriousness, I am aware that there are many existing systems (I use
    Hibernate extensively myself). What this was about was an altogether
    more lightweight solution for simple storage of objects, and as I say I
    only posted out of interest anyway. This kind of thing often pops into
    my head and it's nice for a change to throw it around a bit ...

    Thanks for your input,
    Ross

    --
    [Ross A. Bamford] [ross AT the.website.domain]
    Roscopeco Open Tech ++ Open Source + Java + Apache + CMF
    http://www.roscopec0.f9.co.uk/ + in
     
    Ross Bamford, May 13, 2005
    #6
  7. Ross Bamford

    Ross Bamford Guest

    On Fri, 2005-05-13 at 14:21 +0100, Daniel Dyer wrote:
    > On Fri, 13 May 2005 13:55:33 +0100, Ross Bamford <> wrote:
    >
    > > On Fri, 2005-05-13 at 13:46 +0100, Daniel Dyer wrote:
    > >> On Fri, 13 May 2005 13:27:14 +0100, Ross Bamford <>
    > >> wrote:
    > >>
    > >> > Hi,
    > >> >
    > >> > Last night I was working on some ideas for my current project, but got
    > >> > sidetracked while playing with the object persistence and came up with
    > >> > an idea for a more general persistence system. I've not really given
    > >> it
    > >> > a *lot* of thought yet but would be interested in anyone's thoughts...
    > >>
    > >> What advantages does your suggestion have over Hibernate
    > >> (http://www.hibernate.org), OJB (http://db.apache.org/ojb/), Castor JDO
    > >> (http://www.castor.org) or any of the many other object-relational
    > >> mapping
    > >> tools?

    > >
    > > I don't know. That's the point of discussing it.
    > >
    > > Ross

    >
    > OK, I haven't given it much thought either but here's one thing on the
    > minus side:
    >
    > It's not non-invasive. The source files for the classes you want to
    > persist are coupled to the persistence mechanism. With Hibernate and
    > Castor at least, the mapping is separated out from the model.
    >
    > Dan.
    >


    No I agree, it didn't feel right to me either to give the object the
    opportunity to affect it's mappings.

    I suppose it would be more mapping based on field's name
    (AnObject.aField), which the container is configured to map to given
    database fields. It could try just 'aField' if that's not found.

    This is still broken though, and now the container needs foreknowledge
    of the classes that'll be stored, or allowed to map completely different
    fields to the same database fields.

    Perhaps something like name mangling ... ? (<g>)

    Ross

    --
    [Ross A. Bamford] [ross AT the.website.domain]
    Roscopeco Open Tech ++ Open Source + Java + Apache + CMF
    http://www.roscopec0.f9.co.uk/ + in
     
    Ross Bamford, May 13, 2005
    #7
  8. Ross Bamford

    Daniel Dyer Guest

    On Fri, 13 May 2005 15:13:58 +0100, Ross Bamford <> wrote:
    >
    > This was the point - for more advanced uses you would obviously use
    > Hibernate or similar - this was more a lightweight thing that you could
    > use with e.g:
    >
    > StoreManager m = StoreManager.getManager("myStore",mappings);
    > m.store(someObject);
    >


    Prevayler (http://www.prevayler.org/wiki.jsp) might be an alternative.

    Dan.

    --
    Daniel Dyer
    http://www.footballpredictions.net
     
    Daniel Dyer, May 13, 2005
    #8
  9. Ross Bamford

    enrique Guest

    I guess this is something that could be handy in a server farm where
    individual requests could go anywhere, but you needed to maintain state
    across the session. I've seen something like this in a past project I
    was involved in.

    The thing about it is the cost of the database operations, compared to
    the cost of serialization of objects. So I guess it would depend on
    the application of this strategy to determine whether or not it's a
    good idea.
     
    enrique, May 13, 2005
    #9
  10. Ross Bamford

    CodeFutures Guest

    > It's just that
    > something is telling me there's more in this ...
    >



    It was very interesting to read your thought processes as you stumbled
    onto the vast area of Java persistence. You've taken a good logical
    approach ... however, there's really no point in reinventing
    everything. You should leverage what has been done before. Since
    you're keen on OSS, I'd guess that you're going to end up looking at
    Hibernate pretty soon.



    Choosing a Java persistence strategy:

    http://www.codefutures.com/weblog/andygrove/archives/2005/01/choosing_a_java.html




    Java persistence technology comparison:

    http://www.codefutures.com/weblog/corporate/archives/2005/02/data_persistenc.html


    Regards
    PJ

    PJ Murray
    CodeFutures Software
     
    CodeFutures, May 14, 2005
    #10
  11. Ross Bamford

    Ross Bamford Guest

    On Sat, 2005-05-14 at 05:20 -0700, CodeFutures wrote:
    > > It's just that
    > > something is telling me there's more in this ...
    > >

    >
    >
    > It was very interesting to read your thought processes as you stumbled
    > onto the vast area of Java persistence. You've taken a good logical
    > approach ... however, there's really no point in reinventing
    > everything. You should leverage what has been done before. Since
    > you're keen on OSS, I'd guess that you're going to end up looking at
    > Hibernate pretty soon.
    >
    >


    I've been using Hibernate for quite some time, and in fact use it
    currently as the backing store for my project. It was while working on
    some integration issues that this silly idea popped into my head and I
    thought I'd see if there was mileage in it.

    Not to develop it, you understand, but as a purely theoretical "Bounce
    this around" kind of thing (I'm deprived of communication 8,( )

    I 'stumbled' onto Java Persistence before Java was called Java.

    Ross

    >
    > Choosing a Java persistence strategy:
    >
    > http://www.codefutures.com/weblog/andygrove/archives/2005/01/choosing_a_java.html
    >
    >
    >
    >
    > Java persistence technology comparison:
    >
    > http://www.codefutures.com/weblog/corporate/archives/2005/02/data_persistenc.html
    >
    >
    > Regards
    > PJ
    >
    > PJ Murray
    > CodeFutures Software


    --
    [Ross A. Bamford] [ross AT the.website.domain]
    Roscopeco Open Tech ++ Open Source + Java + Apache + CMF
    http://www.roscopec0.f9.co.uk/ + in
     
    Ross Bamford, May 14, 2005
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. senthil
    Replies:
    0
    Views:
    621
    senthil
    Nov 21, 2003
  2. senthil

    ifft coding in vhdl give idea

    senthil, Dec 9, 2003, in forum: VHDL
    Replies:
    1
    Views:
    1,834
  3. Replies:
    10
    Views:
    1,243
    Big K
    Feb 2, 2005
  4. Dr Mephesto

    App idea, Any idea on implementation?

    Dr Mephesto, Feb 4, 2008, in forum: Python
    Replies:
    3
    Views:
    720
    Dennis Lee Bieber
    Feb 5, 2008
  5. Replies:
    0
    Views:
    635
Loading...

Share This Page