Right use for JTA transactions?

[Scott Balmos]

Hi all,

I've asked this around a few places (especially forum.java.sun.com)
today without being answered. I'm trying to figure out whether my
thought process is right for using JTA transactions.

I'm writing servlets, with most data in SQL, persisted via Hibernate.
However, I also have lots of data in LDAP, IMAP, and the filesystem. I
know JDBC (and really, Hibernate more generally) covers transaction
handling for the SQL side of things. But I was hoping to layer on top a
more general transaction scheme, covering a complete request that may
do changes to all of the above datastores.

My best example is creating a new user. The operation could be creating
the account in LDAP, creating the home directory in the filesystem,
creating the mailbox in IMAP, and then adding some billing info to SQL.
If any one of those operations fails, the whole thing should be rolled
back.

I know LDAP (accessed via JNDI), IMAP (via JavaMail), and the
filesystem don't have a concept of a transaction. So am I thinking
along the right lines of writing an XAResource plugin to JTA for each
of these datastores, which would emulate the commit/rollback/etc
functionality?

Let me know if this seems sane, or if I'm way off base. Thanks!

--Scott

 

[Lee Fesperman]

Hi all,

I've asked this around a few places (especially forum.java.sun.com)
today without being answered. I'm trying to figure out whether my
thought process is right for using JTA transactions.

I'm writing servlets, with most data in SQL, persisted via Hibernate.
However, I also have lots of data in LDAP, IMAP, and the filesystem. I
know JDBC (and really, Hibernate more generally) covers transaction
handling for the SQL side of things. But I was hoping to layer on top a
more general transaction scheme, covering a complete request that may
do changes to all of the above datastores.

My best example is creating a new user. The operation could be creating
the account in LDAP, creating the home directory in the filesystem,
creating the mailbox in IMAP, and then adding some billing info to SQL.
If any one of those operations fails, the whole thing should be rolled
back.

I know LDAP (accessed via JNDI), IMAP (via JavaMail), and the
filesystem don't have a concept of a transaction. So am I thinking
along the right lines of writing an XAResource plugin to JTA for each
of these datastores, which would emulate the commit/rollback/etc
functionality?

Let me know if this seems sane, or if I'm way off base. Thanks!

Yes, it is possible, though JTA is a little hard to get your arms around. Having done an
XAResource for a DBMS, I might be able to offer some aid. Perhaps you could give an
outline on how you intend to emulate transactions on those facilities.

This is certainly an area where it would nice if someone put together some usable
software. I don't know LDAP, but it would be shame if noone provided a transactional
capability for it.

 

[Scott Balmos]

Yes, it is possible, though JTA is a little hard to get your arms
around. Having done an XAResource for a DBMS, I might be able to offer
some aid. Perhaps you could give an outline on how you intend to
emulate transactions on those facilities.

This is certainly an area where it would nice if someone put together
some usable software. I don't know LDAP, but it would be shame if noone
provided a transactional capability for it.

I know OpenLDAP, in some future version, will be implementing LDAP
transactions (it's in their roadmap). IIRC, LDAP transactions is a
proposed protocol extension. But anyway...

I'm not sure how this is handled in general with JDBC, or XAResources
in general. Far as I've read, most people start a UserTransaction, grab
a JDBC connection, and somehow magically JTA tracks all changes. Do
JDBC driver classes have some hook that when they're instantiated, they
check for a JTA transaction, and register themselves there if one
exists?

I used to write a very rudimentary transaction emulator for LDAP, IMAP
folder management (not messages, but folders), and filesystems. It
centered around internally tracking what exact opposite operation would
be neded for a rollback. For a filesystem file/directory delete, for
example, I actually just did a temporary directory rename. If the
transaction committed, the delete actually took place. Otherwise, it's
just a rename back to the original name. For LDAP, if you add an entry,
you note the full DN, and do a delete if needed. For attribute changes,
you note what the previous values were, etc. You get the idea.

I just need to know how XAResources are generally registered. I can't
see something working like foo = ut.getIMAPFolder("folder.blah"), and
the UserTransaction would watch all actions on Folder object foo,
especially since I could do foo.getParent().delete() ... How does the
transaction know that I grabbed the Folder object of the parent?

Any general insight? Thanks!

--Scott

 

[Lee Fesperman]

I know OpenLDAP, in some future version, will be implementing LDAP
transactions (it's in their roadmap). IIRC, LDAP transactions is a
proposed protocol extension. But anyway...

That would be a good thing.

I'm not sure how this is handled in general with JDBC, or XAResources
in general. Far as I've read, most people start a UserTransaction, grab
a JDBC connection, and somehow magically JTA tracks all changes. Do
JDBC driver classes have some hook that when they're instantiated, they
check for a JTA transaction, and register themselves there if one
exists?

An XAResource is basically a passive object ... for the resource (like JDBC, ...)
anyway. The resource does the proper processing when called from the container software.
That is, the resource tracks the changes so it can commit or rollback when directed to
by the holder of the XAResource object. If possible, the resource should implement the
recovery facility of XAResource, but that is not mandatory.

To illustrate:

+ The container calls start() in XAResource. The resource initiates its transaction
mechanism and remembers that subsequent operations are under this distributed
transaction.

+ The container then calls regular modification operations on the resource (outside the
XAResource). For JDBC, it would do executeXXX(), etc. on statements using the Connection
object associated with the XAResource.

+ The container calls end() in XAResource. The resource 'disconnects' future operations
from the distributed transaction but keeps the transaction 'open' for later completion.

+ The container calls prepare()/commit()/rollback()/forget() in XAResource to tell the
resource to complete the 'open' transaction.

For JDBC, the XAResource and Connection object are associated as follows... The JDBC
driver implements javax.sql.XADataSource, which provides for getting an XAConnection.
XAConnection (implemented by the driver) provides methods for getting a Connection
object and an associated XAResource. Normally, the container software retains the
XAResource object for its control and hands off the Connection object to application
software for modification, etc. operations.

You might take a look at JCA which provides a more generic framework for
XAResource/Connection objects that can be used with resources outside JDBC. For
additional info, see my JDJ article - "Understanding JCA"
(http://sys-con.com/story/?storyid=46283&de=1).

I used to write a very rudimentary transaction emulator for LDAP, IMAP
folder management (not messages, but folders), and filesystems. It
centered around internally tracking what exact opposite operation would
be neded for a rollback. For a filesystem file/directory delete, for
example, I actually just did a temporary directory rename. If the
transaction committed, the delete actually took place. Otherwise, it's
just a rename back to the original name. For LDAP, if you add an entry,
you note the full DN, and do a delete if needed. For attribute changes,
you note what the previous values were, etc. You get the idea.

Yes, that seems to be a reasonable approach which you might expand at a later time. I'm
doing some research in that area and talking to XA Vendor about releasing software that
handles file systems in a similiar manner.

I just need to know how XAResources are generally registered. I can't
see something working like foo = ut.getIMAPFolder("folder.blah"), and
the UserTransaction would watch all actions on Folder object foo,
especially since I could do foo.getParent().delete() ... How does the
transaction know that I grabbed the Folder object of the parent?

As I mentioned above, XAResources are passive and don't register themselves. It is the
responsibility of the container software to 'startup' the resource and retrieve its
XAResource. The container software (which you would write in this case) would implement
the UserTransaction and coordinate the resources. You might want to consider using
existing XA control software to provide the external capabilities.

By using XAResource, you would need to implement that class for each of your resources
(LDAP, IMAP, file system), but that would be convenient way to provide the facility. You
also would need to make a 'Connection' object for use by your application software. For
your folder resource, the Connection object would be used to handle such things as your
delete() above. IOW, the application would need to go through your connection wrapper to
do the modifications.

I hope that clarifies the situation but am ready to provide more details that you may
need. We are partners with several vendors of XA control software whom I can enlist to
help you get this together.

 

[Guy Pardon]

There are several ways of improving transactionality of your LDAP
scenario. Here are a few possibilities:

1. The easiest would be to do the following (if possible in your
application model):

Start a JTA transaction
-do your JDBC updates
-post the relevant LDAP and other updates as a JMS message, in the same
transaction
Commit/rollback the transaction
On the LDAP side,pick up the JMS message and process it.

In this case, the JMS message only gets posted (hence received by the
LDAP side) if the transaction commits (you can use our drivers to make
sure it works like that).
Additional requirement: the LDAP processing needs to succeed at some
point in time (possibly after several retries) or you have a problem.
This problem could be reported to an admin tool or so.
As long as the LDAP fails, the JMS message can remain in the queue and
available for retry.

2. The more complex way would be to write XA wrappers for the LDAP. The
complicated thing in this will be how to ensure that prepared
transactions are supported (otherwise it's not transactional at all).
Essentially, if the LDAP has no native XA support then this will amount
to the same transactional characteristics as in solution 1, so I would
opt for 1 if possible (since it is simpler and more standard J2EE).

Also, feel free to experiment with our Transactions product, it allows
you to do most of J2EE without application server (especially if
combined with Spring).

If you need help with any of these suggestions: just drop us an email.
If you decide to go for the XAResource wrapper then I can also
recommend Lee's expertise as a DBMS expert with many years of
accumulated know-how.

Best,
Guy

http://www.atomikos.com
The Transaction Processing Company

 

[sbalmos]

Hi Guy,

First off, it's not just LDAP I'm trying to impose transaction-type
"success guarantees" on. It's filesystem, IMAP, mailing lists, etc. But
anyway... Let's take your JMS-based idea for a moment, since I was
probably going to have to use JMS anyway down the line. The user home
directories, mailing list processing, etc could be on separate boxes,
mind you.

You're suggesting that the "emulated guaranteed success" is produced by
the fact that the JMS message remains in the queue if there's a failure
condition. What would be the point behind using JTA then, as opposed to
straight JDBC transactions and normal JMS message posting?

In using JMS messages, you'd have what I would consider two failure
scenarios: Temporary and permanent. If the failure is temporary (LDAP
server too busy, filesystem server is accessing that part of the
filesystem at the moment), then leaving the request in the JMS queue,
and reattempting processing a few moments later, is fine. If the
failure is permanent, however, then the LDAP server, etc sends back a
JMS reply to the app server. How would the commited JDBC data be rolled
out if we used the JTA/JMS scenario you described?

I guess the flow of processing now would be like such:

- Start JDBC transaction
- Post JDBC data
- Send JMS request to other services (LDAP, IMAP, etc)
- Receive JMS reply
- switch(JMS Reply)
- Success/Temp Fail: commit JDBC transaction
- Perm Fail: rollback JDBC transaction

JTA becomes pointless at this point, I think, unless I'm missing
something. Another thing that just quickly popped into mind... What if
a temporary failure turns into a permanent failure?

I guess it's probably better if we wait until true LDAP transactions
are implemented to do any type of XA wrapper mangling. That's in ver.
2.4 on the OpenLDAP roadmap, due out this fall it looks like. And it
sounds like the JMS idea *might* work for the truly non-transactional
stuff, like IMAP, filesystems, etc.

Thanks for any further pointers and such!

--Scott

 

[nitin_verma]

Hello,
Recently, I have been working on a project that hopefully matches the requirement posted above, from the viewpoint of transactional file-system (not LDAP etc). This project, called XADisk, is hosted on an open source community (java.net) as <url> xadisk.java.net </url>

You would get xa-transactional access to files using xadisk.

Thanks,
Nitin.