How do i handle long-running background tasks in J2EE?

[Tom Anderson]

Hi all,

By 'long-running', i mean 'up to ten seconds'.

The context is a bog-standard e-commerce site. The task in question is
carrying out a credit card authentication and then submitting an order to
a backend system; most of the up-to-ten seconds is spent waiting for the
remote systems to respond. The page flow we want is:

1. User is on the order review page and clicks 'confirm'
2. User goes quickly to a 'were processing ur order lol' page, and waits
for it to sponteneously occur that ...
3. User arrives at the order confirmation page

The idea is that when the confirmation is submitted, we kick off the
authentication and submission as a background task, then serve the
processing page. That page then polls the server (currently by
periodically refreshing itself, in the near future perhaps with some AJAX)
to see if the order has been processed. When it has, the request picks up
the results of the processing and serves the confirmation page (well, or
an error page, etc).

Is there a standard or best-practice way to do this in pure J2EE? If not,
how about in the greater J2EE ecosystem (eg Quartz or something)?
Specifically, when running on JBoss?

As it happens, we know we can do this by spawning a thread. I know that
the J2EE spec contains dire injunctions against doing that, and that it's
not portable, but on our platform, it happens that it works. Our current
implementation is thus to set up a global ExecutorService, and pass the
order processing tasks to it as Callables to be run; we communicate
completion to the request-handling threads via a Future which eventually
delivers a sort of OrderProcessingResult object.

I'd like to know if there is a better way of doing it, though.

The one thing i've come up with is using JMS. We could set up a queue,
have the request threads post orders onto it, then use a message-driven
bean to pull them off and deal with them. I have no idea how we'd make
that multithreaded, nor how we'd communicate completion back to the
request threads. In any case, i lean away from it, because we aren't
currently using JMS or EJB, and i fear they would be a bit of a pain to
incorporate into our architecture. It might be the way to go, but i'm more
interested in hearing about other ideas.

Thanks,
tom

 

[markspace]

I don't know, this is all made up, but...

1. User is on the order review page and clicks 'confirm'
2. User goes quickly to a 'were processing ur order lol' page, and waits
for it to sponteneously occur that ...
3. User arrives at the order confirmation page

I would have page number 2 switch to "long poll" right now to determine
when page 3 is ready. I other words, you have two URLs:

2a. www.mysite.com/lol_processing
2b. www.mysite.com/process_card?n=1234455667890

Page 2a loads, then tries to load page 2b. When it get a response, then
it loads the confirmation page, possibly using some sort of key returned
by 2b.

This is just:

A. The simplest thing I could think of.
B. Doesn't need to use any thread manipulation. 2b can take as long as
it likes.
C. Doesn't bake any UI rules into the servlets. Each URL is an
independent task. The UI itself takes care of timing/presentation.
D. System testing should be easier as a result.

Again, no special experience, just trying to simplify the interaction.

 

[Arne Vajhøj]

By 'long-running', i mean 'up to ten seconds'.

The context is a bog-standard e-commerce site. The task in question is
carrying out a credit card authentication and then submitting an order
to a backend system; most of the up-to-ten seconds is spent waiting for
the remote systems to respond. The page flow we want is:

1. User is on the order review page and clicks 'confirm'
2. User goes quickly to a 'were processing ur order lol' page, and waits
for it to sponteneously occur that ...
3. User arrives at the order confirmation page

The idea is that when the confirmation is submitted, we kick off the
authentication and submission as a background task, then serve the
processing page. That page then polls the server (currently by
periodically refreshing itself, in the near future perhaps with some
AJAX) to see if the order has been processed. When it has, the request
picks up the results of the processing and serves the confirmation page
(well, or an error page, etc).

Is there a standard or best-practice way to do this in pure J2EE? If
not, how about in the greater J2EE ecosystem (eg Quartz or something)?
Specifically, when running on JBoss?

As it happens, we know we can do this by spawning a thread. I know that
the J2EE spec contains dire injunctions against doing that, and that
it's not portable, but on our platform, it happens that it works. Our
current implementation is thus to set up a global ExecutorService, and
pass the order processing tasks to it as Callables to be run; we
communicate completion to the request-handling threads via a Future
which eventually delivers a sort of OrderProcessingResult object.

I'd like to know if there is a better way of doing it, though.

The one thing i've come up with is using JMS. We could set up a queue,
have the request threads post orders onto it, then use a message-driven
bean to pull them off and deal with them. I have no idea how we'd make
that multithreaded, nor how we'd communicate completion back to the
request threads. In any case, i lean away from it, because we aren't
currently using JMS or EJB, and i fear they would be a bit of a pain to
incorporate into our architecture. It might be the way to go, but i'm
more interested in hearing about other ideas.

Creating threads are fully supported in Java EE - it is explicit
forbidden in EJB and a bad idea in servlet but it is a core
feature of JCA.

That said then in this context I think the message queue (via JMS)
and a MDB is a much better solution. The first servlet/Strust Action/
JSF Backing Bean/whatever puts the request in one queue, the MDB
processes and put the response in the other queue, the second
whatever simply checks if the response are in the queue using
message selector.

Arne

 

[Arne Vajhøj]

Creating threads are fully supported in Java EE - it is explicit
forbidden in EJB and a bad idea in servlet but it is a core
feature of JCA.

That said then in this context I think the message queue (via JMS)
and a MDB is a much better solution. The first servlet/Strust Action/
JSF Backing Bean/whatever puts the request in one queue, the MDB
processes and put the response in the other queue, the second
whatever simply checks if the response are in the queue using
message selector.

Both the JCA and the message queue solution are standard and
works fine in JBoss AS.

Arne

 

[Arne Vajhøj]

I don't know, this is all made up, but...


I would have page number 2 switch to "long poll" right now to determine
when page 3 is ready. I other words, you have two URLs:

2a. www.mysite.com/lol_processing
2b. www.mysite.com/process_card?n=1234455667890

Page 2a loads, then tries to load page 2b. When it get a response, then
it loads the confirmation page, possibly using some sort of key returned
by 2b.

This is just:

A. The simplest thing I could think of.
B. Doesn't need to use any thread manipulation. 2b can take as long as
it likes.
C. Doesn't bake any UI rules into the servlets. Each URL is an
independent task. The UI itself takes care of timing/presentation.
D. System testing should be easier as a result.

Again, no special experience, just trying to simplify the interaction.

It is simpler. And it will work in a simple servlet container.

But for a busy site and a pre-"servlet 3.0" container it will
either create a lot of threads or give errors.

Arne

 

[Fredrik Jonson]

That said then in this context I think the message queue (via JMS)
and a MDB is a much better solution. The first servlet/Strust Action/
JSF Backing Bean/whatever puts the request in one queue, the MDB
processes and put the response in the other queue.

While I agree that using MDB for the longrunning task is a good solution,
I would not bother putting the response in a queue. Rather I'd let the
MDB put the response in a database, and have the second "result page" poll
that database until a response appears. That also allows the user to
quit after the initial request, and check the result later in a new
session.

 

[markspace]

But for a busy site and a pre-"servlet 3.0" container it will
either create a lot of threads or give errors.

> That said then in this context I think the message queue (via JMS)
> and a MDB is a much better solution.

Ahhh, OK. You know, I've never done any JMS and I had no idea what it
would even be good for. Now I think you've given me a concrete example
where it is useful. Thanks!

 

[Arne Vajhøj]

Ahhh, OK. You know, I've never done any JMS and I had no idea what it
would even be good for. Now I think you've given me a concrete example
where it is useful. Thanks!

Message queues are good for asynch communications.

Arne

 

[Arne Vajhøj]

While I agree that using MDB for the longrunning task is a good solution,
I would not bother putting the response in a queue. Rather I'd let the
MDB put the response in a database, and have the second "result page" poll
that database until a response appears.

Database:
- always persist
- two operations: read and update

Message queue:
- choice of persist or in memory
- single operation

Message queue is better than database for this.

> That also allows the user to
> quit after the initial request, and check the result later in a new
> session.

So does message queue.

Arne

 

[Tom Anderson]

Interesting! It would indeed make things simpler.

My only concern would be:

But for a busy site and a pre-"servlet 3.0" container it will
either create a lot of threads or give errors.

If it ties up each thread for ten seconds, and we have n requests per
second, then we'll need 10n threads to handle them. I don't have our
statistics to hand, but off the top of my head, i think we handled several
thousand checkouts per hour on the busiest day of last year; that
corresponds, very roughly, to single-figure checkouts per second, which
would mean <100 threads needed to support this model. The application will
probably be split over a few instances, so we're looking at, i would
think, 10-50 threads per instance. That should actually be perfectly
doable. If i remember wrong and it was tens of thousands, then it's a
matter of 100-500 threads; that might be more of a challenge, but in that
case, we ought to have more or beefier boxes to handle the load.

I don't know whether there would be other consequences of having such
long-runnning threads. Timeouts?

I do know that i will face very strong resistance from the colleague with
whom i'm pairing on this to doing it this way: one of his key motivations
in this work has been to avoid long-running request threads, apparently
based on prior experience where they caused a lot of trouble.

Arne: how does servlet 3.0 change this? I'm afraid i know nothing about
it.

tom

 

[Tom Anderson]

Creating threads are fully supported in Java EE - it is explicit
forbidden in EJB and a bad idea in servlet but it is a core feature of
JCA.

Good point, sorry, i was imprecise. I'm not entirely sure what context my
code is running in; it probably counts as a vastly overgrown servlet. I'm
pretty sure it's not a resource adapter.

I suppose one option would be to *write* a resource adapter which is
basically just an Executor, and use that. That would be a modest misuse of
JCA, but hey.

That said then in this context I think the message queue (via JMS) and a
MDB is a much better solution. The first servlet/Strust Action/ JSF
Backing Bean/whatever puts the request in one queue, the MDB processes
and put the response in the other queue, the second whatever simply
checks if the response are in the queue using message selector.

Aha - message selectors. That'd how we'd handle the responses. I'd
forgotten about those. That was one of my key points of puzzlement. And
MDBs can be configured to have multiple instances, which delivers the
concurrency.

So, with JMS/MDB, how do i pass objects from the front end to the back? Do
i have to serialise them into the message? Or can i keep them in some sort
of global map, and just send an ID? I assume either will work; are there
any well-known considerations about the choice? In my case, i'd like the
outcome of the process to include modifications to a mutable object (the
Order), which points to the shared object approach being better. Or can i
pass whole objects somehow?

tom

 

[Arne Vajhøj]

Interesting! It would indeed make things simpler.

My only concern would be:


If it ties up each thread for ten seconds, and we have n requests per
second, then we'll need 10n threads to handle them. I don't have our
statistics to hand, but off the top of my head, i think we handled
several thousand checkouts per hour on the busiest day of last year;
that corresponds, very roughly, to single-figure checkouts per second,
which would mean <100 threads needed to support this model. The
application will probably be split over a few instances, so we're
looking at, i would think, 10-50 threads per instance. That should
actually be perfectly doable. If i remember wrong and it was tens of
thousands, then it's a matter of 100-500 threads; that might be more of
a challenge, but in that case, we ought to have more or beefier boxes to
handle the load.

I don't know whether there would be other consequences of having such
long-runnning threads. Timeouts?

I do know that i will face very strong resistance from the colleague
with whom i'm pairing on this to doing it this way: one of his key
motivations in this work has been to avoid long-running request threads,
apparently based on prior experience where they caused a lot of trouble.

Arne: how does servlet 3.0 change this? I'm afraid i know nothing about it.

Many long running threads is a killer in a web app.

I believe that Servlet 3.0 spec added something
to support long poll.

http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html?page=3

Arne

 

[Arne Vajhøj]

Good point, sorry, i was imprecise. I'm not entirely sure what context
my code is running in; it probably counts as a vastly overgrown servlet.
I'm pretty sure it's not a resource adapter.

I suppose one option would be to *write* a resource adapter which is
basically just an Executor, and use that. That would be a modest misuse
of JCA, but hey.


Aha - message selectors. That'd how we'd handle the responses. I'd
forgotten about those. That was one of my key points of puzzlement. And
MDBs can be configured to have multiple instances, which delivers the
concurrency.

So, with JMS/MDB, how do i pass objects from the front end to the back?
Do i have to serialise them into the message? Or can i keep them in some
sort of global map, and just send an ID? I assume either will work; are
there any well-known considerations about the choice? In my case, i'd
like the outcome of the process to include modifications to a mutable
object (the Order), which points to the shared object approach being
better. Or can i pass whole objects somehow?

I would use a custom message.

Not because I have religious beliefs against global maps, but due to:
- the custom message is a lot more flexible than global map
regarding clustered and multi tiered environments
- the custom message is a lot easier to unit test than global map

Arne