Thread safety in Tomcat

[Alex Molochnikov]

With the deprecation of SingleThreadModel marker, what is the best way to
ensure the thread-safety of the servlet? If the doPost() method runs a
time-consuming section of code, is the "synchronized" block the only answer?

A nice solution would be for the container (Tomcat) to create a new servlet
instance when a concurrent request comes from a different thread, as the
SingleThreadModel implied before it was dropped, but it does not seem to
work this way.

Any suggestions?

Alex Molochnikov
Gestalt Corporation

 

[Alex Molochnikov]

Sorry for following up my own post, but I am wrong on one count: the
SingleThreadModel does cause the Tomcat to create another instance of the
servlet. Still, the question remains: what happens when the
SingleThreadModel interface is gone, and the Java compiler begins to
complain, or Tomcat begins to ignore it? Will it still act as if the
"SingleThreadModel" was implemented by the servlet as a default?

AM

 

[Chris Smith]

With the deprecation of SingleThreadModel marker, what is the best way to
ensure the thread-safety of the servlet? If the doPost() method runs a
time-consuming section of code, is the "synchronized" block the only answer?

There is no problem. SingleThreadModel never made anything thread-safe
that couldn't have been easily fixed anyway.

The way to make things thread-safe is:

1. Don't store thread-private data in shared data storage. That means
that instead of storing data in instance fields, you should use local
variables. When you call other methods, pass the data as parameters.

If that's too cumbersome, then it's time to encapsulate the data into an
object of your own class, initially with appropriate accessors and
mutators. Then it's time to see what methods make more sense when you
move them to that custom class. But I digress; only the previous
paragraph really has anything to do with thread-safety.

2. Use the synchronized keyword whenever you access shared state (that
is, data that is accessed and mutated by several different HTTP
requests).

A nice solution would be for the container (Tomcat) to create a new servlet
instance when a concurrent request comes from a different thread, as the
SingleThreadModel implied before it was dropped, but it does not seem to
work this way.

If you need an object per servlet request, you can create an object per
servlet request. For that object to be the servlet itself is
unbelievably sloppy and an unnecessary hack. That, plus the ambiguity
and limitations in what SingleThreadModel actually means (i.e., does the
Tomcat hack really capture the intent of the interface? And what about
concurrency requirements between several separate servlets?) makes an
excellent argument for avoiding SingleThreadModel even without the
deprecation.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Alex Molochnikov]

2. Use the synchronized keyword whenever you access shared state (that
is, data that is accessed and mutated by several different HTTP
requests).

Well, this was part of my question. Synchronized block guarantees thread
safety, but it also becomes a major bottlencek for handling concurrent
requests.

If you need an object per servlet request, you can create an object per
servlet request. For that object to be the servlet itself is
unbelievably sloppy and an unnecessary hack. That, plus the ambiguity
and limitations in what SingleThreadModel actually means (i.e., does the
Tomcat hack really capture the intent of the interface? And what about
concurrency requirements between several separate servlets?) makes an
excellent argument for avoiding SingleThreadModel even without the
deprecation.

As I said in my own follow-up, the SingleThreadModel marker does cause
Tomcat to create a new instance of the servlet when the requests collide.
And I don't see anything wrong in having a separate object handle the
request - this way, both "conflicting" requests can proceed without waiting
for one another.

What I observe in the Tomcat behavior is how I actually understood the
meaning and purpose of SingleThreadModel. I must be missing something very
fundamental, since I saw a number of caveats similar to yours about it, but
none of them explained what the "ambiguity and limitations" really are, and
how they affect the integrity/performance/whatever of the servlet.

AM

 

[Chris Smith]

Well, this was part of my question. Synchronized block guarantees thread
safety, but it also becomes a major bottlencek for handling concurrent
requests.
[...]

As I said in my own follow-up, the SingleThreadModel marker does cause
Tomcat to create a new instance of the servlet when the requests collide.
And I don't see anything wrong in having a separate object handle the
request - this way, both "conflicting" requests can proceed without waiting
for one another.

I'm not sure you understood what I said. Either your servlet uses
shared state, or it doesn't. SingleThreadModel doesn't magically make
shared state into non-shared state. It just changes where you can put
the two.

If you use shared state, then you should use synchronized blocks to
manage access to it, regardless of whether SingleThreadModel is used or
not. If you're not using shared state, then of course you shouldn't use
synchronized blocks, since that would introduce unnecessary mutual
exclusion in your application.

As I said earlier, if you want an object per request to store shared
state, go ahead and create it; just don't use the servlet class for that
purpose!

What I observe in the Tomcat behavior is how I actually understood the
meaning and purpose of SingleThreadModel. I must be missing something very
fundamental, since I saw a number of caveats similar to yours about it, but
none of them explained what the "ambiguity and limitations" really are, and
how they affect the integrity/performance/whatever of the servlet.

The ambiguity is that the name "SingleThreadModel" really implies that
only a single thread at a time can run. That would be the ultimate
mutual exclusion, and I think is what was originally intended by the
interface. It's not very useful, though, in a production application
that gets any kind of load. Because it's not very useful that way, the
fine folks over at Apache read the spec very closely and decided that
they could follow the letter of the specification by just creating
several objects of the servlet class, and continuing to service multiple
requests at a time. That was a reasonable thing to do -- turning a
completely useless feature into one that's merely bad -- but it changed
the intent of the specification. As far as I know, Sun never chose to
clarify whether that technique should be considered acceptable or not.
Nevertheless, practically everyone does it.

The limitations are fundamental in the idea of SingleThreadModel itself.
The SingleThreadModel marker interface only applies to the servlet
class. Real applications can sometimes have quite a large number of
servlet classes, and put as little code as possible into the servlet
classes themselves. Often the same piece of code is run from several
servlets, and you're back to the same problem again. Basically, the
problem is that SingleThreadModel is a crutch, and like actual crutches,
it limits the amount of stuff you can do while still taking advantage of
the crutch. It excludes certain things that are very good practices.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Alex Molochnikov]

The ambiguity is that the name "SingleThreadModel" really implies that
only a single thread at a time can run. That would be the ultimate
mutual exclusion, and I think is what was originally intended by the
interface. It's not very useful, though, in a production application
that gets any kind of load. Because it's not very useful that way, the
fine folks over at Apache read the spec very closely and decided that
they could follow the letter of the specification by just creating
several objects of the servlet class, and continuing to service multiple
requests at a time. That was a reasonable thing to do -- turning a
completely useless feature into one that's merely bad -- but it changed
the intent of the specification. As far as I know, Sun never chose to
clarify whether that technique should be considered acceptable or not.
Nevertheless, practically everyone does it.

Thank you for the detailed explanation. This is the first time somebody
spelled it out for me in detail.

The limitations are fundamental in the idea of SingleThreadModel itself.
The SingleThreadModel marker interface only applies to the servlet
class. Real applications can sometimes have quite a large number of
servlet classes, and put as little code as possible into the servlet
classes themselves. Often the same piece of code is run from several
servlets, and you're back to the same problem again. Basically, the
problem is that SingleThreadModel is a crutch, and like actual crutches,
it limits the amount of stuff you can do while still taking advantage of
the crutch. It excludes certain things that are very good practices.

I think for the narrow purpose of our application, the SingleThreadModel
marker with the ensuing creation of multiple servlet objects is just fine.
In our case, the "real" processing happens outside of the servlet container
altogether - it is done by a separate headless Java process. The servlet's
role is reduced to dispatching requests from the browser to the Java
process, and routing the results back to the browser. Based on what you
said, having multiple instances of this servlet is the best solution in our
case.

Regards,

AM

 

[Juha Laiho]

I think for the narrow purpose of our application, the SingleThreadModel
marker with the ensuing creation of multiple servlet objects is just fine.
In our case, the "real" processing happens outside of the servlet container
altogether - it is done by a separate headless Java process. The servlet's
role is reduced to dispatching requests from the browser to the Java
process, and routing the results back to the browser. Based on what you
said, having multiple instances of this servlet is the best solution in our
case.

It might be "good enough", but I think it'd be better to get rid of the
SingleThreadModel; something like this:
- write a spearate class and enapsulate the remote access functionality
into this class
- if creating a new connection to the remote system is expensive
(mostly, takes a lot of time), and if a single connection could
handle several requests, consider using some kind of connection
pooling mechanism
- rewrite your servlet so that it doesn't use any instance-level data
(so, class variables) or other non-thread-specific data in the
request service methods; whatever variables are declared within
methods are safe
- have the servlet create a new object of the above remote access
class for each new request (or fetch a free object from the
connection pool, if you ended up using one); so, create/fetch
this remote access class instance within the service method,
and use a method variable to hold the reference, to keep the
reference private to the running thread; if you ended up to use
the connection pool, make sure you return the remote access
objects back to the pool in any case

 

[John C. Bollinger]

The ambiguity is that the name "SingleThreadModel" really implies that
only a single thread at a time can run. That would be the ultimate
mutual exclusion, and I think is what was originally intended by the
interface. It's not very useful, though, in a production application
that gets any kind of load. Because it's not very useful that way, the
fine folks over at Apache read the spec very closely and decided that
they could follow the letter of the specification by just creating
several objects of the servlet class, and continuing to service multiple
requests at a time. That was a reasonable thing to do -- turning a
completely useless feature into one that's merely bad -- but it changed
the intent of the specification. As far as I know, Sun never chose to
clarify whether that technique should be considered acceptable or not.
Nevertheless, practically everyone does it.

Servlet 2.3, section 2.2 ("Number of Instances") includes: "However, for
a servlet implementing the SingleThreadModel interface, the servlet
container may instantiate multiple instances to handle a heavy request
load and serialize requests to a particular instance."

With that said, I fully agree that SingleThreadModel is a kludgy crutch.
In any case where implementing SingleThreadModel is sufficient to
provide thread safety for a servlet, it really isn't that hard to make
the servlet class inherently thread-safe in the first place. Developers
who learn to rely on that crutch are likely to fail to recognize those
cases where it _isn't_ sufficient, and are more likely to be lost when
and if they do recognize it. All-in-all, this is a case where rule
number 3 applies: "Quick hacks generally aren't."*

*J.C.Bollinger's Rules of Programming


John Bollinger
(e-mail address removed)

 

[Alex Molochnikov]

With that said, I fully agree that SingleThreadModel is a kludgy crutch.
In any case where implementing SingleThreadModel is sufficient to
provide thread safety for a servlet, it really isn't that hard to make
the servlet class inherently thread-safe in the first place.

I am not so sure. This kind of a broad generalization implies that the
servlet will always operate in the request-response cycle, typical for a
common web-based application.

I my case, the application in question is a report generator that is
designed to run primarily as a desktop client - backend server office app,
with the certain amount of interactivity between the server and the client.
That is, the server, after starting to work on the client's request to
produce a report, and before completing it, can make a callback on the
client with additional questions (e.g. please select the invoice number for
detailed reporting). While the user is pondering the answer, the server is
in the wait state (there is one server process per client). And the client,
being a single-thread Swing app, is also "frozen" until the user responds to
the modal dialog.

The web-based part of the report generator is just a re-implementation of
the same logic with the servlet replicating the behavior of the Swing-based
client (and the web browser being merely a GUI portion of the "distributed"
client - the invisible part of the client being the servlet). The reasons
for this design choice are partly historical, and partly pragmatic - it was
easier to build it this way, with the mainpower/budget constraints of the
project. All in all, this is how it is done, and the servlet must be
prepared to go into a wait on a single thread, without grinding other users
to a halt.

So, for the servlet's role shifted into becoming essentially a client, the
ability of the container to instantiate a new servlet when the competing
request comes while the original servlet is waiting for the user response,
is a welcome solution to an otherwise awkward problem.

AM

 

[John C. Bollinger]

I am not so sure. This kind of a broad generalization implies that the
servlet will always operate in the request-response cycle, typical for a
common web-based application.

That functional motif is the defining characteristic of a servlet. It
processes requests to produce responses. But that's beside the point.

As Chris Smith also wrote [paraphrased], implementing SingleThreadModel
is not sufficient to guarantee thread safety in most cases where the
servlet accesses shared resources that are outside the scope of the
servlet instance itself. My comment quoted above is essentially the
contrapositive: if SingleThreadModel is sufficient then it is
_necessarily_ the case that the servlet class could have been made
thread-safe in the first place without extraordinary effort. Despite
its broad scope, the statement is in no danger of being over general; it
is an observation on the nature and limitations of the servlet
specification with respect to SingleThreadModel.

I my case, the application in question is a report generator that is
designed to run primarily as a desktop client - backend server office app,
with the certain amount of interactivity between the server and the client.
That is, the server, after starting to work on the client's request to
produce a report, and before completing it, can make a callback on the
client with additional questions (e.g. please select the invoice number for
detailed reporting). While the user is pondering the answer, the server is
in the wait state (there is one server process per client). And the client,
being a single-thread Swing app, is also "frozen" until the user responds to
the modal dialog.

The web-based part of the report generator is just a re-implementation of
the same logic with the servlet replicating the behavior of the Swing-based
client (and the web browser being merely a GUI portion of the "distributed"
client - the invisible part of the client being the servlet). The reasons
for this design choice are partly historical, and partly pragmatic - it was
easier to build it this way, with the mainpower/budget constraints of the
project. All in all, this is how it is done, and the servlet must be
prepared to go into a wait on a single thread, without grinding other users
to a halt.

This is a clear case where implementing SingleThreadModel does not
ensure thread safety. The servlet depends on state shared with the
back-end server and on independent processes running on the server. It
is impossible to guarantee thread safety for the servlet considered in
isolation, and it's darned difficult to ensure it for the combined system.

Moreover, I am not seeing how SingleThreadModel and multiple servlet
instances are particularly helpful to you. If your servlet were
thread-safe and did not implement SingleThreadModel then one particular
server thread blocking in some servlet method would not inconvenience
other users one bit. Are you perhaps not appreciating the implications
of the servlet container being multithreaded? If a servlet does not
implement SingleThreadModel then its container can use one instance to
process any number of simultaneous requests, _in different threads_.

So, for the servlet's role shifted into becoming essentially a client, the
ability of the container to instantiate a new servlet when the competing
request comes while the original servlet is waiting for the user response,
is a welcome solution to an otherwise awkward problem.

No, it's not. At least, nothing you have said so far shows it to be a
solution of any particular merit, relative to the behavior one would
expect of a non-SingleThreadModel servlet in any servlet container worth
using. The blocking problem that you fear is a possibility *caused* by
SingleThreadModel (and sidestepped by most servlet containers), not a
problem solved by it.


John Bollinger
(e-mail address removed)

 

[Chris Smith]

[...]

All in all, this is how it is done, and the servlet must be
prepared to go into a wait on a single thread, without grinding other users
to a halt.

Let's be clear here. No one has ever suggested that the way to migrate
away from SingleThreadModel is to add any kind of synchronized block,
much less to hold a monitor for long periods of time. In fact, BY
DEFINITION, any servlet that is thread-safe with SingleThreadModel can
trivially be made thread-safe without SingleThreadModel, and without a
single use of synchronized, wait/notify, or any other thread
synchronization mechanism. None of your users will grind to a halt.

Now, it sounds like someone misused the servlet class in your example
here. If that's the case, then it can be a pain (though not a very big
one) to fix their mistake by moving code to the appropriate place. If
you have a large number of servlets that do the same thing, then this
could conceivably take some time, and that's why SingleThreadModel is
deprecated rather than removed. In real life, bad code is a fact, and
we do what we can; but the deprecation of SingleThreadModel is welcome
news anyway.

--
www.designacourse.com
The Easiest Way To Train Anyone... Anywhere.

Chris Smith - Lead Software Developer/Technical Trainer
MindIQ Corporation

 

[Alex Molochnikov]

Moreover, I am not seeing how SingleThreadModel and multiple servlet
instances are particularly helpful to you. If your servlet were
thread-safe and did not implement SingleThreadModel then one particular
server thread blocking in some servlet method would not inconvenience
other users one bit. Are you perhaps not appreciating the implications
of the servlet container being multithreaded? If a servlet does not
implement SingleThreadModel then its container can use one instance to
process any number of simultaneous requests, _in different threads_.

It is quite likely that, being used to managing thread-safety in our own
server, I do not have the full appreciation of what the servlet container
can do for me. I am going to take a time out at this point, write a
prototype and see how the container resolves concurrent threads. I will get
back to this discussion later.

No, it's not. At least, nothing you have said so far shows it to be a
solution of any particular merit, relative to the behavior one would
expect of a non-SingleThreadModel servlet in any servlet container worth
using. The blocking problem that you fear is a possibility *caused* by
SingleThreadModel (and sidestepped by most servlet containers), not a
problem solved by it.

I will experiment with both types of servlet in the prototype, and will
hopefully get a better feel for the mechanics of it.

Thank you for the feedback.

AM.

 

[Alex Molochnikov]

Let's be clear here. No one has ever suggested that the way to migrate
away from SingleThreadModel is to add any kind of synchronized block,
much less to hold a monitor for long periods of time. In fact, BY
DEFINITION, any servlet that is thread-safe with SingleThreadModel can
trivially be made thread-safe without SingleThreadModel, and without a
single use of synchronized, wait/notify, or any other thread
synchronization mechanism. None of your users will grind to a halt.

This is a welcome news. I already wrote a very simplistic test case,
colliding two threads, and it appears to hold water so far. I will have to
play more with a more elaborate prototype.

Now, it sounds like someone misused the servlet class in your example
here. If that's the case, then it can be a pain (though not a very big
one) to fix their mistake by moving code to the appropriate place.

This person is no longer with us, and I am going to redo this part of code
myself. I will have to get rid of the instance vars, and move all persistent
objects into the HttpSession, out of the harms way.

If you have a large number of servlets that do the same thing, then this
could conceivably take some time, and that's why SingleThreadModel is
deprecated rather than removed. In real life, bad code is a fact, and
we do what we can; but the deprecation of SingleThreadModel is welcome
news anyway.

The good news in my case is that the number of servlet classes is rather
small, so I should be able to avoid the major pain.

Thank you for your help.

AM