Tomcat - post upload fails for files over 320 KB

[Christoph]

Hi,

I am using a Post Servlet to handle a text file upload. As a test, the
servlet currently just echoes the content of the raw request stream
into the response.

This has worked fine for small files, but the first time I tried to
upload around 1MB, the browser never got beyond the "Sending request"
status.

There appeared to be a precise limit on the length of the text file,
so I experimented and got around 320 KB. It fluctuates slightly, but
it appears to be around 326658 bytes, give or take a few. If the file
is shorter, the servlet will respond instantly. If it is only slightly
longer, the request will never finish sending. The status will remain
stuck at "Sending request to localhost".

Incidentally, if a file is successfully uploaded several times, it
will eventually run into the same problem. The server then becomes
unresponsive until restarted.

----------------------

For a default upload limit, this seems irrationally low, and I didn't
change the configuration. The server also runs locally, so the network
is no issue.

Where should I look for the cause of this problem?

 

[Christoph]

The problem is directly related to the code of the servlet. On trying
to submit the form to an empty servlet that ignores the request, I was
also able to upload several MB. The servlet will only fail when it
tries to loop through the request stream.

Therefore, I post the code of the doPost method here:

{
BufferedReader reader = request.getReader();
ServletOutputStream output = response.getOutputStream();
String line = reader.readLine();
response.getOutputStream().println(line);

// If the following code is commented out, the servlet will run
and print only the first line of the file.
// With the following code, the servlet will fail on files larger
than 320 KB.
while (line != null) {
output.println(line);
line = reader.readLine();
}
}

 

[Christoph]

Starting the server in debug mode somehow gets rid of the "repeated
requests" problem (maybe the server resets on each request when in
debug mode?). However, if a single file is larger than 320 KB, it will
still fail.

 

[Sideswipe]

I forget which it is, but 1 method (post?) has a limit on the size.
The other doesn't (get?) -- did you try changing them up?

 

[Arancaytar]

I forget which it is, but 1 method (post?) has a limit on the size.
The other doesn't (get?) -- did you try changing them up?

Incorrect. GET is limited by the URL length (4KB, I believe), but POST
is only limited by server configuration. Most servers set it to about
10-20MB as a protection, but presumably you could increase it to
several times that.

 

[Lew]

Incorrect. GET is limited by the URL length (4KB, I believe), but POST
is only limited by server configuration. Most servers set it to about
10-20MB as a protection, but presumably you could increase it to
several times that.

GET is not intended for causing change on the server, POST is.
<http://www.w3.org/2001/tag/doc/whenToUseGet.html#checklist>

 

[Christoph]

GET is not intended for causing change on the server, POST is.
<http://www.w3.org/2001/tag/doc/whenToUseGet.html#checklist>

Yeah, that too. Using GET when writing data to the server is one of
the top three WTFs I've seen in inherited code.

But still, POST is supposedly unlimited in length - and besides, since
Tomcat is careful to send even obscure HTTP status codes when apt (it
sends 405 if the servlet doesn't implement a method), it would surely
send 413 (Request Entity Too Large) if it was intentionally limiting
the POST size. That, and as said the error is somehow caused by the
browser, not the servlet.

 

[Christoph]

the error is somehow caused by the
browser, not the servlet.

Argh, tired. What I meant is "the servlet, not the server." Since
other servlets work.

 

[Nigel Wade]

Hi,

I am using a Post Servlet to handle a text file upload. As a test, the
servlet currently just echoes the content of the raw request stream
into the response.

Whilst your client is busy sending the data, and the servlet is echoing it back
to the client, what is emptying the servlet's output buffer? At some point the
output buffer of the servlet will get full and it won't be able send any more
data. Any further attempt to write in the servlet will block, so the servlet is
now no longer reading data from the client. The same then happens to the
client.

This has worked fine for small files, but the first time I tried to
upload around 1MB, the browser never got beyond the "Sending request"
status.

There appeared to be a precise limit on the length of the text file,
so I experimented and got around 320 KB. It fluctuates slightly, but
it appears to be around 326658 bytes, give or take a few. If the file
is shorter, the servlet will respond instantly. If it is only slightly
longer, the request will never finish sending. The status will remain
stuck at "Sending request to localhost".

It looks like you've discovered the size of the servlet's output buffer...

Incidentally, if a file is successfully uploaded several times, it
will eventually run into the same problem. The server then becomes
unresponsive until restarted.

----------------------

For a default upload limit, this seems irrationally low, and I didn't
change the configuration. The server also runs locally, so the network
is no issue.

Where should I look for the cause of this problem?

Don't try to read/write at the same time in one thread.

 

[Christoph]

I was a bit confused about the output buffer not being emptied (after
all, it flushes automatically in usual cases), but I just realized
that the server /can't/ send any response until the request has
finished sending, so I can get a deadlock between read and write
streams. I tried to force a streaming conversation into the stateless
and atomic query-response protocol, which floods the buffers. Thanks
for that tip.

Fortunately, the actual servlet will not have to echo anything (it
writes the input to a database), so the buffer deadlock should only
occur in my test code.