Can you stream HTTP chunked data to Apache?

[alan.snyder]

I would like to do the following. Open a connection to an apache server
using HTTP/1.1 chucking via a POST. Then keep this connection open to
once every few seconds send an update of data (stats) from another
server. So this is essentially streaming data.

Can you do this with Apache Tomcat? Would each "chunk" of data result
in a call-back to doPost().

Could someone please mention if it is possible to do this, and what the
corresponding servlet code that would get each chunk of data look like.
This is not Request/Response, but more streaming of data.

Thanks in advance

 

[Rogan Dawes]

I would like to do the following. Open a connection to an apache server
using HTTP/1.1 chucking via a POST. Then keep this connection open to
once every few seconds send an update of data (stats) from another
server. So this is essentially streaming data.

Can you do this with Apache Tomcat? Would each "chunk" of data result
in a call-back to doPost().

Could someone please mention if it is possible to do this, and what the
corresponding servlet code that would get each chunk of data look like.
This is not Request/Response, but more streaming of data.

Thanks in advance

I sincerely doubt that each chunk would result in a call to doPost().

More likely, you would do something like

InputStream is = request.getInputStream();
byte[] buff = new byte[BUFFSIZE];
int got;
while ((got = is.read(buff) > -1) {
// do something with the data
}

Your InputStream.read(buff) will just block until the next chunk
arrives. Tomcat will take care of unchunking the data as it arrives, so
you would simply see a transparent stream of bytes, with some pauses in
between chunks, perhaps.

Note that you might have to deal with timeouts if the delay between
chunks is too long, and I'm not entirely sure whether streaming a
response at the same time as reading the input is allowed.

Rogan

 

[Andrea Desole]

I would like to do the following. Open a connection to an apache server
using HTTP/1.1 chucking via a POST. Then keep this connection open to
once every few seconds send an update of data (stats) from another
server. So this is essentially streaming data.

Can you do this with Apache Tomcat? Would each "chunk" of data result
in a call-back to doPost().

Could someone please mention if it is possible to do this, and what the
corresponding servlet code that would get each chunk of data look like.
This is not Request/Response, but more streaming of data.

Thanks in advance

I'm not sure, but I don't think it would work. I think the server waits
until the request is complete to give it to your application, and as far
as I know a streaming type for request doesn't exist.
What I would try is probably just to call the servlet every time. That
makes it probably a bit slower, but more stable.

 

[alan.snyder]

...

I sincerely doubt that each chunk would result in a call to doPost().

More likely, you would do something like

InputStream is = request.getInputStream();
byte[] buff = new byte[BUFFSIZE];
int got;
while ((got = is.read(buff) > -1) {
// do something with the data
}

Your InputStream.read(buff) will just block until the next chunk
arrives. Tomcat will take care of unchunking the data as it arrives, so
you would simply see a transparent stream of bytes, with some pauses in
between chunks, perhaps.

Note that you might have to deal with timeouts if the delay between
chunks is too long, and I'm not entirely sure whether streaming a
response at the same time as reading the input is allowed.

Rogan

Thanks, I will give this a try. This would mean that Tomcat keeps that
servlet alive
until the entire stream is closed.

 

[alan.snyder]

I'm not sure, but I don't think it would work. I think the server waits
until the request is complete to give it to your application, and as far
as I know a streaming type for request doesn't exist.
What I would try is probably just to call the servlet every time. That
makes it probably a bit slower, but more stable.

I was given the requirement that data be streamed via HTTP/1.1
chunking, so making one call to the Servlet for each data segment isn't
an option. Ideally the data would be streamed over a simple TCP
connection, but the client has some firewall constrains that makes the
use of HTTP necessary. Just need to figure out of using Tomcat in this
mode is an option.

 

[ohaya]

I was given the requirement that data be streamed via HTTP/1.1
chunking, so making one call to the Servlet for each data segment isn't
an option. Ideally the data would be streamed over a simple TCP
connection, but the client has some firewall constrains that makes the
use of HTTP necessary. Just need to figure out of using Tomcat in this
mode is an option.

Hi,

It's been awhile since I've looked at it, but doesn't the HTTP POST have
a "Content-length" header that specifies the length of the POST'ed data?

If that is the case, you'd need to know the total byte count of the data
you're going to be POSTing ahead of time, and I think that the webserver
would wait for all of the data per the byte count.

Jim

 

[Andrea Desole]

I was given the requirement that data be streamed via HTTP/1.1
chunking, so making one call to the Servlet for each data segment isn't
an option. Ideally the data would be streamed over a simple TCP
connection, but the client has some firewall constrains that makes the
use of HTTP necessary. Just need to figure out of using Tomcat in this
mode is an option.

Never thought of using chunking in a request. Makes kind of sense.
Anyway, the more I think about it the more I have the feeling that the
server wait until the request is complete. It would be nice to know if
it isn't so.