boost::asio, synchronous HTTP read on sslstream

[Torsten Mueller]

I have a problem reading on a sslstream. If the content-length is
specified it works fine, I read exactly the given length and have a
complet answer. If I didn't get a content-length header from the server
I use an algorithm like this:

asio::ssl::stream<asio::ip::tcp::socket&>* m_pSslStream;

// ... initializations ...

// ... read HTTP headers ...
// (get no content-length here)

// read the HTTP body
asio::streambuf sb;
system::error_code error;
size_t n;
while ((n = asio::read(*m_pSslStream, sb,
asio::transfer_exactly(1024), error)) > 0)
{
// ... handle contents ...
}

This should read 1k-blocks until there are no more bytes in the stream.
The last call to the read-function should return less than 1024 bytes
and an error code. Indeed this happens, exactly as specified, but the
last call to the read-function doesn't return before a long timeout is
reached, about one minute or more. What's the reason for this?

I tried also other things: transfer_at_least() or transfer_all(), also
the read_some-method of the sslstream class. All show exactly the same
behaviour.

What can I do to avoid this timeout at the end? Is this related to SSL?
I can't remember that I had problems with this on non-SSL streams.

T.M.

 

[Bart van Ingen Schenau]

I have a problem reading on a sslstream. If the content-length is
specified it works fine, I read exactly the given length and have a
complet answer. If I didn't get a content-length header from the server
I use an algorithm like this:

This should read 1k-blocks until there are no more bytes in the stream.
The last call to the read-function should return less than 1024 bytes
and an error code. Indeed this happens, exactly as specified, but the
last call to the read-function doesn't return before a long timeout is
reached, about one minute or more. What's the reason for this?

I tried also other things: transfer_at_least() or transfer_all(), also
the read_some-method of the sslstream class. All show exactly the same
behaviour.

What can I do to avoid this timeout at the end? Is this related to SSL?
I can't remember that I had problems with this on non-SSL streams.

T.M.

Your problem has nothing to do with C++ and everything with protocol
specifications and typical server behavior.
Both the SSL and the underlying TCP protocols are streaming protocols,
which means that they don't have the notion of messages or message
boundaries. If you send multiple HTTP requests over a single SSL or TCP
connection, then that is a single stream of data as far as SSL and TCP
are concerned and that stream ends when the connection gets closed.
As establishing a SSL connection (and to a lesser extent, also a TCP
connection) is a heavy weight, time consuming operation and because web
pages are typically built from different parts (which means, multiple
requests are needed), web servers will usually not close the connection
immediately after sending the first response, because it is more
economical to re-use the connection for the follow-on requests.

These two factors combined (TCP/SSL doesn't know about messages and
servers not closing the connection immediately) have the effect that if
you don't know how much data to expect, you end up waiting until the
connection gets closed due to a timeout.

Bart v Ingen Schenau