boost::asio for HTTP -> data truncated

T

Torsten Mueller

I use boost::asio classes to perform a GET-request to a HTTP-Server.
The request tells the server to return a JPEG image. This algorithm
works very well with most servers.

Now we have a proxy which is different. This proxy returns a content
length header for the images which seems to be right, perhaps 43kB,
but it does not return the entire contents, only 27kB or something
like that. I use an istream to read from an asio-streambuf and this
istream has the eof-bit set long before I expect it. So the images are
truncated.

Why???

The proxy is (don't ask why) a ccproxy instance. In the log files of
this proxy we can detect that also the proxy himself knows that he
sent only a small part of the data.

Does anybody know what to do? Request again?

T.M.
 
F

Francesco S. Carta

I use boost::asio classes to perform a GET-request to a HTTP-Server.
The request tells the server to return a JPEG image. This algorithm
works very well with most servers.

Now we have a proxy which is different. This proxy returns a content
length header for the images which seems to be right, perhaps 43kB,
but it does not return the entire contents, only 27kB or something
like that. I use an istream to read from an asio-streambuf and this
istream has the eof-bit set long before I expect it. So the images are
truncated.

Why???

The proxy is (don't ask why) a ccproxy instance. In the log files of
this proxy we can detect that also the proxy himself knows that he
sent only a small part of the data.

Does anybody know what to do? Request again?

If the proxy is not sending complete data you cannot really do much with
your requests.

Have you tried requesting the same resource with, say, a regular
browser, and see if the image comes out complete or broken?

If the proxy is actually sending all the data but happens to misbehave
on the way (sending something that is interpreted as an eof by your
program) you might try resetting the stream state and ask for further
input beyond that point.

That's just a speculation, of course... before discarding an eof and
going on requesting more data check the received data against the
content length header value.

Content length headers can lie, of course, but you should be able to
check that out.
 
T

Torsten Mueller

Francesco S. Carta said:
Have you tried requesting the same resource with, say, a regular
browser, and see if the image comes out complete or broken?

It's a bit difficult because the problem happens only on a specific
network behind firewalls - and I, the developer, have no access!
People tell me, that IE never has a problem to get complete data over
the same proxy. But I'm not sure.
If the proxy is actually sending all the data but happens to
misbehave on the way (sending something that is interpreted as an
eof by your program) you might try resetting the stream state and
ask for further input beyond that point.

I had this idea, too. I try this today.

Another idea I had over night is as follows: I've seen that this
network is very very slow. To get the data of a 40kB picture I need 6
or 7 seconds. This opens the possibility to measure the time needed to
get the HTTP header. (The header is always complete.) If I know the
time needed for the HTTP header and I see it's slow I can calculate
the time needed for the HTTP body because I have the Content-Length.
And in this case I could do a break in reading to let the stream the
time needed to fetch it's data before I get an early EOF. Just an
idea.

Thanks for your reply.

T.M.
 
F

Francesco S. Carta

It's a bit difficult because the problem happens only on a specific
network behind firewalls - and I, the developer, have no access!
People tell me, that IE never has a problem to get complete data over
the same proxy. But I'm not sure.

This is getting highly non-C++ related, and as such you have better
chances of getting sensible information by posting details about the
involved proxy in a pertinent group - I'm not sure about which group
would be adequate for such inquiries, though.
I had this idea, too. I try this today.

Another idea I had over night is as follows: I've seen that this
network is very very slow. To get the data of a 40kB picture I need 6
or 7 seconds. This opens the possibility to measure the time needed to
get the HTTP header. (The header is always complete.) If I know the
time needed for the HTTP header and I see it's slow I can calculate
the time needed for the HTTP body because I have the Content-Length.
And in this case I could do a break in reading to let the stream the
time needed to fetch it's data before I get an early EOF. Just an
idea.

I do hope for you that the idea you had will solve your issue, but there
are a lot of things that affect the time needed to fetch a resource over
the Internet, and those things are extremely variable - getting the
header could be a snap while completing the data could be slow as
molasses, but I've happened to see the other way around too, where the
very first response from the server took some good time but then the
rest of the data came fluently and quickly.

If both your computer and the proxy are within the same LAN, then that
could make the "time-needed" estimate more reliable, but I'm always in
the "speculation" field.

I believe there are other things you can do to test the connection, the
response time and the data-completeness but, once more, it's something
extremely OT here - and quite out of my gears too.

All that I can add is to wish you good luck for solving the issue as
soon as possible.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top