ADODB.Stream 'format error: not a pdf or corrupt' only on large file

[iporter]

I use the code below to authorise the download of certain files.
Thus, instead of linking to the file in a wwwroot directory, I link to
this code with the filename as a parameter, and the script streams the
file if the user is authorised.

This has worked fine on PDFs, DOCs, XLS, etc. until today, and 18MB
file presents the error message 'format error: not a pdf or corrupt'.

Is there a file size limit, or a default that needs overridden? Any
thoughts?

Cheers.


The Code:

<%@ language="javascript"%>
<%
if (Session("UserID") == 0) {
Response.Redirect("notauthorized.asp");
} else {
Response.ContentType = "application/x-unknown";
var fn = Request.QueryString("fn");
// Response.Write(fn);
// Response.End();
// fn = "ecpa_efficacy_minutes_08_11_06.pdf"
var FPath = "E:\\Inetpub\\irac-online.org\\documents\\" + fn;
Response.AddHeader("Content-Disposition","attachment; filename=" +
fn);

var adoStream = Server.CreateObject("ADODB.Stream");
adoStream.Open();
adoStream.Type = 1;
adoStream.LoadFromFile(FPath);
Response.BinaryWrite(adoStream.Read());
adoStream.Close();
adoStream = null;

Response.End();
}
%>

 

[Dave Anderson]

I use the code below to authorise the download of certain files.
Thus, instead of linking to the file in a wwwroot directory, I link to
this code with the filename as a parameter, and the script streams the
file if the user is authorised.

This has worked fine on PDFs, DOCs, XLS, etc. until today, and 18MB
file presents the error message 'format error: not a pdf or corrupt'.

What do you mean by "presents the error message"? Is this a client-side
error or a server-side error?

If client-side, then you may need to examine things like your ASP output
buffer size and your script timeouts.

More than likely, you can solve this by writing in chunks. See the example
about halfway down this article:
http://www.aspfaq.com/show.asp?id=2161

 

[iporter]

What do you mean by "presents the error message"? Is this a client-side
error or a server-side error?

If client-side, then you may need to examine things like your ASP output
buffer size and your script timeouts.

More than likely, you can solve this by writing in chunks. See the example
about halfway down this article:http://www.aspfaq.com/show.asp?id=2161

--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms.

Cheers - resolution involved increasing the AspBufferingLimit in
Metabase.xml - see http://clearglass.net/index.php?p=10.

 

[Anthony Jones]

Cheers - resolution involved increasing the AspBufferingLimit in
Metabase.xml - see http://clearglass.net/index.php?p=10.

Here is a port of my VBScript SendFileToResponse function (I haven't tested
this JScript version):-

function SendFileToResponse(FilePath, FileName)
{
var clChunkSize = 1048576 // 1MB

Response.Buffer = false

Response.ContentType = "application/octet-stream"
Response.AddHeader("Content-Disposition",
"attachment; Filename=" + FileName)

var oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = 1 // Binary
oStream.Open()
oStream.LoadFromFile(FilePath)

for (var i = 1, length = Math.floor(oStream.Size / clChunkSize); i <=
length; i++)
Response.BinaryWrite(oStream.Read(clChunkSize))

if ((oStream.Size % clChunkSize) <> 0)
Response.BinaryWrite(oStream.Read(oStream.Size % clChunkSize))
oStream.Close()
}

It has the advantage of using memory more effeciently on the server by
turning off buffering and chunking the file to the response. There is no
need to modify the buffering limit from the default 4MB when using this
function.

 

[Anthony Jones]

Is that fact or speculation? I've never seen evidence that this is more
efficient on the server.

It's not a fact by observation but neither is it speculation.

Here is a fact:-

Response.BinaryWrite doesn't return until all the buffer contents have been
sent.

I'll let you do the math.

 

[Dave Anderson]

It's not a fact by observation but neither is it speculation.

Response.BinaryWrite doesn't return until all the buffer
contents have been sent.

Doesn't return what? From what? To what? This does not make any sense to me.

I'll let you do the math.

OK. In either case, the same amount of content has to be put into the
buffer, and the same amount written from the buffer. You don't even bother
to speculate about the cost of repeatedly using Stream.Read(bytes) over
Stream.Read(adReadAll). So absent the Urim and Thummim neccessary for
scrying the Response Object, there does not seem to be any "math" to do. You
are *clearly* speculating.

I will grant this -- when combined with Response.isClientConnected [1],
using the chunked method allows you to abort the process before the entire
Stream is dealt with, which seems at first glance like a decent idea for
sending very large files. But since your example ignores isClientConnected,
you have not made any sort of "performance" case whatsoever.



[1] Is it a method or a property? The documentation calls it a property, buy
shows it as a method. Stranger still, you can treat it as either one in
JScript.
http://msdn.microsoft.com/library/en-us/iissdk/html/a07ec445-9240-4f23-8fe9-1f9548d3f8b0.asp

 

[Anthony Jones]

It's not a fact by observation but neither is it speculation.

Response.BinaryWrite doesn't return until all the buffer
contents have been sent.

Doesn't return what? From what? To what? This does not make any sense to me.

I'll let you do the math.

OK. In either case, the same amount of content has to be put into the
buffer, and the same amount written from the buffer. You don't even bother
to speculate about the cost of repeatedly using Stream.Read(bytes) over
Stream.Read(adReadAll). So absent the Urim and Thummim neccessary for
scrying the Response Object, there does not seem to be any "math" to do. You
are *clearly* speculating.

I will grant this -- when combined with Response.isClientConnected [1],
using the chunked method allows you to abort the process before the entire
Stream is dealt with, which seems at first glance like a decent idea for
sending very large files. But since your example ignores isClientConnected,
you have not made any sort of "performance" case whatsoever.

Dave,

My apologies I've seem to have managed to irk you once again.

What I meant by 'sent' is sent and acknowledged by the client.

With a buffered response the entire contents is duplicated in memory at the
same time, once in the array read from the stream and once in the buffer.
How much of the file is also duplicated inside the stream object I'm not
certain possibly all of it.

With the 'unbuffered' response only 1MB of the file is duplicated in memory
at the time plus whatever is in the ADODB stream.

Of course in the buffered solution the stream and the array are released
when the client starts to receive bytes. At this time the script context
will also be torn down. Ultimately the average memory requirement for the
buffered solution is likely to be less than my unbuffered one. This would
depend on details of the ADODB stream internals, how much the script context
uses and how big the file is. However it's peak memory requirements are
significantly more.

Another implementation detail that I don't known is whether IIS releases
some of the buffer whilst still using it. It seems an obvious thing to do
which would mean the buffered technique's average memory usage would be
somewhere approaching half the file size. The slower the link the closer to
half the file size it will be.

OTH the 'unbuffered' solution will keep the ADODB.Stream and therefore
potentially a duplicate of the file in memory for the duration of the send.
Hence it's peak requirement is at worst the file size + 2MB. However that is
also it's average requirement as well.

Overall then in a real world I'm probably wrong. Unless spike memory
requirement of the buffered solution is a problem. Although might I risk
your ire once again I do believe ADODB.Stream uses a .tmp file when mem
usage becomes excessive but I could be wrong ... again.

Anthony.