Adodb.Stream - Problem download big files

[Juan]

Hi!


I want to use ASP to download big files using ADODB.STREAM. It works very
fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp is
running. After some time (more or less 2 minutes) I get a response timeout.

Here is the code:

Server.ScriptTimeout = 30000
Response.Buffer = True

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"
Response.AddHeader "Content-Disposition","attachment; filename=" & sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary
oStream.Open
oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
Response.CharSet = "UTF-8"

For i = 0 To oStream.Size
i = i + 128000
Response.BinaryWrite(oStream.Read(128000))
Response.Flush

Next

oStream.Close
Set oStream = Nothing
Response.Flush
Response.End

Do I have to change something in my code - or perhaps a general setting in
IIS / the metabase?

Many thanks in advance



Juan

 

[Anthony Jones]

Hi!


I want to use ASP to download big files using ADODB.STREAM. It works very
fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp is
running. After some time (more or less 2 minutes) I get a response timeout.

Here is the code:

Server.ScriptTimeout = 30000

30,000 seconds is a long script time out

Response.Buffer = True

Change to False, you don't want to buffer anyway (yes I know you are
flushing but this is simpler).

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"

This is not a valid content type if you don't know what the content type is
use the fallback type: "application/octet-stream"

Response.AddHeader "Content-Disposition","attachment; filename=" & sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary
oStream.Open
oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit

Don't add the Content-Length header its a duplication, ASP adds that for
you.

Response.CharSet = "UTF-8"

How come you know the character encoding but don't know the content type?
Consider deleting this line or setting the content-type to the correct text
type.

For i = 0 To oStream.Size
i = i + 128000
Response.BinaryWrite(oStream.Read(128000))
Response.Flush

No need for a flush when buffering is off.

Next

oStream.Close
Set oStream = Nothing
Response.Flush
Response.End

Again no need for a flush and .End is draconian, only use it if you know
something bad will happen if you don't. Even then design out the something
bad rather than use .End.

Do I have to change something in my code - or perhaps a general setting in
IIS / the metabase?

The response timeout is a clientside thing however normally response
timeouts related to the time it takes to get the first chunk of data not the
overall send time.

 

[Daniel Crichton]

Anthony wrote on Thu, 3 Jul 2008 14:33:27 +0100:

Don't add the Content-Length header its a duplication, ASP adds that
for you.

If the stream is being written out in chunks (which it is in the For Next
loop), how does the ASP engine know what the final size will be?

 

[Daniel Crichton]

Juan wrote on Thu, 3 Jul 2008 06:00:01 -0700:

Hi!

I want to use ASP to download big files using ADODB.STREAM. It works
very fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp
is running. After some time (more or less 2 minutes) I get a response
timeout.

Here is the code:

Server.ScriptTimeout = 30000
Response.Buffer = True

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"
Response.AddHeader "Content-Disposition","attachment; filename=" &
sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary oStream.Open oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
Response.CharSet = "UTF-8"

For i = 0 To oStream.Size
i = i + 128000
Response.BinaryWrite(oStream.Read(128000))
Response.Flush
Next

This looks odd - you're looping from 0 to the size of the stream, yet you're
pushing out 128000 bytes at a time. I think you should be using

For i = 0 To oStream.Size Step 128000


Also, in your loop what happens when you reach the end of the stream?


This is how I handle download from one of my applications



Set oStream = Server.CreateObject("ADODB.Stream")
Call oStream.Open()
oStream.Type = 1
call oStream.LoadFromFile(strFilename)

iDownload = 1
If lcase(right(strfilename,4)) = ".pdf" then
Response.ContentType = "application/pdf"
else
Response.ContentType = "application/octet-stream"
end if
Response.AddHeader "Content-Disposition", "filename=" &
strfilename & ";"

Response.Buffer = False

'stream out the file in chunks
Do While Not (oStream.EOS)
Response.BinaryWrite oStream.Read(1024 * 256)
Loop

oStream.Close
Set oStream = Nothing


this reads the file into the stream (just like you already have), sets an
appropriate ContentType (as Anthony points out the one you use is not a
recognised MIME type), turns off buffering, and then writes out the stream
in 256kB chunks until there is nothing left to write to out (using
oStream.EOS to determine if the stream is at the end).

You really do need buffering turned off - it's entirely possible that the
80MB file is going over the defined ASP buffer limit and so you're getting
an ASP error thrown into the data, and that causes what appears to be a
timeout but is actually due to ASP terminating the script because it has
errored. So far with the above code I've had no trouble with files larger
than the defined ASP buffer limit, but I haven't tried it with anything as
large as your file.

 

[Daniel Crichton]

Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:

Anthony wrote on Thu, 3 Jul 2008 14:33:27 +0100:

If the stream is being written out in chunks (which it is in the For
Next loop), how does the ASP engine know what the final size will be?

Actually, I've just realised that the code I use myself doesn't set the
Content-Length header either. ASP doesn't add the Content-Length header
itself either as it has no idea what the file size is when it starts
outputting it, here's an example of the ASP headers from a download from my application:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:26:53 GMT
Transfer-Encoding: chunked
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;


This just means that you don't get the status bar in IE, for instance,
showing the download progress because the final size is not known. Adding
the Content-Length header allows IE (and any other browser that will show a
status bar) to give a progress with a final size value. I tested adding the
content length header to my own application and the headers changed to:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:29:46 GMT
Transfer-Encoding: chunked
Content-Length: 1890660
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;

and appeared to have no detrimental effect on the download.

 

[Daniel Crichton]

Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:

Juan wrote on Thu, 3 Jul 2008 06:00:01 -0700:

This looks odd - you're looping from 0 to the size of the stream, yet
you're pushing out 128000 bytes at a time. I think you should be using

I actually had another idea as to why it might be "timing out" - for 80MB of
data, your loop is running 83886080 times (80 * 1024 * 1024). If you are
using 128kB chunks, it only needs 666 loops to send 80MB. Check your event
logs and see if you have any ASP or IIS events pointing to the process being
terminated.

 

[Anthony Jones]

Daniel wrote to Juan on Thu, 3 Jul 2008 17:22:58 +0100:



I actually had another idea as to why it might be "timing out" - for 80MB of
data, your loop is running 83886080 times (80 * 1024 * 1024). If you are
using 128kB chunks, it only needs 666 loops to send 80MB. Check your event
logs and see if you have any ASP or IIS events pointing to the process being
terminated.

The loop is incrementing 128001 every iteration so potentially it may not
send all the file

 

[Anthony Jones]

Daniel wrote to Anthony Jones on Thu, 3 Jul 2008 17:14:31 +0100:



Actually, I've just realised that the code I use myself doesn't set the
Content-Length header either. ASP doesn't add the Content-Length header
itself either as it has no idea what the file size is when it starts
outputting it, here's an example of the ASP headers from a download from my application:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:26:53 GMT
Transfer-Encoding: chunked
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;


This just means that you don't get the status bar in IE, for instance,
showing the download progress because the final size is not known. Adding
the Content-Length header allows IE (and any other browser that will show a
status bar) to give a progress with a final size value. I tested adding the
content length header to my own application and the headers changed to:

HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 03 Jul 2008 16:29:46 GMT
Transfer-Encoding: chunked
Content-Length: 1890660
Content-Type: application/pdf
Server: Microsoft-IIS/6.0
Content-Disposition: filename=Certifications_at_a_Glance.pdf;

and appeared to have no detrimental effect on the download.

You're right it doesn't have any detrimental affect on IE in fact it serves
a useful purpose. However it is a strictly a breach of the protocol See
http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 para 3.
However the protocol does then indicate that if it is Content-Length is
present it should be ignored in this case.

 

[Anthony Jones]

Hi!


I want to use ASP to download big files using ADODB.STREAM. It works very
fine with files smaller than 80 MB.
On the Webserver I can see that memory allocation and the process w3wp is
running. After some time (more or less 2 minutes) I get a response timeout.

Here is the code:

Server.ScriptTimeout = 30000
Response.Buffer = True

Response.Clear
Response.Expires = 0
Response.ContentType = "Download-File"
Response.AddHeader "Content-Disposition","attachment; filename=" & sfile

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = adTypeBinary
oStream.Open
oStream.LoadFromFile(sfile)
Response.AddHeader "Content-Length", oStream.Size ' -- Schönheit
Response.CharSet = "UTF-8"

For i = 0 To oStream.Size
i = i + 128000
Response.BinaryWrite(oStream.Read(128000))
Response.Flush

Next

oStream.Close
Set oStream = Nothing
Response.Flush
Response.End

Do I have to change something in my code - or perhaps a general setting in
IIS / the metabase?

Many thanks in advance

FYI this is the routine I use for sending a file to response.

Sub SendFileToResponse(FilePath, FileName)

Const clChunkSize = 1048576 ' 1MB

Dim oStream, i
Response.Buffer = False

Response.ContentType = "application/octet-stream"
Response.AddHeader "Content-Disposition", _
"attachment; Filename=" & FileName

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = 1 ' Binary
oStream.Open
oStream.LoadFromFile FilePath

For i = 1 To oStream.Size \ clChunkSize
Response.BinaryWrite oStream.Read(clChunkSize)
Next
If (oStream.Size Mod clChunkSize) <> 0 Then
Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
End If
oStream.Close

End Sub

1MB buffer is a bit on the big side though 128K is a good choice.

 

[Evertjan.]

Anthony Jones wrote on 03 jul 2008 in
microsoft.public.inetserver.asp.general:

FYI this is the routine I use for sending a file to response.

Sub SendFileToResponse(FilePath, FileName)

Const clChunkSize = 1048576 ' 1MB

Dim oStream, i
Response.Buffer = False

Response.ContentType = "application/octet-stream"
Response.AddHeader "Content-Disposition", _
"attachment; Filename=" & FileName

Set oStream = Server.CreateObject("ADODB.Stream")
oStream.Type = 1 ' Binary
oStream.Open
oStream.LoadFromFile FilePath

For i = 1 To oStream.Size \ clChunkSize
Response.BinaryWrite oStream.Read(clChunkSize)
Next
If (oStream.Size Mod clChunkSize) <> 0 Then
Response.BinaryWrite oStream.Read(oStream.Size Mod clChunkSize)
End If

Would an error be given if an empty chunk or is sent?
And if the requested chunk size is larger than the remainder?

If not I would do:

For i = 0 To oStream.Size \ clChunkSize
Response.BinaryWrite oStream.Read(clChunkSize)
Next

 

[Alhambra Eidos Kiquenet]

Mister, any solution about it ??

any sample code that works fine ?? please

thanks in advance...