Strange problem with HttpWebRequest

B

Blackstar

I have a website I am trying to download a file from. This website is
strange in the fact that there is no direct link to the URL of the
file. The way the site works is as follows:

1) Enter in search criteria
2) Click on Download button to go to download page
3) Click on Download button on Download page
4) Server sends 2 bits of data to a script. The data sent does not
include the query data. I'm assuming that the data is stored either in
a server side session variable, or server-side on a temporary database.
5) The script processes and initiates a file download. Inside any of
the web pages, there are NO direct links directly to a file to
download. Once the file is compiled, the webste starts streaming the
data to the standard IE file download dialog. I need to capture this
file that is downloaded into a stream of some sort, then save it out to
a text file. The exported data is a text file.

Here is the code I am trying to use:

<------ CODE -------->
Dim fileURL As New Uri("http://www.somesite.com")
Dim myFileWebRequest As WebRequest =
WebRequest.Create(fileURL)
myFileWebRequest.Method = WebRequestMethods.Http.Post
Dim buffer As Byte() =
Encoding.UTF8.GetBytes("type=normal&cmd=DOWNLOAD")

myFileWebRequest.ContentType =
"application/x-www-form-urlencoded"
myFileWebRequest.ContentLength = buffer.Length
myFileWebRequest.Headers.Add(HttpRequestHeader.Cookie,
wb.Document.Cookie)
'myFileWebRequest.Headers.Add(HttpRequestHeader.Referer,
"http://www.somesite.com/resource/servlet/search")
'myFileWebRequest.Headers.Add(HttpRequestHeader.Accept,
"image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/x-shockwave-flash, application/vnd.ms-excel,
application/vnd.ms-powerpoint, application/msword, */*")

myFileWebRequest.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us")

myFileWebRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,
deflate")
'myFileWebRequest.Headers.Add(HttpRequestHeader.UserAgent,
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR
1.1.4322; .NET CLR 2.0.50727)")
'myFileWebRequest.Headers.Add(HttpRequestHeader.Host,
"www.somesite.com")
'myFileWebRequest.Headers.Add(HttpRequestHeader.Connection,
"Keep-Alive")

myFileWebRequest.Headers.Add(HttpRequestHeader.CacheControl,
"no-cache")

Dim myFileWebResponse As WebResponse =
CType(myFileWebRequest.GetResponse, HttpWebResponse)
Dim receivestream As Stream =
myFileWebResponse.GetResponseStream
Dim encode As Encoding =
System.Text.Encoding.GetEncoding("utf-8")
Dim readStream As New StreamReader(receivestream, encode)
Dim read(256) As Char
Dim count As Integer = readStream.Read(read, 0, 256)
Dim objWriter As New StreamWriter(sFileName)

While count > 0
Dim str As New String(read, 0, count)

objWriter.Write(str)
count = readStream.Read(read, 0, 256)
End While
readStream.Close()
objWriter.Close()

myFileWebResponse.Close()

<--------- END CODE --------->

I am not getting ANY errors or exceptions, the program just hangs on
the following line:
Dim myFileWebResponse As WebResponse =
CType(myFileWebRequest.GetResponse, HttpWebResponse)

It will sit on this line forever if you let it. The other problem is
that the website denies GET requests. It only allows POST requests.
I've had a number of other programmers look at it, and they are just as
stumped as me. If anyone can help, it would be most appreciated!

Also, if you look in the above code, you will notice a large number of
lines commented out that are designed to set items in the header. If I
uncomment them, then the program hangs and tells me that I am not
assigning it correctly and I need to use the name parameter. If the
other header items that are uncomment work, then shouldn't the
commented out ones work the same?

Thanks for the help in advance,
Wade
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

HttpWebRequest 1
HttpWebRequest 2
HttpWebRequest 2
HttpWebRequest & Fiddler 0
HttpWebRequest/.ServerXMLHTTP.4.0 - different results 8
HTTPWebRequest Problem 0
httpwebrequest Vista Error 0
Async HttpWebRequest 1

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top