How to Do This Screen Scraping

T

Tarun

I Have tried to fetch some XML data through screen scraping by the
following code:

System.Net.ServicePointManager.CertificatePolicy = New
MyCertificatePolicy
dim path as String=""
Try
Dim req As HttpWebRequest = CType(WebRequest.Create(path),
HttpWebRequest)
Dim res As HttpWebResponse = CType(req.GetResponse, HttpWebResponse)
Dim strm As Stream = res.GetResponseStream
dim ds as New DataSet
ds.ReadXml(strm)
Catch ex As Exception

End Try

An exception is occuring in ds.ReadXml(strm) The Exception says Root
Element missing.
Exception::
System.Xml.XmlException: The root element is missing.
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlReader.MoveToContent()
at System.Data.DataSet.ReadXml(XmlReader reader, Boolean
denyResolving)
at System.Data.DataSet.ReadXml(Stream stream)
at Flower.Module1.readMyXml(String path)

(Note:The above code is written in readMyXML function in Module1)

But it is not returning the required XML data. Though the same code is
fetching required XML from a Desktop Based application.

The path from which i am fetching XML is Under SSL that is my URL
starts with "https://".
I also have tried with the below code. Here the data to post is been
seperated from the URL.
But here nothing is returned but a timeout is occuring at most probably
in calling "reqStream.Write()" method.

System.Net.ServicePointManager.CertificatePolicy = New
MyCertificatePolicy
Dim b() As Byte
Dim postData As String = path.Substring(path.IndexOf("?") + 1)
path = path.Substring(0, path.IndexOf("?"))
b = System.Text.Encoding.ASCII.GetBytes(postData)
Try
Dim req As HttpWebRequest = CType(WebRequest.Create(path),
HttpWebRequest)
req.Method = "POST"
req.ContentType = "application/x-www-form-urlencoded"
req.ContentLength = b.Length
Dim reqStream As Stream = req.GetRequestStream
reqStream.Write(b, 0, b.Length)

Dim res As HttpWebResponse = CType(req.GetResponse,
HttpWebResponse)
Dim strm As Stream = res.GetResponseStream
Dim ds as New DataSet
ds.ReadXml(strm)

Catch ex As Exception
End Try
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,537
Members
45,022
Latest member
MaybelleMa

Latest Threads

Top