submit form via HttpWebRequest or WebClient

J

John A Grandy

has anyone successfully used HttpWebRequest or WebClient class to simulate
submission of a simple HTML form?

for example: a very simple plain-vanilla form with a textbox and a button.
when the button is clicked the form is submitted with the textbox contents.

could you please post some sample code? thanks.
 
J

Joerg Jooss

John said:
has anyone successfully used HttpWebRequest or WebClient class to
simulate submission of a simple HTML form?

for example: a very simple plain-vanilla form with a textbox and a
button. when the button is clicked the form is submitted with the
textbox contents.

could you please post some sample code? thanks.

That's a full blown WebRequest/WebResponse sample including URL encoding.


public void PostForm(string url, string formData, string encodingName) {
// URL encode the post data, applying the character encoding given by
// encodingName.
MemoryStream content = new MemoryStream(formData.Length * 2);
Encoding encoding = Encoding.GetEncoding(encodingName);
string[] keyValuePairs = formData.Split('&', '=');
int numberOfPairs = keyValuePairs.Length;
bool isKey = true;
foreach (string keyValue in keyValuePairs) {
byte[] bytes = HttpUtility.UrlEncodeToBytes(keyValue, encoding);
content.Write(bytes, 0, bytes.Length);
if (isKey) {
content.WriteByte((byte) '=');
}
else {
content.WriteByte((byte) '&');
}
isKey = !isKey;
}
// We'll end up with one surplus '&' -- shouldn't hurt, but we cut it
anyway
content.SetLength(content.Length - 1);

// That's the actual HTTP part of the sample
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Method = "POST";
request.ContentType = String.Format(
"application/x-www-form-urlencoded; charset={0}", encodingName);
request.ContentLength = content.Length;
using (Stream requestStream = request.GetRequestStream()) {
content.WriteTo(requestStream);
}

HttpWebResponse response = (HttpWebResponse) request.GetResponse();
using (Stream responseStream = response.GetResponseStream()) {
byte[] buffer = new byte[response.ContentLength];
responseStream.Read(buffer, 0, buffer.Length);
// Process buffer... e.g. write to file, decode to string, etc.
}
}

Cheers,
 
J

John A Grandy

so whatever data the user would enter into the text box "textbox1", i
simulate by composing the string

formData="textbox1=thedata"

but how do i know what encodingName should be ?


Joerg Jooss said:
John said:
has anyone successfully used HttpWebRequest or WebClient class to
simulate submission of a simple HTML form?

for example: a very simple plain-vanilla form with a textbox and a
button. when the button is clicked the form is submitted with the
textbox contents.

could you please post some sample code? thanks.

That's a full blown WebRequest/WebResponse sample including URL encoding.


public void PostForm(string url, string formData, string encodingName) {
// URL encode the post data, applying the character encoding given by
// encodingName.
MemoryStream content = new MemoryStream(formData.Length * 2);
Encoding encoding = Encoding.GetEncoding(encodingName);
string[] keyValuePairs = formData.Split('&', '=');
int numberOfPairs = keyValuePairs.Length;
bool isKey = true;
foreach (string keyValue in keyValuePairs) {
byte[] bytes = HttpUtility.UrlEncodeToBytes(keyValue, encoding);
content.Write(bytes, 0, bytes.Length);
if (isKey) {
content.WriteByte((byte) '=');
}
else {
content.WriteByte((byte) '&');
}
isKey = !isKey;
}
// We'll end up with one surplus '&' -- shouldn't hurt, but we cut it
anyway
content.SetLength(content.Length - 1);

// That's the actual HTTP part of the sample
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
request.Method = "POST";
request.ContentType = String.Format(
"application/x-www-form-urlencoded; charset={0}", encodingName);
request.ContentLength = content.Length;
using (Stream requestStream = request.GetRequestStream()) {
content.WriteTo(requestStream);
}

HttpWebResponse response = (HttpWebResponse) request.GetResponse();
using (Stream responseStream = response.GetResponseStream()) {
byte[] buffer = new byte[response.ContentLength];
responseStream.Read(buffer, 0, buffer.Length);
// Process buffer... e.g. write to file, decode to string, etc.
}
}

Cheers,
 
J

Joerg Jooss

John said:
so whatever data the user would enter into the text box "textbox1", i
simulate by composing the string

formData="textbox1=thedata"
Right.

but how do i know what encodingName should be ?

That's the tricky part, because this depends on the web application. But
there's a simple heuristic that should work for most sites. Use the same
encoding that the site uses as character encoding. Open the web page and
check out what character encoding your browser applies (in IE
View->Encoding). Western Europe (ISO) is ISO-8859-1, whereas Western Europe
(Windows) is Windows-1252. Unicoded (UTF-8) is, um, UTF-8 ;->

Unless you're working with a non US or non Western European web site, those
are the ones you'll most likely encounter. In case of doubt, use ISO-8859-1.

Cheers,
 
J

John A Grandy

very interesting ....

unfortunately, i'm in a stickier situation than communicating with a
public-facing website.

i'm communicating over vpn with a private web-server meant for
data-provision to intermediaries.

the web-server does not validate credentials. it just immediately returns
data (interesting, as an xml doc)

the only working tool i have is a java applet written by someone else
(someone who is long gone).

the java applet is very simple. just a textbox and a submit button.

a long alphanumeric string is entered into the textbox (it ends with "^").

clicking the button fires a js function that calls form.submit and the form
is posted.

"view source" on the IE browseer hosting the java applet reveals only a very
simple block of html

a <textarea> element is used for data-entry.

i can't disclose the actual url -- but its similar to
"https://www.somecompany.com:5555/someservice.asmx/somefunction"

.... so apparently the web-server is hosting a .net web-service.


i tried your method. i constructed using the data-string successfully used
in the Java applet :

postData = "myTextArea=dataString";

encodingName = "utf-8";

request.GetResponse();

triggered a runtime error : "500 : internal server error"


do you think the problem might be with encodingName? what do you think
good possibilities are ?

or could the problem be in some other area ?
 
J

Joerg Jooss

John said:
very interesting ....

unfortunately, i'm in a stickier situation than communicating with a
public-facing website. [...]
i can't disclose the actual url -- but its similar to
"https://www.somecompany.com:5555/someservice.asmx/somefunction"

... so apparently the web-server is hosting a .net web-service.
Yes.

i tried your method. i constructed using the data-string
successfully used in the Java applet :

postData = "myTextArea=dataString";

encodingName = "utf-8";

request.GetResponse();

triggered a runtime error : "500 : internal server error"


do you think the problem might be with encodingName? what do you
think good possibilities are ?

or could the problem be in some other area ?

I guess the whole approrach could be wrong then. My sample code showed a
programmatic web form submission, but you probably need a SOAP client.
Actually, if it is a "proper" Web Service, VS .NET should be able to
generate the necessary client-side code for you. The test code you've posted
doesn't fit into the picture at all, though.

I guess you'll need to talk to the Web Service guys to find out what
requests their web service can handle.

Cheers,
 
J

John A Grandy

Hi Joerg ....

I don't think the approach you suggested is wrong.

I have a simple HTML page that submits its form to the web-server, and
successfully receives a response. The HTML form odes not communicate in any
way with any entity other than the following:

The JavaScript function triggered by the button click is very very simple :
all is does is a form.submit. There is no other client-side functionality
occurring.

Whatever method this company is using to process information on their side ,
they do accept posting of a simple form with

"InputData=0123456789abcdefghijklmnopqrstuvwxyz^" (for example)

Is there any problem with URL encoding the "^" character?

I think the encoding-type is probably the villian. I'll have to try various
possibilities.

Also, do you know various common headers that could be added? I see in
many of the postings to this newsgroup that quite a different number of
headers are being used.



Joerg Jooss said:
John said:
very interesting ....

unfortunately, i'm in a stickier situation than communicating with a
public-facing website. [...]
i can't disclose the actual url -- but its similar to
"https://www.somecompany.com:5555/someservice.asmx/somefunction"

... so apparently the web-server is hosting a .net web-service.
Yes.

i tried your method. i constructed using the data-string
successfully used in the Java applet :

postData = "myTextArea=dataString";

encodingName = "utf-8";

request.GetResponse();

triggered a runtime error : "500 : internal server error"


do you think the problem might be with encodingName? what do you
think good possibilities are ?

or could the problem be in some other area ?

I guess the whole approrach could be wrong then. My sample code showed a
programmatic web form submission, but you probably need a SOAP client.
Actually, if it is a "proper" Web Service, VS .NET should be able to
generate the necessary client-side code for you. The test code you've posted
doesn't fit into the picture at all, though.

I guess you'll need to talk to the Web Service guys to find out what
requests their web service can handle.

Cheers,
 
J

John A Grandy

Joerg Jooss said:
John said:
very interesting ....

unfortunately, i'm in a stickier situation than communicating with a
public-facing website. [...]
i can't disclose the actual url -- but its similar to
"https://www.somecompany.com:5555/someservice.asmx/somefunction"

... so apparently the web-server is hosting a .net web-service.
Yes.

i tried your method. i constructed using the data-string
successfully used in the Java applet :

postData = "myTextArea=dataString";

encodingName = "utf-8";

request.GetResponse();

triggered a runtime error : "500 : internal server error"


do you think the problem might be with encodingName? what do you
think good possibilities are ?

or could the problem be in some other area ?

I guess the whole approrach could be wrong then. My sample code showed a
programmatic web form submission, but you probably need a SOAP client.
Actually, if it is a "proper" Web Service, VS .NET should be able to
generate the necessary client-side code for you. The test code you've posted
doesn't fit into the picture at all, though.

I guess you'll need to talk to the Web Service guys to find out what
requests their web service can handle.

Cheers,
 
J

John A Grandy

are the web.config <webServices> settings related to successfully posting to
a remote web-service ?

or do these settings only relate to the hosting of web-services of
web-servers ... ?


for example, by default , the HttpPost setting is not present in web.config
.....



web.config file

<webServices>

<protocols>

<add name="HttpPost" />

</protocols>

</webServices>





Joerg Jooss said:
John said:
very interesting ....

unfortunately, i'm in a stickier situation than communicating with a
public-facing website. [...]
i can't disclose the actual url -- but its similar to
"https://www.somecompany.com:5555/someservice.asmx/somefunction"

... so apparently the web-server is hosting a .net web-service.
Yes.

i tried your method. i constructed using the data-string
successfully used in the Java applet :

postData = "myTextArea=dataString";

encodingName = "utf-8";

request.GetResponse();

triggered a runtime error : "500 : internal server error"


do you think the problem might be with encodingName? what do you
think good possibilities are ?

or could the problem be in some other area ?

I guess the whole approrach could be wrong then. My sample code showed a
programmatic web form submission, but you probably need a SOAP client.
Actually, if it is a "proper" Web Service, VS .NET should be able to
generate the necessary client-side code for you. The test code you've posted
doesn't fit into the picture at all, though.

I guess you'll need to talk to the Web Service guys to find out what
requests their web service can handle.

Cheers,
 
J

Joerg Jooss

John said:
are the web.config <webServices> settings related to successfully
posting to a remote web-service ?

or do these settings only relate to the hosting of web-services of
web-servers ... ?


for example, by default , the HttpPost setting is not present in
web.config ....



web.config file

<webServices>

<protocols>

<add name="HttpPost" />

</protocols>

</webServices>

You're right, these lines control whether an ASP.NET based web service
accepts "pure" (non SOAP) requests. The line <add name="httPost"/> should
enable posting. Note this is *disabled* in ASP.NET 1.1 by default.

Cheers,
 
J

John A Grandy

so, for my purposes of communicating with a remote web service (hosted by a
commercial company) ...

it is *their* responsibility to make sure that their web.config (or
machine.config) is properly setup to accept HTTPPost requests ... ?
 
J

Joerg Jooss

John said:
so, for my purposes of communicating with a remote web service
(hosted by a commercial company) ...

it is *their* responsibility to make sure that their web.config (or
machine.config) is properly setup to accept HTTPPost requests ... ?

*Technically* yes.

But in the end, this is decision between business partners ("We want POST!"
"We have SOAP!") and of course of IT security. If I was concerned: They
offer the service, they make they rules, and they need to tell you what you
can do and what you should do.

Cheers,
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,769
Messages
2,569,580
Members
45,055
Latest member
SlimSparkKetoACVReview

Latest Threads

Top