multipart/form-data Post

[LD]

Hi,

I'm pulling my hair out!!

My problem is,

I need to automatically upload a zip file along with 3 other pieces of text
data to a web server and wait for it's xml response. Basically a
multipart/form-data post. I have tried it using a regular html form and it
works but that is no good because it leaves you at the other server where
you posted the data and I need to process the xml response and redirect. I
have tried the following code below and I either get that the post is
malformed or the connection was terminated. Is there a better way to do
this? I really appreciate any insite into this.

Dim vbCrLf As String = Convert.ToString(Chr(13)) + Convert.ToString(Chr(10))

Dim vbCr As String = Convert.ToString(Chr(13))

Dim vbLf As String = Convert.ToString(Chr(10))

Dim boundary As String =
"aevom43s98jq30m9w492024234ioafwf02439t0j05wa35w5ref"

Dim StrB As New System.Text.StringBuilder()

Dim Tem() As Byte

Dim st As FileStream = File.OpenRead("E:\data.zip")

ReDim Tem(CInt(2 ^ 15))

Do While st.Position < st.Length

st.Read(Tem, 0, Tem.Length - 1)

Loop

st.Close()

StrB.Append(System.Text.Encoding.Default.GetString(Tem))

Dim myWebClient As New System.Net.WebClient()

Dim URL As String

Dim body As String

body = "Data found after header end:" & vbCrLf & vbCrLf & _

boundary & vbCrLf & _

"Content-Disposition: " & "form-data; name=""username""" & vbCrLf & vbCrLf &
_

"""username""" & vbCrLf

body = body & boundary & vbCrLf & _

"Content-Disposition: " & "form-data; name=""password""" & vbCrLf & vbCrLf &
_

"""password""" & vbCrLf

body = body & boundary & vbCrLf & _

"Content-Disposition: " & "form-data; name=""client""" & vbCrLf & vbCrLf & _

"""AUI""" & vbCrLf

body = body & boundary & vbCrLf & _

"Content-Disposition: " & "form-data; name=""data""" & ";
filename=""data.zip""" & vbCrLf & vbCrLf & _

"Content-Type: application/x-zip-compressed" & vbCrLf & vbCrLf & _

StrB.ToString() & vbCrLf & boundary & "--"

URL = "server I need to send to."

Dim webRequest As HttpWebRequest = CType(webRequest.Create(URL),
HttpWebRequest)

Dim encoding As New System.Text.ASCIIEncoding()

Dim byte1 As Byte() = encoding.GetBytes(body)

webRequest.Method = "POST"

webRequest.ContentType = "multipart/form-data; boundary=" & boundary

webRequest.ContentLength = byte1.Length

webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;
Q312461; .NET CLR 1.0.3705)"

webRequest.Accept = "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg,
application/vnd.ms-excel, application/msword, application/x-shockwave-flash,
*/*"

webRequest.KeepAlive = True

Dim newStream As Stream = webRequest.GetRequestStream()

newStream.Write(byte1, 0, byte1.Length)

newStream.Close()

Dim webResponse As HttpWebResponse = CType(webRequest.GetResponse(),
HttpWebResponse)

webRequest.GetResponse()

Dim sReader As New StreamReader(webResponse.GetResponseStream(), False)

Dim rawHTML As String

rawHTML = sReader.ReadToEnd()

Response.Write(rawHTML)

 

[Sherif ElMetainy]

Hello

I think your problem is that you convert the binary content to a string,
concatenate it to the rest of the data and convert back to byte[] array. May
be the problem is that a null byte in the binary file is interpreted as a
string null terminator. So you get an incorrect string. Better convert the
other strings to byte arrays and then write the byte arrays in correct order
to the webrequest object.

Hope this info helps

Best Regards

 

[navyjax2]

Multitype form-data post

I'm writing a similar kind of post procedure in C# .NET for a console app. I am trying to emulate what a webpage is doing when it posts two fields ("datasource" and "feedtype" are the IDs) and an XML file to a server URL ("data" is the ID for that box). (It will work if done that way, but that is not automated, so we are trying to do it this way to automate the process, so please no comments about doing it with a web page "submit" button instead.) Here is my procedure:

        static void postData()
        {
            string dataBoundary = "--xyz";
 
            HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(serverUrl + ":19900/xmlfeed");
            webRequest.ContentType = "multipart/form-data; boundary=" + dataBoundary;
            webRequest.Method = "POST";

            string datasourceStr = String.Format(
                  "--{0}\r\n"
                + "Content-Disposition: form-data; name=\"datasource\"; name=\"files\"\r\n"
                + "Content-Type: application/octet-stream\r\n"
                + "\r\n",
                dataBoundary, "sample");

            string feedtypeStr = String.Format(
                  "--{0}\r\n"
                + "Content-Disposition: form-data; name=\"feedtype\"; name=\"files\"\r\n"
                + "Content-Type: application/octet-stream\r\n"
                + "\r\n",
                dataBoundary, "full");

            string fileStr = String.Format(
                  "--{0}\r\n"
                + "Content-Disposition: form-data; name=\"data\"; filename=\"C:\\content.txt\"\r\n"
                + "Content-Type: application/octet-stream\r\n"
                + "\r\n",
                dataBoundary, @"C:\content.txt");

            // Get the total form post size 
            string totalString = datasourceStr + feedtypeStr + fileStr;
            /*
            byte[] datasourceBytes = Encoding.Default.GetBytes(datasourceStr);
            byte[] feedtypeBytes = Encoding.Default.GetBytes(feedtypeStr);
            byte[] fileBytes = Encoding.Default.GetBytes(fileStr);
            int totalBytes = datasourceBytes.Length + feedtypeBytes.Length + fileStr.Length;
            */
            byte[] totalBytes = Encoding.Default.GetBytes(totalString);
            webRequest.ContentLength = totalBytes.Length;
            
            // Send the data
            Stream webStream = webRequest.GetRequestStream();
  /*        webStream.Write(datasourceBytes, 0, datasourceBytes.Length);
            webStream.Write(feedtypeBytes, 0, feedtypeBytes.Length);
            webStream.Write(fileBytes, 0, fileStr.Length);
   */
            webStream.Write(totalBytes, 0, totalBytes.Length);
            webStream.Close();
           
            // Read the response
            HttpWebResponse res = (HttpWebResponse)webRequest.GetResponse();
            StreamReader sr = new StreamReader(res.GetResponseStream(), Encoding.Default);
            string backstr = sr.ReadToEnd();
            Console.Write(backstr);
            sr.Close();
            res.Close();
            
        }

I have stuff I've commented out, and left in there, to show you what I've tried - I tried combining the strings and getting the total bytes and using that, and I've tried sending them individually using individual byte counts for the stream and leaving webRequest.ContentLength = totalBytes.Length. Honestly, I don't even know if what I have is the right format, what the \r\n's are all about, and what to do with the dataBoundary and all. I only saw that on the bottom of http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2 and thought I'd give it a shot when I saw http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_23620836.html. Please assist. Thanks.

-Tom

 

[navyjax2]

Multitype form-data post

Turns out I wasn't too far off. I found some code that helped me along at http://enterprise-code-samples.googlecode.com/svn/trunk/c-sharp-feed/App_Code/ for what I was trying to do - feed 2 form fields and an XML file to our Google Search Appliance. To make this work, you would need "feeder.cs" from that link, and add the file to your project. Below is the highly revised, working version of the code that interacts/interfaces with it (my version, but based on the old GSAFeedService.cs from that site). This allows it to work without having to be a web service, you would just call "postWebData()" in your application:

using System;
using System.Collections.Generic;
using System.Text;
using System.Xml;
using System.Net;
using System.IO;
using System.Diagnostics;
using System.Web;
using System.Collections;
using System.Collections.Specialized;
using FeedFilePoster;

namespace FeedFilePoster
{
    class Program
    {
        public static string gsaFeedURL = "http://yourGSA.domain.com:19900/xmlfeed";

        static void Main()
        {
            try
            {
                postWebData();
            }
            catch (Exception ex)
            {
                Console.WriteLine("");
                Console.WriteLine("Error");
                Console.WriteLine("=====");
                Console.WriteLine("");
                Console.WriteLine(ex.Message);
                Console.WriteLine("");
                Console.WriteLine("Troubleshooting");
                Console.WriteLine("================");
                Console.WriteLine("");
                Console.WriteLine("This application expects a feed file named FeedFile.xml to be at ");
                Console.WriteLine("the root of the C: drive.  Please ensure it is present.");
                Console.WriteLine("We are posting to " + gsaFeedURL);
                Console.WriteLine("If this is not correct, please enter a hosts entry to the new host.");
                Console.WriteLine("");
            }
        }

        // new one I made from C# web service
        public static void postWebData()
        {
            StringDictionary dictionary = new StringDictionary();
            UploadSpec uploadSpecs = new UploadSpec();
            UTF8Encoding encoding = new UTF8Encoding();
            byte[] bytes;
            Uri gsaURI = new Uri(gsaFeedURL);  // Create new URI to GSA feeder gate
            string sourceURL = @"C:\FeedFile.xml"; // Location of the XML feed file
            // Two parameters to send
            string feedtype = "full";
            string datasource = "test";            

            try
            {
                // Add the parameter values to the dictionary
                dictionary.Add("feedtype", feedtype);
                dictionary.Add("datasource", datasource);
               
                // Load the feed file created and get its bytes
                XmlDocument xml = new XmlDocument();
                xml.Load(sourceURL);
                bytes = Encoding.UTF8.GetBytes(xml.OuterXml);

                // Add data to upload specs
                uploadSpecs.Contents = bytes;
                uploadSpecs.FileName = sourceURL;
                uploadSpecs.FieldName = "data";

                // Post the data
                if ((int)HttpUpload.Upload(gsaURI, dictionary, uploadSpecs).StatusCode == 200)
                {
                    Console.WriteLine("Successful.");
                }
                else
                {
                    // GSA POST not successful
                    Console.WriteLine("Failure.");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
        }
    }
}

Feeder.cs (AKA the real magic I was trying to create before) is below. Credits to author Gary Comstock. Take note, I don't use "HttpDownload" because I don't feel like I need to check to see if the file I'm trying to upload is there using a WebRequest. I could do it other ways if I wanted, and I'm not sure I'm going to need to for my project - I can just use a try...catch for that, like I did above. If you're curious on how to do it their way, it's in the GSAFeedService.cs file, when it calls HttpDownload, however.

using System;
using System.Collections;
using System.Collections.Specialized;
using System.IO;
using System.Net;

/// <summary>
/// Provides functionality for programmatically downloading files 
/// with the HTTP protocol.
/// </summary>
public sealed class HttpDownload
{

    // Prevent construction
    private HttpDownload() { }

    /// <summary>
    /// Downloads the given file into a byte array from a url.
    /// </summary>
    /// <param name="pathname">The http path of the file to download</param>
    public static HttpWebResponse Download(Uri pathname)
    {
        // Initialize the request object
        HttpWebRequest req = (WebRequest.Create(pathname) as HttpWebRequest);
        req.Method = "GET";
        req.ContentType = "text/xml";
        req.Timeout = -1;

        return (req.GetResponse() as HttpWebResponse);
    }
}

  /// <summary>
  /// Provides functionality for programmatically uploading files 
  /// with the HTTP protocol.
  /// </summary>
  public sealed class HttpUpload
  {

    // Prevent construction
    private HttpUpload() { }

    /// <summary>
    /// Uploads the given data.
    /// </summary>
    /// <param name="uri">The URI to which the data shall be sent.</param>
    /// <param name="formFields">Form fields to be posted along with the file.</param>
    /// <param name="objects">The data to be sent.</param>
    public static HttpWebResponse Upload(Uri uri, StringDictionary formFields, params UploadSpec[] objects) {

      // Initialize the request object
      HttpWebRequest req = (WebRequest.Create(uri) as HttpWebRequest);
      
      string boundary = Guid.NewGuid().ToString().Replace("-", "");
      req.ContentType = "multipart/form-data; boundary=" + boundary;
      req.Method = "POST";

      MemoryStream postData = new MemoryStream();
      string newLine = "\r\n";
      StreamWriter sw = new StreamWriter(postData);

      if (formFields != null) 
        foreach (DictionaryEntry de in formFields) {
          sw.Write("--" + boundary + newLine);
          sw.Write("Content-Disposition: form-data; name=\"{0}\"{1}{1}{2}{1}",
            de.Key,
            newLine,
            de.Value
          );
        }
      
      foreach (UploadSpec us in objects) {
        sw.Write("--" + boundary + newLine);
        sw.Write("Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"{2}",
          us.FieldName,
          us.FileName,
          newLine
        );
        sw.Write("Content-Type: application/octet-stream" + newLine + newLine);
        sw.Flush();

        postData.Write(us.Contents, 0, us.Contents.Length);
        sw.Write(newLine);
      }

      sw.Write("--{0}--{1}", boundary, newLine);
      sw.Flush();

      req.ContentLength = postData.Length;
      using (Stream s = req.GetRequestStream())
        postData.WriteTo(s);
      postData.Close();

      return (req.GetResponse() as HttpWebResponse);
    }
  }

  /// <summary>
  /// Holds the information about the file(s) to be uploaded.
  /// </summary>
  public struct UploadSpec {
      
    private byte[] contents;
    private string fileName;
    private string fieldName;

    /// <summary>
    /// The byte array content to be uploaded.
    /// </summary>
    public byte[] Contents {
      get { return contents; }
      set { contents = value; }
    }

    /// <summary>
    /// The name of the file to be uploaded.
    /// </summary>
    public string FileName {
      get { return fileName; }
      set { fileName = value; }
    }

    /// <summary>
    /// The HTML form field the file should be uploaded into.
    /// </summary>
    public string FieldName {
      get { return fieldName; }
      set { fieldName = value; }
    }

    /// <summary>
    /// Creates a new upload spec based on a byte array.
    /// </summary>
    /// <param name="contents">The contents to be uploaded.</param>
    /// <param name="fileName">The file to be uploaded.</param>
    /// <param name="fieldName">The field name as which this file shall be sent to.</param>
    public UploadSpec(byte[] contents, string fileName, string fieldName) {
      this.contents = contents;
      this.fileName = fileName;
      this.fieldName = fieldName;
    }
  }

Hope it helps someone.

-Tom