multipart/form-data in an HTTP client

Discussion in 'Python' started by Nelson Minar, Sep 1, 2004.

  1. Nelson Minar

    Nelson Minar Guest

    I'm writing some code to upload photos to Flickr. The Photo Upload API
    requires documents be POSTed via a multipart/form-data request. I was
    surprised to learn that Python 2.3's HTTP clients don't support this
    form of POSTs. There is support in cgi.py, for servers.

    There are some implementations of multipart/form-data on ASPN:
    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
    urllib2_file seems to meet my needs, but I'm not wild about how it's
    implemented. Is there some other recommended way to do
    multipart/form-data uploads with HTTP in Python?


    References:
    http://www.flickr.com/services/api/
    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
    http://www.faqs.org/rfcs/rfc1867.html
     
    Nelson Minar, Sep 1, 2004
    #1
    1. Advertising

  2. On Wed, 01 Sep 2004 14:52:33 GMT, Nelson Minar <> wrote:
    > There are some implementations of multipart/form-data on ASPN:
    > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
    > urllib2_file seems to meet my needs, but I'm not wild about how it's
    > implemented. Is there some other recommended way to do
    > multipart/form-data uploads with HTTP in Python?


    I like ClientForm. I hope one day it or something like it will be in the stdlib.
     
    Anthony Baxter, Sep 1, 2004
    #2
    1. Advertising

  3. Nelson Minar

    John J. Lee Guest

    Nelson Minar <> writes:

    > I'm writing some code to upload photos to Flickr. The Photo Upload API
    > requires documents be POSTed via a multipart/form-data request. I was
    > surprised to learn that Python 2.3's HTTP clients don't support this
    > form of POSTs. There is support in cgi.py, for servers.


    "Not supported" is an exaggeration, perhaps: there isn't special
    support to make it especially easy, true, but neither is there
    anything about urllib2 that makes it harder than it should be given
    the level of the interface exposed: you just have to add the right
    HTTP headers and HTTP request body data.


    > There are some implementations of multipart/form-data on ASPN:
    > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
    > urllib2_file seems to meet my needs, but I'm not wild about how it's
    > implemented.


    What's wrong with that implementation? Looks reasonable to me, though
    it seems to have a hack to work around bad servers that is different to
    the one I have in my own code. I can well believe that both hacks are
    required to work with as many servers as possible :-(


    John
     
    John J. Lee, Sep 1, 2004
    #3
  4. Nelson Minar

    Jeff Shannon Guest

    Nelson Minar wrote:

    >I'm writing some code to upload photos to Flickr. The Photo Upload API
    >requires documents be POSTed via a multipart/form-data request. I was
    >surprised to learn that Python 2.3's HTTP clients don't support this
    >form of POSTs. There is support in cgi.py, for servers.
    >
    >There are some implementations of multipart/form-data on ASPN:
    > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/146306
    >urllib2_file seems to meet my needs, but I'm not wild about how it's
    >implemented. Is there some other recommended way to do
    >multipart/form-data uploads with HTTP in Python?
    >
    >


    I've been using something closely modelled on that Cookbook recipe,
    without any real problems. (I've updated it to use HTTPConnection()
    and return the response object, and in my case I'm connecting to an
    HTTPS server, but these are trivial modifications.)

    I, too, was surprised that the existing Python libraries don't directly
    support multipart/form-data already, and I hope that this gets added in
    soon.

    Jeff Shannon
    Technician/Programmer
    Credit International
     
    Jeff Shannon, Sep 1, 2004
    #4
  5. Nelson Minar

    John J. Lee Guest

    Jeff Shannon <> writes:
    [...]
    > I've been using something closely modelled on that Cookbook recipe,
    > without any real problems. (I've updated it to use
    > HTTPConnection() and return the response object, and in my case I'm
    > connecting to an HTTPS server, but these are trivial modifications.)
    >
    > I, too, was surprised that the existing Python libraries don't
    > directly support multipart/form-data already, and I hope that this
    > gets added in soon.

    [...]

    Have you published your function? If not, please do.

    Several people would like to see such a function in 2.5 (including
    me), and your function sounds a good candidate. Maybe a little
    polishing is required, but don't let that stop you: if you don't get
    time to do that polishing, this is a rare occasion when somebody else
    is likely to push it the small additional distance it would need to go
    in order to get into the Python stdlib!


    John
     
    John J. Lee, Sep 1, 2004
    #5
  6. Nelson Minar

    Jeff Shannon Guest

    John J. Lee wrote:

    >Jeff Shannon <> writes:
    >[...]
    >
    >
    >>I've been using something closely modelled on that Cookbook recipe,
    >>without any real problems. (I've updated it to use
    >>HTTPConnection() and return the response object, and in my case I'm
    >>connecting to an HTTPS server, but these are trivial modifications.)
    >>
    >>

    >[...]
    >
    >Have you published your function? If not, please do.
    >
    >


    Here it is, for what it's worth. (Hopefully my mailer won't mangle the
    indentation...) As I noted, the differences between this and the
    Cookbook recipe are minimal -- I literally copied the recipe into my
    editor and made a few changes.

    If I were going to put further effort into polishing it for library use
    (which I probably won't have much opportunity to do), I'd make at least
    two changes. One would be to set it up to select between HTTP and
    HTTPS. (It currently tries to determine the protocol from the hostname
    string, but does nothing with that information.) The other would be to
    enable the file-subpart to add a Content-type header. (This is
    irrelevant for my particular application, as the form(s) I'm uploading
    to don't care.)

    In case it matters to anyone, inasmuch as this code is nearly identical
    to the published recipe, my changes can be considered to be under the
    same license as the original recipe. If someone feels like using this
    in any way, feel free.

    Jeff Shannon
    Technician/Programmer
    Credit International


    #! /usr/bin/python

    import httplib

    def post_multipart(host, selector, fields, files):
    """
    Post fields and files to an http host as multipart/form-data.
    fields is a sequence of (name, value) elements for regular form fields.
    files is a sequence of (name, filename, value) elements for data to be uploaded as files.
    Return an appropriate httplib.HTTPResponse object.
    """
    content_type, body = encode_multipart_formdata(fields, files)
    protocol = host.split(':')[0]
    h = httplib.HTTPSConnection(host)
    h.putrequest('POST', selector)
    h.putheader('content-type', content_type)
    h.putheader('content-length', str(len(body)))
    h.endheaders()
    h.send(body)
    response = h.getresponse()
    return response

    def encode_multipart_formdata(fields, files):
    """
    fields is a sequence of (name, value) elements for regular form fields.
    files is a sequence of (name, filename, value) elements for data to be uploaded as files.
    Return (content_type, body) ready for httplib.HTTPConnection instance
    """
    BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_---$---'
    CRLF = '\r\n'
    L = []
    for (key, value) in fields:
    L.append('--' + BOUNDARY)
    L.append('Content-Disposition: form-data; name="%s"' % key)
    L.append('')
    L.append(value)
    for (key, filename, value) in files:
    L.append('--' + BOUNDARY)
    L.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
    L.append('')
    L.append(value)
    L.append('--' + BOUNDARY + '--')
    L.append('')
    body = CRLF.join(L)
    content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
    return content_type, body
     
    Jeff Shannon, Sep 2, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ames Andreas (MPA/DF)

    Re: multipart/form-data in an HTTP client

    Ames Andreas (MPA/DF), Sep 2, 2004, in forum: Python
    Replies:
    1
    Views:
    488
    Jeff Shannon
    Sep 2, 2004
  2. Bruno Dilly
    Replies:
    0
    Views:
    455
    Bruno Dilly
    Aug 18, 2006
  3. Gabriel Genellina
    Replies:
    0
    Views:
    967
    Gabriel Genellina
    Aug 19, 2006
  4. Kevin DeValck
    Replies:
    1
    Views:
    762
    7stud --
    May 17, 2011
  5. Replies:
    1
    Views:
    441
Loading...

Share This Page