PYTHON HTTP POST

L

lilanidhaval

Hi,

I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
.... 'User-Agent':'Mozilla/4.0',
.... 'Content-Length':'7'}405 Method Not Allowed

Regards,
Dhaval
 
K

koranthala

Does google accept POST?

Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
body = urllib.urlencode(dict(<fields>))
content_type = "application/x-www-form-urlencoded"
return content_type, body

If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
"""
fields is a sequence of (name, value) elements for regular form
fields.
files is a sequence of (name, filename, value) elements for data
to be uploaded as files
Return (content_type, body) ready for httplib.HTTP instance
"""
BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s";
filename="%s"' % (key, filename))
L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body

Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.

def post(host, selector, fields, files):
if files:
content_type, body = encode_multipart_formdata(fields, files)
else:
content_type, body = encode_formdata(fields)

h = httplib.HTTPConnection(host)
#Spoof Mozilla
headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
'Content-Type': content_type
}
h.request('POST', selector, body, headers)
res = h.getresponse()
return res.status, res.reason, res.read()

Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.

Hope this helps.
 
K

koranthala

Does google accept POST?

Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
        body = urllib.urlencode(dict(<fields>))
        content_type = "application/x-www-form-urlencoded"
        return content_type, body

If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
    """
    fields is a sequence of (name, value) elements for regular form
fields.
    files is a sequence of (name, filename, value) elements for data
to be uploaded as files
    Return (content_type, body) ready for httplib.HTTP instance
    """
    BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
    CRLF = '\r\n'
    L = []
    for (key, value) in fields:
        L.append('--' + BOUNDARY)
        L.append('Content-Disposition: form-data; name="%s"' % key)
        L.append('')
        L.append(value)
    for (key, filename, value) in files:
        L.append('--' + BOUNDARY)
        L.append('Content-Disposition: form-data; name="%s";
filename="%s"' % (key, filename))
        L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
        L.append('')
        L.append(value)
    L.append('--' + BOUNDARY + '--')
    L.append('')
    body = CRLF.join(L)
    content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
    return content_type, body

Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.

def post(host, selector, fields, files):
    if files:
       content_type, body = encode_multipart_formdata(fields, files)
    else:
       content_type, body = encode_formdata(fields)

    h = httplib.HTTPConnection(host)
    #Spoof Mozilla
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
        'Content-Type': content_type
        }
    h.request('POST', selector, body, headers)
    res = h.getresponse()
    return res.status, res.reason, res.read()

Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.

Hope this helps.

I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
...         'User-Agent':'Mozilla/4.0',
...         'Content-Length':'7'}
405 Method Not Allowed
Regards,
Dhaval

oops - Forgot to mention that POSTing files mechanism is taken from a
recipe in active state -
http://code.activestate.com/recipes/146306/
 
D

dhaval

The action part of the field is not set to anything.
I need any site with working example that accepts POST.

Does google accept POST?

Anyways, if you dont need to post files, you can use urlencode itself.
def encode_formdata(fields):
body = urllib.urlencode(dict(<fields>))
content_type = "application/x-www-form-urlencoded"
return content_type, body

If you need to post files too, then you will have to use multipart
data
def encode_multipart_formdata(fields, files):
"""
fields is a sequence of (name, value) elements for regular form
fields.
files is a sequence of (name, filename, value) elements for data
to be uploaded as files
Return (content_type, body) ready for httplib.HTTP instance
"""
BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
L = []
for (key, value) in fields:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s"' % key)
L.append('')
L.append(value)
for (key, filename, value) in files:
L.append('--' + BOUNDARY)
L.append('Content-Disposition: form-data; name="%s";
filename="%s"' % (key, filename))
L.append('Content-Type: %s' % mimetypes.guess_type(filename)
[0] or 'application/octet-stream'
L.append('')
L.append(value)
L.append('--' + BOUNDARY + '--')
L.append('')
body = CRLF.join(L)
content_type = 'multipart/form-data; boundary=%s' % BOUNDARY
return content_type, body

Since POST files doesnt work with urllib, you might have to use
httplib - or go for very high level tools like twisted.
I here show an example with httplib.

def post(host, selector, fields, files):
if files:
content_type, body = encode_multipart_formdata(fields, files)
else:
content_type, body = encode_formdata(fields)

h = httplib.HTTPConnection(host)
#Spoof Mozilla
headers = {
'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.0.4) Gecko/2008102920 Firefox/3.0.4',
'Content-Type': content_type
}
h.request('POST', selector, body, headers)
res = h.getresponse()
return res.status, res.reason, res.read()

Please note that you can use multipart whether or not files are there,
but parsing multipart usually is slower.

Hope this helps.

I need one complete example of how to do a http post to any site.
I have tried making a POST to google but all I am returned with is a
405 error.
I don't want to use Pygoogle as I want to try and do this with other
sites.
I am also having problems inputing with the param
I have tried Mechanize. There are no problems with getting data only
posting.
... 'User-Agent':'Mozilla/4.0',
... 'Content-Length':'7'}
405 Method Not Allowed
Regards,
Dhaval
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,008
Latest member
HaroldDark

Latest Threads

Top