sending a file chunk by chunk instead as a whole to a web server

S

Sanjeeb

Hi,
I have a web client which send a file to a server as multipart form
data, the sending of data is fromhttp://code.activestate.com/recipes/146306-http-client-to-post-using-....

I dont want to open the whole file to memory(at cliend end) and then
send, i just want to send part by part, say chunk of 1024 bytes to the
server and then assemble at the server end.

Could some one suggest what would be the best way to do this?

Regards
-Sanjeeb

Hey any taker for this question???
 
K

Kushal Kumaran

Hi,
I have a web client which send a file to a server as multipart form
data, the sending of data is from
http://code.activestate.com/recipes/146306-http-client-to-post-using-multipartform-data/.

I dont want to open the whole file to memory(at cliend end) and then
send, i just want to send part by part, say chunk of 1024 bytes to the
server and then assemble at the server end.

Could some one suggest what would be the best way to do this?

There's no reason why sending the whole file implies reading the whole
file into memory at one time. You can just read your desired chunk
size and send it, then read the next chunk, and so on. You might have
to first find the total size to calculate what to set Content-Length
to.

ISTR questions of this nature have been asked in the past on this list.
 
R

Ryan Kelly

There's no reason why sending the whole file implies reading the whole
file into memory at one time. You can just read your desired chunk
size and send it, then read the next chunk, and so on. You might have
to first find the total size to calculate what to set Content-Length
to.

More concretely, you would restructure the encode_multipart_formdata()
function as a generator, yielding chunks of data to send one at a time.
Something like this:

def post_multipart(host,selector,fields,files):
...snip...
h.endheaders()
for chunk in encode_multipart_formdata_chunks(fields,files):
h.send(chunk)
errcode, errmsg, headers = h.getreply()
return h.file.read()


def encode_multipart_formdata(fields,files):
BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
for (key, value) in fields:
yield '--' + BOUNDARY
yield 'Content-Disposition: form-data; name="%s"' % key
yield ''
yield value
for (key, filename, value) in files:
yield '--' + BOUNDARY
yield 'Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename)
...etc...
...etc...

There are many improvements to make, but this should get you started.
For example, you'll need to calculate the total content-length rather
than just calling len(body) to obtain it. That's left as an exercise to
the reader.


Ryan


--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
(e-mail address removed) | http://www.rfk.id.au/ramblings/gpg/ for details


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEABECAAYFAkxXrpwACgkQfI5S64uP50q+fACgzcvVbFuG25WI1B/9Daqk+Zvp
LVIAoMbwFNlvrPsz5vZqwjR8s8P5FHhY
=rhPT
-----END PGP SIGNATURE-----
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,734
Messages
2,569,441
Members
44,832
Latest member
GlennSmall

Latest Threads

Top