Re: OverflowError while sending large file via socket

Discussion in 'Python' started by Steven D'Aprano, Apr 13, 2009.

  1. On Mon, 13 Apr 2009 00:21:34 +0200, Ryniek90 wrote:

    > When i wanted to send an .iso file of 4GB length, i had traceback:
    > "OverflowError: requested number of bytes is more than a Python string
    > can hold"
    >
    > Sockets are being used in every network app, i.e: p2p progs (like
    > BitTorrent), and exchanged data is often bigger than 4GB.


    But they don't transfer the entire file as ONE packet. Split your data
    into smaller packets. I don't know what a good size for each packet would
    be, but if I were doing this, I'd probably start with 4096 or 8192
    *bytes*.

    http://www.amk.ca/python/howto/sockets/


    > So why i've
    > had that Traceback? How many number of bytes Python string can hold?


    The documentation doesn't seem to specify a maximum string length:

    http://docs.python.org/library/stdtypes.html

    which suggests to me that it will be implementation dependent. However,
    I'd guess that the current CPython implementation will have a hard limit
    of 2**32 bytes (4GB), and a soft limit on the amount of memory that you
    have. You're trying to create a single, continuous block of memory 4GB in
    size! Unless you've got *at least* 4GB of RAM, this is impossible even in
    principle, and in practice you need more than that to allow for the
    overhead of the operating system, Python, and any other applications you
    have running.



    --
    Steven
     
    Steven D'Aprano, Apr 13, 2009
    #1
    1. Advertising

  2. Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:
    >
    > which suggests to me that it will be implementation dependent


    The length of sequences is constrained by sys.maxsize
    (and no, you can't change it).
     
    Benjamin Peterson, Apr 13, 2009
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Paolo Alexis Falcone
    Replies:
    3
    Views:
    1,106
    Paolo Alexis Falcone
    Jun 29, 2003
  2. Luke
    Replies:
    3
    Views:
    475
    Duncan Grisby
    Jun 11, 2004
  3. =?ISO-8859-15?Q?Martin_B=FCrkle?=

    No error while sending via TCP Socket

    =?ISO-8859-15?Q?Martin_B=FCrkle?=, Jun 30, 2006, in forum: Python
    Replies:
    10
    Views:
    600
    Grant Edwards
    Jul 3, 2006
  4. Tzury Bar Yochay
    Replies:
    5
    Views:
    847
    Steven D'Aprano
    Oct 23, 2008
  5. .nLL
    Replies:
    1
    Views:
    177
    Anthony Jones
    Feb 13, 2007
Loading...

Share This Page