socket.sendall(), non-blocking sockets, and multi-threaded socket sending

Discussion in 'Python' started by Tim Black, Aug 2, 2004.

  1. Tim Black

    Tim Black Guest

    My application requires sending a large piece (~2MB) of data to
    several devices on a network via TCP sockets. I have experimented with
    different methods for doing this and this has raised some questions
    about the implementation of Python sockets.

    (both methods use blocking sockets)

    Method 1: Calls socket.sendall(data) for each device in sequence, all
    in a single thread.

    Method 2: Each socket has its own thread that calls
    socket.send(datachunk) iteratively until all data is sent.

    So Method 1 sends all data to device1, then send all data to device2,
    etc... until all data is sent to all devices. Method 2 sends chunks of
    data to all devices in parallel. What I am seeing is that Method 1 is
    faster than Method 2. Given that the bottleneck is the actual sending
    of data on the sockets and not some latency in processing the message
    at the other end, this result makes sense to me. Method 2 involves
    sending the data in smaller chunks, whereas with Method 1 the
    chunksize is selected by the TCP/IP stack implementation, which is
    probably more efficient. Also, Method 2 entails context switching
    between the send threads.

    It turns out that using Method 1, all sendall() calls return
    immediately. I am assuming at this point that my app is in the hands
    of the TCP/IP stack implementation on my machine (win32-XP). Can
    anyone explain what happens inside sendall() for Win32?

    Also, I would like to get advice and opinions on what is the most
    efficient way to broadcast data to several devices on a network.

    Thanks,
    T
    Tim Black, Aug 2, 2004
    #1
    1. Advertising

  2. Tim Black

    Alan Kennedy Guest

    Re: socket.sendall(), non-blocking sockets, and multi-threaded socketsending

    [Tim Black]
    > My application requires sending a large piece (~2MB) of data to
    > several devices on a network via TCP sockets. I have experimented with
    > different methods for doing this and this has raised some questions
    > about the implementation of Python sockets.
    >
    > (both methods use blocking sockets)
    >
    > Method 1: Calls socket.sendall(data) for each device in sequence, all
    > in a single thread.
    >
    > Method 2: Each socket has its own thread that calls
    > socket.send(datachunk) iteratively until all data is sent.
    >
    > So Method 1 sends all data to device1, then send all data to device2,
    > etc... until all data is sent to all devices. Method 2 sends chunks of
    > data to all devices in parallel. What I am seeing is that Method 1 is
    > faster than Method 2. Given that the bottleneck is the actual sending
    > of data on the sockets and not some latency in processing the message
    > at the other end, this result makes sense to me. Method 2 involves
    > sending the data in smaller chunks, whereas with Method 1 the
    > chunksize is selected by the TCP/IP stack implementation, which is
    > probably more efficient. Also, Method 2 entails context switching
    > between the send threads.
    >
    > It turns out that using Method 1, all sendall() calls return
    > immediately. I am assuming at this point that my app is in the hands
    > of the TCP/IP stack implementation on my machine (win32-XP). Can
    > anyone explain what happens inside sendall() for Win32?
    >
    > Also, I would like to get advice and opinions on what is the most
    > efficient way to broadcast data to several devices on a network.


    I'd love to have enough time to get into discussing your findings
    about sockets. But unfortunately, I don't :-(

    But I did want to point out a module that you may not have come across
    that makes the job of distributing information peer-to-peer over LANs
    very easy: the spread module. I highly recommend that you take a look
    over it: it could save you a lot of wheel-reinvention.

    http://www.python.org/other/spread/

    HTH,

    --
    alan kennedy
    ------------------------------------------------------
    check http headers here: http://xhaus.com/headers
    email alan: http://xhaus.com/contact/alan
    Alan Kennedy, Aug 3, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Irmen de Jong
    Replies:
    0
    Views:
    327
    Irmen de Jong
    Jan 8, 2005
  2. nukleus
    Replies:
    14
    Views:
    814
    Chris Uppal
    Jan 22, 2007
  3. Thomas Rachel
    Replies:
    0
    Views:
    187
    Thomas Rachel
    Jan 7, 2013
  4. Steven D'Aprano
    Replies:
    1
    Views:
    112
    Chris Angelico
    Jan 7, 2013
  5. Philipp Hagemeister
    Replies:
    0
    Views:
    148
    Philipp Hagemeister
    Jan 8, 2013
Loading...

Share This Page