socket programming

Discussion in 'Python' started by Pedro, May 4, 2013.

  1. Pedro

    Pedro Guest

    I'm writing a simple app that uses socket programming, I'm using the following from tutorialspoint as a framework but I'm having a couple of issues implementing it.

    First - this code constantly loops around an open socket. Is there a way touse something like an interrupt so I don't have to loop constantly to monitor the socket?

    Second, if any part of the program fails execution (crashes) the port oftenremains open on my windows machine and the only way to close it that i know of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.

    Thanks for looking



    SERVER:

    import socket # Import socket module

    s = socket.socket() # Create a socket object
    host = socket.gethostname() # Get local machine name
    port = 12345 # Reserve a port for your service.
    s.bind((host, port)) # Bind to the port

    s.listen(5) # Now wait for client connection.
    while True:
    c, addr = s.accept() # Establish connection with client.
    print 'Got connection from', addr
    c.send('Thank you for connecting')
    c.close() # Close the connection

    CLIENT:
    import socket # Import socket module

    s = socket.socket() # Create a socket object
    host = socket.gethostname() # Get local machine name
    port = 12345 # Reserve a port for your service.

    s.connect((host, port))
    print s.recv(1024)
    s.close # Close the socket when done
    Pedro, May 4, 2013
    #1
    1. Advertising

  2. On Sat, May 4, 2013 at 12:13 PM, Pedro <> wrote:
    > First - this code constantly loops around an open socket. Is there a way to use something like an interrupt so I don't have to loop constantly to monitor the socket?


    The accept() call should block. It's not going to spin or anything. If
    you need to monitor multiple sockets, have a look at select().

    > Second, if any part of the program fails execution (crashes) the port often remains open on my windows machine and the only way to close it that i know of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.


    It remains for a short time to ensure that there's no lurking
    connections. You can bypass this check by setting the SO_REUSEADDR
    option - lemme hunt that down in the Python docs, haven't done that in
    Python for a while...

    http://docs.python.org/3.3/library/socket.html#socket.socket.setsockopt

    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    That should do the job.

    ChrisA
    Chris Angelico, May 4, 2013
    #2
    1. Advertising

  3. Pedro

    Pedro Guest

    On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:
    > On Sat, May 4, 2013 at 12:13 PM, Pedro <> wrote:
    >
    > > First - this code constantly loops around an open socket. Is there a way to use something like an interrupt so I don't have to loop constantly to monitor the socket?

    >
    >
    >
    > The accept() call should block. It's not going to spin or anything. If
    >
    > you need to monitor multiple sockets, have a look at select().
    >
    >
    >
    > > Second, if any part of the program fails execution (crashes) the port often remains open on my windows machine and the only way to close it that iknow of is through task manager or by rebooting the machine. Is there an easy way around this problem ? If I don't close the port the program can't open it again and crashes.

    >
    >
    >
    > It remains for a short time to ensure that there's no lurking
    >
    > connections. You can bypass this check by setting the SO_REUSEADDR
    >
    > option - lemme hunt that down in the Python docs, haven't done that in
    >
    > Python for a while...
    >
    >
    >
    > http://docs.python.org/3.3/library/socket.html#socket.socket.setsockopt
    >
    >
    >
    > s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    >
    >
    >
    > That should do the job.
    >
    >
    >
    > ChrisA


    Thanks Chris, can you elaborate on the accept() call should block?
    Pedro, May 4, 2013
    #3
  4. On Sat, May 4, 2013 at 1:37 PM, Pedro <> wrote:
    > On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:
    >> The accept() call should block. It's not going to spin or anything. If
    >>
    >> you need to monitor multiple sockets, have a look at select().

    >
    > Thanks Chris, can you elaborate on the accept() call should block?


    When you call accept(), your program stops running until there's a
    connection. It's like calling input() (or raw_input()) and your
    program stopping until you type something. You can disable that by
    setting the socket nonblocking, but I don't think you're doing that
    here (and you probably don't want to).

    Consider the accept() call to be, effectively, like reading from the
    bound socket. In many ways it functions that way.

    ChrisA
    Chris Angelico, May 4, 2013
    #4
  5. Pedro

    Pedro Guest

    On Friday, May 3, 2013 11:56:01 PM UTC-4, Chris Angelico wrote:
    > On Sat, May 4, 2013 at 1:37 PM, Pedro <> wrote:
    >
    > > On Friday, May 3, 2013 10:23:38 PM UTC-4, Chris Angelico wrote:

    >
    > >> The accept() call should block. It's not going to spin or anything. If

    >
    > >>

    >
    > >> you need to monitor multiple sockets, have a look at select().

    >
    > >

    >
    > > Thanks Chris, can you elaborate on the accept() call should block?

    >
    >
    >
    > When you call accept(), your program stops running until there's a
    >
    > connection. It's like calling input() (or raw_input()) and your
    >
    > program stopping until you type something. You can disable that by
    >
    > setting the socket nonblocking, but I don't think you're doing that
    >
    > here (and you probably don't want to).
    >
    >
    >
    > Consider the accept() call to be, effectively, like reading from the
    >
    > bound socket. In many ways it functions that way.
    >
    >
    >
    > ChrisA


    Brilliant explanation Chris, I swear I read three different docs on that question and could not deduce what you just wrote. Thank you.
    Pedro, May 4, 2013
    #5
  6. On 4-5-2013 4:13, Pedro wrote:
    > SERVER:
    >
    > import socket # Import socket module
    >
    > s = socket.socket() # Create a socket object
    > host = socket.gethostname() # Get local machine name
    > port = 12345 # Reserve a port for your service.
    > s.bind((host, port)) # Bind to the port


    This won't always work as expected, particularly on machines with multiple network
    interfaces. Depending on what your situation requires, a simpler

    s.bind(('', port))

    can be more suitable. (empty string means INADDR_ANY)

    >
    > s.listen(5) # Now wait for client connection.
    > while True:
    > c, addr = s.accept() # Establish connection with client.
    > print 'Got connection from', addr
    > c.send('Thank you for connecting')


    The send() call returns the number of bytes actually sent. This number can be LESS than
    the length of the buffer you wanted to send! So you will need a loop here to make sure
    all data is actually transmitted.

    The easiest way out is to use this instead:

    c.sendall('Thank you for connecting')

    However, this might fail due to some I/O error and then it leaves your socket in an
    undetermined state because you can't tell how much of the data was actually transmitted.
    (Recovering from errors is problematic if not impossible with sendall, as far as I know
    the only thing you can do is just close the bad socket and create a new one)


    > c.close() # Close the connection
    >
    > CLIENT:
    > import socket # Import socket module
    >
    > s = socket.socket() # Create a socket object
    > host = socket.gethostname() # Get local machine name
    > port = 12345 # Reserve a port for your service.
    >
    > s.connect((host, port))
    > print s.recv(1024)


    While this usually seems to work (especially with small buffer sizes) it has the same
    problem as pointed out above with send(): recv() might not retrieve all data at once. It
    returns the amount of data actually received in that call.

    [warning, hairy details below]

    You HAVE to make a loop here to get all chunks of data until they add up to the total
    data size that you expected. (There's a flag MSG_WAITALL you can pass to recv to get
    around this. But it is not available on all systems, and on some systems where it is
    provided, it doesn't work correctly.) Also you need to be careful with the buffer size
    passed to recv, too large and it causes problems on some systems. Around 60kb seems a
    good upper bound here.

    This usually also means you need to somehow tell the receiving end of the socket how
    much data is to be expected, and only then send that actual data. One way to do this is
    to send the length first as a struct-packed integer (4 bytes), and the data after that.
    Note that even reading the 4 bytes on the receiving side to determine the expected
    length might be broken up in multiple chunks: you can't even expect recv(4) to always
    return those 4 bytes. So even that needs to be in a loop...

    Other ways to know on the receiving end when to stop reading from the socket is to
    standardize on some sort of termination character (such as '\0' or perhaps '\n'), fixed
    length buffers, or to always close the socket after every single message (but that is
    hugely inefficient if you need to send multiple messages).


    > s.close # Close the socket when done
    >


    Oh, you forgot the parentheses here:

    s.close()


    Bottom line:
    Socket programming on this level is hugely complicated. It doesn't seem too bad if you
    start of with these simple example programs, but that's false hope. If at all possible,
    avoid direct socket programming, and use a high-level protocol or library instead
    (ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer.


    Regards

    Irmen de Jong
    Irmen de Jong, May 4, 2013
    #6
  7. On Sat, May 4, 2013 at 7:37 PM, Irmen de Jong <> wrote:
    > Bottom line:
    > Socket programming on this level is hugely complicated. It doesn't seem too bad if you
    > start of with these simple example programs, but that's false hope. If at all possible,
    > avoid direct socket programming, and use a high-level protocol or library instead
    > (ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer.
    >


    It's not quite as bad as this paragraph seems to imply. Sure, there
    are a million things that can go wrong, but once you master that, it's
    the very easiest way to do cross-language, cross-platform,
    cross-computer communication. Practically EVERY language can do basic
    TCP sockets, but not every language has higher level bindings.

    ChrisA
    Chris Angelico, May 4, 2013
    #7
  8. Pedro

    Pedro Guest

    On Saturday, May 4, 2013 5:37:42 AM UTC-4, Irmen de Jong wrote:
    > On 4-5-2013 4:13, Pedro wrote: > SERVER: > > import socket # Import socket module > > s = socket.socket() # Create a socket object > host = socket.gethostname() # Get local machine name > port = 12345 # Reserve a portfor your service. > s.bind((host, port)) # Bind to the port This won't always work as expected, particularly on machines with multiple network interfaces. Depending on what your situation requires, a simpler s.bind(('', port)) can be more suitable. (empty string means INADDR_ANY) > > s.listen(5) # Now wait for client connection. > while True: > c, addr = s.accept() # Establish connection with client. > print 'Got connection from', addr > c.send('Thank you for connecting') The send() call returns the number of bytes actually sent. This number can be LESS than the length of the buffer you wanted to send! So you will need a loop here to make sure all data is actuallytransmitted. The easiest way out is to use this instead: c.sendall('Thank you for connecting') However, this might fail due to some I/O error and then it leaves your socket in an undetermined state because you can't tell howmuch of the data was actually transmitted. (Recovering from errors is problematic if not impossible with sendall, as far as I know the only thing youcan do is just close the bad socket and create a new one) > c.close() # Close the connection > > CLIENT: > import socket # Import socket module > > s= socket.socket() # Create a socket object > host = socket.gethostname() # Get local machine name > port = 12345 # Reserve a port for your service. > > s.connect((host, port)) > print s.recv(1024) While this usually seems to work (especially with small buffer sizes) it has the same problem aspointed out above with send(): recv() might not retrieve all data at once.It returns the amount of data actually received in that call. [warning, hairy details below] You HAVE to make a loop here to get all chunks of data until they add up to the total data size that you expected. (There's a flag MSG_WAITALL you can pass to recv to get around this. But it is not available on all systems, and on some systems where it is provided, it doesn't workcorrectly.) Also you need to be careful with the buffer size passed to recv, too large and it causes problems on some systems. Around 60kb seems a good upper bound here. This usually also means you need to somehow tell the receiving end of the socket how much data is to be expected, and only then send that actual data. One way to do this is to send the length first as a struct-packed integer (4 bytes), and the data after that. Note that even reading the 4 bytes on the receiving side to determine the expected length might be broken up in multiple chunks: you can't even expect recv(4) to alwaysreturn those 4 bytes. So even that needs to be in a loop... Other ways to know on the receiving end when to stop reading from the socket is to standardize on some sort of termination character (such as '\0' or perhaps '\n'),fixed length buffers, or to always close the socket after every single message (but that is hugely inefficient if you need to send multiple messages).. > s.close # Close the socket when done > Oh, you forgot the parentheses here: s.close() Bottom line: Socket programming on this level is hugely complicated. It doesn't seem too bad if you start of with these simple example programs, but that's false hope. If at all possible, avoid direct socket programming, and use a high-level protocol or library instead (ftp/http/some IPC library/Twisted). Let them deal with the complexity of the socket layer.. Regards Irmen de Jong


    Thanks for the reply. I'm sending short strings as commands to my server machine so the socket module seems to be doing the trick reliably. I'll try to add Twisted to my arsenal though.
    Cheers
    Pedro, May 6, 2013
    #8
  9. On Tue, May 7, 2013 at 1:54 AM, Pedro <> wrote:
    > Thanks for the reply. I'm sending short strings as commands to my server machine so the socket module seems to be doing the trick reliably. I'll try to add Twisted to my arsenal though.
    > Cheers


    I've never used Twisted, so I can't say how good it is. All I know is
    that what I learned about socket programming in C on OS/2 is still
    valid on Windows and on Linux, and in every language I've ever used
    (bar JavaScript and ActionScript, which are deliberately sandboxed and
    thus don't have direct socket calls available). So take your pick: Go
    with Twisted, and bind yourself to a particular library, or go with
    BSD sockets, and do the work yourself. Like everything else in
    programming, it's a tradeoff.

    ChrisA
    Chris Angelico, May 6, 2013
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    4,811
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    962
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    542
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    659
    Steve Holden
    Feb 1, 2009
  5. Steve Holden
    Replies:
    1
    Views:
    714
Loading...

Share This Page