need a thread to keep a socket connection alive?

Discussion in 'Python' started by nephish@xit.net, Apr 22, 2006.

  1. Guest

    hey there,

    i have a script that waits for message packets from a data server over
    a socket.
    it goes a little like this:

    while 1:
    x+=1
    databack = sockobj.recv(158)
    if databack:

    print 'caught a message %s bytes ' % len(databack)
    if len(databack) > 120:
    message = databack[3:-3] #strip stx and enx
    print '\n\n%s' % message
    else:
    break
    print 'end data ack'


    it works fine for a while, but the server requires that i send a
    heartbeat ping every 600 seconds or it will terminate the connection.

    so i also need something like
    while 1:
    sockobj.send(ping)
    ping_acknowlage = sockobj.recv(48)
    time.sleep(550)



    should i do this with threads? i dont want to interrupt the listening
    cycle to send a ping.

    appreciate any tips on how would be the best way to pull this off.
    , Apr 22, 2006
    #1
    1. Advertising

  2. Rene Pijlman Guest

    :
    >i have a script that waits for message packets from a data server over
    >a socket.


    Using what network protocol?

    >it works fine for a while, but the server requires that i send a
    >heartbeat ping every 600 seconds or it will terminate the connection.

    [...]
    >should i do this with threads? i dont want to interrupt the listening
    >cycle to send a ping.


    If this is a TCP connection with a conversational protocol, you can't have
    two threads reading bytes of the socket, without some sort of
    coordination. When one thread parses the bytes it received, some bytes may
    be part of the next message for the other thread.

    You may be better off with asynchronous I/O and a state machine model.
    http://squirl.nightmare.com/medusa/async_sockets.html
    http://www.python.org/doc/lib/module-asyncore.html
    http://twistedmatrix.com/projects/core/documentation/howto/clients.html

    --
    René Pijlman
    Rene Pijlman, Apr 22, 2006
    #2
    1. Advertising

  3. Guest

    thanks for the info, i will likely use the first link you posted with
    the async module just to get it going, but i want to learn more about
    twisted for later. there is even an O'Reilly book on it i see.
    thanks for the tips,
    sk
    , Apr 22, 2006
    #3
  4. Ben Sizer Guest

    wrote:
    > i have a script that waits for message packets from a data server over
    > a socket.


    If you're using TCP, bear in mind that you do not receive packets - you
    receive a stream of data, which may usually come in the same quantities
    as it was sent, but not always. If you don't take that into account,
    you may end up missing a valid message because it arrived in several
    parts.

    > it works fine for a while, but the server requires that i send a
    > heartbeat ping every 600 seconds or it will terminate the connection.


    It is probably worth just reconnecting if necessary. After all, you
    could be disconnected for other reasons too.

    --
    Ben Sizer
    Ben Sizer, Apr 24, 2006
    #4
  5. Roy Smith Guest

    wrote:

    > hey there,
    >
    > i have a script that waits for message packets from a data server over
    > a socket.
    > it goes a little like this:
    >
    > while 1:
    > x+=1
    > databack = sockobj.recv(158)
    > if databack:
    >
    > print 'caught a message %s bytes ' % len(databack)
    > if len(databack) > 120:
    > message = databack[3:-3] #strip stx and enx
    > print '\n\n%s' % message
    > else:
    > break
    > print 'end data ack'


    You need to go review how TCP works. All that it guarantees is that you
    will receive bytes in the same order they were sent. It says nothing about
    maintaining record boundaries. Just because you did a send(n) at one end,
    it doesn't mean that you can expect to read n bytes in a single recv() call
    at this end. Multiple send() calls could have their contents accumlated
    into a single recv() call, or a single send() could get broken up into
    several recv() calls.

    If you want to read fixed-length messages (as you appear to be trying to do
    with your recv(158)), you need to build a buffering layer which reads from
    the socket into a buffer and then doles out messages to a higher layer from
    that buffer.

    > it works fine for a while, but the server requires that i send a
    > heartbeat ping every 600 seconds or it will terminate the connection.
    >
    > so i also need something like
    > while 1:
    > sockobj.send(ping)
    > ping_acknowlage = sockobj.recv(48)
    > time.sleep(550)


    This needs to be burried in a lower layer as well. You want to build some
    kind of bufferedConnection class which hides all this gunk from your
    application. You probably will want sendMessage() and recvMessage()
    methods for your class. You probably want to have this class create a
    thread to handle the low-level I/O asyncronously, and put the heatbeat
    processing in there.

    This is not a trivial problem. By the time you're done with it, you will
    have learned a lot about how to communicate over a network.
    Roy Smith, Apr 24, 2006
    #5
  6. Roy Smith wrote:

    > If you want to read fixed-length messages (as you appear to be trying to do
    > with your recv(158)), you need to build a buffering layer which reads from
    > the socket into a buffer and then doles out messages to a higher layer from
    > that buffer.


    > This is not a trivial problem. By the time you're done with it, you will
    > have learned a lot about how to communicate over a network.


    however, creating a buffered layer for reading is a trivial problem: just call
    makefile on the socket object, and use the resulting object as a file handle:

    >>> s = socket.socket()
    >>> s.connect(("www.python.org", 80))
    >>> s.send("GET / HTTP/1.0\n\n")

    16
    >>> f = s.makefile()
    >>> f.readline()

    'HTTP/1.1 200 OK\r\n'
    >>> f.readline()

    'Date: Mon, 24 Apr 2006 12:37:46 GMT\r\n'
    >>> f.read(10)

    'Server: Ap'
    >>> f.read(10)

    'ache/2.0.5'
    >>> f.readline()

    '4 (Debian GNU/Linux) DAV/2 SVN/1.1.4 mod_python/3.1.3 ...
    >>> f.readline()

    'Last-Modified: Mon, 24 Apr 2006 04:52:53 GMT\r\n'

    etc.

    </F>
    Fredrik Lundh, Apr 24, 2006
    #6
  7. Roy Smith Guest

    "Fredrik Lundh" <> wrote:
    > however, creating a buffered layer for reading is a trivial problem: just call
    > makefile on the socket object, and use the resulting object as a file handle:


    The problem with that is that makefile() requires the socket to be in
    blocking mode. If you're going to be implementing heartbeat, that's
    probably not what you want.
    Roy Smith, Apr 24, 2006
    #7
  8. Serge Orlov Guest

    wrote:
    > hey there,
    >
    > i have a script that waits for message packets from a data server over
    > a socket.
    > it goes a little like this:
    >
    > while 1:
    > x+=1
    > databack = sockobj.recv(158)
    > if databack:
    >
    > print 'caught a message %s bytes ' % len(databack)
    > if len(databack) > 120:
    > message = databack[3:-3] #strip stx and enx
    > print '\n\n%s' % message
    > else:
    > break
    > print 'end data ack'
    >
    >
    > it works fine for a while, but the server requires that i send a
    > heartbeat ping every 600 seconds or it will terminate the connection.
    >
    > so i also need something like
    > while 1:
    > sockobj.send(ping)
    > ping_acknowlage = sockobj.recv(48)
    > time.sleep(550)
    >
    >
    >
    > should i do this with threads? i dont want to interrupt the listening
    > cycle to send a ping.
    >
    > appreciate any tips on how would be the best way to pull this off.


    sockobj.settimeout(550)

    before the loop and later in the loop:

    try:
    databack = sockobj.recv(158)
    except socket.timeout:
    ping_server(sockobj)
    continue

    Also, as other people pointed out, you'd better make buffered socket
    with .makefile() socket method.
    Serge Orlov, Apr 24, 2006
    #8
  9. Roy Smith Guest

    Serge Orlov <> wrote:
    > sockobj.settimeout(550)
    > [...]
    > Also, as other people pointed out, you'd better make buffered socket
    > with .makefile() socket method.


    If I understand the docs for the socket module correctly, these two
    suggestions are mutually incompatable.
    Roy Smith, Apr 24, 2006
    #9
  10. Serge Orlov Guest

    Roy Smith wrote:
    > Serge Orlov <> wrote:
    > > sockobj.settimeout(550)
    > > [...]
    > > Also, as other people pointed out, you'd better make buffered socket
    > > with .makefile() socket method.

    >
    > If I understand the docs for the socket module correctly, these two
    > suggestions are mutually incompatable.


    Perhaps this restriction was lifted?

    >>> s = socket.socket()
    >>> s.settimeout(15)
    >>> s.connect(("www.python.org", 80))
    >>> f = s.makefile()
    >>> f.readline()


    Traceback (most recent call last):
    File "<pyshell#21>", line 1, in -toplevel-
    f.readline()
    File "C:\Python24\lib\socket.py", line 340, in readline
    data = self._sock.recv(self._rbufsize)
    timeout: timed out
    >>>
    Serge Orlov, Apr 24, 2006
    #10
  11. Guest

    ok, thanks for all the suggestions, gents, i clearly have more to read
    on this.
    i have discovered that the server will send a request for the heartbeat
    ping if its almost timed out, so i use the length of the message to
    determine what to do with it.

    msg = sockobj.recv(1024)

    if len(msg) == 158:
    record the data
    elif len(msg) == (34): # length of request for ping
    ping the server
    else:
    yada yada.

    each real message ( according to their docs ) should be exactly 158
    bytes.

    i know i need to look more into all of this.. but thanks for all of
    your help

    -shawn
    , Apr 24, 2006
    #11
  12. Roy Smith Guest

    <> wrote:
    >elif len(msg) == (34): # length of request for ping
    > ping the server


    This seems really dangerous. You are obviously writing to some
    already defined (and hopefully, documented) protocol. What does the
    protocol spec say about ping request messages?

    I would be very surprised if the spec says that a ping request is
    defined as any message which is 34 bytes long. More likely, it says
    something like, "A ping request is defined by the command code being
    set to 5" or something like that. That's what you should be testing,
    not the length of the message.
    Roy Smith, Apr 24, 2006
    #12
  13. Rene Pijlman Guest

    :
    >i have discovered that the server will send a request for the heartbeat
    >ping if its almost timed out, so i use the length of the message to
    >determine what to do with it.
    >
    >msg = sockobj.recv(1024)
    >
    >if len(msg) == 158:
    > record the data
    >elif len(msg) == (34): # length of request for ping
    > ping the server


    Incorrect. There are no 'messages' with TCP, only bytes.

    --
    René Pijlman
    Rene Pijlman, Apr 24, 2006
    #13
  14. Guest

    ok, every message starts with "ENX" and ends with "STX"
    in between are several parts. the first is the message length sent as
    an unsigned long int (according to the docs) this is four bytes. The
    next is the message type - another 4 bytes that corrospond to a certain
    chart. for example, the login is message type 200. Then there is the
    message. different messages have different lengths and types.
    the data part, i am still trying to break apart. Some of it (when
    printed to the screen) is a string, then some very funny characters,
    then another string, then more funny looking characters, then 'ETX'

    up untill now, all i have had to process is the strings, so i split the
    message up, and used the strings as my data. But now, i do need the
    other info. -whew.

    i guess i am learning a lot. i just ordered Python Essential Reference
    from amazon. ;)

    anyway. from what i have found on line, i am going to be using the
    struct module.



    oh, that was a typo earlier. not 34 bytes, but 14.
    the data comming in is alway in 158 bytes though.

    thanks guys.
    , Apr 25, 2006
    #14
  15. Ben Sizer Guest

    wrote:
    > the data comming in is alway in 158 bytes though.


    And one day it may not. :) Consider yourself warned! (In a friendly
    manner.)

    --
    Ben Sizer
    Ben Sizer, Apr 25, 2006
    #15
  16. Guest

    yeah, he he
    , Apr 25, 2006
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Luiz Alberto Gibson da Costa

    web chat development, http header connection: keep-alive

    Luiz Alberto Gibson da Costa, Aug 6, 2003, in forum: ASP .Net
    Replies:
    0
    Views:
    597
    Luiz Alberto Gibson da Costa
    Aug 6, 2003
  2. twscott
    Replies:
    1
    Views:
    5,028
  3. peter brugel

    IIS needs Connection:Keep-Alive??

    peter brugel, Sep 21, 2003, in forum: HTML
    Replies:
    0
    Views:
    2,907
    peter brugel
    Sep 21, 2003
  4. Leon
    Replies:
    0
    Views:
    334
  5. hisan
    Replies:
    1
    Views:
    1,301
    Dan Stromberg
    Jun 25, 2012
Loading...

Share This Page