help with concurrency control (threads/processes & signals)

Discussion in 'Python' started by Sori Schwimmer, Oct 27, 2005.

  1. Hi,

    I am working on an application which involves
    interprocess communication. More to the point,
    processes should be able to notify other processes
    about certain situations, so the "notifyees" would be
    able to act in certain ways.

    As processes are distributed on several machines, in
    different physical locations, my thinking was:
    a) set a message manager (MM)
    b) all the participants will register with MM, so MM
    will have their host address and their pid on host
    c) when someone needs to send a notification, it is
    sent to MM, and MM it's doing the job

    To start with, I created processes, psrv.py and
    clnt.py, and tried to sketch the above scenario.

    Here is the server, psrv.py:


    #! /usr/bin/python

    from socket import *
    from os import *
    from signal import *
    from cPickle import *
    #import pdb
    import thread

    #pdb.set_trace()

    def process(ms):
    global c,regtab,mssgs
    print 'ms[0]=',ms[0]
    if ms[0]=='0':
    regtab.append(ms[1])
    mssgs.append([])
    elif ms[0]=='1':
    for i in range(len(regtab)):
    mssgs.append(ms[1])
    kill(regtab,SIGUSR1)
    elif ms[0]=='2':
    for i in range(len(regtab)):
    if regtab!=ms[2]:
    mssgs.append(ms[1])
    kill(regtab,SIGUSR1)
    elif ms[0]=='3':
    print 'executing ms[0]==3'
    for i in range(len(regtab)):
    if regtab==ms[1]:
    c,p=s.accept()
    for j in mssgs:
    c.send(dumps(j,True))
    c.close()
    print 'connection closed'

    regtab=[]
    mssgs=[]

    s=socket()
    s.bind(('127.0.0.1',9691))
    s.listen(5)

    while True:
    print 'listening ...'
    c,p=s.accept()
    while True:
    m=c.recv(1024)
    if not m: break
    ms=loads(m)
    c.close()

    thread.start_new_thread(process,(ms,))
    # ***************************************************

    And here is the client side, clnt.py:


    #! /usr/bin/python

    from socket import *
    from os import *
    from signal import *
    from cPickle import *
    #import pdb

    #pdb.set_trace()

    def k3(a,b):
    print 'Executing k3'
    s=socket()
    s.connect(('127.0.0.1',9691))
    s.send(dumps(('3',getpid()),True))
    while True:
    print '"3" sent'
    m1=s.recv(1024)
    print m1
    if not m1: break
    m=loads(m1)
    s.close()
    print 'connection closed in k3'
    print 'm is ',m

    signal(SIGUSR1,k3)

    m=''
    k=raw_input()
    while k!='4':
    s=socket()
    s.connect(('127.0.0.1',9691))
    if k=='0':
    s.send(dumps(('0',getpid()),True))
    elif k=='1':
    s.send(dumps(('1','For all'),True))
    elif k=='2':
    s.send(dumps(('2','For some',getpid()),True))
    elif k=='3':
    s.send(dumps(('3',getpid()),True))
    while True:
    m1=s.recv(1024)
    if not m1: break
    m=loads(m1)
    s.close()
    print 'connection closed in main'
    print 'm is',m
    k=raw_input()
    # ***************************************************


    Comments: Clients register themselves with server by
    sending a message of type '0'. If they want to
    broadcast a message, clients will ask nicely the
    server to do it for them through a message of type
    '1'. The message is supposed to be kept in
    eachclient's "mailbox", then the server should notify
    each client by sending a SIGUSR1 signal, and then
    waiting for each clientto ask for its own copy of the
    notification. When a client is ready to do it, will
    just send a message of type '3' to the server.
    Messages of type '2' are just a variation of type '1'
    which dosen't include the sender in the list of
    receivers.

    My first version was forking a new process in the
    server to serve requests, but I soon realized that, as
    each process receives a copy of the original data,
    updating the "mailboxes" won't work (the "mailboxes"
    updated are local copies). In the threads version the
    "mailboxes" are shared, but so are the sockets, which
    will force me to add some locks and make sure that an
    open socket in the server will talk only with a
    certain client.

    For the moment I am toying with one server, and one
    client. Still, it is not working as expected. '0' goes
    through, so does'1', but not '3' ('3' as a result of
    '1', or '3' requested from keyboard).

    Life is a struggle. Programming in Python shouldn't
    be. Ergo, I'm doing something wrong.

    Any advice?

    Thanks,
    Sorin

    Environment: Gentoo Linux, Python 2.4.1



    __________________________________
    Yahoo! FareChase: Search multiple travel sites in one click.
    http://farechase.yahoo.com
    Sori Schwimmer, Oct 27, 2005
    #1
    1. Advertising

  2. Sori Schwimmer

    Alan Kennedy Guest

    [Sori Schwimmer]
    > I am working on an application which involves
    > interprocess communication. More to the point,
    > processes should be able to notify other processes
    > about certain situations, so the "notifyees" would be
    > able to act in certain ways.
    >
    > As processes are distributed on several machines, in
    > different physical locations, my thinking was:
    > a) set a message manager (MM)
    > b) all the participants will register with MM, so MM
    > will have their host address and their pid on host
    > c) when someone needs to send a notification, it is
    > sent to MM, and MM it's doing the job


    [snip]

    > Life is a struggle. Programming in Python shouldn't
    > be. Ergo, I'm doing something wrong.
    >
    > Any advice?


    Rather than rolling your own, have you considered using the spread
    module: robust, tested, efficient and no infrastructure development
    required.

    http://www.zope.org/Members/tim_one/spread/
    http://www.python.org/other/spread/

    The latter page has links to the original C spread module, which has
    documentation, FAQs, etc.

    --
    alan kennedy
    ------------------------------------------------------
    email alan: http://xhaus.com/contact/alan
    Alan Kennedy, Oct 27, 2005
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. default
    Replies:
    3
    Views:
    455
    Shaomin
    Oct 24, 2003
  2. Replies:
    1
    Views:
    500
    Gordon Beaton
    Feb 19, 2004
  3. Jeff Rodriguez
    Replies:
    23
    Views:
    1,116
    David Schwartz
    Dec 9, 2003
  4. Srikanth Mandava
    Replies:
    1
    Views:
    394
    Michael Hudson
    Feb 19, 2004
  5. Mário Lopes

    Traping signals on child processes

    Mário Lopes, Jun 2, 2008, in forum: Ruby
    Replies:
    12
    Views:
    287
    ara.t.howard
    Jun 3, 2008
Loading...

Share This Page