fork, threads and proper closing

Discussion in 'Python' started by Tomasz Pajor, Jun 28, 2009.

  1. Tomasz Pajor

    Tomasz Pajor Guest

    Hello,

    Configuration is as follows.

    I have a starter process which creates 3 sub processes (forks) and each
    of this processes creates a number of threads.
    Threads in that processes have semaphore so on KeyboardInterrupt without
    sending a sigterm to the subprocess i'm not able to close threads.
    Is there any work around? Can I somehow run join for the thread on
    keyboard interrupt?

    --
    Best regards
    Tomasz Pajor
    Tomasz Pajor, Jun 28, 2009
    #1
    1. Advertising

  2. Tomasz Pajor

    OdarR Guest

    On 28 juin, 23:26, Tomasz Pajor <> wrote:
    > Hello,
    >
    > Configuration is as follows.
    >
    > I have a starter process which creates 3 sub processes (forks) and each
    > of this processes creates a number of threads.
    > Threads in that processes have semaphore so on KeyboardInterrupt without
    > sending a sigterm to the subprocess i'm not able to close threads.
    > Is there any work around? Can I somehow run join for the thread on
    > keyboard interrupt?


    When creating a thread you can add a Queue parameter to communicate
    with threads:
    http://docs.python.org/library/queue.html
    easy and reliable.

    give them a "poison pill" in the queue: a recognizable object placed
    on the queue that means "when you get this, stop."

    better to not rely on keyboard for thread stopping.

    Olivier
    OdarR, Jun 29, 2009
    #2
    1. Advertising

  3. Tomasz Pajor

    Tomasz Pajor Guest


    > On 28 juin, 23:26, Tomasz Pajor <> wrote:
    >
    >> Hello,
    >>
    >> Configuration is as follows.
    >>
    >> I have a starter process which creates 3 sub processes (forks) and each
    >> of this processes creates a number of threads.
    >> Threads in that processes have semaphore so on KeyboardInterrupt without
    >> sending a sigterm to the subprocess i'm not able to close threads.
    >> Is there any work around? Can I somehow run join for the thread on
    >> keyboard interrupt?
    >>

    >
    > When creating a thread you can add a Queue parameter to communicate
    > with threads:
    > http://docs.python.org/library/queue.html
    > easy and reliable.
    >
    > give them a "poison pill" in the queue: a recognizable object placed
    > on the queue that means "when you get this, stop."
    >

    can You provide any working example?
    > better to not rely on keyboard for thread stopping.
    >

    i use keyboard interrupt only for debuging purposes
    > Olivier
    >
    Tomasz Pajor, Jun 29, 2009
    #3
  4. On 29 Giu, 07:10, OdarR <> wrote:
    > On 28 juin, 23:26, Tomasz Pajor <> wrote:
    >
    > > Hello,

    >
    > > Configuration is as follows.

    >
    > > I have a starter process which creates 3 sub processes (forks) and each
    > > of this processes creates a number of threads.
    > > Threads in that processes have semaphore so on KeyboardInterrupt without
    > > sending a sigterm to the subprocess i'm not able to close threads.
    > > Is there any work around? Can I somehow run join for the thread on
    > > keyboard interrupt?

    >
    > When creating a thread you can add a Queue parameter to communicate
    > with threads:http://docs.python.org/library/queue.html
    > easy and reliable.
    >
    > give them a "poison pill" in the queue: a recognizable object placed
    > on the queue that means "when you get this, stop."
    >


    This is the way I usually go, but it has one important limitation: if
    the thread is waiting
    for a blocking I/O operation to complete, like reading from a socket
    with no data or waiting
    for a locked resource (i.e. semaphore) to be unlocked, it will not
    service the queue and will
    not read the 'quit command' (the poison pill), and therefore will not
    quit until the blocking
    I/O terminates (and it could be never).

    ASAIK, there is no way - in python - to solve this.



    > Olivier


    Ciao
    ----
    FB
    Francesco Bochicchio, Jun 29, 2009
    #4
  5. Tomasz Pajor

    OdarR Guest

    On 29 juin, 14:44, Francesco Bochicchio <> wrote:
    > On 29 Giu, 07:10, OdarR <> wrote:
    >
    >
    >
    > > On 28 juin, 23:26, Tomasz Pajor <> wrote:

    >
    > > > Hello,

    >
    > > > Configuration is as follows.

    >
    > > > I have a starter process which creates 3 sub processes (forks) and each
    > > > of this processes creates a number of threads.
    > > > Threads in that processes have semaphore so on KeyboardInterrupt without
    > > > sending a sigterm to the subprocess i'm not able to close threads.
    > > > Is there any work around? Can I somehow run join for the thread on
    > > > keyboard interrupt?

    >
    > > When creating a thread you can add a Queue parameter to communicate
    > > with threads:http://docs.python.org/library/queue.html
    > > easy and reliable.

    >
    > > give them a "poison pill" in the queue: a recognizable object placed
    > > on the queue that means "when you get this, stop."

    >
    > This is the way I usually go, but it has one important limitation: if
    > the thread is waiting
    > for a blocking I/O operation to complete, like reading from a socket
    > with no data or waiting


    add a small wait (time.sleep()), and also, I/O function in Python can
    often releas the GIL...

    > for a locked resource (i.e. semaphore) to be unlocked, it will not
    > service the queue and will
    > not read the 'quit command' (the poison pill), and therefore will not
    > quit until the blocking
    > I/O terminates (and it could be never).
    >
    > ASAIK, there is no way - in python - to solve this.


    no easy way, yes...
    but I think I gave a good advice to our friend Tomasz :)

    Olivier
    OdarR, Jun 29, 2009
    #5
  6. In message <cf35773f-600b-44a0-9a5b-
    >, Francesco Bochicchio wrote:

    > ... if the thread is waiting for a blocking I/O operation to complete,
    > like reading from a socket with no data or waiting for a locked resource
    > (i.e. semaphore) to be unlocked, it will not service the queue and will
    > not read the 'quit command' (the poison pill), and therefore will not
    > quit until the blocking I/O terminates (and it could be never).


    Under Linux systems, threads are little different from processes. In
    particular, sending a thread/process a signal will cause any blocking I/O
    operation in progress to abort, either with only some bytes read/written, or
    if nothing was read/written, with an EINTR error.
    Lawrence D'Oliveiro, Jun 30, 2009
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?ISO-8859-1?Q?Andreas_M=FCller?=

    fork or threads

    =?ISO-8859-1?Q?Andreas_M=FCller?=, May 10, 2005, in forum: C++
    Replies:
    3
    Views:
    6,393
    Kanenas
    May 18, 2005
  2. Ajay Bakhshi
    Replies:
    0
    Views:
    404
    Ajay Bakhshi
    May 3, 2004
  3. =?Utf-8?B?Vk1J?=

    Closing popup window when closing parent window?

    =?Utf-8?B?Vk1J?=, Feb 14, 2007, in forum: ASP .Net
    Replies:
    3
    Views:
    649
    Thomas Hansen
    Feb 15, 2007
  4. Eric Snow

    os.fork and pty.fork

    Eric Snow, Jan 8, 2009, in forum: Python
    Replies:
    0
    Views:
    569
    Eric Snow
    Jan 8, 2009
  5. thomas
    Replies:
    0
    Views:
    225
    thomas
    Oct 23, 2003
Loading...

Share This Page