Re: Let child process to run while parent is out (multiprocessing)

Discussion in 'Python' started by Kushal Kumaran, Jul 20, 2012.

  1. On Fri, Jul 20, 2012 at 2:04 AM, John Wong <> wrote:
    > def main(...):
    > build_id = create_build_id(...)
    > build_stuff
    > return build_id
    >
    > Suppose build_stuff compiles a C program. It could take days to finish, and
    > notify users their builds are ready. I was thinking about using
    > mutliprocessing to handle the build_stuff.
    >
    > So here is a sample:
    >
    > #!/usr/bin/python
    >
    > import multiprocessing as mp
    > import time
    >
    > def build():
    > print 'I am building HUGE things'
    > time.sleep(10)
    >
    > def main():
    > build_p = mp.Process(name='build process', target=build)
    > build_p.start()
    > return 'abcd12345'
    >
    > if __name__ == '__main__':
    >
    > v = main()
    > print v
    > print 'done'
    >
    > Here is output:
    > yeukhon@fermat:~$ python c2.py
    > abcd12345
    > done [now hangs for 10 seconds]
    > I build things
    >
    > When I looked at `ps -elf|grep python`, I can see two processes running, and
    > one of the python c2.py process is `wait`. But this is not ideal,
    > especially this is a web app. I can't implement any advanced queue / event
    > system right now (I use Pylon, I believe I have gevent installed). But just
    > with multiprocessing, is it possible to send the id first, while running
    > child in the backgroud?
    >
    > Right now it hangs there as long as the child process is alive. Any
    > solutions?
    >


    >From the documentation, there does not seem to be any way of

    "detaching" a multiprocessing Process. But it is doable by using the
    underlying os.fork directly (CAUTION: not ready for being invoked from
    a web app):

    #!/usr/bin/python

    import os
    import time

    def build_and_send_email():
    print 'I am building HUGE things'
    time.sleep(10)

    def main():
    child_pid = os.fork()
    if child_pid == 0:
    build_and_send_email()
    os._exit(0)
    return 'abcd12345'

    if __name__ == '__main__':
    v = main()
    print v
    print 'done'

    To make it work correctly with web app will require a bit more work.
    At the least, you will have to close all file descriptors to make sure
    the request processing finishes. You can turn it into a proper
    background process (a daemon) using the python-daemon library with
    very little code, I think.

    --
    regards,
    kushal
     
    Kushal Kumaran, Jul 20, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. rabbits77
    Replies:
    1
    Views:
    516
    John C. Bollinger
    Feb 25, 2004
  2. Noel Dolan
    Replies:
    0
    Views:
    249
    Noel Dolan
    Jul 18, 2004
  3. Zhidian Du
    Replies:
    2
    Views:
    116
    David Efflandt
    Feb 21, 2004
  4. Chris Angelico
    Replies:
    0
    Views:
    176
    Chris Angelico
    Jul 20, 2012
  5. Bitswapper
    Replies:
    5
    Views:
    162
    Prasad, Ramit
    Aug 27, 2013
Loading...

Share This Page