Re: reloading code and multiprocessing

Discussion in 'Python' started by andrea crotti, Jul 27, 2012.

  1. 2012/7/25 andrea crotti <>:
    >
    > I would also like to avoid this in general, but we have many
    > subprocesses to launch and some of them might take weeks, so we need
    > to have a process which is always running, because there is never a
    > point in time where we can just say let's stop everything and start again..
    >
    > Anyway if there are better solutions I'm still glad to hear them, but
    > I would also like to keep it simple..
    >
    > Another thing which now we need to figure out is how to communicate
    > with the live process.. For example we might want to submit something
    > manually, which should pass from the main process.
    >
    > The first idea is to have a separate process that opens a socket and
    > listens for data on a local port, with a defined protocol.
    >
    > Then the main process can parse these commands and run them.
    > Are there easier ways otherwise?



    So I was trying to do this, removing the module from sys.modules and
    starting a new process (after modifying the file), but it doesn't work
    as I expected.
    The last assertion fails, but how?

    The pyc file is not generated, the module is actually not in
    sys.modules, and the function doesn't in the subprocess doesn't fail
    but still returns the old value.
    Any idea?

    old_a = "def ret(): return 0"
    new_a = "def ret(): return 1"


    def func_no_import(queue):
    queue.put(a_glob.ret())


    class TestMultiProc(unittest.TestCase):

    def test_reloading_with_global_import(self):
    """In this case the import is done before the process are started,
    so we need to clean sys.modules to make sure we reload everything
    """
    queue = Queue()
    open(path.join(CUR_DIR, 'old_a.py'), 'w').write(old_a)

    p1 = Process(target=func_no_import, args=(queue, ))
    p1.start()
    p1.join()
    self.assertEqual(queue.get(), 0)

    open(path.join(CUR_DIR, 'old_a.py'), 'w').write(new_a)
    del sys.modules['auto_tester.tests.a_glob']

    p2 = Process(target=func_no_import, args=(queue, ))
    p2.start()
    p2.join()
    self.assertEqual(queue.get(), 1)
     
    andrea crotti, Jul 27, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. andrea crotti

    reloading code and multiprocessing

    andrea crotti, Jul 19, 2012, in forum: Python
    Replies:
    2
    Views:
    179
    88888 Dihedral
    Jul 19, 2012
  2. Chris Angelico

    Re: reloading code and multiprocessing

    Chris Angelico, Jul 20, 2012, in forum: Python
    Replies:
    0
    Views:
    124
    Chris Angelico
    Jul 20, 2012
  3. andrea crotti

    Re: reloading code and multiprocessing

    andrea crotti, Jul 23, 2012, in forum: Python
    Replies:
    0
    Views:
    152
    andrea crotti
    Jul 23, 2012
  4. Chris Angelico

    Re: reloading code and multiprocessing

    Chris Angelico, Jul 23, 2012, in forum: Python
    Replies:
    0
    Views:
    139
    Chris Angelico
    Jul 23, 2012
  5. andrea crotti

    Re: reloading code and multiprocessing

    andrea crotti, Jul 25, 2012, in forum: Python
    Replies:
    0
    Views:
    203
    andrea crotti
    Jul 25, 2012
Loading...

Share This Page