Re: Passing functions as parameter (multiprocessing)

Discussion in 'Python' started by Oscar Benjamin, Nov 13, 2012.

  1. On 13 November 2012 12:51, Peter Otten <> wrote:
    > Jean-Michel Pichavant wrote:
    >
    >> I'm having problems understanding an issue with passing function as
    >> parameters.

    >
    >> Here's a code that triggers the issue:
    >>
    >>
    >> import multiprocessing
    >>
    >> def f1():
    >> print 'I am f1'
    >> def f2(foo):
    >> print 'I am f2 %s' % foo
    >>
    >> workers = [
    >> (f1,tuple()),
    >> (f2,(5,)),
    >> ]
    >>
    >> procs=[]
    >> for func, parameters in workers:

    >
    >> def subproc(*args, **kwargs):
    >> return func(*args, **kwargs)
    >> procs.append(multiprocessing.Process(target=subproc, args=parameters))


    I don't know if this is to do with the way that the code was
    simplified before posting but this subproc function wrapper does
    nothing (even after Peter fixed it below). This code is needlessly
    complicated for what it does.

    > Python has late binding, and when the loop has finished the name func is
    > bound to f2. You have created multiple subproc functions, but that doesn't
    > matter as they all invoke func aka f2.
    >
    > A possible fix:
    >
    > def make_subproc(func):
    > def subproc(*args, **kwargs):
    > return func(*args, **kwargs)
    > return subproc
    >
    > procs=[]
    > for func, parameters in workers:
    > procs.append(multiprocessing.Process(target=make_subproc(func),
    > args=parameters))


    The above is one way to wrap a function but it wraps the function into
    a function that is exactly the same as the original function. I assume
    that the reason for having the subproc function was to be able to bind
    parameters to the function like so:

    def make_subproc(func, args):
    def subproc():
    return func(*args)
    return subproc

    procs=[]
    for func, parameters in workers:
    procs.append(multiprocessing.Process(target=make_subproc(func, parameters)))

    However, as you have already noticed, multiprocess.Process already has
    the machinery to do this. If you pass in a value for args you can
    leave out the subproc/make_subproc function and write:

    procs=[]
    for func, parameters in workers:
    procs.append(multiprocessing.Process(target=func, args=parameters))

    A loop like this can just be a list comprehension:

    procs = [multiprocessing.Process(target=func, args=args) for func,
    args in workers]


    Oscar
     
    Oscar Benjamin, Nov 13, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Metalone
    Replies:
    0
    Views:
    366
    Metalone
    Jan 6, 2010
  2. Mister B
    Replies:
    8
    Views:
    604
    Nick Keighley
    Aug 26, 2010
  3. MRAB
    Replies:
    0
    Views:
    211
  4. Peter Otten
    Replies:
    0
    Views:
    163
    Peter Otten
    Nov 13, 2012
  5. Peter Otten
    Replies:
    0
    Views:
    206
    Peter Otten
    Nov 13, 2012
Loading...

Share This Page