Function application optimization.

Discussion in 'Python' started by Jacek Generowicz, Dec 12, 2003.

  1. Given

    fncs = [func1, func2, ..., funcN]
    args = [arg1, arg2, ..., argN]

    How should one spell

    results = map(lambda f,a: f(a), fncs, args)

    in order to get the result most quickly ?



    Unfortunately "apply" takes a tuple of arguments, and there is no
    "funcall"[*] in Python.


    [*] def funcall(fn, *args):
    return fn(*args)
     
    Jacek Generowicz, Dec 12, 2003
    #1
    1. Advertising

  2. On 12 Dec 2003 10:47:50 +0100, Jacek Generowicz <> wrote:

    >Given
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    >
    >How should one spell
    >
    > results = map(lambda f,a: f(a), fncs, args)
    >
    >in order to get the result most quickly ?
    >
    >
    >
    >Unfortunately "apply" takes a tuple of arguments, and there is no
    >"funcall"[*] in Python.
    >
    >
    >[*] def funcall(fn, *args):
    > return fn(*args)
    >


    >>> fncs = [lambda x,f='func_%s(%%s)'%i:f%x for i in xrange(4)]
    >>> args = 'zero one two three'.split()
    >>> map(lambda f,a: f(a), fncs, args)

    ['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

    I'd probably try a list comprehension

    >>> [f(a) for f,a in zip(fncs,args)]

    ['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']

    Regards,
    Bengt Richter
     
    Bengt Richter, Dec 12, 2003
    #2
    1. Advertising

  3. Jacek Generowicz

    Duncan Booth Guest

    Jacek Generowicz <> wrote in
    news::

    > Given
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    >
    > How should one spell
    >
    > results = map(lambda f,a: f(a), fncs, args)
    >
    > in order to get the result most quickly ?
    >

    Well, the way you wrote it isn't actually bad. The obvious alternative
    (using a list comprehension) is slightly slower, but there isn't an awful
    lot to choos between any of them, I suspect most of the time goes in the
    actual f(a) function call. Any of these methods runs at about 300,000
    function calls/second on my slow laptop.

    >>> setup = """import itertools

    def fn1(a): pass
    fns = [fn1] * 100
    args = [0] * 100
    """
    >>> stmt1 = "result = [ f(a) for (f,a) in itertools.izip(fns, args) ]"
    >>> stmt2 = "result = [ f(a) for (f,a) in zip(fns, args) ]"
    >>> t = timeit.Timer(stmt1, setup)
    >>> t.repeat(3,10000)

    [3.5571303056673855, 3.5537404893639177, 3.5594278043718077]
    >>> t = timeit.Timer(stmt2, setup)
    >>> t.repeat(3,10000)

    [3.893281967400867, 3.87834794645687, 3.8829105375124868]
    >>> setup = """import itertools

    def fn1(a): pass
    fns = [fn1] * 1000
    args = [0] * 1000
    """
    >>> t = timeit.Timer(stmt1, setup)
    >>> t.repeat(3,1000)

    [3.3503928571927304, 3.3343195853104248, 3.3495254285111287]
    >>> t = timeit.Timer(stmt2, setup)
    >>> t.repeat(3,1000)

    [3.8062683944467608, 3.7946001516952492, 3.7881063096007779]
    >>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
    >>> t = timeit.Timer(stmt3, setup)
    >>> t.repeat(3,1000)

    [3.3275902384241363, 3.3010907810909202, 3.3174872784110789]

    The f(a) call is taking about half the time in any of these methods, so you
    aren't going to get very much improvement whatever you do to the loop.

    --
    Duncan Booth
    int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
    "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
     
    Duncan Booth, Dec 12, 2003
    #3
  4. Jacek Generowicz wrote:
    > Given
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    >
    > How should one spell
    >
    > results = map(lambda f,a: f(a), fncs, args)
    >
    > in order to get the result most quickly ?
    >


    I dont know if it will be faster (timeit might tell you this), but what
    about something as simple stupid as:

    results = []
    nbfuns = len(fncs)
    for i in range(len):
    results.append(fncs(args))

    Note that you must have at least len(fncs) args.


    HTH,
    Bruno
     
    Bruno Desthuilliers, Dec 12, 2003
    #4
  5. (Bengt Richter) writes:

    > I'd probably try a list comprehension
    >
    > >>> [f(a) for f,a in zip(fncs,args)]

    > ['func_0(zero)', 'func_1(one)', 'func_2(two)', 'func_3(three)']


    Yup, been there, done that, it's slower. (I guess I should have
    mentioned that in the original post.)
     
    Jacek Generowicz, Dec 12, 2003
    #5
  6. Jacek Generowicz

    Peter Otten Guest

    Jacek Generowicz wrote:

    > Given
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    >
    > How should one spell
    >
    > results = map(lambda f,a: f(a), fncs, args)
    >
    > in order to get the result most quickly ?
    >


    If you can afford to destroy the args list, a tiny speed gain might be in
    for you (not verified):

    for index, func in enumerate(fncs):
    args[index] = func[index]

    Peter
     
    Peter Otten, Dec 12, 2003
    #6
  7. Jacek Generowicz

    John Roth Guest

    "Jacek Generowicz" <> wrote in message
    news:...
    > Given
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    >
    > How should one spell
    >
    > results = map(lambda f,a: f(a), fncs, args)
    >
    > in order to get the result most quickly ?
    >
    >
    >
    > Unfortunately "apply" takes a tuple of arguments, and there is no
    > "funcall"[*] in Python.
    >
    >
    > [*] def funcall(fn, *args):
    > return fn(*args)



    Building on a couple of other responses:

    Untested code:

    fncs = [func1, func2, ..., funcN]
    args = [arg1, arg2, ..., argN]
    results = []
    for function, arguement in zip(fncs, args):
    results.append(function(arguement))

    Notice the use of zip() to put the two lists together.
    I haven't timed it, but removing an extra layer of
    function call has got to be faster. Likewise, zip
    and tuple unpacking is most likely going to be
    faster than indexing every time through the loop.
    The append, on the other hand, might slow things
    down a bit.

    John Roth

    >
     
    John Roth, Dec 12, 2003
    #7
  8. Jacek Generowicz

    Duncan Booth Guest

    "John Roth" <> wrote in
    news::

    > Building on a couple of other responses:
    >
    > Untested code:
    >
    > fncs = [func1, func2, ..., funcN]
    > args = [arg1, arg2, ..., argN]
    > results = []
    > for function, arguement in zip(fncs, args):
    > results.append(function(arguement))
    >
    > Notice the use of zip() to put the two lists together.
    > I haven't timed it, but removing an extra layer of
    > function call has got to be faster. Likewise, zip
    > and tuple unpacking is most likely going to be
    > faster than indexing every time through the loop.
    > The append, on the other hand, might slow things
    > down a bit.


    Yes, getting rid of the append does indeed speed things up. On the same
    system as I posted timings for the list comprehension, the fastest way I've
    found so far is to get rid of the appends by preallocating the list, and to
    go for the plain old simple technique of writing a for loop out explicitly.
    Using 'enumerate' to avoid the lookup on one of the input lists is nearly
    as fast, but not quite.

    >>> setup = """import itertools

    def fn1(a): pass
    fns = [fn1] * 1000
    args = [0] * 1000
    """

    >>> stmt1 = """result = args[:]

    for i in range(len(fns)):
    result = fns(args)
    """
    >>> min(timeit.Timer(stmt1, setup).repeat(3,1000))

    2.9747384094916924
    >>> stmt3 = "results = map(lambda f,a: f(a), fns, args)"
    >>> min(timeit.Timer(stmt3, setup).repeat(3,1000))

    3.3257092731055309
    >>>


    --
    Duncan Booth
    int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
    "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
     
    Duncan Booth, Dec 12, 2003
    #8
  9. Jacek Generowicz

    Peter Otten Guest

    Peter Otten wrote:

    > for index, func in enumerate(fncs):
    > args[index] = func[index]


    Oops,
    args[index] = func(args[index])

    And it's much slower than result.append(func(args[index]), too :-(

    Peter
     
    Peter Otten, Dec 12, 2003
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Denis Remezov
    Replies:
    20
    Views:
    725
    Jakob Bieling
    Apr 20, 2004
  2. pt
    Replies:
    8
    Views:
    780
  3. Ravikiran

    Zero Optimization and Sign Optimization???

    Ravikiran, Nov 17, 2008, in forum: C Programming
    Replies:
    22
    Views:
    869
    Thad Smith
    Nov 24, 2008
  4. Anthony Goubard

    ann: New website optimization application

    Anthony Goubard, Jan 16, 2009, in forum: HTML
    Replies:
    0
    Views:
    421
    Anthony Goubard
    Jan 16, 2009
  5. Anthony Goubard
    Replies:
    2
    Views:
    88
    Thomas 'PointedEars' Lahn
    Jan 16, 2009
Loading...

Share This Page