Re: list comprehension question

Discussion in 'Python' started by Dave Angel, Oct 17, 2012.

  1. Dave Angel

    Dave Angel Guest

    On 10/17/2012 12:43 AM, Kevin Anthony wrote:
    > Is it not true that list comprehension is much faster the the for loops?
    >
    > If it is not the correct way of doing this, i appoligize.
    > Like i said, I'm learing list comprehension.
    >

    (Please don't top-post; it ruins the ordering. In these forums, put
    your response after the part you quote from earlier messages. Or even
    better, after each part you quote. Then trim off the parts you didn't
    reference.)

    list comprehensions CAN be much faster, but not necessarily. The most
    complex a loop, the less likely it'll help much.

    In any case, only the inner loop will be affected. Nesting two list
    comprehensions will make a trivial difference.

    On the other hand, Hans Mulder shows some other factoring which seems
    much more readable than yours.

    Studying (and testing) those could teach you a lot about comprehensions,
    as well as about the libraries that can help. Note especially what
    zip(*b) yields, and think about what it means.


    --

    DaveA
     
    Dave Angel, Oct 17, 2012
    #1
    1. Advertising

  2. Dave Angel

    rusi Guest

    On Oct 17, 5:33 pm, Dave Angel <> wrote:
    > On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
    >
    > > If it is not the correct way of doing this, i appoligize.
    > > Like i said, I'm learing list comprehension.


    > list comprehensions CAN be much faster, but not necessarily.  The most
    > complex a loop, the less likely it'll help much.


    One-lining the comprehension seems to make a difference of about 10%
    out here. Maybe Ive missed something? Seems too large…

    # My original suggestion
    def dot(p,q): return sum (x*y for x,y in zip(p,q))
    def transpose(m): return zip(*m)
    def mm(a,b): return mmt(a, transpose(b))
    def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

    # One-liner (Thanks Hans for reminding me of sum)

    def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
    zip(*b)] for ra in a]

    >>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
    >>> t1.timeit(1000)

    12.276363849639893
    >>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
    >>> t0.timeit(1000)

    13.453603029251099
     
    rusi, Oct 17, 2012
    #2
    1. Advertising

  3. Dave Angel

    rusi Guest

    On Oct 17, 7:06 pm, rusi <> wrote:
    > On Oct 17, 5:33 pm, Dave Angel <> wrote:
    >
    > > On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?

    >
    > > > If it is not the correct way of doing this, i appoligize.
    > > > Like i said, I'm learing list comprehension.

    > > list comprehensions CAN be much faster, but not necessarily.  The most
    > > complex a loop, the less likely it'll help much.

    >
    > One-lining the comprehension seems to make a difference of about 10%
    > out here. Maybe Ive missed something? Seems too large…
    >
    > # My original suggestion
    > def dot(p,q): return sum (x*y for x,y in zip(p,q))
    > def transpose(m): return zip(*m)
    > def mm(a,b): return mmt(a, transpose(b))
    > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
    >
    > # One-liner (Thanks Hans for reminding me of sum)
    >
    > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
    > zip(*b)] for ra in a]
    >
    > >>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
    > >>> t1.timeit(1000)

    > 12.276363849639893
    > >>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
    > >>> t0.timeit(1000)

    >
    > 13.453603029251099


    In case anyone wants to try out with the same data, I used:

    m = [range(i,i+30) for i in range(30)]
     
    rusi, Oct 17, 2012
    #3
  4. Dave Angel

    Dave Angel Guest

    On 10/17/2012 10:06 AM, rusi wrote:
    > On Oct 17, 5:33 pm, Dave Angel <> wrote:
    >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
    >>
    >>> If it is not the correct way of doing this, i appoligize.
    >>> Like i said, I'm learing list comprehension.

    >> list comprehensions CAN be much faster, but not necessarily. The most
    >> complex a loop, the less likely it'll help much.

    > One-lining the comprehension seems to make a difference of about 10%
    > out here. Maybe Ive missed something? Seems too large…
    >
    > # My original suggestion
    > def dot(p,q): return sum (x*y for x,y in zip(p,q))
    > def transpose(m): return zip(*m)
    > def mm(a,b): return mmt(a, transpose(b))
    > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
    >
    > # One-liner (Thanks Hans for reminding me of sum)
    >
    > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
    > zip(*b)] for ra in a]
    >
    >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
    >>>> t1.timeit(1000)

    > 12.276363849639893
    >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
    >>>> t0.timeit(1000)

    > 13.453603029251099


    And I'd wager all the improvement is in the inner loop, the dot() function.


    --

    DaveA
     
    Dave Angel, Oct 17, 2012
    #4
  5. Dave Angelæ–¼ 2012å¹´10月17日星期三UTC+8下åˆ10時37分01秒寫é“:
    > On 10/17/2012 10:06 AM, rusi wrote:
    >
    > > On Oct 17, 5:33 pm, Dave Angel <> wrote:

    >
    > >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that listcomprehension is much faster the the for loops?

    >
    > >>

    >
    > >>> If it is not the correct way of doing this, i appoligize.

    >
    > >>> Like i said, I'm learing list comprehension.

    >
    > >> list comprehensions CAN be much faster, but not necessarily. The most

    >
    > >> complex a loop, the less likely it'll help much.

    >
    > > One-lining the comprehension seems to make a difference of about 10%

    >
    > > out here. Maybe Ive missed something? Seems too large�

    >
    > >

    >
    > > # My original suggestion

    >
    > > def dot(p,q): return sum (x*y for x,y in zip(p,q))

    >
    > > def transpose(m): return zip(*m)

    >
    > > def mm(a,b): return mmt(a, transpose(b))

    >
    > > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

    >
    > >

    >
    > > # One-liner (Thanks Hans for reminding me of sum)

    >
    > >

    >
    > > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in

    >
    > > zip(*b)] for ra in a]

    >
    > >

    >
    > >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")

    >
    > >>>> t1.timeit(1000)

    >
    > > 12.276363849639893

    >
    > >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")

    >
    > >>>> t0.timeit(1000)

    >
    > > 13.453603029251099

    >
    >
    >
    > And I'd wager all the improvement is in the inner loop, the dot() function.
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Thanks for the tips of matrix operations over some fields or rings
    other than the real field and the complex field.
     
    88888 Dihedral, Oct 17, 2012
    #5
  6. Dave Angelæ–¼ 2012å¹´10月17日星期三UTC+8下åˆ10時37分01秒寫é“:
    > On 10/17/2012 10:06 AM, rusi wrote:
    >
    > > On Oct 17, 5:33 pm, Dave Angel <> wrote:

    >
    > >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that listcomprehension is much faster the the for loops?

    >
    > >>

    >
    > >>> If it is not the correct way of doing this, i appoligize.

    >
    > >>> Like i said, I'm learing list comprehension.

    >
    > >> list comprehensions CAN be much faster, but not necessarily. The most

    >
    > >> complex a loop, the less likely it'll help much.

    >
    > > One-lining the comprehension seems to make a difference of about 10%

    >
    > > out here. Maybe Ive missed something? Seems too large�

    >
    > >

    >
    > > # My original suggestion

    >
    > > def dot(p,q): return sum (x*y for x,y in zip(p,q))

    >
    > > def transpose(m): return zip(*m)

    >
    > > def mm(a,b): return mmt(a, transpose(b))

    >
    > > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

    >
    > >

    >
    > > # One-liner (Thanks Hans for reminding me of sum)

    >
    > >

    >
    > > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in

    >
    > > zip(*b)] for ra in a]

    >
    > >

    >
    > >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")

    >
    > >>>> t1.timeit(1000)

    >
    > > 12.276363849639893

    >
    > >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")

    >
    > >>>> t0.timeit(1000)

    >
    > > 13.453603029251099

    >
    >
    >
    > And I'd wager all the improvement is in the inner loop, the dot() function.
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Thanks for the tips of matrix operations over some fields or rings
    other than the real field and the complex field.
     
    88888 Dihedral, Oct 17, 2012
    #6
  7. Dave Angel

    rusi Guest

    On Oct 17, 7:37 pm, Dave Angel <> wrote:

    > And I'd wager all the improvement is in the inner loop, the dot() function.


    Sorry -- red herring!

    Changing

    def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
    zip(*b)] for ra in a]

    to

    def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
    zip(*b)] for ra in a]

    makes the speed diff vanish
     
    rusi, Oct 17, 2012
    #7
  8. rusiæ–¼ 2012å¹´10月17日星期三UTC+8下åˆ10時50分11秒寫é“:
    > On Oct 17, 7:37 pm, Dave Angel <> wrote:
    >
    >
    >
    > > And I'd wager all the improvement is in the inner loop, the dot() function.

    >
    >
    >
    > Sorry -- red herring!
    >
    >
    >
    > Changing
    >
    >
    >
    > def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
    >
    > zip(*b)] for ra in a]
    >
    >
    >
    > to
    >
    >
    >
    > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
    >
    > zip(*b)] for ra in a]
    >
    >
    >
    > makes the speed diff vanish


    I think a lot people don't work on computations over
    fields other real and complex.

    That is why a lot people keep complaining about the speeds
    of python programs executed.
     
    88888 Dihedral, Oct 17, 2012
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Elston

    List Comprehension question

    Mark Elston, Dec 11, 2003, in forum: Python
    Replies:
    1
    Views:
    333
    Francis Avila
    Dec 11, 2003
  2. Eelco Hoekema

    [Newby question] List comprehension

    Eelco Hoekema, Aug 6, 2004, in forum: Python
    Replies:
    7
    Views:
    306
    Eelco Hoekema
    Aug 6, 2004
  3. Shane Geiger
    Replies:
    4
    Views:
    408
    bullockbefriending bard
    Mar 25, 2007
  4. Debajit Adhikary
    Replies:
    17
    Views:
    724
    Debajit Adhikary
    Oct 18, 2007
  5. Vedran Furac(
    Replies:
    4
    Views:
    357
    Marc 'BlackJack' Rintsch
    Dec 19, 2008
Loading...

Share This Page