# Re: list comprehension question

Discussion in 'Python' started by Dave Angel, Oct 17, 2012.

1. ### Dave AngelGuest

On 10/17/2012 12:43 AM, Kevin Anthony wrote:
> Is it not true that list comprehension is much faster the the for loops?
>
> If it is not the correct way of doing this, i appoligize.
> Like i said, I'm learing list comprehension.
>

(Please don't top-post; it ruins the ordering. In these forums, put
your response after the part you quote from earlier messages. Or even
better, after each part you quote. Then trim off the parts you didn't
reference.)

list comprehensions CAN be much faster, but not necessarily. The most
complex a loop, the less likely it'll help much.

In any case, only the inner loop will be affected. Nesting two list
comprehensions will make a trivial difference.

On the other hand, Hans Mulder shows some other factoring which seems

Studying (and testing) those could teach you a lot about comprehensions,
as well as about the libraries that can help. Note especially what
zip(*b) yields, and think about what it means.

--

DaveA

Dave Angel, Oct 17, 2012

2. ### rusiGuest

On Oct 17, 5:33 pm, Dave Angel <> wrote:
> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
>
> > If it is not the correct way of doing this, i appoligize.
> > Like i said, I'm learing list comprehension.

> list comprehensions CAN be much faster, but not necessarily.  The most
> complex a loop, the less likely it'll help much.

One-lining the comprehension seems to make a difference of about 10%
out here. Maybe Ive missed something? Seems too large…

# My original suggestion
def dot(p,q): return sum (x*y for x,y in zip(p,q))
def transpose(m): return zip(*m)
def mm(a,b): return mmt(a, transpose(b))
def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

# One-liner (Thanks Hans for reminding me of sum)

def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]

>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
>>> t1.timeit(1000)

12.276363849639893
>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
>>> t0.timeit(1000)

13.453603029251099

rusi, Oct 17, 2012

3. ### rusiGuest

On Oct 17, 7:06 pm, rusi <> wrote:
> On Oct 17, 5:33 pm, Dave Angel <> wrote:
>
> > On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?

>
> > > If it is not the correct way of doing this, i appoligize.
> > > Like i said, I'm learing list comprehension.

> > list comprehensions CAN be much faster, but not necessarily.  The most
> > complex a loop, the less likely it'll help much.

>
> One-lining the comprehension seems to make a difference of about 10%
> out here. Maybe Ive missed something? Seems too large…
>
> # My original suggestion
> def dot(p,q): return sum (x*y for x,y in zip(p,q))
> def transpose(m): return zip(*m)
> def mm(a,b): return mmt(a, transpose(b))
> def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
>
> # One-liner (Thanks Hans for reminding me of sum)
>
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> zip(*b)] for ra in a]
>
> >>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
> >>> t1.timeit(1000)

> 12.276363849639893
> >>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
> >>> t0.timeit(1000)

>
> 13.453603029251099

In case anyone wants to try out with the same data, I used:

m = [range(i,i+30) for i in range(30)]

rusi, Oct 17, 2012
4. ### Dave AngelGuest

On 10/17/2012 10:06 AM, rusi wrote:
> On Oct 17, 5:33 pm, Dave Angel <> wrote:
>> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that list comprehension is much faster the the for loops?
>>
>>> If it is not the correct way of doing this, i appoligize.
>>> Like i said, I'm learing list comprehension.

>> list comprehensions CAN be much faster, but not necessarily. The most
>> complex a loop, the less likely it'll help much.

> One-lining the comprehension seems to make a difference of about 10%
> out here. Maybe Ive missed something? Seems too large…
>
> # My original suggestion
> def dot(p,q): return sum (x*y for x,y in zip(p,q))
> def transpose(m): return zip(*m)
> def mm(a,b): return mmt(a, transpose(b))
> def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]
>
> # One-liner (Thanks Hans for reminding me of sum)
>
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
> zip(*b)] for ra in a]
>
>>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")
>>>> t1.timeit(1000)

> 12.276363849639893
>>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")
>>>> t0.timeit(1000)

> 13.453603029251099

And I'd wager all the improvement is in the inner loop, the dot() function.

--

DaveA

Dave Angel, Oct 17, 2012
5. ### 88888 DihedralGuest

Dave Angelæ–¼ 2012å¹´10æœˆ17æ—¥æ˜ŸæœŸä¸‰UTC+8ä¸‹åˆ10æ™‚37åˆ†01ç§’å¯«é“ï¼š
> On 10/17/2012 10:06 AM, rusi wrote:
>
> > On Oct 17, 5:33 pm, Dave Angel <> wrote:

>
> >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that listcomprehension is much faster the the for loops?

>
> >>

>
> >>> If it is not the correct way of doing this, i appoligize.

>
> >>> Like i said, I'm learing list comprehension.

>
> >> list comprehensions CAN be much faster, but not necessarily. The most

>
> >> complex a loop, the less likely it'll help much.

>
> > One-lining the comprehension seems to make a difference of about 10%

>
> > out here. Maybe Ive missed something? Seems too largeÃ¯Â¿Â½

>
> >

>
> > # My original suggestion

>
> > def dot(p,q): return sum (x*y for x,y in zip(p,q))

>
> > def transpose(m): return zip(*m)

>
> > def mm(a,b): return mmt(a, transpose(b))

>
> > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

>
> >

>
> > # One-liner (Thanks Hans for reminding me of sum)

>
> >

>
> > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in

>
> > zip(*b)] for ra in a]

>
> >

>
> >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")

>
> >>>> t1.timeit(1000)

>
> > 12.276363849639893

>
> >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")

>
> >>>> t0.timeit(1000)

>
> > 13.453603029251099

>
>
>
> And I'd wager all the improvement is in the inner loop, the dot() function.
>
>
>
>
>
> --
>
>
>
> DaveA

Thanks for the tips of matrix operations over some fields or rings
other than the real field and the complex field.

88888 Dihedral, Oct 17, 2012
6. ### 88888 DihedralGuest

Dave Angelæ–¼ 2012å¹´10æœˆ17æ—¥æ˜ŸæœŸä¸‰UTC+8ä¸‹åˆ10æ™‚37åˆ†01ç§’å¯«é“ï¼š
> On 10/17/2012 10:06 AM, rusi wrote:
>
> > On Oct 17, 5:33 pm, Dave Angel <> wrote:

>
> >> On 10/17/2012 12:43 AM, Kevin Anthony wrote:> Is it not true that listcomprehension is much faster the the for loops?

>
> >>

>
> >>> If it is not the correct way of doing this, i appoligize.

>
> >>> Like i said, I'm learing list comprehension.

>
> >> list comprehensions CAN be much faster, but not necessarily. The most

>
> >> complex a loop, the less likely it'll help much.

>
> > One-lining the comprehension seems to make a difference of about 10%

>
> > out here. Maybe Ive missed something? Seems too largeÃ¯Â¿Â½

>
> >

>
> > # My original suggestion

>
> > def dot(p,q): return sum (x*y for x,y in zip(p,q))

>
> > def transpose(m): return zip(*m)

>
> > def mm(a,b): return mmt(a, transpose(b))

>
> > def mmt(a,b): return [[dot(ra, rb) for rb in b] for ra in a]

>
> >

>
> > # One-liner (Thanks Hans for reminding me of sum)

>
> >

>
> > def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in

>
> > zip(*b)] for ra in a]

>
> >

>
> >>>> t1=Timer("res=mm1(m,m)", setup="from __main__ import mm1, m")

>
> >>>> t1.timeit(1000)

>
> > 12.276363849639893

>
> >>>> t0=Timer("res=mm(m,m)", setup="from __main__ import mm, m")

>
> >>>> t0.timeit(1000)

>
> > 13.453603029251099

>
>
>
> And I'd wager all the improvement is in the inner loop, the dot() function.
>
>
>
>
>
> --
>
>
>
> DaveA

Thanks for the tips of matrix operations over some fields or rings
other than the real field and the complex field.

88888 Dihedral, Oct 17, 2012
7. ### rusiGuest

On Oct 17, 7:37 pm, Dave Angel <> wrote:

> And I'd wager all the improvement is in the inner loop, the dot() function.

Sorry -- red herring!

Changing

def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
zip(*b)] for ra in a]

to

def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
zip(*b)] for ra in a]

makes the speed diff vanish

rusi, Oct 17, 2012
8. ### 88888 DihedralGuest

rusiæ–¼ 2012å¹´10æœˆ17æ—¥æ˜ŸæœŸä¸‰UTC+8ä¸‹åˆ10æ™‚50åˆ†11ç§’å¯«é“ï¼š
> On Oct 17, 7:37Â pm, Dave Angel <> wrote:
>
>
>
> > And I'd wager all the improvement is in the inner loop, the dot() function.

>
>
>
> Sorry -- red herring!
>
>
>
> Changing
>
>
>
> def mm1(a,b): return [[sum(x*y for x,y in zip(ra,rb)) for rb in
>
> zip(*b)] for ra in a]
>
>
>
> to
>
>
>
> def mm1(a,b): return [[sum([x*y for x,y in zip(ra,rb)]) for rb in
>
> zip(*b)] for ra in a]
>
>
>
> makes the speed diff vanish

I think a lot people don't work on computations over
fields other real and complex.

That is why a lot people keep complaining about the speeds
of python programs executed.

88888 Dihedral, Oct 17, 2012