map, LC, genexp

B

Bryan

i thought that LC and genexp were supposed to be faster than map. i also thought i read somewhere in this group that
the slowest looping mechanism in 2.4 is 20% faster than the fastest looping mechanism in 2.3. if i'm wrong about this,
remembered wrong or did something wrong, please let me know. here are some timeit tests i did on my machine. notice
that map is faster in both python versions and that LC24/LC23 is only a 5.5% speed improvement.

python 2.3

timeit "for i in map(str, xrange(100)): pass"
10000 loops, best of 3: 118 usec per loop

timeit "for i in [str(x) for x in xrange(100)]: pass"
1000 loops, best of 3: 200 usec per loop

python 2.4b1

timeit "for i in map(str, xrange(100)): pass"
10000 loops, best of 3: 144 usec per loop

timeit "for i in [str(x) for x in xrange(100)]: pass"
10000 loops, best of 3: 189 usec per loop

timeit "for i in (str(x) for x in xrange(100)): pass"
10000 loops, best of 3: 185 usec per loop


thanks,

bryan
 
R

Robert Kern

Bryan said:
i thought that LC and genexp were supposed to be faster than map. i
also thought i read somewhere in this group that the slowest looping
mechanism in 2.4 is 20% faster than the fastest looping mechanism in
2.3. if i'm wrong about this, remembered wrong or did something wrong,
please let me know. here are some timeit tests i did on my machine.
notice that map is faster in both python versions and that LC24/LC23 is
only a 5.5% speed improvement.

python 2.3

timeit "for i in map(str, xrange(100)): pass"
10000 loops, best of 3: 118 usec per loop

map still beats list comprehensions and generator expressions when the
function is builtin. Try again with a Python function that gets
"inlined" in the LC and genexp:

E.g.

def neg(x):
return -x

map(neg, xrange(100))
[-x for x in xrange(100)]
(-x for x in xrange(100))

I believe (though I haven't tested this specific example) that is the
case for which they are faster than map.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
A

Alex Martelli

Bryan said:
the slowest looping mechanism in 2.4 is 20% faster than the fastest
looping mechanism in 2.3. if i'm wrong about this, remembered wrong or
did something wrong, please let me know. here are some timeit tests i did
on my machine. notice that map is faster in both python versions and that
LC24/LC23 is only a 5.5% speed improvement.

It may depend on your machine and how well you're optimizing the
building of the various version. My sweet little laptop is no match for
the general performance of your box, but relative performances are (best
one in each case):

With 2.4:

$ python -mtimeit 'for i in (str(x)for x in xrange(100)):pass'
1000 loops, best of 3: 534 usec per loop

$ python -mtimeit 'for i in [str(x)for x in xrange(100)]:pass'
1000 loops, best of 3: 526 usec per loop

$ python -mtimeit 'for i in map(str,xrange(100)):pass'
1000 loops, best of 3: 400 usec per loop

--> genexp and LC within noise, map of built-in over 20% faster

with 2.3:

$ /usr/bin/python timeit.py 'for i in map(str,xrange(100)):pass'
1000 loops, best of 3: 540 usec per loop

$ /usr/bin/python timeit.py 'for i in[str(x)for x in xrange(100)]:pass'
1000 loops, best of 3: 741 usec per loop

2.3 way slower than 2.4 for same ops, higher map gain wrt LC.


But this is for tiny loops. Try less-tiny ones:

$ python -mtimeit 'for i in [str(x)for x in xrange(10000)]:pass'
10 loops, best of 3: 59.9 msec per loop

$ python -mtimeit 'for i in (str(x)for x in xrange(10000)):pass'
10 loops, best of 3: 52.8 msec per loop

now difference is NOT 'in the noise': it's repeatably over 12%.

$ python -mtimeit 'for i in map(str, xrange(10000)):pass'
10 loops, best of 3: 45.5 msec per loop

Still faster (with a built-in), but slowing down like the LC, due to
similar allocation issues, while the genexp scales up better.


The genexp will really shine if your loop is huge, so the saving of
memory allocation translates into huge time savings.

But if you're REALLY in a hurry, the fastest approach is another:

$ python -mtimeit -s'from itertools import imap' 'for i in imap(str,
xrange(10000)):pass'
10 loops, best of 3: 39.1 msec per loop

As you see, even for this modest loop itertools' advantage on map is
already comparable to map's on the genexp. AND imap scales up like the
genexp -- no big memory allocation.


Ratios very similar in 2.3:

$ /usr/bin/python timeit.py -s'from itertools import imap' 'for i in
imap(str, xrange(10000)):pass'
10 loops, best of 3: 4.91e+04 usec per loop

$ /usr/bin/python timeit.py 'for i in map(str, xrange(10000)):pass'
10 loops, best of 3: 5.58e+04 usec per loop

2.4 has an easy 20% gain, but, if you're stuck with 2.3, you can still
get some 15% gain by using itertools rather than map. Here, too, the
same scaling considerations apply.


It _is_ peculiar that the fastest solution isn't even in the subject;-).
Lesson to retain: *** itertools roolz *** !!!-)


Alex
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,900
Latest member
Nell636132

Latest Threads

Top