# map vs. list-comprehension

Discussion in 'Python' started by Mandus, Jun 29, 2005.

1. ### MandusGuest

Hi there,

inspired by a recent thread where the end of reduce/map/lambda in Python was
discussed, I looked over some of my maps, and tried to convert them to
list-comprehensions.

This one I am not sure how to conver:

Given three tuples of length n, b,i and d, I now do:

map(lambda bb,ii,dd: bb+ii*dd,b,i,d)

which gives a list of length n.

Anyone have an idea what the equivalent list comprehension will be?

take care,
--
Mandus - the only mandus around.

Mandus, Jun 29, 2005

2. ### F. PetitjeanGuest

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :
> Hi there,
>
> inspired by a recent thread where the end of reduce/map/lambda in Python was
> discussed, I looked over some of my maps, and tried to convert them to
> list-comprehensions.
>
> This one I am not sure how to conver:
>
> Given three tuples of length n, b,i and d, I now do:
>
> map(lambda bb,ii,dd: bb+ii*dd,b,i,d)
>
> which gives a list of length n.

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

F. Petitjean, Jun 29, 2005

3. ### =?utf-8?q?Bj=C3=B6rn_Lindstr=C3=B6m?=Guest

"F. Petitjean" <> writes:

> res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]
>
> Hoping that zip will not be deprecated.

Nobody has suggested that. The ones that are planned to be removed are
lambda, reduce, filter and map. Here's GvR's blog posting that explains
the reasons:

--
BjÃ¶rn LindstrÃ¶m <>
Student of computational linguistics, Uppsala University, Sweden

=?utf-8?q?Bj=C3=B6rn_Lindstr=C3=B6m?=, Jun 29, 2005
4. ### MandusGuest

29 Jun 2005 10:04:40 GMT skrev F. Petitjean:
> Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :
>> Hi there,
>>
>> inspired by a recent thread where the end of reduce/map/lambda in Python was
>> discussed, I looked over some of my maps, and tried to convert them to
>> list-comprehensions.
>>
>> This one I am not sure how to conver:
>>
>> Given three tuples of length n, b,i and d, I now do:
>>
>> map(lambda bb,ii,dd: bb+ii*dd,b,i,d)
>>
>> which gives a list of length n.

>
> res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

aha - thanks! I wasn't aware of zip. Guess I have to put python in a
nutshell on my nightstand (again)

seem to be a tad slower than the map, but nothing serious. Guess it's
the extra zip.

>
> Hoping that zip will not be deprecated.

hopefully not...

--
Mandus - the only mandus around.

Mandus, Jun 29, 2005
5. ### Carl BanksGuest

F. Petitjean wrote:
> Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :
> > Hi there,
> >
> > inspired by a recent thread where the end of reduce/map/lambda in Python was
> > discussed, I looked over some of my maps, and tried to convert them to
> > list-comprehensions.
> >
> > This one I am not sure how to conver:
> >
> > Given three tuples of length n, b,i and d, I now do:
> >
> > map(lambda bb,ii,dd: bb+ii*dd,b,i,d)
> >
> > which gives a list of length n.

>
> res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]
>
> Hoping that zip will not be deprecated.

Notice that zip doesn't do any functional stuff--it merely manipulates
data structures--so it ought not to be lumped in with map, filter, and
reduce.

Fear not, people: just as the BDFL does not indiscriminately add
features, also he does not indiscriminately remove them. zip, though
it feels a little exotic, is very useful and serves a purpose that no
language feature serves(*), so rest assured it's not going to
disappear.

(*) Excepting izip, of course, which is more useful than zip and
probably should also be a builtin.

--
Carl Banks

Carl Banks, Jun 29, 2005
6. ### George SakkisGuest

"Carl Banks" <> wrote in message

> Fear not, people: just as the BDFL does not indiscriminately add
> features, also he does not indiscriminately remove them. zip, though
> it feels a little exotic, is very useful and serves a purpose that no
> language feature serves(*), so rest assured it's not going to
> disappear.
>
> (*) Excepting izip, of course, which is more useful than zip and
> probably should also be a builtin.

One of the envisioned changes in Python 3000
(http://www.python.org/peps/pep-3000.html) is to return iterators
instead of lists in several contexts (e.g. dict.keys(), dict.items()),
so perhaps zip will stand for what itertools.izip is today.

George

George Sakkis, Jun 29, 2005
7. ### Scott David DanielsGuest

Mandus wrote:
> 29 Jun 2005 10:04:40 GMT skrev F. Petitjean:
>
>>Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :
>>
>>res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

>
> seem to be a tad slower than the map, but nothing serious. Guess it's
> the extra zip.

You could try timing it using itertools.izip rather than zip.

--Scott David Daniels

Scott David Daniels, Jun 29, 2005
8. ### Terry ReedyGuest

"F. Petitjean" <> wrote in message
news:42c27238\$0\$26269\$...
>res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

> Hoping that zip will not be deprecated.

Cease worrying. Zip was added to replace the zipping behavior of map and
the idiom map(None, a, b, ...). It simultaneously altered the behavior for
unequal length inputs to stop with the shortest instead of padding to the
longest.

tjr

Terry Reedy, Jun 29, 2005
9. ### MandusGuest

Wed, 29 Jun 2005 08:33:58 -0700 skrev Scott David Daniels:
> Mandus wrote:
>> 29 Jun 2005 10:04:40 GMT skrev F. Petitjean:
>>
>>>Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :
>>>
>>>res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

>>
>> seem to be a tad slower than the map, but nothing serious. Guess it's
>> the extra zip.

> You could try timing it using itertools.izip rather than zip.

jepp - faster, but still slower than the map.

1000000 iterations:
zip+list-comprehension: 8.1s
izip+list-comprehension: 7.5s
map: 7.0s

--
Mandus - the only mandus around.

Mandus, Jun 30, 2005
10. ### Mike P.Guest

"Björn Lindström" <> wrote in message
news:...
> "F. Petitjean" <> writes:
>
> > res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]
> >
> > Hoping that zip will not be deprecated.

>
> Nobody has suggested that. The ones that are planned to be removed are
> lambda, reduce, filter and map. Here's GvR's blog posting that explains
> the reasons:
>
>

That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
much, but I use lambda, map and filter all the time. These are some of the
features of Python that I love the best. I can get some pretty compact and
easy to read code with them. And no, I'm not a Lisp programmer (never
programmed in Lisp). My background being largely C++, I discovered lambda,
apply, map and filter in Python, although I had seen similar stuff in other
functional languages like Miranda and Haskell.

Also, I don't necessarily think list comprehensions are necessarily easier
to read. I don't use them all that much to be honest.

IMHO I'm not particularly happy with the way Python is going language wise.
I mean, I don't think I'll ever use decorators, for example. Personally, in
terms of language features and capabilities I think the language is fine.
Not much (if anything needs to be added).

I think at this stage the Python community and Python programmers would be
better served by building a better, more standardised, cross platform, more
robust, better documented, and more extensive standard library. I would be
happy to contribute in this regard, rather than having debates about the
addition and removal of language features which don't improve my
productivity.

I would love it if modules like PyOpenGL, PyOSG (Open Scene Graph), PyQt, a
graph library etc, were all part of the standard python library, and that
worked out of the box on all major platforms -Windows, Unix, Linux, Mac. All
these modules which are C/C++ based are all at different versions and at
different stages, requiring different versions of Python working on
different operating systems.
It's not as transparent as it should be. For example, why aren't PIL,
Numeric and a host of other fairly mainstream Python modules not part of the
standard library? Compare that with the huge SDK that comes with Java.

Then there is always issues of performance, better standard development
tools, better documentation. There are lots of things to do, to make the
Python programmers life better without touching the actual features of the
language.

Sorry, I've probably gone way off topic, and probably stirred up political
issues which I'm not aware of, but, man when I hear stuff like the proposed
removal of reduce, lambda, filter and map, all I see ahead of me is a waste
of time as a programmer.

I don't program in Python for it's own sake. I program in Python because it
lets me get my job done quicker and it saves me time. The proposed removals
are going to waste my time. Why? Because my team and myself are going to
have to go through all our code and change stuff like maps to ugly looking
list comprehensions or whatever when Python 3000 comes out. Sure some of you
will say you don't have to update, just stick with Python 2.3/2.4 or
whatever. That is fine in theory, but in practice I'm going to have to use
some third party module which will require Python 3000 (this happened to me
recently with a module which had a serious bug with the Python 2.3 version,
but worked with the Python 2.4 version - I had to upgrade every single third
party module I was using - I was lucky the ones I was using had 2.4
versions, but there are still a lot of modules out there that don't).

Sorry for the OT long rant.

Mike

Mike P., Jun 30, 2005
11. ### Tom AndersonGuest

On Fri, 1 Jul 2005, Mike P. wrote:

> "Björn Lindström" <> wrote in message
> news:...
>> "F. Petitjean" <> writes:
>>
>>> res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]
>>>
>>> Hoping that zip will not be deprecated.

>>
>> Nobody has suggested that. The ones that are planned to be removed are
>> lambda, reduce, filter and map. Here's GvR's blog posting that explains
>> the reasons:
>>

>
> That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
> much, but I use lambda, map and filter all the time. These are some of
> the features of Python that I love the best. I can get some pretty
> compact and easy to read code with them.

Same here.

> And no, I'm not a Lisp programmer (never programmed in Lisp). My
> background being largely C++, I discovered lambda, apply, map and filter
> in Python, although I had seen similar stuff in other functional
> languages like Miranda and Haskell.

Same here too!

> Also, I don't necessarily think list comprehensions are necessarily
> easier to read. I don't use them all that much to be honest.

And here!

However, i also felt that way about generator functions - until the other
day, when i realised one was the best solution to a problem i had. That
made me realise that the same was probably true of list comprehensions.

That said, i do still think that map etc are better than list comps,
because they involve less language. Once you have the idea of a function
and a list, you can understand map as a function that operates on lists;
list comprehensions provide a whole new splodge of arbitrary syntax to
learn. I guess you could say the same about lambda, which is really an
essential part of the whole map way of life, but i don't think that's fair
- list comprehensions are a structure for doing just one thing, whereas
lambda is a construct of enormous general power.

I'd be happy for the lambda syntax to be tidied up, though - perhaps it
could be merged with def? Like:

some_statements
return some_expression

def name(args): return some_expression # one-line form

def name(args): some_statements; return some_expression

def name(args) = some_expression # shorthand one-line form

Then an anonymous form, which is an expression rather than a statement:

def (args):
some_statements
return some_expression

def (args): return some_expression

def (args) = some_expression

The latter form is like a lambda; i'm not sure how the former forms would
work inside enclosing expressions; i think it would look pretty sick:

area = 4.0 * math.pi * (radius ** 2)
volume = 4.0 / 3.0 * math.pi * (radius ** 2)
return area / volume

It works, but i admit it's not hugely pretty. But then, i would't advise
anyone to actually do this; it's just there for completeness.

You might also want to allow:

def name(args) = some_statements; some_expression

And the anonymous counterpart. But i'm not sure about that one. Multiple
expressions inside lambdas would sometimes be useful, but you can get
those with the shorthand form.

> I think at this stage the Python community and Python programmers would
> be better served by building a better, more standardised, cross
> platform, more robust, better documented, and more extensive standard
> library. I would be happy to contribute in this regard, rather than
> having debates about the addition and removal of language features which
> don't improve my productivity.

Same here.

> Sorry, I've probably gone way off topic, and probably stirred up
> political issues which I'm not aware of, but, man when I hear stuff like
> the proposed removal of reduce, lambda, filter and map, all I see ahead
> of me is a waste of time as a programmer.

Same here.

> Sorry for the OT long rant.

Yeah, that was really off-topic for a python newsgroup. You didn't even
mention regional accents once!

tom

--
How did i get here?

Tom Anderson, Jun 30, 2005
12. ### Terry HancockGuest

On Thursday 30 June 2005 09:49 am, Mike P. wrote:
> That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
> much, but I use lambda, map and filter all the time.
> [...]
> Also, I don't necessarily think list comprehensions are necessarily easier
> to read. I don't use them all that much to be honest.

Actually, I prefer list comprehension syntax, after having a chance to experience
both, both as a writer and a reader of other people's code. So I disagree with
this particular issue.

> IMHO I'm not particularly happy with the way Python is going language wise.
> I mean, I don't think I'll ever use decorators, for example. Personally, in
> terms of language features and capabilities I think the language is fine.
> Not much (if anything needs to be added).

This though, I mostly agree with. I'm not sure if I'll ever find use for decorators,
but the "@" signs bother me, I must admit. I liked the fact that there was a
way to do this without fiddling with the language syntax.

> I think at this stage the Python community and Python programmers would be
> better served by building a better, more standardised, cross platform, more
> robust, better documented, and more extensive standard library. I would be
> happy to contribute in this regard, rather than having debates about the
> addition and removal of language features which don't improve my
> productivity.

Now this, I agree with wholeheartedly! The library is where the real work ought
to be happening.

should be improved and the "batteries included" focus should be sidelined.

One of the strengths of Python has been that the language itself is small (which
it shares with C and (if I understand correctly, not being a lisp programmer?) Lisp),
but with all the syntax enhancements going on, Python is getting pretty complicated.
I have to wonder if new users won't begin to find it just as intimidating as Perl
or other big languages.

I know something like that happened with HTML. When I learned it, HTML was
very simple (version 1? I think or maybe 2?). Later extensions in 3.2 and 4.0 didn't
bother me, because they were incremental increases over what I already knew. But
new users are finding HTML much more daunting, which encourages them to
reach for WYSIWYG tools and the like, rather than just coding it.

So my former experience of HTML as "dead simple" seems to have been undermined
by the later developments. It still seems "dead simple" to me, but new users don't
seem to have the reaction that I did.

I strongly doubt this is because "new users are wimpier than we were".

Likewise, part of the thing that attracted me to Python and which I still consider
a "selling point" is that it is not only easy for me to use, but easy for potential
collaborators to learn. Since the projects I am most interested in are free-software
projects primarily interesting to non-programmers or semi-skilled programmers,
the need to make the code as easy for them to understand as possible is important
to me. I want to actually encourage users to *become* programmers to work on
my projects, not just interest existing programmers.

Before I encountered Python, that didn't seem possible. But I was really impressed
with how easy Python was to learn (that is, Python 1.5.9). Even 2.1 was pretty easy,
and I acknowledge the benefits of list comps and the like, even though I also see that
they can be abused. Now, though, I'm starting to wonder if that newfound advantage
is disappearing.

Frankly, I feel like slamming on the brakes. Simplicity is really, really good,
and we shouldn't forget that in our zeal to make Python "a better systems programming
language". ISTM that it's already pretty good at those tasks, and if it's a little harder
to do a few things, maybe that's the cost of keeping most things easy.

In any case, the Python object model is very flexible, and most things you want
to do can be done using it. So it seems to me like compartmentalizing changes by
keeping them in modules would be really good.

I guess I've become a python conservative. ;-)

--
Terry Hancock ( hancock at anansispaceworks.com )
Anansi Spaceworks http://www.anansispaceworks.com

Terry Hancock, Jul 1, 2005
13. ### Roy SmithGuest

Terry Hancock <> wrote:
> One of the strengths of Python has been that the language itself is
> small (which it shares with C and (if I understand correctly, not being
> a lisp programmer?) Lisp), but with all the syntax enhancements going
> on, Python is getting pretty complicated. I have to wonder if new users
> won't begin to find it just as intimidating as Perl or other big
> languages.

+1

Even some of the relatively recent library enhancements have been kind of
complicated. The logging module, for example, seems way over the top.

Look at what happened to C when it mutated into C++. In isolation, most of
the features of C++ seem like good ideas. Taken together, it's a huge
hairy mess that most people only understand increasingly larger subsets of.
Fred Brooks called it the second system sy

Roy Smith, Jul 1, 2005
14. ### Roy SmithGuest

Terry Hancock <> wrote:
> One of the strengths of Python has been that the language itself is
> small (which it shares with C and (if I understand correctly, not being
> a lisp programmer?) Lisp), but with all the syntax enhancements going
> on, Python is getting pretty complicated. I have to wonder if new users
> won't begin to find it just as intimidating as Perl or other big
> languages.

+1

Even some of the relatively recent library enhancements have been kind of
complicated. The logging module, for example, seems way over the top.

Look at what happened to C when it mutated into C++. In isolation, most of
the features of C++ seem like good ideas. Taken together, it's a huge
hairy mess that most people only understand increasingly larger subsets of.
Fred Brooks called it the second system syndrome.

Roy Smith, Jul 1, 2005
15. ### Robert KernGuest

Roy Smith wrote:
> Terry Hancock <> wrote:
>
>>One of the strengths of Python has been that the language itself is
>>small (which it shares with C and (if I understand correctly, not being
>>a lisp programmer?) Lisp), but with all the syntax enhancements going
>>on, Python is getting pretty complicated. I have to wonder if new users
>>won't begin to find it just as intimidating as Perl or other big
>>languages.

>
> +1
>
> Even some of the relatively recent library enhancements have been kind of
> complicated. The logging module, for example, seems way over the top.
>
> Look at what happened to C when it mutated into C++. In isolation, most of
> the features of C++ seem like good ideas. Taken together, it's a huge
> hairy mess that most people only understand increasingly larger subsets of.
> Fred Brooks called it the second system sy

Looks like the PSU got to yoNO CARRIER

Robert Kern, Jul 1, 2005
16. ### Roy SmithGuest

Robert Kern <> wrote:

> Looks like the PSU got to yoNO CARRIER

No, the trackpad on my PowerBook seems to have gone a little haywire and
I'm getting the occasional random mouse click. In that case, it seemed to
have clicked the "Post" button.

Roy Smith, Jul 1, 2005
17. ### George SakkisGuest

"Terry Hancock" wrote:

> On Thursday 30 June 2005 09:49 am, Mike P. wrote:
>
> > IMHO I'm not particularly happy with the way Python is going language wise.
> > I mean, I don't think I'll ever use decorators, for example. Personally, in
> > terms of language features and capabilities I think the language is fine.
> > Not much (if anything needs to be added).

>
> This though, I mostly agree with. I'm not sure if I'll ever find use for decorators,
> but the "@" signs bother me, I must admit. I liked the fact that there was a
> way to do this without fiddling with the language syntax.

What way ? The foo=decorator(foo) workaround after the function's body
? That's hardly acceptable for common decorators such as staticmethod,
classmethod and other suggested ones (synchronized, cached, traced,
etc.). I don't mind the @ so much - I don't like it particularly either
- and I don't compare it to the alternative rejected decorator syntax
forms, but it's certainly an improvement over the pre-2.4 explicit
assignment *after* the function.

Properties are in the same boat with decorators. Although they are
great semantically, the current syntax is less than optimal.
Interestingly, Pyrex does much better in the readability department
since 0.9
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/version/Doc/extension_types.html#Properties).
The Property decorator recipe
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410698) comes
pretty close to it, but I wouldn't mind a small addition to the
language syntax for something so useful as properties.

Keeping the language small is a worthwhile goal, but it should be
content with s-expressions.

Just my \$0.02,

George

George Sakkis, Jul 1, 2005
18. ### Tom AndersonGuest

On Thu, 30 Jun 2005, Roy Smith wrote:

> Terry Hancock <> wrote:
>
>> One of the strengths of Python has been that the language itself is
>> small (which it shares with C and (if I understand correctly, not being
>> a lisp programmer?) Lisp), but with all the syntax enhancements going
>> on, Python is getting pretty complicated. I have to wonder if new users
>> won't begin to find it just as intimidating as Perl or other big
>> languages.

>
> +1
>
> Even some of the relatively recent library enhancements have been kind
> of complicated. The logging module, for example, seems way over the
> top.

Exactly the same thing happened with Java. if you look at the libraries
that were in 1.1, they're very clean and simple (perhaps with the
exception of AWT). 1.2 added a load of stuff that was much less
well-designed (with the notable exception of the collections stuff, which
is beautiful), and a lot of the extension packages that have been written
since then are seriously crappy. My particular bugbear is JAI, the imaging
library, the most gratuitously badly-designed library it has ever been my
misfortune to work with. EJB is another great example.

I imagine the reason for this degradation has been the expansion of the
java design team: it started off with James Gosling, who is an incredibly
smart guy and an awesome engineer, and a relatively small team of crack
troops; they were capable of writing good code, and really cared about
doing that. Over the years, as it's grown, it's had to absorb a lot of
people who don't have that combination of intelligence and good taste, and
they've written a lot of crap. I suspect a trend away from gifted lone
hackers and towards design by committee hasn't helped, either.

How this applies to python, where the BDFL is still very much at the helm,
is not clear. I wonder if analogies to Linux, also a despotism, are more
useful?

tom

--
In-jokes for out-casts

Tom Anderson, Jul 1, 2005
19. ### Tom AndersonGuest

On Fri, 1 Jul 2005, George Sakkis wrote:

> "Terry Hancock" wrote:
>
> Keeping the language small is a worthwhile goal, but it should be traded
> off with conciseness and readability; otherwise we could well be content
> with s-expressions.

There's quite a number of satisfied LISP programmers out there who *are*
content with S-expressions ...

tom

--
In-jokes for out-casts

Tom Anderson, Jul 1, 2005
20. ### George SakkisGuest

"Tom Anderson" wrote:

> On Fri, 1 Jul 2005, George Sakkis wrote:
>
> > Keeping the language small is a worthwhile goal, but it should be traded
> > off with conciseness and readability; otherwise we could well be content
> > with s-expressions.

>
> There's quite a number of satisfied LISP programmers out there who *are*
> content with S-expressions ...
>
> tom

"We" referred mostly to python programmers, and although quite a few
may also be happy users of one or more lisp-ish dialect, I doubt they
think it beats python in the syntax department ;-)

George

George Sakkis, Jul 1, 2005