map vs. list-comprehension

[Mandus]

Hi there,

inspired by a recent thread where the end of reduce/map/lambda in Python was
discussed, I looked over some of my maps, and tried to convert them to
list-comprehensions.

This one I am not sure how to conver:

Given three tuples of length n, b,i and d, I now do:

map(lambda bb,ii,dd: bb+ii*dd,b,i,d)

which gives a list of length n.

Anyone have an idea what the equivalent list comprehension will be?

take care,

 

[F. Petitjean]

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :

Hi there,

inspired by a recent thread where the end of reduce/map/lambda in Python was
discussed, I looked over some of my maps, and tried to convert them to
list-comprehensions.

This one I am not sure how to conver:

Given three tuples of length n, b,i and d, I now do:

map(lambda bb,ii,dd: bb+ii*dd,b,i,d)

which gives a list of length n.

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

 

[Guest]

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

Nobody has suggested that. The ones that are planned to be removed are
lambda, reduce, filter and map. Here's GvR's blog posting that explains
the reasons:

http://www.artima.com/weblogs/viewpost.jsp?thread=98196

 

[Mandus]

29 Jun 2005 10:04:40 GMT skrev F. Petitjean:

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :

Hi there,

inspired by a recent thread where the end of reduce/map/lambda in Python was
discussed, I looked over some of my maps, and tried to convert them to
list-comprehensions.

This one I am not sure how to conver:

Given three tuples of length n, b,i and d, I now do:

map(lambda bb,ii,dd: bb+ii*dd,b,i,d)

which gives a list of length n.

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

aha - thanks! I wasn't aware of zip. Guess I have to put python in a
nutshell on my nightstand (again)

seem to be a tad slower than the map, but nothing serious. Guess it's
the extra zip.

Hoping that zip will not be deprecated.

hopefully not...

 

[Carl Banks]

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :

Hi there,

inspired by a recent thread where the end of reduce/map/lambda in Python was
discussed, I looked over some of my maps, and tried to convert them to
list-comprehensions.

This one I am not sure how to conver:

Given three tuples of length n, b,i and d, I now do:

map(lambda bb,ii,dd: bb+ii*dd,b,i,d)

which gives a list of length n.

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

Notice that zip doesn't do any functional stuff--it merely manipulates
data structures--so it ought not to be lumped in with map, filter, and
reduce.

Fear not, people: just as the BDFL does not indiscriminately add
features, also he does not indiscriminately remove them. zip, though
it feels a little exotic, is very useful and serves a purpose that no
language feature serves(*), so rest assured it's not going to
disappear.

(*) Excepting izip, of course, which is more useful than zip and
probably should also be a builtin.

 

[George Sakkis]

Fear not, people: just as the BDFL does not indiscriminately add
features, also he does not indiscriminately remove them. zip, though
it feels a little exotic, is very useful and serves a purpose that no
language feature serves(*), so rest assured it's not going to
disappear.

(*) Excepting izip, of course, which is more useful than zip and
probably should also be a builtin.

One of the envisioned changes in Python 3000
(http://www.python.org/peps/pep-3000.html) is to return iterators
instead of lists in several contexts (e.g. dict.keys(), dict.items()),
so perhaps zip will stand for what itertools.izip is today.

George

 

[Scott David Daniels]

29 Jun 2005 10:04:40 GMT skrev F. Petitjean:

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

seem to be a tad slower than the map, but nothing serious. Guess it's
the extra zip.

You could try timing it using itertools.izip rather than zip.

--Scott David Daniels
(e-mail address removed)

 

[Terry Reedy]

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

Cease worrying. Zip was added to replace the zipping behavior of map and
the idiom map(None, a, b, ...). It simultaneously altered the behavior for
unequal length inputs to stop with the shortest instead of padding to the
longest.

tjr

 

[Mandus]

Wed, 29 Jun 2005 08:33:58 -0700 skrev Scott David Daniels:

29 Jun 2005 10:04:40 GMT skrev F. Petitjean:

Le Wed, 29 Jun 2005 09:46:15 +0000 (UTC), Mandus a écrit :

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

seem to be a tad slower than the map, but nothing serious. Guess it's
the extra zip.

You could try timing it using itertools.izip rather than zip.

jepp - faster, but still slower than the map.

1000000 iterations:
zip+list-comprehension: 8.1s
izip+list-comprehension: 7.5s
map: 7.0s

 

[Mike P.]

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

Nobody has suggested that. The ones that are planned to be removed are
lambda, reduce, filter and map. Here's GvR's blog posting that explains
the reasons:

http://www.artima.com/weblogs/viewpost.jsp?thread=98196

That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
much, but I use lambda, map and filter all the time. These are some of the
features of Python that I love the best. I can get some pretty compact and
easy to read code with them. And no, I'm not a Lisp programmer (never
programmed in Lisp). My background being largely C++, I discovered lambda,
apply, map and filter in Python, although I had seen similar stuff in other
functional languages like Miranda and Haskell.

Also, I don't necessarily think list comprehensions are necessarily easier
to read. I don't use them all that much to be honest.

IMHO I'm not particularly happy with the way Python is going language wise.
I mean, I don't think I'll ever use decorators, for example. Personally, in
terms of language features and capabilities I think the language is fine.
Not much (if anything needs to be added).

I think at this stage the Python community and Python programmers would be
better served by building a better, more standardised, cross platform, more
robust, better documented, and more extensive standard library. I would be
happy to contribute in this regard, rather than having debates about the
addition and removal of language features which don't improve my
productivity.

I would love it if modules like PyOpenGL, PyOSG (Open Scene Graph), PyQt, a
graph library etc, were all part of the standard python library, and that
worked out of the box on all major platforms -Windows, Unix, Linux, Mac. All
these modules which are C/C++ based are all at different versions and at
different stages, requiring different versions of Python working on
different operating systems.
It's not as transparent as it should be. For example, why aren't PIL,
Numeric and a host of other fairly mainstream Python modules not part of the
standard library? Compare that with the huge SDK that comes with Java.

Then there is always issues of performance, better standard development
tools, better documentation. There are lots of things to do, to make the
Python programmers life better without touching the actual features of the
language.

Sorry, I've probably gone way off topic, and probably stirred up political
issues which I'm not aware of, but, man when I hear stuff like the proposed
removal of reduce, lambda, filter and map, all I see ahead of me is a waste
of time as a programmer.

I don't program in Python for it's own sake. I program in Python because it
lets me get my job done quicker and it saves me time. The proposed removals
are going to waste my time. Why? Because my team and myself are going to
have to go through all our code and change stuff like maps to ugly looking
list comprehensions or whatever when Python 3000 comes out. Sure some of you
will say you don't have to update, just stick with Python 2.3/2.4 or
whatever. That is fine in theory, but in practice I'm going to have to use
some third party module which will require Python 3000 (this happened to me
recently with a module which had a serious bug with the Python 2.3 version,
but worked with the Python 2.4 version - I had to upgrade every single third
party module I was using - I was lucky the ones I was using had 2.4
versions, but there are still a lot of modules out there that don't).

Sorry for the OT long rant.

Mike

 

[Tom Anderson]

res = [ bb+ii*dd for bb,ii,dd in zip(b,i,d) ]

Hoping that zip will not be deprecated.

Nobody has suggested that. The ones that are planned to be removed are
lambda, reduce, filter and map. Here's GvR's blog posting that explains
the reasons:

http://www.artima.com/weblogs/viewpost.jsp?thread=98196

That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
much, but I use lambda, map and filter all the time. These are some of
the features of Python that I love the best. I can get some pretty
compact and easy to read code with them.

Same here.

And no, I'm not a Lisp programmer (never programmed in Lisp). My
background being largely C++, I discovered lambda, apply, map and filter
in Python, although I had seen similar stuff in other functional
languages like Miranda and Haskell.

Same here too!

Also, I don't necessarily think list comprehensions are necessarily
easier to read. I don't use them all that much to be honest.

And here!

However, i also felt that way about generator functions - until the other
day, when i realised one was the best solution to a problem i had. That
made me realise that the same was probably true of list comprehensions.

That said, i do still think that map etc are better than list comps,
because they involve less language. Once you have the idea of a function
and a list, you can understand map as a function that operates on lists;
list comprehensions provide a whole new splodge of arbitrary syntax to
learn. I guess you could say the same about lambda, which is really an
essential part of the whole map way of life, but i don't think that's fair
- list comprehensions are a structure for doing just one thing, whereas
lambda is a construct of enormous general power.

I'd be happy for the lambda syntax to be tidied up, though - perhaps it
could be merged with def? Like:

def name(args): # traditional form
some_statements
return some_expression

def name(args): return some_expression # one-line form

def name(args): some_statements; return some_expression

def name(args) = some_expression # shorthand one-line form

Then an anonymous form, which is an expression rather than a statement:

def (args):
some_statements
return some_expression

def (args): return some_expression

def (args) = some_expression

The latter form is like a lambda; i'm not sure how the former forms would
work inside enclosing expressions; i think it would look pretty sick:

surfaceAreaToVolumeRatios = map(def (radius):
area = 4.0 * math.pi * (radius ** 2)
volume = 4.0 / 3.0 * math.pi * (radius ** 2)
return area / volume
, radii)

It works, but i admit it's not hugely pretty. But then, i would't advise
anyone to actually do this; it's just there for completeness.

You might also want to allow:

def name(args) = some_statements; some_expression

And the anonymous counterpart. But i'm not sure about that one. Multiple
expressions inside lambdas would sometimes be useful, but you can get
those with the shorthand form.

I think at this stage the Python community and Python programmers would
be better served by building a better, more standardised, cross
platform, more robust, better documented, and more extensive standard
library. I would be happy to contribute in this regard, rather than
having debates about the addition and removal of language features which
don't improve my productivity.

Same here.

Sorry, I've probably gone way off topic, and probably stirred up
political issues which I'm not aware of, but, man when I hear stuff like
the proposed removal of reduce, lambda, filter and map, all I see ahead
of me is a waste of time as a programmer.

Same here.

Sorry for the OT long rant.

Yeah, that was really off-topic for a python newsgroup. You didn't even
mention regional accents once!

tom

 

[Terry Hancock]

That really sucks, I wasn't aware of these plans. Ok, I don't use reduce
much, but I use lambda, map and filter all the time.
[...]
Also, I don't necessarily think list comprehensions are necessarily easier
to read. I don't use them all that much to be honest.

Actually, I prefer list comprehension syntax, after having a chance to experience
both, both as a writer and a reader of other people's code. So I disagree with
this particular issue.

IMHO I'm not particularly happy with the way Python is going language wise.
I mean, I don't think I'll ever use decorators, for example. Personally, in
terms of language features and capabilities I think the language is fine.
Not much (if anything needs to be added).

This though, I mostly agree with. I'm not sure if I'll ever find use for decorators,
but the "@" signs bother me, I must admit. I liked the fact that there was a
way to do this without fiddling with the language syntax.

I think at this stage the Python community and Python programmers would be
better served by building a better, more standardised, cross platform, more
robust, better documented, and more extensive standard library. I would be
happy to contribute in this regard, rather than having debates about the
addition and removal of language features which don't improve my
productivity.

Now this, I agree with wholeheartedly! The library is where the real work ought
to be happening.

Either that, or previous suggestions about making access to external packages
should be improved and the "batteries included" focus should be sidelined.

One of the strengths of Python has been that the language itself is small (which
it shares with C and (if I understand correctly, not being a lisp programmer?) Lisp),
but with all the syntax enhancements going on, Python is getting pretty complicated.
I have to wonder if new users won't begin to find it just as intimidating as Perl
or other big languages.

I know something like that happened with HTML. When I learned it, HTML was
very simple (version 1? I think or maybe 2?). Later extensions in 3.2 and 4.0 didn't
bother me, because they were incremental increases over what I already knew. But
new users are finding HTML much more daunting, which encourages them to
reach for WYSIWYG tools and the like, rather than just coding it.

So my former experience of HTML as "dead simple" seems to have been undermined
by the later developments. It still seems "dead simple" to me, but new users don't
seem to have the reaction that I did.

I strongly doubt this is because "new users are wimpier than we were".

Likewise, part of the thing that attracted me to Python and which I still consider
a "selling point" is that it is not only easy for me to use, but easy for potential
collaborators to learn. Since the projects I am most interested in are free-software
projects primarily interesting to non-programmers or semi-skilled programmers,
the need to make the code as easy for them to understand as possible is important
to me. I want to actually encourage users to *become* programmers to work on
my projects, not just interest existing programmers.

Before I encountered Python, that didn't seem possible. But I was really impressed
with how easy Python was to learn (that is, Python 1.5.9). Even 2.1 was pretty easy,
and I acknowledge the benefits of list comps and the like, even though I also see that
they can be abused. Now, though, I'm starting to wonder if that newfound advantage
is disappearing.

Frankly, I feel like slamming on the brakes. Simplicity is really, really good,
and we shouldn't forget that in our zeal to make Python "a better systems programming
language". ISTM that it's already pretty good at those tasks, and if it's a little harder
to do a few things, maybe that's the cost of keeping most things easy.

In any case, the Python object model is very flexible, and most things you want
to do can be done using it. So it seems to me like compartmentalizing changes by
keeping them in modules would be really good.

I guess I've become a python conservative. ;-)

 

[Roy Smith]

One of the strengths of Python has been that the language itself is
small (which it shares with C and (if I understand correctly, not being
a lisp programmer?) Lisp), but with all the syntax enhancements going
on, Python is getting pretty complicated. I have to wonder if new users
won't begin to find it just as intimidating as Perl or other big
languages.

+1

Even some of the relatively recent library enhancements have been kind of
complicated. The logging module, for example, seems way over the top.

Look at what happened to C when it mutated into C++. In isolation, most of
the features of C++ seem like good ideas. Taken together, it's a huge
hairy mess that most people only understand increasingly larger subsets of.
Fred Brooks called it the second system sy

 

[Roy Smith]

One of the strengths of Python has been that the language itself is
small (which it shares with C and (if I understand correctly, not being
a lisp programmer?) Lisp), but with all the syntax enhancements going
on, Python is getting pretty complicated. I have to wonder if new users
won't begin to find it just as intimidating as Perl or other big
languages.

+1

Even some of the relatively recent library enhancements have been kind of
complicated. The logging module, for example, seems way over the top.

Look at what happened to C when it mutated into C++. In isolation, most of
the features of C++ seem like good ideas. Taken together, it's a huge
hairy mess that most people only understand increasingly larger subsets of.
Fred Brooks called it the second system syndrome.

 

[Robert Kern]

+1

Even some of the relatively recent library enhancements have been kind of
complicated. The logging module, for example, seems way over the top.

Look at what happened to C when it mutated into C++. In isolation, most of
the features of C++ seem like good ideas. Taken together, it's a huge
hairy mess that most people only understand increasingly larger subsets of.
Fred Brooks called it the second system sy

Looks like the PSU got to yoNO CARRIER

 

[Roy Smith]

Looks like the PSU got to yoNO CARRIER

No, the trackpad on my PowerBook seems to have gone a little haywire and
I'm getting the occasional random mouse click. In that case, it seemed to
have clicked the "Post" button.

 

[George Sakkis]

This though, I mostly agree with. I'm not sure if I'll ever find use for decorators,
but the "@" signs bother me, I must admit. I liked the fact that there was a
way to do this without fiddling with the language syntax.

What way ? The foo=decorator(foo) workaround after the function's body
? That's hardly acceptable for common decorators such as staticmethod,
classmethod and other suggested ones (synchronized, cached, traced,
etc.). I don't mind the @ so much - I don't like it particularly either
- and I don't compare it to the alternative rejected decorator syntax
forms, but it's certainly an improvement over the pre-2.4 explicit
assignment *after* the function.

Properties are in the same boat with decorators. Although they are
great semantically, the current syntax is less than optimal.
Interestingly, Pyrex does much better in the readability department
since 0.9
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/version/Doc/extension_types.html#Properties).
The Property decorator recipe
(http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/410698) comes
pretty close to it, but I wouldn't mind a small addition to the
language syntax for something so useful as properties.

Keeping the language small is a worthwhile goal, but it should be
traded off with conciseness and readability; otherwise we could well be
content with s-expressions.

Just my $0.02,

George

 

[Tom Anderson]

+1

Even some of the relatively recent library enhancements have been kind
of complicated. The logging module, for example, seems way over the
top.

Exactly the same thing happened with Java. if you look at the libraries
that were in 1.1, they're very clean and simple (perhaps with the
exception of AWT). 1.2 added a load of stuff that was much less
well-designed (with the notable exception of the collections stuff, which
is beautiful), and a lot of the extension packages that have been written
since then are seriously crappy. My particular bugbear is JAI, the imaging
library, the most gratuitously badly-designed library it has ever been my
misfortune to work with. EJB is another great example.

I imagine the reason for this degradation has been the expansion of the
java design team: it started off with James Gosling, who is an incredibly
smart guy and an awesome engineer, and a relatively small team of crack
troops; they were capable of writing good code, and really cared about
doing that. Over the years, as it's grown, it's had to absorb a lot of
people who don't have that combination of intelligence and good taste, and
they've written a lot of crap. I suspect a trend away from gifted lone
hackers and towards design by committee hasn't helped, either.

How this applies to python, where the BDFL is still very much at the helm,
is not clear. I wonder if analogies to Linux, also a despotism, are more
useful?

tom

 

[Tom Anderson]

:

Keeping the language small is a worthwhile goal, but it should be traded
off with conciseness and readability; otherwise we could well be content
with s-expressions.

There's quite a number of satisfied LISP programmers out there who *are*
content with S-expressions ...

tom

 

[George Sakkis]

There's quite a number of satisfied LISP programmers out there who *are*
content with S-expressions ...

tom

"We" referred mostly to python programmers, and although quite a few
may also be happy users of one or more lisp-ish dialect, I doubt they
think it beats python in the syntax department ;-)

George