# an ingrate newbie complains

Discussion in 'Python' started by Elaine Jackson, Feb 4, 2004.

1. ### Elaine JacksonGuest

I have some minor complaints about the Python language, and I'm interested to
know how other people feel about them. Here they are:

1) I find the following behavior puzzling and disappointing:

>>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
>>> Y=dict(X)
>>> Z=list(Y)
>>> Z==X

False
>>> Z==Y.keys()

True

2) How come you can't write...

if y = f(x) > 3:
<do a bunch of things with x and y>

....as an equivalent of...

y = f(x)
if y > 3:
<do a bunch of things with x and y>

....?

That kind of syntax would be especially welcome in list comprehensions:

[f(x,y) for x in list if y=g(x)<=0]

If g(x) is a fairly complicated expression, and y occurs several times in
f(x,y), considerable clarity could be gained.

Elaine Jackson, Feb 4, 2004

2. ### Christopher A. CraigGuest

"Elaine Jackson" <> writes:

> 1) I find the following behavior puzzling and disappointing:
>
> >>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
> >>> Y=dict(X)
> >>> Z=list(Y)
> >>> Z==X

> False
> >>> Z==Y.keys()

> True

the list() constructor treats its argument as an iterator. Because
the overwhelmingly most common use case for an iterator on a dict is
to iterate over the keys, this creates a list of keys when given a
dict.

--
Christopher A. Craig <>
Love does no wrong to a neighbor; therefore love is
the fulfillment of the law - Romans 13:10

Christopher A. Craig, Feb 4, 2004

3. ### Peter OttenGuest

Elaine Jackson wrote:

> I have some minor complaints about the Python language, and I'm interested
> to know how other people feel about them. Here they are:
>
> 1) I find the following behavior puzzling and disappointing:
>
>>>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
>>>> Y=dict(X)
>>>> Z=list(Y)
>>>> Z==X

> False
>>>> Z==Y.keys()

> True

I too would have expected iter(d) to generate the same list as d.iteritems()
instead of d.iterkeys(). However, for me this is not a problem in practice
as I tend to be explicit, e. g

Z = Y.items()

which is also a good reminder that I'm converting a dictionary.

>
> 2) How come you can't write...
>
> if y = f(x) > 3:
> <do a bunch of things with x and y>
>
> ...as an equivalent of...
>
> y = f(x)
> if y > 3:
> <do a bunch of things with x and y>
>
> ...?

Too much C programming considered harmful. What is the advantage of the
first version over the second? Suppose you knew Python and wanted to learn
C, wouldn't you complain on comp.lang.c about the unnecessary complexity?

> That kind of syntax would be especially welcome in list comprehensions:
>
> [f(x,y) for x in list if y=g(x)<=0]
>
> If g(x) is a fairly complicated expression, and y occurs several times in
> f(x,y), considerable clarity could be gained.

Is the above list comprehension that frequent? Then how about

[f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]

With the arrival of generator expressions, some of the overhead (the
intermediate list) is bound to go away. In the mean time, there's still
that good old for loop, which IMHO is still the most readible solution if
things get really complicated.

Peter

Peter Otten, Feb 4, 2004
4. ### Bruno DesthuilliersGuest

Elaine Jackson wrote:
> I have some minor complaints about the Python language, and I'm interested to
> know how other people feel about them. Here they are:
>
> 1) I find the following behavior puzzling and disappointing:
>
>
>>>>X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
>>>>Y=dict(X)
>>>>Z=list(Y)
>>>>Z==X

>
> False
>
>>>>Z==Y.keys()

>
> True

If you iterate over a dict, you in fact iterate over the keys (which
makes perfect sens ihmo, just think of the cases where you would iterate
over a dict...). Now if you read the doc for the list() constructor,
you'll see that it takes either a sequence or an iterable. Since a dict
is not a sequence, but an iterable, with a defined behavior, I don't see
anything puzzling nor disappointing here

> 2) How come you can't write...
>
> if y = f(x) > 3:
> <do a bunch of things with x and y>
>
> ...as an equivalent of...
>
> y = f(x)
> if y > 3:
> <do a bunch of things with x and y>
>
> ...?

In C, an assignement *is* an expression, and this leads to

1/ a very common bug, ie :
char *buf = malloc(100);
if (buf = NULL) {
/* some error code here */
}

char *buf;
if ((buf = malloc(100)) != NULL) {
/* proceed with buf */
}

The BDFL designed Python to have a clear, readable, and as less
error-prone syntax as possible.

> That kind of syntax would be especially welcome in list comprehensions:
>
> [f(x,y) for x in list if y=g(x)<=0]
>
> If g(x) is a fairly complicated expression, and y occurs several times in
> f(x,y), considerable clarity could be gained.
>

if g(x) is a fairly complicated expression, you'd better stick with the
common idiom anyway :

result = []
for x in thelist: #BTW, dont use 'list' as an identifier
y = g(x)
if y <= 0:
result.append(f(x, y))

Now I must confess that I'd sometime like to have assignement as
expressions too. But one particular feature of dictatorship is that the
dictator dictates !-)

My 2 cents

Bruno Desthuilliers, Feb 4, 2004
5. ### Dang GriffithGuest

On Wed, 04 Feb 2004 19:46:28 +0100, Peter Otten <>
wrote:

>> That kind of syntax would be especially welcome in list comprehensions:
>>
>> [f(x,y) for x in list if y=g(x)<=0]
>>
>> If g(x) is a fairly complicated expression, and y occurs several times in
>> f(x,y), considerable clarity could be gained.

>
>Is the above list comprehension that frequent? Then how about
>
>[f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]
>
>With the arrival of generator expressions, some of the overhead (the
>intermediate list) is bound to go away. In the mean time, there's still
>that good old for loop, which IMHO is still the most readible solution if
>things get really complicated.

I couldn't get your example to run.

If g is a generator expression, this works for me:

[f(x, y) for x in lst for y in g(x) if cond(y)]

--dang
p.s.
>>> def f(x, y):

.... return '%d%d' % (x, y)
....
>>> def g(x):

.... yield 2 * x
....
>>> def cond(y):

.... return y % 10 == 2
....
>>> lst = range(10)
>>> print [f(x, y) for x in lst for y in g(x) if cond(y)]

['12', '612']

Dang Griffith, Feb 4, 2004
6. ### Skip MontanaroGuest

Elaine> 1) I find the following behavior puzzling and disappointing:

>>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
>>> Y=dict(X)
>>> Z=list(Y)
>>> Z==X

False
>>> Z==Y.keys()

True

That list(Y) returns the keys of X is perhaps unfortunate, but the same
behavior allows you to write:

for key in Y:
print (key, y[key])

which can be an efficiency gain if Y is large (not having to build a list of
all the keys ahead of time). You'll find this to be true though:

W = Y.items()
W.sort()
W == X

Elaine> 2) How come you can't write...

Elaine> if y = f(x) > 3:
Elaine> <do a bunch of things with x and y>

There is a common class of errors in C code:

if (c = 0) {
...
}

Is that supposed to be an assignment or a test? Python avoids that problem
by not allowing assignments within expressions.

Elaine> That kind of syntax would be especially welcome in list
Elaine> comprehensions:

Elaine> [f(x,y) for x in list if y=g(x)<=0]

I think you can recast that as:

[f(x,y) for (x,y) in zip(list, [g(z) for z in list]) if y <= 0]

but do you really want to? (This won't work if list is an iterator.)

Elaine> If g(x) is a fairly complicated expression, and y occurs several
Elaine> times in f(x,y), considerable clarity could be gained.

If g(x) is a fairly complicated expression and y occurs several times in
f(x,y), perhaps you should be using a for loop instead of a list
comprehension:

result = []
for x in list:
y = g(x)
if y <= 0:
result.append(f(x,y))
return result

(FYI, you shouldn't use "list" as a variable name. You're shadowing a
builtin, a practice that can lead to confusing error messages, if nothing
else.)

Skip

Skip Montanaro, Feb 4, 2004
7. ### Peter OttenGuest

Dang Griffith wrote:

> On Wed, 04 Feb 2004 19:46:28 +0100, Peter Otten <>
> wrote:
>
>>> That kind of syntax would be especially welcome in list comprehensions:
>>>
>>> [f(x,y) for x in list if y=g(x)<=0]
>>>
>>> If g(x) is a fairly complicated expression, and y occurs several times
>>> in f(x,y), considerable clarity could be gained.

>>
>>Is the above list comprehension that frequent? Then how about
>>
>>[f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]
>>
>>With the arrival of generator expressions, some of the overhead (the
>>intermediate list) is bound to go away. In the mean time, there's still
>>that good old for loop, which IMHO is still the most readible solution if
>>things get really complicated.

>
> I couldn't get your example to run.

Checking...

Python 2.3.3 (#1, Jan 3 2004, 13:57:08)
[GCC 3.2] on linux2
>>> def f(x, y):

.... return '%d%d' % (x, y)
....
>>> def g(x): return 2*x

....
>>> def cond(x): return x % 10 == 2

....
>>> lst = range(10)
>>> [f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]

['12', '612']
>>>

Seems to work. Maybe you overlooked the nested list comps?

> If g is a generator expression, this works for me:
>
> [f(x, y) for x in lst for y in g(x) if cond(y)]

This is elegant.

Peter

Peter Otten, Feb 4, 2004
8. ### Elaine JacksonGuest

Thanks to everyone who responded. Digging into this matter of list
comprehensions a bit more, I turned up some interesting facts. Consider the
following example:

>>> f = lambda x,y: x*y+2*pow(y,2)
>>> g = lambda x: 3*pow(x,2)
>>> list = [1,2,3,4,5]

The kind of expression I have in mind would be equivalent to...

>>> [f(x,y) for (x,y) in [(x,g(x)) for x in list] if y>19]

[1539, 4800, 11625]

....which is equivalent to...

>>> [f(x,y) for x in list for y in [g(x)] if y>19]

[1539, 4800, 11625]

....but it's not equivalent to...

>>> [f(x,y) for y in [g(x)] if y>19 for x in list]

[11325, 11400, 11475, 11550, 11625]

....or to...

>>> [f(x,y) for y in [g(x)] for x in list if y>19]

[11325, 11400, 11475, 11550, 11625]

I imagine a good explanation of this discrepancy can only be gotten by dipping
into the documentation and finding out what an "iterator" is, and that's
unfortunate (imho), because it means that list comprehensions fail to
"intuitively suggest the proper meaning to a human reader who has not yet been
introduced to the construct" (quote taken from the FAQ).

del list
## Peace

"Skip Montanaro" <> wrote in message
news:...
|
| Elaine> 1) I find the following behavior puzzling and disappointing:
|
| >>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
| >>> Y=dict(X)
| >>> Z=list(Y)
| >>> Z==X
| False
| >>> Z==Y.keys()
| True
|
| That list(Y) returns the keys of X is perhaps unfortunate, but the same
| behavior allows you to write:
|
| for key in Y:
| print (key, y[key])
|
| which can be an efficiency gain if Y is large (not having to build a list of
| all the keys ahead of time). You'll find this to be true though:
|
| W = Y.items()
| W.sort()
| W == X
|
| Elaine> 2) How come you can't write...
|
| Elaine> if y = f(x) > 3:
| Elaine> <do a bunch of things with x and y>
|
| There is a common class of errors in C code:
|
| if (c = 0) {
| ...
| }
|
| Is that supposed to be an assignment or a test? Python avoids that problem
| by not allowing assignments within expressions.
|
| Elaine> That kind of syntax would be especially welcome in list
| Elaine> comprehensions:
|
| Elaine> [f(x,y) for x in list if y=g(x)<=0]
|
| I think you can recast that as:
|
| [f(x,y) for (x,y) in zip(list, [g(z) for z in list]) if y <= 0]
|
| but do you really want to? (This won't work if list is an iterator.)
|
| Elaine> If g(x) is a fairly complicated expression, and y occurs several
| Elaine> times in f(x,y), considerable clarity could be gained.
|
| If g(x) is a fairly complicated expression and y occurs several times in
| f(x,y), perhaps you should be using a for loop instead of a list
| comprehension:
|
| result = []
| for x in list:
| y = g(x)
| if y <= 0:
| result.append(f(x,y))
| return result
|
| (FYI, you shouldn't use "list" as a variable name. You're shadowing a
| builtin, a practice that can lead to confusing error messages, if nothing
| else.)
|
| Skip
|

Elaine Jackson, Feb 5, 2004
9. ### Rich KrauterGuest

Are some of these examples only working because and y have been
initialized by previous runs?
I tried deleting x and y between list comprehensions and they don't work
anymore. Maybe I'm missing the point.

Rich

On Wed, 2004-02-04 at 19:22, Elaine Jackson wrote:
> Thanks to everyone who responded. Digging into this matter of list
> comprehensions a bit more, I turned up some interesting facts. Consider the
> following example:
>
> >>> f = lambda x,y: x*y+2*pow(y,2)
> >>> g = lambda x: 3*pow(x,2)
> >>> list = [1,2,3,4,5]

>
> The kind of expression I have in mind would be equivalent to...
>
> >>> [f(x,y) for (x,y) in [(x,g(x)) for x in list] if y>19]

> [1539, 4800, 11625]
>
> ...which is equivalent to...
>
> >>> [f(x,y) for x in list for y in [g(x)] if y>19]

> [1539, 4800, 11625]
>
> ...but it's not equivalent to...
>
> >>> [f(x,y) for y in [g(x)] if y>19 for x in list]

> [11325, 11400, 11475, 11550, 11625]
>
> ...or to...
>
> >>> [f(x,y) for y in [g(x)] for x in list if y>19]

> [11325, 11400, 11475, 11550, 11625]
>
> I imagine a good explanation of this discrepancy can only be gotten by dipping
> into the documentation and finding out what an "iterator" is, and that's
> unfortunate (imho), because it means that list comprehensions fail to
> "intuitively suggest the proper meaning to a human reader who has not yet been
> introduced to the construct" (quote taken from the FAQ).
>
> del list
> ## Peace
>
> "Skip Montanaro" <> wrote in message
> news:...
> |
> | Elaine> 1) I find the following behavior puzzling and disappointing:
> |
> | >>> X=[(1,1),(2,4),(3,9),(4,16),(5,25)]
> | >>> Y=dict(X)
> | >>> Z=list(Y)
> | >>> Z==X
> | False
> | >>> Z==Y.keys()
> | True
> |
> | That list(Y) returns the keys of X is perhaps unfortunate, but the same
> | behavior allows you to write:
> |
> | for key in Y:
> | print (key, y[key])
> |
> | which can be an efficiency gain if Y is large (not having to build a list of
> | all the keys ahead of time). You'll find this to be true though:
> |
> | W = Y.items()
> | W.sort()
> | W == X
> |
> | Elaine> 2) How come you can't write...
> |
> | Elaine> if y = f(x) > 3:
> | Elaine> <do a bunch of things with x and y>
> |
> | There is a common class of errors in C code:
> |
> | if (c = 0) {
> | ...
> | }
> |
> | Is that supposed to be an assignment or a test? Python avoids that problem
> | by not allowing assignments within expressions.
> |
> | Elaine> That kind of syntax would be especially welcome in list
> | Elaine> comprehensions:
> |
> | Elaine> [f(x,y) for x in list if y=g(x)<=0]
> |
> | I think you can recast that as:
> |
> | [f(x,y) for (x,y) in zip(list, [g(z) for z in list]) if y <= 0]
> |
> | but do you really want to? (This won't work if list is an iterator.)
> |
> | Elaine> If g(x) is a fairly complicated expression, and y occurs several
> | Elaine> times in f(x,y), considerable clarity could be gained.
> |
> | If g(x) is a fairly complicated expression and y occurs several times in
> | f(x,y), perhaps you should be using a for loop instead of a list
> | comprehension:
> |
> | result = []
> | for x in list:
> | y = g(x)
> | if y <= 0:
> | result.append(f(x,y))
> | return result
> |
> | (FYI, you shouldn't use "list" as a variable name. You're shadowing a
> | builtin, a practice that can lead to confusing error messages, if nothing
> | else.)
> |
> | Skip
> |
>

Rich Krauter, Feb 5, 2004
10. ### Dan Dang GriffithGuest

Peter Otten <> wrote in message news:<bvrvsn\$pdn\$02\$-online.com>...
> Dang Griffith wrote:
>
> > On Wed, 04 Feb 2004 19:46:28 +0100, Peter Otten <>
> > wrote:
> >
> >>> That kind of syntax would be especially welcome in list comprehensions:
> >>>
> >>> [f(x,y) for x in list if y=g(x)<=0]
> >>>
> >>> If g(x) is a fairly complicated expression, and y occurs several times
> >>> in f(x,y), considerable clarity could be gained.
> >>
> >>Is the above list comprehension that frequent? Then how about
> >>
> >>[f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]
> >>
> >>With the arrival of generator expressions, some of the overhead (the
> >>intermediate list) is bound to go away. In the mean time, there's still
> >>that good old for loop, which IMHO is still the most readible solution if
> >>things get really complicated.

> >
> > I couldn't get your example to run.

>
> Checking...
>
> Python 2.3.3 (#1, Jan 3 2004, 13:57:08)
> [GCC 3.2] on linux2
> >>> def f(x, y):

> ... return '%d%d' % (x, y)
> ...
> >>> def g(x): return 2*x

> ...
> >>> def cond(x): return x % 10 == 2

> ...
> >>> lst = range(10)
> >>> [f(x, y) for x, y in [(x, g(x)) for x in lst] if cond(y)]

> ['12', '612']
> >>>

>
> Seems to work. Maybe you overlooked the nested list comps?
>
> > If g is a generator expression, this works for me:
> >
> > [f(x, y) for x in lst for y in g(x) if cond(y)]

>
> This is elegant.
>
> Peter

OIC--you mentioned generators and I defined g as yield 2*x.
You're right--yours works with non-generator, mine works with generator.

Maybe this should be a new thread, but... is defining a generator
like I did here, i.e. one that really only returns one value considered
an abuse of generators? It sure seemed convenient for this application.
def g(x): yield 2*x
--dang

Dan Dang Griffith, Feb 5, 2004
11. ### Peter OttenGuest

Dan Dang Griffith wrote:

> Maybe this should be a new thread, but... is defining a generator
> like I did here, i.e. one that really only returns one value considered
> an abuse of generators? It sure seemed convenient for this application.
> def g(x): yield 2*x

Now you're asking it - yes, I was a little confused at first. I even removed
the condition to provoke the m x n behaviour of the two for statements,
expecting 100 items in the resulting list - and only then realized n=1.
Also if g() really were something complicated as specified by the OP, you
wouldn't want to limit it to for loops and list comprehensions. This could
be remedied by a little adapter:

>>> def makeGen(f):

.... def g(*x): yield f(*x)
.... return g
....

>>> [(x, y) for x in range(3) for y in makeGen(g)(x)]

[(0, 0), (1, 2), (2, 4)]

However, the resulting code is no longer beautiful, and I'd probably rather
reintroduce a temporary list:

>>> [(x, y) for x in range(3) for y in [g(x)]]

[(0, 0), (1, 2), (2, 4)]

My personal conclusion: use only list comprehensions containing one for,
stick to good old for loops for the more complicated things.

Peter

PS: I mentioned generator expressions as of PEP 289, see
www.python.org/peps/pep-0289.html

Peter Otten, Feb 7, 2004