PEP 289: Generator Expressions (please comment)

[rm]

Toward the end of the PEP, we have the following:

combined with >reduction functions like sum(), min(), and max().
Separate proposals are >forthcoming that recommend several new
accumulation functions possibly >including: product(), average(),
alltrue(), anytrue(), nlargest(), nsmallest().


This is great. I'd like to request that alltrue() and anytrue() be
renamed forall() and exists(), repsectively. It would allow:

def prime(x):
return forall(x%y for y in xrange(2, math.sqrt(x)))

etc. This is a an opportunity to be mathematically precise, and I
think we should be. Python is going to have the universal and
existential operators in a very general syntax, so let's celebrate it
a little.

later,
Jess

maybe allfalse() should be included aswell?

rm

 

[Mark Hahn]

Python doesn't use iftrue and whentrue, so why should it use alltrue and
anytrue? I vote for all and any.

 

[Erik Max Francis]

Python doesn't use iftrue and whentrue, so why should it use alltrue
and
anytrue? I vote for all and any.

Well said.

 

[Jess Austin]

maybe allfalse() should be included aswell?

rm

allfalse() isn't necessary; it's equivalent to "not any()", "not
exists()", "not anytrue()", or whatever depending on what is decided
for the existential form. Similarly, anyfalse() is equivalent to "not
all()", "not forall()", etc. Actually, that last example sounds
terrible, so I'm withdrawing my "forall" suggestion for the universal
form.

In the interests of orthogonality, I'd advise against the {any,
all}false() forms. It seems like the consensus is "all" and "any".
This makes sense to me, and I think this will be a great addition to
the language.

later,
Jess

 

[Michele Simionato]

It seems like the consensus is "all" and "any".

Not necessarely. I prefer "alltrue" and "anytrue", for instance.

Michele

 

[rm]

(e-mail address removed) (Jess Austin) wrote >

It seems like the consensus is "all" and "any".

which exists as well in SQL

e.g.

select ...
from ...
where x > any ( sub-select )

or

select ...
from ...
where x > all ( sub-select )

but SQL supports the following as well

select ...
from ...
where exists ( sub-select )

bye,
rm

 

[Alex Martelli]

(e-mail address removed) (Jess Austin) wrote >

which exists as well in SQL

....with a subtly different meaning...

where x > any ( sub-select )

I can just imagine the fun somebody's going to have after
coding, e.g.:

while x > any( ... ) :
...

where exists ( sub-select )

That's another function, in Python terms. We could
code the SQL any, all and exists (taking any iterator or
iterable argument) roughly as follows:


class any(object):

def __init__(self, it):
" invoked by any(it) "
self.it = iter(it)

def __lt__(self, x):
" invoked by x > any(it) &c "
for item in self.it:
if x > item: return True
else: return False
# assuming x > any([]) is to be false

# and similarly for other comparisons


class all(object):

def __init__(self, it):
" invoked by all(it) "
self.it = iter(it)

def __lt__(self, x):
" invoked by x > all(it) &c "
for item in self.it:
if not (x > item): return False
else: return True
# assuming x > all([]) is to be true

# and similarly for other comparisons


def exists(it):
" invoked by exists(it) "
for item in it: return True
else: return False
# assuming exists([]) is to be false


I don't think the SQL semantics for any and all merge _well_ with
those that people want around here, though it would technically be
possible to also define a __nonnull__ in each class to do the
obvious checks -- I think it would be rather confusing to allow
both "if x > all(...):" and "if all(...):" with 'all' meaning such
different things in the two cases. Similarly for 'any'. And
'exists' to mean 'has some true items' is of course very different
from the SQL meaning of 'exists' to mean "it's not empty"...!

I'd just love to have such meanings for any, all, and exists, but
I believe it's not going happen -- because people around here don't
like the spelling 'alltrue' to mean "all items are true" and so on.
Ah well, there will be other ways to spell similar semantics,
though goofy rather than elegant ones -- e.g., instead of:

if x > all(theitems):

we'll just have to code:

if all(x>item for item in theitems):


In any case, the SQL meaning isn't extensible to Python's chaining
of comparisons when we only have an iterator (not an iterable) --
in that case,

if x > all(itemsiterator) > y :

can't be made to mean what we'd obviously _like_ it to mean
without spending the memory to save all the items as they go,
since it's evaluated equivalently to:

_aux = all(itemsiterator)

if (x > _aux) and (_aux > y) :
...

and if it comes to the comparison after the implied and, the
itemsiterator is exhausted by that time. So, maybe, it _is_
for the better that the apparently popular preference for 'all'
to mean 'alltrue' stops us from having 'all' work like in SQL
instead - it does save us from potentially non-obvious behavior
when both comparison chaining and iterators-as-opposed-to-
iterables are simultaneously involved.


Alex

 

[Alex Martelli]

(e-mail address removed) (Jess Austin) wrote in message


Not necessarely. I prefer "alltrue" and "anytrue", for instance.

Me too, FWIW. But more important is, where are these operators
going to _live_ -- the builtins are definitely _out_ , judging by
Guido's recent reactions.

I think we need a companion module to itertools. The functions
in itertools *produce* useful iterators; we need a module for
functions that usefully *consume* iterators. Yes, a few, in each
case, judged more useful than others, are built-ins (iter,
enumerate, reversed as producers; sum, min, and max as consumers).

But, most others won't be -- so we have itertools and need [good
name for "iterator consumers" sought -- accumulators?). This, btw,
will usefully afford e.g.
from accumulators import any as anytrue
or viceversa, mitigating the effect of a dlsliked name being chosen.


Other candidates (names can surely be enhanced...) in random order:

average(numbers)
average_and_variance(numbers) # returns tuple -- average, variance
median(numbers)

highest(N, numbers) # return list of N highest numbers
lowest(N, numbers) # ditto for lowest
# note: highest and lowest might get optional key= -- like lists' sort
# method has added in 2.4

take(N, numbers) # ditto for the first N
exists(xs) # true if xs is non-empty

repeatcall(f, xs, result=None)
# like: for x in xs: f(x)
return result
# typical uses:
# # just concatenate a lots of sequences into list allitems
# allitems = []
# repeatcall(allitems.extend, lotsofsequences)
# # merge a lot of dictionaries and loop on merged result dict d
# d = {}
# for k in repeatcall(d.update, ds, d): ...


Alex

 

[Will Stuyvesant]

[Jess Austin]

...It seems like the consensus is "all" and "any".

Oh no! Their mathematical names are "forall" and "exists" and that is
what they should be IMO. Any programmer I can think of is familiar
with "forall" and "exists" in mathematics.

 

[Georgy Pruss]

If it's a vote, I'm voting for 'all' and 'any'. They are short, natural and symmetric.
G-:

[Jess Austin]
...It seems like the consensus is "all" and "any".

 

[Bruno Desthuilliers]

http://www.python.org/peps/pep-0289.html

In brief, the PEP proposes a list comprehension style syntax for
creating fast, memory efficient generator expressions on the fly:

sum(x*x for x in roots)
min(d.temperature()*9/5 + 32 for d in days)
Set(word.lower() for word in text.split() if len(word) < 5)
dict( (k, somefunc(k)) for k in keylist )
dotproduct = sum(x*y for x,y in itertools.izip(xvec, yvec))
bestplayer, bestscore = max( (p.score, p.name) for p in players )

Each of the above runs without creating a full list in memory,
which saves allocation time, conserves resources, and exploits
cache locality.

+3.1415926535897931 for me.

Bruno

 

[John J. Lee]

[Jess Austin]
...It seems like the consensus is "all" and "any".

Oh no! Their mathematical names are "forall" and "exists" and that is
what they should be IMO. Any programmer I can think of is familiar
with "forall" and "exists" in mathematics.

Far from all Python programmers have been exposed to quantificational
logic.


John

 

[John J. Lee]

(e-mail address removed) (Jess Austin) wrote in message

Not necessarely. I prefer "alltrue" and "anytrue", for instance.

+1


John

 

[Greg Ewing (using news.cis.dfn.de)]

Oh no! Their mathematical names are "forall" and "exists" and that is
what they should be IMO.

Actually, their mathematical names are an upside-down "A"
and an upside-down "E", which are often pronounced "for
all" and "there exists", but other pronunciations are
possible.

In Python, "all" and "any" would be closest to the
conciseness of the mathematical symbols, IMO. Until
we get to use Unicode in identifiers, anyway...

 

[John Roth]

Actually, their mathematical names are an upside-down "A"
and an upside-down "E", which are often pronounced "for
all" and "there exists", but other pronunciations are
possible.

In Python, "all" and "any" would be closest to the
conciseness of the mathematical symbols, IMO. Until
we get to use Unicode in identifiers, anyway...

Hasten the day!

As far as I'm concerned, 3.0 seems to be about as
likely as my seeing a real unicorn.

John Roth

 

[Magnus Lie Hetland]

Actually, their mathematical names are an upside-down "A"
and an upside-down "E", which are often pronounced "for
all" and "there exists", but other pronunciations are
possible.

While we're nit-picking, I'd like to point out that the "E" isn't
upside-down -- it's mirrored (or rotated 180 degrees or whatever), so
it's horizontal edges point left.

As for the "forall" and "exists" -- these are the LaTeX names for
these symbols. Given that Guido has already used LaTeX as a motivation
for enumerate(), that might speak in favor of these names (even though
"all" and "any" seem better to me too...)