a few extensions for the itertools

[Mathias Panzenboeck]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

isum(iterable, start=0) -> value
Returns the sum of the elements of a iterable
plus the value of parameter 'start'. When the
iterable is empty, returns start.


iproduct(iterable, start=0) -> value
Returns the product of the elements of a iterable
times the value of parameter 'start'. When the
iterable is empty, returns start.


forall(predicate, iterable, default=True) -> bool
Returns True, when for all elements x in iterable
predicate(x) is True. When the iterable is empty,
returns default.


forany(predicate, iterable, default=False) -> bool
Returns True, when for any element x in iterable
predicate(x) is True. When the iterable is empty,
returns default.


take(n,iterable) -> iterator
returns a iterator over the first n
elements of the iterator


drop(n,iterable) -> iterable
drops the first n elemetns of iterable and
return a iterator over the rest


heads(iterable) -> iterator over all heads
example:
for head in heads([1,2,3,4,5,6,7,8,9]):
print head

output:
[]
[1]
[1, 2]
[1, 2, 3]
[1, 2, 3, 4]
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5, 6]
[1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7, 8]
[1, 2, 3, 4, 5, 6, 7, 8, 9]


tails(iterable) -> iterator over all tails
example:
for tail in tails([1,2,3,4,5,6,7,8,9]):
print tail

output:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
[2, 3, 4, 5, 6, 7, 8, 9]
[3, 4, 5, 6, 7, 8, 9]
[4, 5, 6, 7, 8, 9]
[5, 6, 7, 8, 9]
[6, 7, 8, 9]
[7, 8, 9]
[8, 9]
[9]
[]


fcain(funct,*functs) -> function(...,***)
fcain(f1,f2,...,fn)(*args,*kwargs) equals f1(f2(...fn(*args,*kwargs)))

 

[Paul McGuire]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

Just a couple of questions:

iproduct(iterable, start=0) -> value
Returns the product of the elements of a iterable
times the value of parameter 'start'. When the
iterable is empty, returns start.

Wouldn't 1 be a better default value for start?

forall(predicate, iterable, default=True) -> bool
Returns True, when for all elements x in iterable
predicate(x) is True. When the iterable is empty,
returns default.


forany(predicate, iterable, default=False) -> bool
Returns True, when for any element x in iterable
predicate(x) is True. When the iterable is empty,
returns default.

How are these different from all and any in Python 2.5?

-- Paul

 

[Steven D'Aprano]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

What does it mean for an iterable to be less than another iterable? That
it has fewer items? How do these two iterables compare?

iter([1, 2, None, "foo", 3+2j])

def ones():
while 1:
yield 1

Which is smaller?

isum(iterable, start=0) -> value
Returns the sum of the elements of a iterable
plus the value of parameter 'start'. When the
iterable is empty, returns start.

You mean just like the built-in sum()?

1066


iproduct(iterable, start=0) -> value
Returns the product of the elements of a iterable
times the value of parameter 'start'. When the
iterable is empty, returns start.

If I recall, product() was requested about the same time that sum() was
introduced, and Guido rejected it as a built-in because it was really only
useful for calculating geometric means, and it is easy to do if you need
it:

def product(it, start=1):
# default value of 1 is more sensible than 0
# 1 is the multiplicative identity
p = start
for x in it:
p *= x
return p

forall(predicate, iterable, default=True) -> bool
Returns True, when for all elements x in iterable
predicate(x) is True. When the iterable is empty,
returns default.


forany(predicate, iterable, default=False) -> bool
Returns True, when for any element x in iterable
predicate(x) is True. When the iterable is empty,
returns default.

I vaguely recall plans for all() and any() builtins -- perhaps for Python
2.5?

take(n,iterable) -> iterator
returns a iterator over the first n
elements of the iterator

Just like itertools.islice(iterable, n).
[0, 1, 2, 3, 4]

drop(n,iterable) -> iterable
drops the first n elemetns of iterable and
return a iterator over the rest

Just like itertools.islice(iterable, n, None)
[15, 16, 17, 18, 19]

(Aside: I think islice would be so much cleaner if it took keyword
arguments.)

heads(iterable) -> iterator over all heads
tails(iterable) -> iterator over all tails

What would you use these for?

fcain(funct,*functs) -> function(...,***)
fcain(f1,f2,...,fn)(*args,*kwargs) equals f1(f2(...fn(*args,*kwargs)))

The usual term for this is function composition.

 

[Carl Banks]

Just a couple of questions:

Wouldn't 1 be a better default value for start?

I concur; start should default to 1.

How are these different from all and any in Python 2.5?

1. These functions apply a predicate to the items. It's simple enough
to do with any/all and a genexp, but by the same argument, it's simple
enough to do imap and ifilter with a plain genexp.
2. They have default values. Default values for any and all don't make
sense, and I don't think they make sense here, either. All of nothing
is always True; any of nothing is always False.


Carl Banks

 

[Gabriel Genellina]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

isum(iterable, start=0) -> value
Returns the sum of the elements of a iterable
plus the value of parameter 'start'. When the
iterable is empty, returns start.

Isn't the same as the builtin sum?

iproduct(iterable, start=0) -> value

As others said, start should be 1

fcain(funct,*functs) -> function(...,***)
fcain(f1,f2,...,fn)(*args,*kwargs) equals
f1(f2(...fn(*args,*kwargs)))

I don't understand it, nor even the signature. Perhaps it tries to be
"fchain", function composition? But what has it to do with iterables?


--
Gabriel Genellina
Softlab SRL

__________________________________________________
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

 

[George Sakkis]

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

What does it mean for an iterable to be less than another iterable? That
it has fewer items? How do these two iterables compare?

iter([1, 2, None, "foo", 3+2j])

def ones():
while 1:
yield 1

Which is smaller?

Haven't checked the specific implementation, but I would expect it to
behave like sequences of the same type, i.e. first compare the first
elements of the iterables; if they are equal compare the second
elements, and so on, until the first inequality or until the shorter
one ends. In your example, the second iterable is smaller. Needless to
say, you'd better not compare an infinite iterable with itself ;-)

drop(n,iterable) -> iterable
drops the first n elemetns of iterable and
return a iterator over the rest

Just like itertools.islice(iterable, n, None)
[15, 16, 17, 18, 19]

(Aside: I think islice would be so much cleaner if it took keyword
arguments.)

How about slice notation ? I just posted in the Cookbook an OO wrapper
of itertools that, among other functions, uses slice notation for
islice and "+" for chain. Admittedly, my proposal in the py-3k list to
make iter() return itertools-enabled iterators was overwhelmingly shot
down, but I still like it anyway. FWIW, here's the Cookbook link:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498272

George

 

[Duncan Booth]

take(n,iterable) -> iterator
returns a iterator over the first n
elements of the iterator

Isn't this just the same as itertools.islice(iterable, n) ?

drop(n,iterable) -> iterable
drops the first n elemetns of iterable and
return a iterator over the rest

and this looks to be the same as itertools.islice(iterable, n, None)

Can you give use cases for 'heads' and 'tails'? I'm curious why you would
want them.

 

[Mathias Panzenboeck]

> At Sunday 19/11/2006 17:35, Mathias Panzenboeck wrote:
>
>
> Isn't the same as the builtin sum?
>

No, because the builtin sum want's a list. This can also handle any kind of iterable, so this would
work:

isum(i**2 for i in xrange(100))

sum would need firs the whole list to be generated:

sum([i**2 for i in xrange(100)])

>
> As others said, start should be 1
>

Indeed. Can't believe I made that mistake... the mistake is only in the documentation.

>
> I don't understand it, nor even the signature. Perhaps it tries to be
> "fchain", function composition? But what has it to do with iterables?
>

Ups, missed out the 'h'. (Also only in the documentation.)

It's like the . operator in haskell:

fchain(f,g,h) is the same like lambda *args,**kwargs: f(g(h(*args,**kwargs)))

 

[Mathias Panzenboeck]

Isn't this just the same as itertools.islice(iterable, n) ?

ok, that's true.

and this looks to be the same as itertools.islice(iterable, n, None)

same here.

Can you give use cases for 'heads' and 'tails'? I'm curious why you would
want them.

I use them in haskell all the time. But in haskell the lists are all "generators".

In haskell you would implement naive string-search like this:

import List
findIndex (isPrefixOf "bla") (tails "dfvbdbblaesre")

 

[Mathias Panzenboeck]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

What does it mean for an iterable to be less than another iterable? That
it has fewer items? How do these two iterables compare?

iter([1, 2, None, "foo", 3+2j])

def ones():
while 1:
yield 1

Which is smaller?

it's like cmp on lists, but on iterables.

[1,2,3] < [1,2,4]
[1,2,3] < [1,2,3,0]
....

You mean just like the built-in sum()?

No, because the builtin sum can't handle iterables other than lists. Or dose it? Hmm, maby it dose
since any new version and I didn't mention it.

If I recall, product() was requested about the same time that sum() was
introduced, and Guido rejected it as a built-in because it was really only
useful for calculating geometric means, and it is easy to do if you need
it:

def product(it, start=1):
# default value of 1 is more sensible than 0
# 1 is the multiplicative identity
p = start
for x in it:
p *= x
return p




I vaguely recall plans for all() and any() builtins -- perhaps for Python
2.5?

all() and any() don't get predicate functions as arguments.

take(n,iterable) -> iterator
returns a iterator over the first n
elements of the iterator

Just like itertools.islice(iterable, n).
[0, 1, 2, 3, 4]

ok, ok, ok. I have overseen that.

drop(n,iterable) -> iterable
drops the first n elemetns of iterable and
return a iterator over the rest

Just like itertools.islice(iterable, n, None)
[15, 16, 17, 18, 19]

(Aside: I think islice would be so much cleaner if it took keyword
arguments.)

heads(iterable) -> iterator over all heads
tails(iterable) -> iterator over all tails

What would you use these for?

fcain(funct,*functs) -> function(...,***)
fcain(f1,f2,...,fn)(*args,*kwargs) equals f1(f2(...fn(*args,*kwargs)))

The usual term for this is function composition.

 

[Mathias Panzenboeck]

I concur; start should default to 1.

Bug in the documentation, not in the function.

 

[Fredrik Lundh]

No, because the builtin sum want's a list.

the one in Python doesn't:
.... for i in range(3):
.... print "yield", i
.... yield i
....yield 0
yield 1
yield 2
3

</F>

 

[Duncan Booth]

No, because the builtin sum want's a list. This can also handle any
kind of iterable, so this would work:

isum(i**2 for i in xrange(100))

sum would need firs the whole list to be generated:

sum([i**2 for i in xrange(100)])
Really?

2666666466666670000000L

seems to work fine, and judging by the memory usage it pretty obviously
doesn't create an intermediate list.

 

[Fredrik Lundh]

No, because the builtin sum can't handle iterables other than lists.
> Or dose it? Hmm, maby it dose since any new version and I didn't
> mention it.

sum() was added in 2.3, and has always supported arbitrary iterables.

</F>

 

[Steven D'Aprano]

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

What does it mean for an iterable to be less than another iterable? That
it has fewer items? How do these two iterables compare?

iter([1, 2, None, "foo", 3+2j])

def ones():
while 1:
yield 1

Which is smaller?

it's like cmp on lists, but on iterables.

[1,2,3] < [1,2,4]
[1,2,3] < [1,2,3,0]

But that meaningless, as far as I can see. Lists and iterators aren't the
same thing. A list is a collection; an iterator is not, but it can be
accumulated into a collection.

If equality is meaningful for an object, you should be able to test for
equality without changing the object. But that isn't true for iterators.

Worse, because comparing an iterator consumes items, you can easily get
crazy results like the following:

L = [1, 2, 3, 4, 5]
def it():

.... yield 5; yield 0; yield 0; yield 0; yield 0
....
Now, we can compare the first item of L with the first item of it:

L[0] < it.next() # if True, L < it

True

So L must be less than it, right? Let's print both objects out in full to
check:

print L [1, 2, 3, 4, 5]
print list(it) # if L < it, it must be > L

[0, 0, 0, 0]

Oops.

I think the GENERAL concept of comparison between iterators is
meaningless. A lazy comparison between the items of an iterator and some
other iterable may be a useful thing to do, but as a general concept,
saying that an iterator compares bigger or smaller or equal to something
else doesn't make sense, since the mere fact that you make that comparison
will change the iterator.

 

[Carl Banks]

Worse, because comparing an iterator consumes items, you can easily get
crazy results like the following:

[snip]

However, that doesn't stop the "in" operator:
[4]

I'm not sure if I like it, but at least it's probably a lot more useful
than icmp.

Carl Banks

 

[Roberto Bonvallet]

all() and any() don't get predicate functions as arguments.

all(predicate(x) for x in iterable)
any(predicate(x) for x in iterable)

 

[Mathias Panzenboeck]

On Sun, 19 Nov 2006 21:35:24 +0100, Mathias Panzenboeck wrote:

I wrote a few functions which IMHO are missing in python(s itertools).

You can download them here:
http://sourceforge.net/project/showfiles.php?group_id=165721&package_id=212104

A short description to all the functions:

icmp(iterable1, iterable2) -> integer
Return negative if iterable1 < iterable2,
zero if iterable1 == iterable1,
positive if iterable1 > iterable1.

What does it mean for an iterable to be less than another iterable? That
it has fewer items? How do these two iterables compare?

iter([1, 2, None, "foo", 3+2j])

def ones():
while 1:
yield 1

Which is smaller?

it's like cmp on lists, but on iterables.

[1,2,3] < [1,2,4]
[1,2,3] < [1,2,3,0]

But that meaningless, as far as I can see. Lists and iterators aren't the
same thing. A list is a collection; an iterator is not, but it can be
accumulated into a collection.

If equality is meaningful for an object, you should be able to test for
equality without changing the object. But that isn't true for iterators.

Worse, because comparing an iterator consumes items, you can easily get
crazy results like the following:

L = [1, 2, 3, 4, 5]
def it():

... yield 5; yield 0; yield 0; yield 0; yield 0
...
Now, we can compare the first item of L with the first item of it:

L[0] < it.next() # if True, L < it

True

So L must be less than it, right? Let's print both objects out in full to
check:

print L [1, 2, 3, 4, 5]
print list(it) # if L < it, it must be > L

[0, 0, 0, 0]

Oops.

I think the GENERAL concept of comparison between iterators is
meaningless. A lazy comparison between the items of an iterator and some
other iterable may be a useful thing to do, but as a general concept,
saying that an iterator compares bigger or smaller or equal to something
else doesn't make sense, since the mere fact that you make that comparison
will change the iterator.

I see. Yes, in general you are right, but I implemented it to use it for that (and similar things):

icmp(open("foo.txt"),open("bar.txt"))

Thats cool, I think.


panzi

 

[Mathias Panzenboeck]

all(predicate(x) for x in iterable)
any(predicate(x) for x in iterable)

Very true, but 2.4 is still very common and forall(perdicate,iterable) is less to write.
(And haskell has both, to. all == and, forall == all, ...)

Well, but I think you are right. Don't reinvent the wheel, I guess.


panzi

 

[Mathias Panzenboeck]

No, because the builtin sum want's a list. This can also handle any
kind of iterable, so this would work:

isum(i**2 for i in xrange(100))

sum would need firs the whole list to be generated:

sum([i**2 for i in xrange(100)])
Really?

sum(i**2 for i in xrange(20000000))

2666666466666670000000L

seems to work fine, and judging by the memory usage it pretty obviously
doesn't create an intermediate list.

Very strange. I must have made some strange error. I tried that (more than once) and it failed.
Don't know why. But now it works here, too. What did I write, when it failed?