reduce to be removed?

[Dustan]

According to the following page on Wikipedia:
http://en.wikipedia.org/wiki/Python_(programming_language)#Future_development
reduce is going to be removed in python 3.0. It talks of an
accumulation loop; I have no idea what that's supposed to mean. So,

===============================[[1,2,3],
[4,5,6],
[7,8,9]]

reduce(lambda a,b:a+b, x, [])

[1, 2, 3, 4, 5, 6, 7, 8, 9]
===============================

What's an accumulation loop, and how would I convert this code so it's
compatible with the future 3.0 (preferably in a short sweet expression
that I can embed in a list comprehension)?

 

[Fredrik Lundh]

What's an accumulation loop, and how would I convert this code so it's
compatible with the future 3.0

the release of Python 3.0 is far away, and nobody knows how it's going
to look. trying to be future-compatible at this time is a major waste
of time and (not quite as wasteful as reopening yet another old "let's
make some pointless change to the language" thread, but almost).

surely you must have something better to do with your time ?

</F>

 

[Dustan]

the release of Python 3.0 is far away, and nobody knows how it's going
to look. trying to be future-compatible at this time is a major waste
of time and (not quite as wasteful as reopening yet another old "let's
make some pointless change to the language" thread, but almost).

surely you must have something better to do with your time ?

</F>

It's always nice to know there are such good-natured people ready to
help on this group. Anyway, I figured out a way to get the builtin
function 'sum' to work as I need:
sum([[1,2,3],[4,5,6],[7,8,9]], [])

On an unrelated note, can anyone explain this unpredictable behavior on
IDLE? I swear I never hit ctrl-c...
==============================Help on built-in function sum in module __builtin__:

sum(...)
sum(sequence, start=0) -> value

Returns the sum of a sequence of numbers (NOT strings) plus the
value
of parameter 'start'. When the sequence is empty, returns start.
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
help(sum)
File "C:\Python25\lib\site.py", line 346, in __call__
return pydoc.help(*args, **kwds)
File "C:\Python25\lib\pydoc.py", line 1642, in __call__
self.help(request)
File "C:\Python25\lib\pydoc.py", line 1687, in help
self.output.write('\n')
File "C:\Python25\lib\idlelib\PyShell.py", line 1246, in write
self.shell.write(s, self.tags)
File "C:\Python25\lib\idlelib\PyShell.py", line 1235, in write
raise KeyboardInterrupt
KeyboardInterrupt
==============================

 

[Virgil Dupras]

According to the following page on Wikipedia:
http://en.wikipedia.org/wiki/Python_(programming_language)#Future_development
reduce is going to be removed in python 3.0. It talks of an
accumulation loop; I have no idea what that's supposed to mean. So,

===============================

x =\

[[1,2,3],
[4,5,6],
[7,8,9]]

reduce(lambda a,b:a+b, x, [])

[1, 2, 3, 4, 5, 6, 7, 8, 9]
===============================

What's an accumulation loop, and how would I convert this code so it's
compatible with the future 3.0 (preferably in a short sweet expression
that I can embed in a list comprehension)?

itertools.chain or sum(x,[])

 

[Fredrik Lundh]

It's always nice to know there are such good-natured people ready to
help on this group.

any special reason why you keep pretending that some random wikipedia
editor knows more about a future Python release than the people that
develops Python ?

Anyway, I figured out a way to get the builtin
function 'sum' to work as I need:
sum([[1,2,3],[4,5,6],[7,8,9]], [])

sum() is designed for adding numbers, not sequences. abusing it
for sequences leads to inefficient code, and extremely bad worst-
case behaviour, since you end up copying the same data over and
over and over again -- the function even checks for strings for
this very reason:

>>> sum(["123", "456", "789"], "")

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: sum() can't sum strings [use ''.join(seq) instead]

(maybe it should check for other well-known containers as well?)

if you care about writing robust code, why not just use a for-loop,
and the list extend method?

</F>

 

[Dustan]

any special reason why you keep pretending that some random wikipedia
editor knows more about a future Python release than the people that
develops Python ?

Be careful how you word that - this is the first time I've ever
referenced wikipedia in a question on this forum.

Anyway, I figured out a way to get the builtin
function 'sum' to work as I need:
sum([[1,2,3],[4,5,6],[7,8,9]], [])

sum() is designed for adding numbers, not sequences. abusing it
for sequences leads to inefficient code, and extremely bad worst-
case behaviour, since you end up copying the same data over and
over and over again

Thanks for some quality feedback for a change. Why can't you do that
more often?

-- the function even checks for strings for
this very reason:

sum(["123", "456", "789"], "")

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: sum() can't sum strings [use ''.join(seq) instead]

(maybe it should check for other well-known containers as well?)

if you care about writing robust code, why not just use a for-loop,
and the list extend method?

Because I'm embedding this expression in a list comprehension (as I
stated in my original post), and last time I checked, it's not possible
to treat a for-loop as an expression (which is what a list
comprehension requires).

 

[Fredrik Lundh]

> Because I'm embedding this expression in a list comprehension

because?

</F>

 

[Dustan]

because?

</F>

Because I thought I would be able to get an answer without revealing
the exact details of what I am doing. I didn't realize that wasn't an
option. I'll try once more to give you an idea of what I'm trying to
accomplish without letting on the details.
[[[1,2,3],[4,5,6],[7,8,9]],
[[3,2,1],[6,5,4],[9,8,7]]]

Here, foo appears to be a 3-dimensional list - except it's supposed to
be 2-dimensional. The inner-list-of-lists is a result of how I'm
producing the data, and now I want to do a mass-concatenation (or
extending) of the inner-list-of-lists, and come up with this:

foo == [[1,2,3,4,5,6,7,8,9],[3,2,1,6,5,4,9,8,7]]

True

What's the best way to accomplish this?

It's not quite this simple, but let's just see what you can come up
with the information at hand, and I'll see if I can adapt it to my
needs.

 

[Fredrik Lundh]

>
Because I thought I would be able to get an answer without revealing
the exact details of what I am doing.

alright, let's try again: why do you need a self-contained reduce
replacement that can be embedded inside a list comprehension ?

</F>

 

[Dustan]

alright, let's try again: why do you need a self-contained reduce
replacement that can be embedded inside a list comprehension ?

</F>

[[[1,2,3],[4,5,6],[7,8,9]],
[[3,2,1],[6,5,4],[9,8,7]]]

Here, foo appears to be a 3-dimensional list - except it's supposed to
be 2-dimensional. The inner-list-of-lists is a result of how I'm
producing the data, and now I want to do a mass-concatenation (or
extending) of the inner-list-of-lists, and come up with this result:

foo == [[1,2,3,4,5,6,7,8,9],[3,2,1,6,5,4,9,8,7]]

True

What's the best way to accomplish this?

It's not quite this simple, but let's just see what you can come up
with the information at hand, and I'll see if I can adapt it to my
needs.

 

[Robert Kern]

Fredrik Lundh wrote:

Because I'm embedding this expression in a list comprehension (as I
stated in my original post), and last time I checked, it's not possible
to treat a for-loop as an expression (which is what a list
comprehension requires).

As with all such things, you stick the implementation in a well-named function
and simply call the function everywhere. The implementation never needs to be a
one-liner expression.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Fredrik Lundh]

[[[1,2,3],[4,5,6],[7,8,9]],
[[3,2,1],[6,5,4],[9,8,7]]]

Here, foo appears to be a 3-dimensional list - except it's supposed to
be 2-dimensional. The inner-list-of-lists is a result of how I'm
producing the data, and now I want to do a mass-concatenation (or
extending) of the inner-list-of-lists, and come up with this result:

foo == [[1,2,3,4,5,6,7,8,9],[3,2,1,6,5,4,9,8,7]]

True

that still doesn't explain your "the expression must be used in a list
comprehension" requirement, though. assuming that the sizes are
varying, and at least sometimes a lot larger than 3x3, I'd probably
write the above as

for index, item in enumerate(foo):
this = []
for i in item:
this.extend(i)
foo[index] = this

which should be pretty efficient, since it avoids unnecessary function
calls, and is amortized linear time instead of O(N**2).

or, if I was in a hurry, and didn't really care if the inner sequences
were lists or tuples:

foo = map(Tkinter._flatten, foo)

</F>

 

[Steven D'Aprano]

alright, let's try again: why do you need a self-contained reduce
replacement that can be embedded inside a list comprehension ?

</F>

[[[1,2,3],[4,5,6],[7,8,9]],
[[3,2,1],[6,5,4],[9,8,7]]]

Here, foo appears to be a 3-dimensional list - except it's supposed to
be 2-dimensional. The inner-list-of-lists is a result of how I'm
producing the data, and now I want to do a mass-concatenation (or
extending) of the inner-list-of-lists, and come up with this result:

foo == [[1,2,3,4,5,6,7,8,9],[3,2,1,6,5,4,9,8,7]]

True

What's the best way to accomplish this?

I don't know if this is the best, but it didn't take long to come up with
it:

foo = [[[1,2,3],[4,5,6],[7,8,9]], .... [[3,2,1],[6,5,4],[9,8,7]]]

def unroll(list3d):

.... newl = []
.... for sublist in list3d:
.... newl.append(sum(sublist, []))
.... return newl
....[[1, 2, 3, 4, 5, 6, 7, 8, 9], [3, 2, 1, 6, 5, 4, 9, 8, 7]]


Repeat after me:

"Not everything has to be a one-liner."


If sum() is too slow, because your sub-lists are huge, you can easily
factor that out and replace it with something using extend.

 

[Dustan]

As with all such things, you stick the implementation in a well-named function
and simply call the function everywhere. The implementation never needs to be a
one-liner expression.

It's already in a function, but in order to convert the reduce function
into a for-loop that I can use as an expression, I would have to create
another independent function, which would make that code more cluttered
than it already is.

Unless you're saying to perform the list comprehension manually. That
would be two for-loops I use in the list comprehension, plus another
one to take place of the reduce function. Sure, that spreads out each
individual step a little more, but it also makes it more difficult to
understand what the overall goal of the code is (I'm going for
readability as well as easy maintenance here).

 

[Dustan]

foo =\

[[[1,2,3],[4,5,6],[7,8,9]],
[[3,2,1],[6,5,4],[9,8,7]]]

Here, foo appears to be a 3-dimensional list - except it's supposed to
be 2-dimensional. The inner-list-of-lists is a result of how I'm
producing the data, and now I want to do a mass-concatenation (or
extending) of the inner-list-of-lists, and come up with this result:

foo == [[1,2,3,4,5,6,7,8,9],[3,2,1,6,5,4,9,8,7]]

True

that still doesn't explain your "the expression must be used in a list
comprehension" requirement, though.

Oh, right; sorry about the confusion. The list isn't quite that simple,
and in order to pull the right pieces together, I use a list
comprehension.

assuming that the sizes are
varying, and at least sometimes a lot larger than 3x3, I'd probably
write the above as

for index, item in enumerate(foo):
this = []
for i in item:
this.extend(i)
foo[index] = this

I'll see if that works.

which should be pretty efficient, since it avoids unnecessary function
calls, and is amortized linear time instead of O(N**2).

or, if I was in a hurry, and didn't really care if the inner sequences
were lists or tuples:

foo = map(Tkinter._flatten, foo)

</F>

I don't know if this is the best, but it didn't take long to come up with
it:

foo = [[[1,2,3],[4,5,6],[7,8,9]], ... [[3,2,1],[6,5,4],[9,8,7]]]

def unroll(list3d):

... newl = []
... for sublist in list3d:
... newl.append(sum(sublist, []))
... return newl

deja vu...

[[1, 2, 3, 4, 5, 6, 7, 8, 9], [3, 2, 1, 6, 5, 4, 9, 8, 7]]


Repeat after me:

"Not everything has to be a one-liner."

Not everything has to be a one-liner. But readability helps.

 

[Robert Kern]

It's already in a function, but in order to convert the reduce function
into a for-loop that I can use as an expression, I would have to create
another independent function, which would make that code more cluttered
than it already is.

Unless you're saying to perform the list comprehension manually. That
would be two for-loops I use in the list comprehension, plus another
one to take place of the reduce function. Sure, that spreads out each
individual step a little more, but it also makes it more difficult to
understand what the overall goal of the code is (I'm going for
readability as well as easy maintenance here).

Let's just say that we disagree entirely on what constitutes readability and
maintainability.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Paddy]

Anyway, I figured out a way to get the builtin
function 'sum' to work as I need:
sum([[1,2,3],[4,5,6],[7,8,9]], [])

Hah!
No-one expects sum to be used on anything but numbers.

Except lists as above.

No-one expects sum to be used on anything but numbers, and maybe lists
too.

;-)


(My apologies for my poor grasp of the Spanish Inquisition sketch; and
I was alluding to
this thread:
http://groups.google.com/group/comp.lang.python/msg/f34525384f8028b5 )

 

[Ben Finney]

Not everything has to be a one-liner. But readability helps.

Indeed. Complex one-liners are rarely as readable as a well-named
function call, implemented with several explicit readable lines in
one place, and then re-used as many times as necessary.

 

[Dustan]

Alright, I can see I'm a bit outvoted here. I tried your suggestions
and it worked fine.

I'll also try to consider in the future that part of the problem might
be lack of information conveyed on my part.

 

[George Sakkis]

Alright, I can see I'm a bit outvoted here. I tried your suggestions
and it worked fine.

I'll also try to consider in the future that part of the problem might
be lack of information conveyed on my part.

If you insist on one-liners, it can be done without sum(), though it
probably doesn't buy you much in readability:

from itertools import chain
[list(chain(*row)) for row in foo]

By the way, if this was not a toy example and you're doing serious work
with n-dimensional arrays, make yourself a favor and install NumPy;
it's usually both faster and more elegant for array manipulations than
pure python.

George