yet another noob question

[bearophileHUGS]

Gerard Flanagan:

mod5 = ['1','2','3','4','5']
X = [ ''.join([a,b,c,d,e])
for a in mod5
for b in mod5
for c in mod5
for d in mod5
for e in mod5 ]

A modified version of your one is the faster so far:

v = "12345"
r = [a+b+c+d+e for a in v for b in v for c in v for d in v for e in v]

Bye,
bearophile

 

[Jason Nordwick]

Somehow my other response to the list got lost. I'm still learning Python, but this seems much better than my first attempt:

def pr(x): print x
def cross(x,y): return [a+b for a in x for b in y]
x=map(pr, reduce(cross, [map(str,range(1,6))]*5))

-j

Or without filter:

from operator import add
def pr(x): print x
def cross(x,y): return reduce(add, [[a+b for b in y] for a in x])
x=map(pr, reduce(cross, [map(str,range(1,6))]*5))

[...]

reduce(add, list) is the same as sum(list) and is only half as fast as sum:add").repeat(200, 1000))/200
0.10820861031220445
Also note that reduce will be removed in Python 3000.

 

[Jason Nordwick]

Also note that reduce will be removed in Python 3000.

What will replace it?

-j

 

[bearophileHUGS]

Jason Nordwick:

What will replace it?

Nothing, I presume. You will have to write a function to find another
way to solve problems.

Bye,
bearophile

 

[Jason Nordwick]

*mouth agape*

Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.
25.817132345032689

where

bits = map(lambda x:randint(0,1), xrange(1000000))

def reduceadd(v):
return reduce(add, v)

def loopadd(v):
sum = 0
for x in v:
sum = add(sum, x)
return sum



(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)
-j

 

[AlbaClause]

I would like to generate every unique combination of numbers 1-5 in a 5
digit number and follow each combo with a newline. So i'm looking at
generating combinations such as: (12345) , (12235), (55554) and so on.
What would be the best way to do this? So, basically i'm looking for a
list of all combinations of 1-5 in a 5 digit unique number. Also, when
I send the list to print there can't be any duplicates of the combos.


Thanks,

Mike

This is one of those questions that reminded me that I don't know Python.
I launched my Python interpreter, and forgot how to even print a string.
Duh!

Oh well...

--

 

[Stargaming]

Jason Nordwick wrote:

Well, *perhaps* there will be a new function product() -- and there we
go having replaced every arithmetical use of reduce() because Guido
cannot read them. Nice.

Read on: "The fate of reduce() in Python 3000"
http://www.artima.com/weblogs/viewpost.jsp?thread=98196 -- found on "PEP
3100 -- Miscellaneous Python 3.0 Plans"
http://www.python.org/dev/peps/pep-3100/#id63

Regards,
Stargaming
--

 

[starGaming]

*mouth agape*

Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.

25.817132345032689

where

bits = map(lambda x:randint(0,1), xrange(1000000))

def reduceadd(v):
return reduce(add, v)

def loopadd(v):
sum = 0
for x in v:
sum = add(sum, x)
return sum



(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)

[...]

Assuming Guido van Rossum is right and reduce() is only useful for
arithmetic expressions (could you show an example where you use it
else?), you could take advantage of more "built-ins".
To spoiler the result of this, the loop-variant is slightly (in this
test: 2ms) faster.
The results on my amd64 3k+ were:
Reduce: 0.5686828074520.56633643382
For-loop: 0.56633643382

#!/usr/bin/env python
#
#

from random import randint
from timeit import Timer
from operator import add

def bits():
return [randint(0,1) for x in xrange(1000)]
# use def & list comprehension since lambda/map will be removed in
Python 3.0, too


print bits()[0:5]
print bits()[0:5] # it's working.

def reducefunc(x):
return reduce(add,x)
def loopfunc(x):
y = 0
for i in x: y += i
return y

x = bits()
print reducefunc(x)
print loopfunc(x) # both give proper output

print sum(Timer("reducefunc(bits())", "from __main__ import bits,
reducefunc").repeat(20,100))/20
print sum(Timer("loopfunc(bits())", "from __main__ import bits,
loopfunc").repeat(20,100))/20

 

[Jason Nordwick]

I use reduce to also do indexing, hashing with upsert semantics of lists of key-value pairs, transitioning through a state table, etc...

Somebody else pointed out to me how odd it is of Python to be ditching reduce when Guido van Rossum was hired by Google, and Google is literally built on map and reduce (http://en.wikipedia.org/wiki/Mapreduce).

Here is a code fragment I pulled from what I am currently working on (with a little modification to make things simple):

First build a tree:

def byn(a,n): return [[a for i in x] for x in zip(*[range(y,len(a),n) for y in range(n)])] ....
def whilep(pred,f,x):

.... while pred(x): x = f(x)
.... return x
....[[[[['a', 'b'], ['c', 'd']], [['e', 'f'], ['g', 'h']]], [[['i', 'j'], ['k', 'l']], [['m', 'n'], ['o', 'p']]]]]

Now if I had the list [0,1,0,0,1] and needed to find what letter it encodes:

t[0][1][0][0][1]

'j'

But sometimes, I want to be able to index branches or the leaves are at different depths, so I essentially have a list of indexes to follow:

def indexDepth(a,i): return reduce(lambda x,y: x[y], i, a) ....
indexDepth(t,[0,1,0,0,1])

'j'


Also, I completely understand your timings, but you are not timing reduce against a hand coded loop, but instead the operator '+' against the function add, as the function symbol lookup and call seem to have a heavy price. Reduce was one of the nice ways to eliminate some of those lookups.

-j

*mouth agape*

Wow. That really sucks. I make extensive use of reduce. It seems to run more than twice as fast as a for loop.

25.817132345032689

where

bits = map(lambda x:randint(0,1), xrange(1000000))

def reduceadd(v):
return reduce(add, v)

def loopadd(v):
sum = 0
for x in v:
sum = add(sum, x)
return sum



(Yes, I know there are better ways to sum up a list, but I just wanted to test the performance of the loop against reduce. In most of my reduce usages, the function passed to reduce is much more complex.)

[...]

Assuming Guido van Rossum is right and reduce() is only useful for
arithmetic expressions (could you show an example where you use it
else?), you could take advantage of more "built-ins".
To spoiler the result of this, the loop-variant is slightly (in this
test: 2ms) faster.
The results on my amd64 3k+ were:
Reduce: 0.5686828074520.56633643382
For-loop: 0.56633643382

#!/usr/bin/env python
#
#

from random import randint
from timeit import Timer
from operator import add

def bits():
return [randint(0,1) for x in xrange(1000)]
# use def & list comprehension since lambda/map will be removed in
Python 3.0, too


print bits()[0:5]
print bits()[0:5] # it's working.

def reducefunc(x):
return reduce(add,x)
def loopfunc(x):
y = 0
for i in x: y += i
return y

x = bits()
print reducefunc(x)
print loopfunc(x) # both give proper output

print sum(Timer("reducefunc(bits())", "from __main__ import bits,
reducefunc").repeat(20,100))/20
print sum(Timer("loopfunc(bits())", "from __main__ import bits,
loopfunc").repeat(20,100))/20

 

[Steve Holden]

I use reduce to also do indexing, hashing with upsert semantics of lists of key-value pairs, transitioning through a state table, etc...

Somebody else pointed out to me how odd it is of Python to be ditching reduce when Guido van Rossum was hired by Google, and Google is literally built on map and reduce (http://en.wikipedia.org/wiki/Mapreduce).

That seems a bit literal. Just because they use a tool called MapReduce
that doesn't imply that thay chose to implement it with the Python map()
and reduce() functions. It's a distributed application, in case you
hadn't noticed ...

regards
Steve

 

[Jason Nordwick]

That isn't what I meant. If there was a a point (and I'm not really sure that I'm even trying to make one), the point was that Google makes heavy use of reduce-like functionality, essentially implementing a distributed reduce across a cluster. From what I hear, they use a lot of Python and hired van Rossum for a reason. It just seems odd (don't read anything into this than mere cocked eyebrows) that the language designer that they hired and obviously have a lot of trust in would decide that reduce was essentially pointless. Google's distributed reduce seems to point in opposite way.

However, if reduce could be rolled into the list comprehension syntax, that would be even better. Or take it that extra step and roll a grouping functionality in there too, then you would have map, reduce, group, and filter all in one construct. You could call it select (joins are merely indexes into other structures).

-j