quickly looping over a 2D array?

M

Martin

Hi,

I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...

for i in range(numrows):
for j in range(numcols):

if array_1[i, j] == some_value or array_2[i, j] >= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value

Many Thanks,

Martin
 
P

Peter Otten

Martin said:
I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...

for i in range(numrows):
for j in range(numcols):

if array_1[i, j] == some_value or array_2[i, j] >= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value

array_1[(array_1 == some_value) | (array_2 >= array_1 * some_other_value)] =
some_new_value

maybe?
 
M

Martin

Martin said:
I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...
for i in range(numrows):
for j in range(numcols):
if array_1[i, j] == some_value or array_2[i, j] >= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value

array_1[(array_1 == some_value) | (array_2 >= array_1 * some_other_value)] =
some_new_value

maybe?

So I tried...

band_1[(array_1 == 255) or (array_2 >= array_1 * factor)] = 0

which led to

ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()

so not sure that works?
 
P

Peter Otten

Martin said:
Martin said:
I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...
for i in range(numrows):
for j in range(numcols):
if array_1[i, j] == some_value or array_2[i, j] >= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value

array_1[(array_1 == some_value) | (array_2 >= array_1 *
some_other_value)] = some_new_value

maybe?

So I tried...

band_1[(array_1 == 255) or (array_2 >= array_1 * factor)] = 0

which led to

ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()

so not sure that works?

Copy and paste -- or replace "or" with "|".
 
M

Martin

Martin said:
Martin wrote:
I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...
for i in range(numrows):
for j in range(numcols):
if array_1[i, j] == some_value or array_2[i, j] >= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value
array_1[(array_1 == some_value) | (array_2 >= array_1 *
some_other_value)] = some_new_value
maybe?
So I tried...
band_1[(array_1 == 255) or (array_2 >= array_1 * factor)] = 0
which led to
ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()
so not sure that works?

Copy and paste -- or replace "or" with "|".

apologies - I mistook that for a type for "or"

I now get the following error...

ValueError: shape mismatch: objects cannot be broadcast to a single
shape
 
P

Peter Otten

Martin said:
Martin said:
Martin wrote:
I am new to python and I was wondering if there was a way to speed
up the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy)
something like the following...
for i in range(numrows):
for j in range(numcols):
if array_1[i, j] == some_value or array_2[i, j] >=
array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value
array_1[(array_1 == some_value) | (array_2 >= array_1 *
some_other_value)] = some_new_value

So I tried...
band_1[(array_1 == 255) or (array_2 >= array_1 * factor)] = 0
which led to
ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()
so not sure that works?

Copy and paste -- or replace "or" with "|".

apologies - I mistook that for a type for "or"

I now get the following error...

ValueError: shape mismatch: objects cannot be broadcast to a single
shape

It seems array_1 and array_2 have a -- dada! -- different shape.
Assuming array_1 is the smaller one:

numrows, numcols = array_1.shape
array_1[(array_1 == some_value) | (array_2[:numrows,:numcols] >= array_1 *
some_other_value)] = some_new_value

There may be other options, but I'm not a numpy expert.

Peter
 
M

Martin

Martin said:
Martin wrote:
Martin wrote:
I am new to python and I was wondering if there was a way to speed
up the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy)
something like the following...
for i in range(numrows):
for j in range(numcols):
if array_1[i, j] == some_value or array_2[i, j] >=
array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value
array_1[(array_1 == some_value) | (array_2 >= array_1 *
some_other_value)] = some_new_value
maybe?
So I tried...
band_1[(array_1 == 255) or (array_2 >= array_1 * factor)] = 0
which led to
ValueError: The truth value of an array with more than one element is
ambiguous. Use a.any() or a.all()
so not sure that works?
Copy and paste -- or replace "or" with "|".
apologies - I mistook that for a type for "or"
I now get the following error...
ValueError: shape mismatch: objects cannot be broadcast to a single
shape

It seems array_1 and array_2 have a -- dada! -- different shape.
Assuming array_1 is the smaller one:

numrows, numcols = array_1.shape
array_1[(array_1 == some_value) | (array_2[:numrows,:numcols] >= array_1 *
some_other_value)] = some_new_value

There may be other options, but I'm not a numpy expert.

Peter

My mistake - the incorrect size in the arrays was my error.

The statement works now, but it doesn't give the same results as my
original logic, strangely!?

in my logic:

data = np.zeros((numrows, numcols), dtype = np.uint8, order ='C')

for i in range(numrows):
for j in range(numcols):
if band3[i,j] == 255 or band3[i,j] >= band6[i,j] * factor:
data[i,j] = 0
else:
data[i,j] = 1

to do the same with what you suggested...

data = np.ones((numrows, numcols), dtype = np.uint8, order ='C')
data[(band3 == 255) | (band6 >= band3 * factor)] = 0

Thanks
 
P

Peter Otten

Martin said:
The statement works now, but it doesn't give the same results as my
original logic, strangely!?

in my logic:

data = np.zeros((numrows, numcols), dtype = np.uint8, order ='C')

for i in range(numrows):
for j in range(numcols):
if band3[i,j] == 255 or band3[i,j] >= band6[i,j] * factor:
data[i,j] = 0
else:
data[i,j] = 1

to do the same with what you suggested...

data = np.ones((numrows, numcols), dtype = np.uint8, order ='C')
data[(band3 == 255) | (band6 >= band3 * factor)] = 0

Did you swap band3 and band6? If that's the case, it is an error you should
be able to find yourself.

Please be a bit more careful.

Also, it is good practice to write a few test cases where you have
precalculated the result. How would you otherwise know which version is
correct? Writing a third one to break the tie?

Peter
 
M

Martin

Martin said:
The statement works now, but it doesn't give the same results as my
original logic, strangely!?
in my logic:
data = np.zeros((numrows, numcols), dtype = np.uint8, order ='C')
for i in range(numrows):
for j in range(numcols):
if band3[i,j] == 255 or band3[i,j] >= band6[i,j] * factor:
data[i,j] = 0
else:
data[i,j] = 1
to do the same with what you suggested...
data = np.ones((numrows, numcols), dtype = np.uint8, order ='C')
data[(band3 == 255) | (band6 >= band3 * factor)] = 0

Did you swap band3 and band6? If that's the case, it is an error you should
be able to find yourself.

Please be a bit more careful.

Also, it is good practice to write a few test cases where you have
precalculated the result. How would you otherwise know which version is
correct? Writing a third one to break the tie?

Peter

apologies... it was the way I typed it up, I wasn't copying and
pasting from my text editor!
Thanks Peter.
 
R

Robert Kern

Hi,

I am new to python and I was wondering if there was a way to speed up
the way I index 2D arrays when I need to check two arrays
simultaneously? My current implementations is (using numpy) something
like the following...

for i in range(numrows):
for j in range(numcols):

if array_1[i, j] == some_value or array_2[i, j]>= array_1[i,
j] * some_other_value
array_1[i, j] = some_new_value

Peter has given you good answers, but you will probably want to ask future numpy
questions on the numpy mailing list.

http://www.scipy.org/Mailing_Lists

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,439
Members
44,829
Latest member
PIXThurman

Latest Threads

Top