concatenate Numeric

[Sheldon]

Hi,

I am trying to build a large array using concatenate function in
python.
So as I loop over the number of arrays, of which there are 12 (4 down
and 3 across), I create 3 long arrays by concatenating them at the
bottom and then concatenating them side by side:
for ind in range(num_arrays):
if ind == 0:
bias_down1 = array(bias)
ppslon1 = array(ppslon)

ppslat1 = array(ppslat)

elif ind > 0 and <= 3:
bias_down1 = concatenate((bias_down1,bias),0)

ppslon1 = concatenate((ppslon1,ppslon),0)

ppslat1 = concatenate((ppslat1,ppslat),0)
elif ind == 4:
bias_down2 = array(bias)
ppslon2 = array(ppslon)
ppslat2 = array(ppslat)
elif ind > 4 and ind <= 7:
bias_down2 = concatenate((bias_down2,bias),0)

ppslon2 = concatenate((ppslon2,ppslon),0)

ppslat2 = concatenate((ppslat2,ppslat),0)

elif ind == 8:
bias_down3 = array(bias)

ppslon3 = array(ppslon)

ppslat3 = array(ppslat)

elif ind > 8:
bias_down3 = concatenate((bias_down3,bias),0)
ppslon3 = concatenate((ppslon3,ppslon),0)
ppslat3 = concatenate((ppslat3,ppslat),0)

bias_a = concatenate((bias_down1,bias_down2),1) # joining the arrays
side by side
bias_all = concatenate((bias_a,bias_down3),1)

ppslat_a = concatenate((ppslat1,ppslat2),1)
ppslat_all = concatenate((ppslat_a,ppslat3),1)

ppslon_a = concatenate((ppslon1,ppslon2),1)
ppslon_all = concatenate((ppslon_a,ppslon3),1)

print 'Max lat', max(max(ppslat_all)), '\t','Min lat',
min(min(ppslat_all))
print 'Max lon', max(max(ppslon_all)), '\t','Min lon',
min(min(ppslon_all))

*****************
Now this works, the array size is correct but the longitude values
given for max and min are wrong. What is even stranger to me is that
when I write the array in binary format to a file and read it with
Matlab, the max and min are correct but when I read it back with python
the max and min are again incorrect for only the longitude data. I
saved the max and min for the longitude for each array and then check
it in the python program and they are correct at the end but the
max(max(ppslon)) values is incorrect. Does anyone knows why this is
so?
If I was doing something wrong then Matlab would not have returned
correct values.

Any help is appreciated!

/Sheldon

 

[Sheldon]

Sheldon skrev:

Hi,

I am trying to build a large array using concatenate function in
python.
So as I loop over the number of arrays, of which there are 12 (4 down
and 3 across), I create 3 long arrays by concatenating them at the
bottom and then concatenating them side by side:
for ind in range(num_arrays):
if ind == 0:
bias_down1 = array(bias)
ppslon1 = array(ppslon)

ppslat1 = array(ppslat)

elif ind > 0 and <= 3:
bias_down1 = concatenate((bias_down1,bias),0)

ppslon1 = concatenate((ppslon1,ppslon),0)

ppslat1 = concatenate((ppslat1,ppslat),0)
elif ind == 4:
bias_down2 = array(bias)
ppslon2 = array(ppslon)
ppslat2 = array(ppslat)
elif ind > 4 and ind <= 7:
bias_down2 = concatenate((bias_down2,bias),0)

ppslon2 = concatenate((ppslon2,ppslon),0)

ppslat2 = concatenate((ppslat2,ppslat),0)

elif ind == 8:
bias_down3 = array(bias)

ppslon3 = array(ppslon)

ppslat3 = array(ppslat)

elif ind > 8:
bias_down3 = concatenate((bias_down3,bias),0)
ppslon3 = concatenate((ppslon3,ppslon),0)
ppslat3 = concatenate((ppslat3,ppslat),0)

bias_a = concatenate((bias_down1,bias_down2),1) # joining the arrays
side by side
bias_all = concatenate((bias_a,bias_down3),1)

ppslat_a = concatenate((ppslat1,ppslat2),1)
ppslat_all = concatenate((ppslat_a,ppslat3),1)

ppslon_a = concatenate((ppslon1,ppslon2),1)
ppslon_all = concatenate((ppslon_a,ppslon3),1)

print 'Max lat', max(max(ppslat_all)), '\t','Min lat',
min(min(ppslat_all))
print 'Max lon', max(max(ppslon_all)), '\t','Min lon',
min(min(ppslon_all))

*****************
Now this works, the array size is correct but the longitude values
given for max and min are wrong. What is even stranger to me is that
when I write the array in binary format to a file and read it with
Matlab, the max and min are correct but when I read it back with python
the max and min are again incorrect for only the longitude data. I
saved the max and min for the longitude for each array and then check
it in the python program and they are correct at the end but the
max(max(ppslon)) values is incorrect. Does anyone knows why this is
so?
If I was doing something wrong then Matlab would not have returned
correct values.

Any help is appreciated!

/Sheldon

Sorry, there is a small error when I wrote this part of the program:
should say "ind <= 3"

/sheldon

 

[Robert Kern]

Hi,

I am trying to build a large array using concatenate function in
python.
So as I loop over the number of arrays, of which there are 12 (4 down
and 3 across), I create 3 long arrays by concatenating them at the
bottom and then concatenating them side by side:
[snip]

print 'Max lat', max(max(ppslat_all)), '\t','Min lat',
min(min(ppslat_all))
print 'Max lon', max(max(ppslon_all)), '\t','Min lon',
min(min(ppslon_all))

*****************
Now this works, the array size is correct but the longitude values
given for max and min are wrong. What is even stranger to me is that
when I write the array in binary format to a file and read it with
Matlab, the max and min are correct but when I read it back with python
the max and min are again incorrect for only the longitude data. I
saved the max and min for the longitude for each array and then check
it in the python program and they are correct at the end but the
max(max(ppslon)) values is incorrect. Does anyone knows why this is
so?
If I was doing something wrong then Matlab would not have returned
correct values.

Don't use min() and max() on multidimensional arrays. They won't give sensible
answers.


In [11]: a = RA.random([3,5])

In [12]: a
Out[12]:
array([[ 0.01721657, 0.64291363, 0.33210659, 0.89887972, 0.24437849],
[ 0.88205348, 0.00839329, 0.35999039, 0.9966411 , 0.54957126],
[ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076]])

In [13]: min(a)
Out[13]: array([ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076])


The builtin min() and max() compare the values in the sequence. In this case,
those values are the rows of the arrays. Numeric uses rich comparisons, so the
result of a comparison is a boolean array. Numeric also has the convention that
if any of the elements of an array are considered to be True, then the whole
array is.


In [16]: a[1] < a[2]
Out[16]: array([0, 1, 0, 0, 0])

In [17]: bool(_)
Out[17]: True

In [18]: a[2] < a[1]
Out[18]: array([1, 0, 1, 1, 1])

In [19]: bool(_)
Out[19]: True


This makes min(a) incorrect when len(a.shape) > 1. Instead, use the minimum and
maximum ufuncs provided with Numeric:


In [21]: N.minimum.reduce(a.flat)
Out[21]: 0.0083932917161983426

In [22]: N.maximum.reduce(a.flat)
Out[22]: 0.99664110397663608


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Sheldon]

Robert Kern skrev:

Hi,

I am trying to build a large array using concatenate function in
python.
So as I loop over the number of arrays, of which there are 12 (4 down
and 3 across), I create 3 long arrays by concatenating them at the
bottom and then concatenating them side by side:
[snip]

print 'Max lat', max(max(ppslat_all)), '\t','Min lat',
min(min(ppslat_all))
print 'Max lon', max(max(ppslon_all)), '\t','Min lon',
min(min(ppslon_all))

*****************
Now this works, the array size is correct but the longitude values
given for max and min are wrong. What is even stranger to me is that
when I write the array in binary format to a file and read it with
Matlab, the max and min are correct but when I read it back with python
the max and min are again incorrect for only the longitude data. I
saved the max and min for the longitude for each array and then check
it in the python program and they are correct at the end but the
max(max(ppslon)) values is incorrect. Does anyone knows why this is
so?
If I was doing something wrong then Matlab would not have returned
correct values.

Don't use min() and max() on multidimensional arrays. They won't give sensible
answers.


In [11]: a = RA.random([3,5])

In [12]: a
Out[12]:
array([[ 0.01721657, 0.64291363, 0.33210659, 0.89887972, 0.24437849],
[ 0.88205348, 0.00839329, 0.35999039, 0.9966411 , 0.54957126],
[ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076]])

In [13]: min(a)
Out[13]: array([ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076])


The builtin min() and max() compare the values in the sequence. In this case,
those values are the rows of the arrays. Numeric uses rich comparisons, so the
result of a comparison is a boolean array. Numeric also has the convention that
if any of the elements of an array are considered to be True, then the whole
array is.


In [16]: a[1] < a[2]
Out[16]: array([0, 1, 0, 0, 0])

In [17]: bool(_)
Out[17]: True

In [18]: a[2] < a[1]
Out[18]: array([1, 0, 1, 1, 1])

In [19]: bool(_)
Out[19]: True


This makes min(a) incorrect when len(a.shape) > 1. Instead, use the minimum and
maximum ufuncs provided with Numeric:


In [21]: N.minimum.reduce(a.flat)
Out[21]: 0.0083932917161983426

In [22]: N.maximum.reduce(a.flat)
Out[22]: 0.99664110397663608


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Thanks a million!!!!!!!!!!!!!!!!!!!!!!!

Now I can sleep better.

/Sheldon

 

[Sheldon]

Hi,

I am trying to build a large array using concatenate function in
python.
So as I loop over the number of arrays, of which there are 12 (4 down
and 3 across), I create 3 long arrays by concatenating them at the
bottom and then concatenating them side by side:
[snip]

print 'Max lat', max(max(ppslat_all)), '\t','Min lat',
min(min(ppslat_all))
print 'Max lon', max(max(ppslon_all)), '\t','Min lon',
min(min(ppslon_all))

*****************
Now this works, the array size is correct but the longitude values
given for max and min are wrong. What is even stranger to me is that
when I write the array in binary format to a file and read it with
Matlab, the max and min are correct but when I read it back with python
the max and min are again incorrect for only the longitude data. I
saved the max and min for the longitude for each array and then check
it in the python program and they are correct at the end but the
max(max(ppslon)) values is incorrect. Does anyone knows why this is
so?
If I was doing something wrong then Matlab would not have returned
correct values.

Don't use min() and max() on multidimensional arrays. They won't give sensible
answers.


In [11]: a = RA.random([3,5])

In [12]: a
Out[12]:
array([[ 0.01721657, 0.64291363, 0.33210659, 0.89887972, 0.24437849],
[ 0.88205348, 0.00839329, 0.35999039, 0.9966411 , 0.54957126],
[ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076]])

In [13]: min(a)
Out[13]: array([ 0.59983864, 0.18983323, 0.13727718, 0.8987289 , 0.05425076])


The builtin min() and max() compare the values in the sequence. In this case,
those values are the rows of the arrays. Numeric uses rich comparisons, so the
result of a comparison is a boolean array. Numeric also has the convention that
if any of the elements of an array are considered to be True, then the whole
array is.


In [16]: a[1] < a[2]
Out[16]: array([0, 1, 0, 0, 0])

In [17]: bool(_)
Out[17]: True

In [18]: a[2] < a[1]
Out[18]: array([1, 0, 1, 1, 1])

In [19]: bool(_)
Out[19]: True


This makes min(a) incorrect when len(a.shape) > 1. Instead, use the minimum and
maximum ufuncs provided with Numeric:


In [21]: N.minimum.reduce(a.flat)
Out[21]: 0.0083932917161983426

In [22]: N.maximum.reduce(a.flat)
Out[22]: 0.99664110397663608


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Hi Robert,

Thanks again for showing me this. I have been trying to read up on
reduce() as I have never used it before. I would like to know what it
does. So far my search has found nothing that I can grasp. The
reference library notes are estoteric at best.
Can you enlighten me on this matter?'

/Sheldon

 

[Robert Kern]

Thanks again for showing me this. I have been trying to read up on
reduce() as I have never used it before. I would like to know what it
does. So far my search has found nothing that I can grasp. The
reference library notes are estoteric at best.
Can you enlighten me on this matter?'

The .reduce() method on ufuncs works pretty much like the reduce() builtin
function. It applies the binary ufunc along the given axis of the array (the
first one by default) cumulatively.

a = [3, 2, 1, 0]
minimum.reduce(a) == minimum(minimum(minimum(a[0], a[1]), a[2]), a[3])

I will note, in the form of enticement to get you to try the currently active
array package instead of Numeric, that in numpy, arrays have methods to do
minimums and maximum rather more conveniently.
array([[ 0.49892358, 0.11931907, 0.37146848, 0.07494308, 0.91973863],
[ 0.92049698, 0.35016683, 0.01711571, 0.59542456, 0.49897077],
[ 0.57449315, 0.99592033, 0.20549262, 0.25135288, 0.04111402]])

>>> a.min() 0.017115711878847639
>>> a.min(axis=0) array([ 0.49892358, 0.11931907, 0.01711571, 0.07494308, 0.04111402])
>>> a.min(axis=1)

array([ 0.07494308, 0.01711571, 0.04111402])

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Sheldon]

Robert Kern skrev:

Thanks again for showing me this. I have been trying to read up on
reduce() as I have never used it before. I would like to know what it
does. So far my search has found nothing that I can grasp. The
reference library notes are estoteric at best.
Can you enlighten me on this matter?'

The .reduce() method on ufuncs works pretty much like the reduce() builtin
function. It applies the binary ufunc along the given axis of the array (the
first one by default) cumulatively.

a = [3, 2, 1, 0]
minimum.reduce(a) == minimum(minimum(minimum(a[0], a[1]), a[2]), a[3])

I will note, in the form of enticement to get you to try the currently active
array package instead of Numeric, that in numpy, arrays have methods to do
minimums and maximum rather more conveniently.
array([[ 0.49892358, 0.11931907, 0.37146848, 0.07494308, 0.91973863],
[ 0.92049698, 0.35016683, 0.01711571, 0.59542456, 0.49897077],
[ 0.57449315, 0.99592033, 0.20549262, 0.25135288, 0.04111402]])

a.min() 0.017115711878847639
a.min(axis=0) array([ 0.49892358, 0.11931907, 0.01711571, 0.07494308, 0.04111402])
a.min(axis=1)

array([ 0.07494308, 0.01711571, 0.04111402])

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

Thanks for the explanation! Super.
I am trying to get my bosses to purchase the Numpy documentation and
upgrade to Numpy as well as matplotlib and other necessary scientific
modules. But it is not entirely up to me. Still I need to learn more
about Python and installing these modules myself.