Numpy array index handling

[phishboh]

Being a Matlab user wanting to switch to Python/SciPy, I'd like to
know how the following Matlab code would be written in Python:

% First, just some artificial data
N = 1000000 ;
input = sign(randn(1, N)) ;
a = (1 : N) ;
% This is what I'd like to do;
% for indices where the input has a certain value, compute and store
something
% for indices where the input has another certain value, compute and
store something else
output(input < 0) = 10 * a(input < 0) ;
output(input > 0) = 20 * a(input > 0) ;

I have tried the following in Python:
N = 1000000
input = sign(randn(N))
a = arange(N)
output = zeros(N)
output[input < 0] = 10 * a[input < 0]
output[input > 0] = 20 * a[input > 0]

However, that gives me in IndexError.

Of course I could use a for-loop, but I assume that there is another
way (although I haven't been able to find it in any documentation).

Many thanks in advance!

 

[Robert Kern]

Being a Matlab user wanting to switch to Python/SciPy,

Fantastic! You might be interested in joining the numpy mailing list. There are
a lot more of us numpy devs and users there than here.

http://www.scipy.org/Mailing_Lists

I'd like to
know how the following Matlab code would be written in Python:

% First, just some artificial data
N = 1000000 ;
input = sign(randn(1, N)) ;
a = (1 : N) ;
% This is what I'd like to do;
% for indices where the input has a certain value, compute and store
something
% for indices where the input has another certain value, compute and
store something else
output(input < 0) = 10 * a(input < 0) ;
output(input > 0) = 20 * a(input > 0) ;

I have tried the following in Python:
N = 1000000
input = sign(randn(N))
a = arange(N)
output = zeros(N)
output[input < 0] = 10 * a[input < 0]
output[input > 0] = 20 * a[input > 0]

However, that gives me in IndexError.

When reporting that you get an error, copy-and-pasting the full traceback is
highly recommended. We can give you an answer much more quickly if you do. In
this case, your example works fine for me with numpy 1.0.3 (after supplying the
missing imports).


In [1]: from numpy import *

In [2]: N = 1000000

In [3]: input = sign(random.randn(N))

In [4]: a = arange(N)

In [5]: output = zeros(N)

In [6]: output[input < 0] = 10 * a[input < 0]

In [7]: output[input > 0] = 20 * a[input > 0]


If you are still having problems, please join us on numpy-discussion with the
full traceback.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[sturlamolden]

from numpy import *
from numpy.random import *
N = 1000000
input = sign(randn(N))
a = arange(N)
output = zeros(N)
output[input < 0] = 10 * a[input < 0]
output[input > 0] = 20 * a[input > 0]

Worked fine for me.


S.M.

 

[phishboh]

Thanks a lot for your reply.
I'll have a look at the numpy-discussion for future issues.

FYI, I'm using Python 2.4.3 for Windows (Enthought Edition) and the
included IPython shell. I found my mistake; importing of pylab.
E.g., this works
from pylab import * ; from scipy import * ; y = arange(3) ; y[y>1] = 0
but not
from scipy import * ; from pylab import * ; z = arange(3) ; z[z>1] = 0

Using the 'whos' command, I can see that y is an ndarray type, whereas
z is an array type.
(I also see a lot of functions, etc. How can I list just the
variables?)

It seems a bit risky to use 'from scipy import *'. Maybe it's better
to use 'import scipy' and then scipy.arange, to be sure what is
actually being used? Would there be any disadvanages with that
approach, other than more use of the keyboard?

 

[sturlamolden]

It seems a bit risky to use 'from scipy import *'. Maybe it's better
to use 'import scipy' and then scipy.arange, to be sure what is
actually being used? Would there be any disadvanages with that
approach, other than more use of the keyboard?

Generally the idiom

from something import *

is discouraged as it clutters the namespace (it can e.g. shadow
globals).

I prefer something like:

import numpy as np

Also I suggest you don't use SciPy and Matplotlib (pylab) unless you
mean it. Import from numpy instead. Import pylab if you need graphs
and scipy if you need something special that is not in numpy.

Get the latest installer (or sources) from:

Python 2.5.1: www.python.org
NumPy 1.0.3: www.scipy.org
SciPy 0.5.2: www.scipy.org
Matplotlib 0.90.1: matplotlib.sourceforge.net
PIL 1.1.6: www.pythonware.com/products/pil/
wxPython 2.8.4.0: www.wxpython.org
gcc and gfortran (for scipy.wave): http://gcc.gnu.org/wiki/GFortranBinaries

At least that is what makes me as a scientist happy with Python

Enthought and similar distros are in my experience "unclean". They
don't always work and they are difficult to update. I rather download
the binary installers (for Windows) and install the packages I need.

I also suggest you buy a copy of "Guide to NumPy" by Travis Oliphant.
http://www.tramy.us/guidetoscipy.html

S.M.

 

[Robert Kern]

Thanks a lot for your reply.
I'll have a look at the numpy-discussion for future issues.

FYI, I'm using Python 2.4.3 for Windows (Enthought Edition) and the
included IPython shell. I found my mistake; importing of pylab.
E.g., this works
from pylab import * ; from scipy import * ; y = arange(3) ; y[y>1] = 0
but not
from scipy import * ; from pylab import * ; z = arange(3) ; z[z>1] = 0

Using the 'whos' command, I can see that y is an ndarray type, whereas
z is an array type.

Ah, this means that matplotlib is configured to use Numeric, numpy's
predecessor, which does not have the capability to index with boolean masks. You
need to change matplotlib's "numerix" setting to "numpy". See its documentation
on how to do that. Also, the Enthought Edition that you downloaded is quite old.
I recommend getting newer versions of the packages.

(I also see a lot of functions, etc. How can I list just the
variables?)

Functions are first-class objects in Python, so you are seeing "just the
variables". However, you are probably only interested in a few types. You can
look at the help for %who, which will show you how to restrict the types of
objects that are displayed by that command. Unfortunately, it doesn't have a way
to specifically exclude types. Most likely, you can get by with the following:


In [15]: %who ndarray float int complex str
ALLOW_THREADS BUFSIZE CLIP ERR_CALL ERR_DEFAULT ERR_DEFAULT2
ERR_IGNORE ERR_LOG ERR_PRINT
ERR_RAISE ERR_WARN FLOATING_POINT_SUPPORT FPE_DIVIDEBYZERO
FPE_INVALID FPE_OVERFLOW FPE_UNDERFLOW Inf Infinity
MAXDIMS NAN NINF NZERO NaN PINF PZERO RAISE SHIFT_DIVIDEBYZERO
SHIFT_INVALID SHIFT_OVERFLOW SHIFT_UNDERFLOW UFUNC_BUFSIZE_DEFAULT
UFUNC_PYVALS_NAME WRAP e inf infty
nan pi z


You can assign that to a macro to make it easier to type:

In [16]: %macro mywho 15
Macro `mywho` created. To execute, type its name (without quotes).
Macro contents:
_ip.magic("who ndarray float int complex str")

In [17]: mywho
-------> mywho()
ALLOW_THREADS BUFSIZE CLIP ERR_CALL ERR_DEFAULT ERR_DEFAULT2
ERR_IGNORE ERR_LOG ERR_PRINT
ERR_RAISE ERR_WARN FLOATING_POINT_SUPPORT FPE_DIVIDEBYZERO
FPE_INVALID FPE_OVERFLOW FPE_UNDERFLOW Inf Infinity
MAXDIMS NAN NINF NZERO NaN PINF PZERO RAISE SHIFT_DIVIDEBYZERO
SHIFT_INVALID SHIFT_OVERFLOW SHIFT_UNDERFLOW UFUNC_BUFSIZE_DEFAULT
UFUNC_PYVALS_NAME WRAP e inf infty
nan pi z

It seems a bit risky to use 'from scipy import *'. Maybe it's better
to use 'import scipy' and then scipy.arange, to be sure what is
actually being used? Would there be any disadvanages with that
approach, other than more use of the keyboard?

There really aren't any besides the extra verbosity. Using "from foo import *"
is often fine for interactive use, but for code that you write in a file, it is
*highly* recommended that you import modules and use them explicitly for the
reasons you mention. You can reduce the typing burden somewhat like so:

import numpy as N
a = N.arange(10)

That's a fairly common idiom in the numpy community. You can also import
specific symbols if you know what you are going to use. If you're doing a lot of
trig, for example:

from numpy import sin, cos, tan, pi

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

Enthought and similar distros are in my experience "unclean". They
don't always work and they are difficult to update. I rather download
the binary installers (for Windows) and install the packages I need.

Right, that's why we (I'm an Enthought employee) haven't been updating the
Enthought Edition. We are moving to a less monolithic approach with Python eggs.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco