NumArray array-indexing

[Michael Drumheller]

(If you're not interested in NumArray, please skip this message.)

I am new to NumArray and I wonder if someone can help me with
array-indexing. Here's the basic situation: Given a rank-2 array
(i.e., a matrix) it seems to be trivial, with array indexing,
to extract a subset of its *columns*. But it does not seem
to be trivial to extract a subset of its *rows*. The code
snippet below describes the problem (if it really is a problem)
in detail. Note that the "problem" has an obvious, quick
solution via take(), but I wish it could be done with
the much more compact method of array indexing. I hope
my little snippet conveys what I'm after.

Basically, it seems to me that NumArray simply does not support
the distinction between a column vector and a row vector. That
is, if you have x=[1,2,3], then transpose(x) is a no-op. True?
(Note that doing x.shape=[3,1] does not do what I want; it produces
an awkward object that does not have the desired effect from
an array-indexing point of view.) Does this strike anyone
else as a rather serious limitation for someone (like me)
who would love to use Python/NumArray for my daily math
instead of, say, Matlab?

Thank you.

Mike D.

-----------------------cut here-----------------------------

Demo snippet:

from numarray import *
x = array(range(1,10), type=None, shape=[3,3])
print "(A) Original 3x3 array:\n", x
i = [1,2]
print "(B) An index set:\n", i
print "(C) 2nd and 3rd rows of x w/ take(x, i, 0):\n", take(x, i, 0)
print "(D) 2nd and 3rd cols of x w/ take(x, i, 1):\n", take(x, i, 1)
print "(E) 2nd and 3rd rows of x w/ x:\n", x
print "(F) 2nd and 3rd rows of x w/ transpose(transpose(x)):\n",
transpose(transpose(x))
print "(G) Wish x[transpose(i)] would work, but alas:\n",
x[transpose(i)]

It has this output:

(A) Original 3x3 array:
[[1 2 3]
[4 5 6]
[7 8 9]]
(B) An index set:
[1, 2]
(C) 2nd and 3rd rows of x w/ take(x, i, 0):
[[4 5 6]
[7 8 9]]
(D) 2nd and 3rd cols of x w/ take(x, i, 1):
[[2 3]
[5 6]
[8 9]]
(E) 2nd and 3rd rows of x w/ x:
[[4 5 6]
[7 8 9]]
(F) 2nd and 3rd rows of x w/ transpose(transpose(x)):
[[2 3]
[5 6]
[8 9]]
(G) Wish x[transpose(i)] would work, but alas:
[[4 5 6]
[7 8 9]]

 

[Christopher T King]

I am new to NumArray and I wonder if someone can help me with
array-indexing. Here's the basic situation: Given a rank-2 array
(i.e., a matrix) it seems to be trivial, with array indexing,
to extract a subset of its *columns*. But it does not seem
to be trivial to extract a subset of its *rows*.

You can do this using slices, like so:

from numarray import *
a = array([[1, 2], [3, 4]])
a[:,1]

array([2, 4])

':' means 'take all values along this axis', just like how with standard
Python lists it means 'take all values in the list'.

Basically, it seems to me that NumArray simply does not support
the distinction between a column vector and a row vector. That
is, if you have x=[1,2,3], then transpose(x) is a no-op. True?

False. You have to supply numarray with a two-dimensional array in order
to perform a two-dimensional transpose:

transpose([[1, 2, 3]])

array([[1],
[2],
[3]])

 

[Christopher T King]

Basically, it seems to me that NumArray simply does not support
the distinction between a column vector and a row vector. That
is, if you have x=[1,2,3], then transpose(x) is a no-op. True?

False. You have to supply numarray with a two-dimensional array in order
to perform a two-dimensional transpose:

Where by "False" I meant "The first sentence is false, but the second
sentence is true". Column vectors and row vectors must be represented as
two-dimensional arrays; transpose() of a one-dimensional array is a no-op
since all transpose() does (by default) is reverse the order of the axes.

 

[Michael Drumheller]

I am new to NumArray and I wonder if someone can help me with
array-indexing. Here's the basic situation: Given a rank-2 array
(i.e., a matrix) it seems to be trivial, with array indexing,
to extract a subset of its *columns*. But it does not seem
to be trivial to extract a subset of its *rows*.

You can do this using slices, like so:

from numarray import *
a = array([[1, 2], [3, 4]])
a[:,1]

array([2, 4])

':' means 'take all values along this axis', just like how with standard
Python lists it means 'take all values in the list'.

But that just gets me *one* column. I was trying to extract
an arbitrary subset of columns. Like I pointed out in the example
code I gave, this is trivial get take(), but does not seem to
be trivial with array indexing (and impossible with slicing).
Mike

 

[Michael Drumheller]

Basically, it seems to me that NumArray simply does not support
the distinction between a column vector and a row vector. That
is, if you have x=[1,2,3], then transpose(x) is a no-op. True?

False. You have to supply numarray with a two-dimensional array in order
to perform a two-dimensional transpose:

Where by "False" I meant "The first sentence is false, but the second
sentence is true". Column vectors and row vectors must be represented as
two-dimensional arrays; transpose() of a one-dimensional array is a no-op
since all transpose() does (by default) is reverse the order of the axes.

I understand that, but I think it just goes to support my contention that
NumArray does not support transposed vectors *from an array-indexing point
of view.* Here is what I mean:

As you pointed out in your previous message, you can get "the transpose
of a row vector" if the "row vector" is actually a single-row matrix,
i.e.,

transpose([[1, 2, 3]])

array([[1],
[2],
[3]])

However, whereas [1, 2, 3] passed as an index-array will get you the
second, third, and fourth rows of a rank-2 matrix, [[1,2,3]] will *not*
do that. (It gets you some other weird thing that I can't remember.)
That is, a single-row matrix may be the same thing as a row vector
in a mathematical context, but it is not the same thing in an
array-indexing context. Similarly, passing [[1],
[2],
[3]]
as an index array doesn't get you anything remotely like the second
third, and fourth columns. So it seems to me that array indexing
can easily get you an arbitrary subset of rows, but not an arbitrary
subset of columns. Would you agree?
By the way: Thank you for your attention to my problem!
Mike

 

[Peter Otten]

array-indexing. Here's the basic situation: Given a rank-2 array
(i.e., a matrix) it seems to be trivial, with array indexing,
to extract a subset of its columns. But it does not seem
to be trivial to extract a subset of its rows. The code

Could it be you are looking for the Ellipsis (...)?

a = numarray.array(range(9), shape=[3,3])
a

array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])

a[0:2]

array([[0, 1, 2],
[3, 4, 5]])

a[..., 1] array([1, 4, 7])
a[..., 1:]

array([[1, 2],
[4, 5],
[7, 8]])

a[..., 0:3:2]

array([[0, 2],
[3, 5],
[6, 8]])

Of course in 2D you do not really need it:

a[:, :2]

array([[0, 1],
[3, 4],
[6, 7]])


But at some point it may make things clearer:

a = numarray.array(range(2**10), shape=[2]*10)
a[...,:][(-1,)*7]

array([[[1016, 1017],
[1018, 1019]],

[[1020, 1021],
[1022, 1023]]])

Peter

 

[Christopher T King]

However, whereas [1, 2, 3] passed as an index-array will get you the
second, third, and fourth rows of a rank-2 matrix, [[1,2,3]] will *not*
do that. (It gets you some other weird thing that I can't remember.)
That is, a single-row matrix may be the same thing as a row vector
in a mathematical context, but it is not the same thing in an
array-indexing context. Similarly, passing [[1],
[2],
[3]]
as an index array doesn't get you anything remotely like the second
third, and fourth columns. So it seems to me that array indexing
can easily get you an arbitrary subset of rows, but not an arbitrary
subset of columns. Would you agree?

Forgive the poor flow of the following; I've rewritten it a couple of
times.

Think about what you're trying to do for a second. You want to pass a
list of indices to extract along a given axis. You need three pieces of
information to do this: which indices, which axis, and possibly the
geometry of the output vector. You're supplying the wrong information;
you're supplying which indices, and a geometry. Matlab guesses the right
information (which axis) from the wrong information (the geometry).

When you index a 2-dimensional array in Matlab, what happens? If the
index array is a row vector, then the indices you supply index the first
dimension. If the index array is a column vector, then the indices index
the second dimension. But what if the index array is two-dimensional?
Which dimension should the indices index? (For the record, they seem to
index the second dimension.)

numarray, on the other hand, provides an exact mechanism for supplying
index arrays thusly:

a=array([[1,2,3],[4,5,6],[7,8,9]])
a[[1,2],[0,0]]

array([4,7])

In this example, [1,2] are the row indices, and [0,0] are the column
indices (the information Matlab guesses for you). You can abbreviate
[0,0] as just 0:

a[[1,2],0]

array([4,7])

(try the previous in Matlab; you will get similar results)

You can index along another column vector if you want:

a[[1,2],1]

array([5,8])

Or you can arbitrarily index both dimensions (AFAICT, something not
possible in Matlab):

a[[1,2],[0,1]]

array([4,8])

Note that doing the above in Matlab results in a 2x2 matrix, probably not
what was wanted. The explicit numarray equivalent to that Matlab
construction is a[[[1],[2]],[[0,1]]].

Note also that the output vector takes the shape of the index vector:

a[[[1,2]],0] array([[4,7]])
a[[[1],[2]],0]

array([[4],
[7]])

So the answer to your question is, yes, not only can you use arbitrary
array indices in numarray, they're more powerful than the Matlab
equivalent. Yet another reason why explicit is better than implicit.