weird behavior. bug perhaps?

[zoom]

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the following
output:

(4, 2)
[2, 1]
(1, 8)
..
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

m = [[1,2],[3,4],[3,4],[3,4]]

from scipy import matrix, tile, mean, shape
print shape(m) (4, 2)
print [shape(m)[1],1] [2, 1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

 

[rusi]

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

     def setUp(self):
         self.m = [[1,2],[3,4],[3,4],[3,4]]

     def test_simplify(self):
         m = matrix(self.m)
         print shape(m)
         print [shape(m)[1],1]
         print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
     unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to reproduce the weird behavior that I'm  about to describe.)

If i run it in terminal via "python test.py" command I get the following
output:

(4, 2)
[2, 1]
(1, 8)
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Now comes the funny part.
Let's try to run the following code in python interpreter:

 >>> m = [[1,2],[3,4],[3,4],[3,4]]
 >>>
 >>> from scipy import matrix, tile, mean, shape
 >>> print shape(m)
(4, 2)
 >>> print [shape(m)[1],1]
[2, 1]
 >>> print shape(tile(mean(m,1),[shape(m)[1],1]).T)
(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

[Never used scipy so pls excuse if I am off...]

Given list m, in the class you are doing m -> matrix(m)
which you dont seem to be doing in the interpreter.

 

[Marcel Rodrigues]

Note that

print [shape(m)[1],1]

just prints a list with two elements where the first element is shape(m)[1]
and the second is the number 1 (regardless of the value of m). I'm pretty
sure that's not what you want.

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(**unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)**[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the following
output:

(4, 2)
[2, 1]
(1, 8)
.
------------------------------**------------------------------**----------
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

m = [[1,2],[3,4],[3,4],[3,4]]

from scipy import matrix, tile, mean, shape
print shape(m) (4, 2)
print [shape(m)[1],1] [2, 1]
print shape(tile(mean(m,1),[shape(m)**[1],1]).T)

(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)**[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

 

[Robert Kern]

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file, sufficient to
reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the following output:

(4, 2)
[2, 1]
(1, 8)
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

m = [[1,2],[3,4],[3,4],[3,4]]

from scipy import matrix, tile, mean, shape
print shape(m) (4, 2)
print [shape(m)[1],1] [2, 1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

As rusi noted, the difference between your two snippets is that in one, you
converted the list of lists to a matrix object in your test suite but not in
your interactive session. Most numpy functions like mean() will convert their
arguments to regular numpy.ndarray objects rather than matrix objects. matrix is
a subclass of ndarray that adds special behavior: in particular, operations on
matrix objects retain their 2D-ness even when an ndarray would flatten down to a
1D array.

[~]
|1> import numpy as np

[~]
|2> m = [[1,2],[3,4],[3,4],[3,4]]

[~]
|3> a = np.array(m)

[~]
|4> b = np.matrix(m)

[~]
|5> np.mean(a, axis=1)
array([ 1.5, 3.5, 3.5, 3.5])

[~]
|6> np.mean(b, axis=1)
matrix([[ 1.5],
[ 3.5],
[ 3.5],
[ 3.5]])

[~]
|7> np.mean(a, axis=1).shape
(4,)

[~]
|8> np.mean(b, axis=1).shape
(4, 1)


This will propagate through the rest of your computation.

Personally, I recommend avoiding the matrix type. It causes too many problems.
Stick to plain ndarrays.

You will probably want to ask further numpy questions on the numpy-discussion
mailing list:

http://www.scipy.org/scipylib/mailing-lists.html

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[zoom]

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the following
output:

(4, 2)
[2, 1]
(1, 8)
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Now comes the funny part.
Let's try to run the following code in python interpreter:

m = [[1,2],[3,4],[3,4],[3,4]]

from scipy import matrix, tile, mean, shape
print shape(m) (4, 2)
print [shape(m)[1],1] [2, 1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

[Never used scipy so pls excuse if I am off...]

Given list m, in the class you are doing m -> matrix(m)
which you dont seem to be doing in the interpreter.

yes, that's the thing.

thanks a lot

FYI this happens because
(4,)

thanks again

 

[zoom]

Hi, I have a strange problem here. Perhaps someone would care to help me.

In the file test.py I have the following code:

from scipy import matrix, tile, mean, shape
import unittest

class TestSequenceFunctions(unittest.TestCase):

def setUp(self):
self.m = [[1,2],[3,4],[3,4],[3,4]]

def test_simplify(self):
m = matrix(self.m)
print shape(m)
print [shape(m)[1],1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

if __name__ == '__main__':
unittest.main()

(Note that test.py, is just a simplification of my testing file,
sufficient to
reproduce the weird behavior that I'm about to describe.)

If i run it in terminal via "python test.py" command I get the
following output:

(4, 2)
[2, 1]
(1, 8)
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK


Now comes the funny part.
Let's try to run the following code in python interpreter:

m = [[1,2],[3,4],[3,4],[3,4]]

from scipy import matrix, tile, mean, shape
print shape(m) (4, 2)
print [shape(m)[1],1] [2, 1]
print shape(tile(mean(m,1),[shape(m)[1],1]).T)

(4, 2)

Note the difference between outputs of:
print shape(tile(mean(m,1),[shape(m)[1],1]).T)


I mean, WTF?
This is definitely not the expected behavior.
Anybody knows what just happened here?

As rusi noted, the difference between your two snippets is that in one,
you converted the list of lists to a matrix object in your test suite
but not in your interactive session. Most numpy functions like mean()
will convert their arguments to regular numpy.ndarray objects rather
than matrix objects. matrix is a subclass of ndarray that adds special
behavior: in particular, operations on matrix objects retain their
2D-ness even when an ndarray would flatten down to a 1D array.

[~]
|1> import numpy as np

[~]
|2> m = [[1,2],[3,4],[3,4],[3,4]]

[~]
|3> a = np.array(m)

[~]
|4> b = np.matrix(m)

[~]
|5> np.mean(a, axis=1)
array([ 1.5, 3.5, 3.5, 3.5])

[~]
|6> np.mean(b, axis=1)
matrix([[ 1.5],
[ 3.5],
[ 3.5],
[ 3.5]])

[~]
|7> np.mean(a, axis=1).shape
(4,)

[~]
|8> np.mean(b, axis=1).shape
(4, 1)


This will propagate through the rest of your computation.

Personally, I recommend avoiding the matrix type. It causes too many
problems. Stick to plain ndarrays.

You will probably want to ask further numpy questions on the
numpy-discussion mailing list:

http://www.scipy.org/scipylib/mailing-lists.html

didn't see it earlier.

thanks.

so ndarrays, you say...

 

[rusi]

yes, that's the hing.

thanks a lot

FYI this happens because
 >>> shape(mean(m,1))
(4, 1)
 >>> shape(mean(array(m),1))
(4,)

thanks again

And thank you for the 'Thank you' !!
Given the noob-questions the list is currently dealing with, your
question was a relief and a pleasure.
You have (unwittingly maybe??) followed http://sscce.org/ Thats great.
Also the 'Thanks' at end helps not just to tickle the ego of the
responder but also to confirm the answer which is often not sure.

The only small suggestion I would make is next time please spend a
minute on choosing a pointed subject line.