Re: numpy.frombuffer != unpack() ??

Discussion in 'Python' started by Gary Herron, May 18, 2008.

  1. Gary Herron

    Gary Herron Guest

    Marlin Rowley wrote:
    > Actually in my traversal of the never-ending maze of understanding
    > arrays in python, I stumbled that the data in the wrong sequence.
    >
    > Let's get back to this transpose() deal, say we have these values as
    > integers (representing rgba again):
    >
    > [[[0,1,2,3]
    > [4,5,6,7]
    > [8,9,10,11]
    > [12,13,14,15]]
    >
    > [[16,17,18,19]
    > [20,21,22,23]
    > [24,25,26,27]
    > [28,29,30,31]]]
    >
    > Now if I do this: transpose((2,0,1)), I get this:
    >
    > [[[0,4,8,12] [16,20,24,28]]
    > [[1,5,9,13] [17,21,25,29]]
    > [[2,6,10,14][18,22,26,30]]
    > [[3,7,11,15][19,23,27,31]]]
    >
    > This is NOT what I want. I want the new array to be:
    >
    > [0,4,8,12][1,5,9,13]
    > [2,6,10,14][3,7,11,15]
    > [16,20,24,28][17,21,25,29]
    > [18,22,26,30][19,23,27,31]
    >
    > How do I do this?


    That's s little ambiguous, but one of the following two
    transpose-reshape-print's might be what you want.

    Gary Herron


    import numpy

    a = numpy.array([[[0,1,2,3],
    [4,5,6,7],
    [8,9,10,11],
    [12,13,14,15]],

    [[16,17,18,19],
    [20,21,22,23],
    [24,25,26,27],
    [28,29,30,31]]])


    print numpy.reshape(a.transpose(0,2,1), (8,4))
    print numpy.reshape(a.transpose(0,2,1), (4,2,4))

    output is:

    [[ 0 4 8 12]
    [ 1 5 9 13]
    [ 2 6 10 14]
    [ 3 7 11 15]
    [16 20 24 28]
    [17 21 25 29]
    [18 22 26 30]
    [19 23 27 31]]

    and

    [[[ 0 4 8 12]
    [ 1 5 9 13]]

    [[ 2 6 10 14]
    [ 3 7 11 15]]

    [[16 20 24 28]
    [17 21 25 29]]

    [[18 22 26 30]
    [19 23 27 31]]]



    >
    > -M
    >
    >
    >
    >
    >
    > ------------------------------------------------------------------------
    >
    > > Date: Sat, 17 May 2008 08:58:08 -0700
    > > From:
    > > To:
    > > CC:
    > > Subject: Re: numpy.frombuffer != unpack() ??
    > >
    > > Marlin Rowley wrote:
    > > >
    > > > Very cool.
    > > >
    > > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
    > > > a represents a tile with height of 2 and width of 4 with 4 bits/pixel
    > > > for each color.
    > > >
    > > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
    > > > this seperates the stream into individual values - Check
    > > >
    > > > > >>> b.shape=(2,4,4)
    > > >
    > > > This reshapes the array so that b.shape=(height,width,#bits/pixel)

    > - Check
    > > >
    > > > >>> c = b.transpose((2,0,1))
    > > >
    > > > What does the (2,0,1) represent in terms of width and height and
    > > > number of bytes?

    > >
    > > The (2,0,1) tells how to exchange the axes. For instance in a 2D array,
    > > a normal transpose exchanges rows and columns. It will change a (a by
    > > b) sized array into a (b by a) sized array. This would be equivalent to
    > > the more saying interchange axes 0,1 to the new order of 1,0.
    > >
    > > In numpy with higher dimension arrays, the default transpose just
    > > exchanges the first two axes, and the full transpose allows you to
    > > specify exactly the new ordering of the exes.
    > >
    > > So transpose((2,0,1)) means take axes (0,1,2) to the new order
    > > (2,1,0). In terms of sizes, an (a by b by c) sized array will end being
    > > of size (c by a by b) in size.
    > >
    > > In terms of implementation, there may not be *any* data re-arrangement
    > > in a transpose. The only thing that needs changing is how the indices
    > > are converted to an actual machine address of an indexed item. The
    > > documentation notes this by saying transpose returns a "new view" of

    > the
    > > array. This explains why I copied the array before extracting bytes
    > > out of it -- you really do need the elements in the new order for the
    > > next operation.
    > >
    > > Gary Herron
    > >
    > > >
    > > >
    > > >
    > > >
    > > >
    > > >

    > ------------------------------------------------------------------------
    > > >
    > > > > Date: Fri, 16 May 2008 17:08:20 -0700
    > > > > From:
    > > > > To: ;
    > > > > Subject: Re: numpy.frombuffer != unpack() ??
    > > > >
    > > > > Marlin Rowley wrote:
    > > > > > All:
    > > > > >
    > > > > > Say I have an array:
    > > > > >
    > > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
    > > > > >
    > > > > > How do I make it so that I now have:
    > > > > >
    > > > > > starting with first element (a[0])
    > > > > > new_arr[0] = 'r'
    > > > > > new_arr[1] = 'g'
    > > > > > new_arr[2] = 'b'
    > > > > > new_arr[3] = 'a'
    > > > > > new_arr[4] = 'r'
    > > > > > .....
    > > > > >
    > > > > > continuing "through" a[1] with the same new_arr
    > > > > > new_arr[N] = 'r'
    > > > > > new_arr[N+1] = 'g'
    > > > > > ....
    > > > > >
    > > > > > -M
    > > > >
    > > > > Numpy can do this for you. First, do you really mean the array to
    > > > > contain lists of one string each? If so:
    > > > >
    > > > > >>> import numpy
    > > > > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
    > > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
    > > > > kludge here
    > > > > >>> b
    > > > > array(['r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b',

    > 'b', 'a',
    > > > > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
    > > > > 'b', 'b', 'a', 'a', 'a', 'a'],
    > > > > dtype='|S1')
    > > > > >>> b.shape=(2,4,4)
    > > > > >>> b
    > > > > array([[['r', 'r', 'r', 'r'],
    > > > > ['g', 'g', 'g', 'g'],
    > > > > ['b', 'b', 'b', 'b'],
    > > > > ['a', 'a', 'a', 'a']],
    > > > >
    > > > > [['r', 'r', 'r', 'r'],
    > > > > ['g', 'g', 'g', 'g'],
    > > > > ['b', 'b', 'b', 'b'],
    > > > > ['a', 'a', 'a', 'a']]],
    > > > > dtype='|S1')
    > > > > >>> c = b.transpose((2,0,1))
    > > > > >>> c
    > > > > array([[['r', 'g', 'b', 'a'],
    > > > > ['r', 'g', 'b', 'a']],
    > > > >
    > > > > [['r', 'g', 'b', 'a'],
    > > > > ['r', 'g', 'b', 'a']],
    > > > >
    > > > > [['r', 'g', 'b', 'a'],
    > > > > ['r', 'g', 'b', 'a']],
    > > > >
    > > > > [['r', 'g', 'b', 'a'],
    > > > > ['r', 'g', 'b', 'a']]],
    > > > > dtype='|S1')
    > > > > >>> d=c.copy() # To make it contiguous
    > > > > >>> d.shape = (32,)
    > > > > >>> d
    > > > > array(['r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b',

    > 'a', 'r',
    > > > > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
    > > > > 'b', 'a', 'r', 'g', 'b', 'a'],
    > > > > dtype='|S1')
    > > > >
    > > > > Done. Cool no?
    > > > >
    > > > > Gary Herron
    > > > >
    > > > > >
    > > > > >
    > > > > >
    > > > > >
    > > > > >
    > > >

    > ------------------------------------------------------------------------
    > > > > > From:
    > > > > > To: ;
    > > > > > Subject: RE: numpy.frombuffer != unpack() ??
    > > > > > Date: Fri, 16 May 2008 17:31:30 -0500
    > > > > >
    > > > > > Thank you! That solved it!
    > > > > >
    > > > > > -M
    > > > > >
    > > > > >
    > > > > >
    > > >

    > ------------------------------------------------------------------------
    > > > > >
    > > > > > > To:
    > > > > > > From:
    > > > > > > Subject: Re: numpy.frombuffer != unpack() ??
    > > > > > > Date: Fri, 16 May 2008 17:25:00 -0500
    > > > > > >
    > > > > > > Marlin Rowley wrote:
    > > > > > > > All:
    > > > > > > >
    > > > > > > > I'm getting different floating point values when I use numpy
    > > > > > vs. unpack().
    > > > > > > >
    > > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
    > > > > > > > buffer = unpack("!f", byte)
    > > > > > > >
    > > > > > > > frgba[0] != buffer[0]
    > > > > > > >
    > > > > > > > why? This is forcing me use the unpack() function since it's
    > > > > > giving me
    > > > > > > > the correct values. What am I doing wrong?
    > > > > > >
    > > > > > > Endianness, perhaps? '!' specifies big-endian data (an alias for
    > > > > > '>'). Most
    > > > > > > likely, you are on a little-endian platform. All of the dtypes
    > > > > > in numpy default
    > > > > > > to the native-endianness unless specified. If you want to read
    > > > > > big-endian data
    > > > > > > using numpy, do this:
    > > > > > >
    > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
    > > > > > >
    > > > > > > If you have any more problems with numpy, please join us on the
    > > > > > numpy mailing
    > > > > > > list. When reporting problems, please try to provide a small but
    > > > > > complete
    > > > > > > snippet of self-contained code, the output that you got, and
    > > > > > explain the output
    > > > > > > that you expected to get. Thank you.
    > > > > > >
    > > > > > > http://www.scipy.org/Mailing_Lists
    > > > > > >
    > > > > > > --
    > > > > > > Robert Kern
    > > > > > >
    > > > > > > "I have come to believe that the whole world is an enigma, a
    > > > > > harmless enigma
    > > > > > > that is made terrible by our own mad attempt to interpret it as
    > > > > > though it had
    > > > > > > an underlying truth."
    > > > > > > -- Umberto Eco
    > > > > > >
    > > > > > > --
    > > > > > > http://mail.python.org/mailman/listinfo/python-list
    > > > > >
    > > > > >
    > > > > >
    > > >

    > ------------------------------------------------------------------------
    > > > > > E-mail for the greater good. Join the i’m Initiative from
    > > > > > Microsoft.
    > > > > >
    > > >

    > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
    > > > > >
    > > > > >
    > > > > >
    > > > > >
    > > >

    > ------------------------------------------------------------------------
    > > > > > E-mail for the greater good. Join the i’m Initiative from

    > Microsoft.
    > > > > >
    > > >

    > <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
    >
    > > >
    > > > > >
    > > > > >
    > > >

    > ------------------------------------------------------------------------
    > > > > >
    > > > > > --
    > > > > > http://mail.python.org/mailman/listinfo/python-list
    > > > >
    > > >
    > > >
    > > >

    > ------------------------------------------------------------------------
    > > > Keep your kids safer online with Windows Live Family Safety. Help
    > > > protect your kids.
    > > >

    > <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008>
    >
    > > >
    > > >

    > ------------------------------------------------------------------------
    > > >
    > > > --
    > > > http://mail.python.org/mailman/listinfo/python-list

    > >

    >
    >
    > ------------------------------------------------------------------------
    > Keep your kids safer online with Windows Live Family Safety. Help
    > protect your kids.
    > <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008>
    >
    > ------------------------------------------------------------------------
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
     
    Gary Herron, May 18, 2008
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ivan.dm
    Replies:
    2
    Views:
    1,119
    ivan.dm
    Jan 9, 2006
  2. drife
    Replies:
    1
    Views:
    369
    Travis E. Oliphant
    Mar 1, 2006
  3. Duncan Smith
    Replies:
    3
    Views:
    424
    Duncan Smith
    Apr 25, 2007
  4. Replies:
    2
    Views:
    492
    Robert Kern
    Nov 13, 2007
  5. News123

    Image.frombuffer and warning

    News123, Mar 3, 2010, in forum: Python
    Replies:
    2
    Views:
    1,570
    News123
    Mar 3, 2010
Loading...

Share This Page