parsing tab separated data efficiently into numpy/pylab arrays

Discussion in 'Python' started by per, Mar 13, 2009.

  1. per

    per Guest

    hi all,

    what's the most efficient / preferred python way of parsing tab
    separated data into arrays? for example if i have a file containing
    two columns one corresponding to names the other numbers:

    col1 \t col 2
    joe \t 12.3
    jane \t 155.0

    i'd like to parse into an array() such that i can do: mydata[:, 0] and
    mydata[:, 1] to easily access all the columns.

    right now i can iterate through the file, parse it manually using the
    split('\t') command and construct a list out of it, then convert it to
    arrays. but there must be a better way?

    also, my first column is just a name, and so it is variable in length
    -- is there still a way to store it as an array so i can access: mydata
    [:, 0] to get all the names (as a list)?

    thank you.
    per, Mar 13, 2009
    #1
    1. Advertising

  2. per

    Matteo Guest

    On 13 Mar, 23:19, per <> wrote:
    > hi all,
    >
    > what's the most efficient / preferred python way of parsing tab
    > separated data into arrays? for example if i have a file containing
    > two columns one corresponding to names the other numbers:
    >
    > col1    \t     col 2
    > joe    \t  12.3
    > jane   \t 155.0
    >
    > i'd like to parse into an array() such that i can do: mydata[:, 0] and
    > mydata[:, 1] to easily access all the columns.
    >
    > right now i can iterate through the file, parse it manually using the
    > split('\t') command and construct a list out of it, then convert it to
    > arrays. but there must be a better way?
    >
    > also, my first column is just a name, and so it is variable in length
    > -- is there still a way to store it as an array so i can access: mydata
    > [:, 0] to get all the names (as a list)?
    >
    > thank you.


    I think you can do it through:

    array.fromfile()
    array.reshape()

    but you should look up the reference for those.
    Matteo, Mar 13, 2009
    #2
    1. Advertising

  3. per

    Guest

    You could take a look/use the very handy csv2rec function in
    matplotlib.mlab, which creates numpy struct arrays.

    Marco

    On Mar 13, 10:19 pm, per <> wrote:
    > hi all,
    >
    > what's the most efficient / preferred python way ofparsingtab
    > separated data intoarrays? for example if i have a file containing
    > two columns one corresponding to names the other numbers:
    >
    > col1    \t     col 2
    > joe    \t  12.3
    > jane   \t 155.0
    >
    > i'd like to parse into an array() such that i can do: mydata[:, 0] and
    > mydata[:, 1] to easily access all the columns.
    >
    > right now i can iterate through the file, parse it manually using the
    > split('\t') command and construct a list out of it, then convert it toarrays. but there must be a better way?
    >
    > also, my first column is just a name, and so it is variable in length
    > -- is there still a way to store it as an array so i can access: mydata
    > [:, 0] to get all the names (as a list)?
    >
    > thank you.
    , Mar 24, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Martijn van Exel

    WebService output > Tab-separated text?

    Martijn van Exel, Nov 2, 2004, in forum: XML
    Replies:
    1
    Views:
    431
    Joris Gillis
    Nov 3, 2004
  2. timw.google

    pylab doesn't find numpy on Windows

    timw.google, Jun 13, 2006, in forum: Python
    Replies:
    2
    Views:
    635
    timw.google
    Jun 13, 2006
  3. Cliff  Martin

    Parsing tab separated file

    Cliff Martin, Jan 31, 2007, in forum: C++
    Replies:
    1
    Views:
    332
    rossum
    Jan 31, 2007
  4. Phil

    pylab numpy zeros

    Phil, Nov 5, 2007, in forum: Python
    Replies:
    0
    Views:
    432
  5. Adam

    Parsing delimiter-separated data.

    Adam, Nov 14, 2003, in forum: Perl Misc
    Replies:
    4
    Views:
    130
Loading...

Share This Page