CSV to matrix array

A

Ana Dionísio

Hello!

I have a CSV file with 20 rows and 12 columns and I need to store it as a matrix. I already created an array with zeros, but I don't know how to fill it with the data from the csv file. I have this script:

import numpy
from numpy import array
from array import *
import csv

input = open('Cenarios.csv','r')
cenario = csv.reader(input)

array=numpy.zeros([20, 12])


I know I have to use for loops but I don't know how to use it to put the data the way I want. Can you help me?

Thanks!
 
M

Mark Lawrence

Hello!

I have a CSV file with 20 rows and 12 columns and I need to store it as a matrix. I already created an array with zeros, but I don't know how to fill it with the data from the csv file. I have this script:

import numpy
from numpy import array
from array import *
import csv

input = open('Cenarios.csv','r')
cenario = csv.reader(input)

array=numpy.zeros([20, 12])


I know I have to use for loops but I don't know how to use it to put the data the way I want. Can you help me?

Thanks!

I'm no expert on numpy but there is a loadtxt function see
http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html
 
A

Ana Dionísio

That only puts the data in one column, I wanted to separate it.

For example:
data in csv file:

1 2 3 4 5
7 8 9 10 11
a b c d e

I wanted an array where I could pick an element in each position. In the case above if I did print array[0][3] it would pick 4
 
A

Ana Dionísio

That only puts the data in one column, I wanted to separate it.

For example:
data in csv file:

1 2 3 4 5
7 8 9 10 11
a b c d e

I wanted an array where I could pick an element in each position. In the case above if I did print array[0][3] it would pick 4
 
R

rusi

Hi, thanks for yor answer! ;)

Anyone has more suggestions?

My suggestions:

1. Tell us what was lacking in Mark's suggestion (to use loadtxt)
2. Read his postscript (for googlegroup posters).
[In case you did not notice your posts are arriving in doubles]
 
D

Dave Angel

That only puts the data in one column, I wanted to separate it.

For example:
data in csv file:

1 2 3 4 5
7 8 9 10 11
a b c d e

I wanted an array where I could pick an element in each position. In the case above if I did print array[0][3] it would pick 4

I know nothing about numpy.

How about something like:
import csv
myreader = csv.reader(....)
mylist = list(myreader)


Presumably you can fill in the details. Anyway, I think this will give
you a list of lists, and perhaps you can convert that to a numpy array,
if you really need one of those.
 
J

Javier Miranda

Keep the flattened data array others suggested, and then just split it like
this: *(replace `example_data`, `_array`, and `columns`)*

.. . . [_array[i:i + colums] for i in \

.. . . xrange(0, len(_array), colums)]

[[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13, 14]]

Gist <https://gist.github.com/anonymous/f9064e4c8790ae037ec6>

What do you guys think?


That only puts the data in one column, I wanted to separate it.

For example:
data in csv file:

1 2 3 4 5
7 8 9 10 11
a b c d e

I wanted an array where I could pick an element in each position. In the
case above if I did print array[0][3] it would pick 4
I know nothing about numpy.

How about something like:
import csv
myreader = csv.reader(....)
mylist = list(myreader)


Presumably you can fill in the details. Anyway, I think this will give
you a list of lists, and perhaps you can convert that to a numpy array, if
you really need one of those.
 
G

giacomo boffi

Ana Dionísio said:
Hello!

I have a CSV file with 20 rows and 12 columns and I need to store it
as a matrix.

array=numpy.array([row for row in csv.reader(open('Cenarios.csv'))])

NB: i used "array=" as in your sample code, BUT
 
A

Ana Dionísio

It's still not working. I still have one column with all the data inside, like this:

2999;T3;3;1;1;Off;ON;OFF;ON;ON;ON;ON;Night;;;;;;

How can I split this data in a way that if I want to print "T3" I would just do "print array[0][1]"?
 
M

Mark Lawrence

It's still not working. I still have one column with all the data inside, like this:

2999;T3;3;1;1;Off;ON;OFF;ON;ON;ON;ON;Night;;;;;;

How can I split this data in a way that if I want to print "T3" I would just do "print array[0][1]"?

I said before I'm no expert on numpy but my understanding is that all
arrays are homogeneous, hence you can't load the data you show above
without some form of mapping. In that case you'd have to read the data
with the csv module as others have already suggested, apply your mapping
and then write this to your array. The obvious alternative is to use a
list of lists.
 
C

cantorp

Dear Ana,

your example data could be transformed into a matrix with
import csv
rows = csv.reader(open("your_data_file.csv"), delimiter=" ")
array = [row for row in rows]
array[0][3]
4

HTH
Paolo

Am Freitag, 12. April 2013 19:29:05 UTC+2 schrieb Ana Dionísio:
That only puts the data in one column, I wanted to separate it.



For example:

data in csv file:



1 2 3 4 5

7 8 9 10 11

a b c d e



I wanted an array where I could pick an element in each position. In the case above if I did print array[0][3] it would pick 4
 
C

cantorp

Dear Ana,

your example data could be transformed into a matrix with
import csv
rows = csv.reader(open("your_data_file.csv"), delimiter=" ")
array = [row for row in rows]
array[0][3]
4

HTH
Paolo

Am Freitag, 12. April 2013 19:29:05 UTC+2 schrieb Ana Dionísio:
That only puts the data in one column, I wanted to separate it.



For example:

data in csv file:



1 2 3 4 5

7 8 9 10 11

a b c d e



I wanted an array where I could pick an element in each position. In the case above if I did print array[0][3] it would pick 4
 
O

Oscar Benjamin

It's still not working. I still have one column with all the data inside,like this:

2999;T3;3;1;1;Off;ON;OFF;ON;ON;ON;ON;Night;;;;;;

How can I split this data in a way that if I want to print "T3" I would just do "print array[0][1]"?

You initially reported that your data was a CSV file, which normally
means that the values in each row are separated by comma characters
e.g. ','. Actually the data here are separated by semicolons e.g. ';'.
This means that whether you use numpy or the csv module you will need
to specify that the data is separated by semicolons. In numpy you
would do this with

import numpy
data = numpy.loadtxt('file.csv', dtype=int, delimiter=';')

You need to set dtype to be whatever data type you want to convert the
values to e.g. int or float. This is because numpy arrays are
homogeneous. In your case the data (presumably a channel/event header
from an EEG file) is not homogeneous as you have integer data '2999'
followed by the channel name 'T3' which is a string. You can load all
values as strings with

data = numpy.loadtxt('file.csv', dtype=str, delimiter=';')

It is possible to have heterogeneous types in a numpy array using
dtype=object but if you use that with the loadtxt function it will
just use strings for all values.

Alternatively you can use the csv module in the standard library to
load all the data as strings

import csv
with open('file.csv', 'rb') as csvfile:
data = list(csv.reader(csvfile, delimiter=';'))

This will give you a list of lists of strings rather than a numpy
array. Afterwards you can convert the integer values to int if you
want like so:

for row in data:
row[0] = int(row[0])

This works because lists can store heterogeneous data, unlike numpy arrays.

Either of the above will let you access the data with e.g. data[2][7]
to get the value from the 8th column of the 3rd row. However, I think
that the better thing to do though would be to use a csv.DictReader
and store your data as a list of dicts. This would look like:

# Change this to names that actually describe each column of your data
columns = ['sample_rate', 'channel_name', 'electrode_number',
'lowpass_filter',...]

data = []
with open('file.csv') as csvfile:
for row in csv.DictReader(csvfile, fieldnames=columns, delimiter=';'):
# Convert non-string data here e.g.:
row['sample_rate'] = int(row['sample_rate'])
data.append(row)

Now you can access the data using e.g. data[0]['channel_name'] which I
think is better than data[0][1] and you can store data of
heterogeneous type e.g. int, str, etc. in the same row.


Oscar
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,770
Messages
2,569,583
Members
45,072
Latest member
trafficcone

Latest Threads

Top