J
Jia Hu
Hi,
My 20 csv files has string header, and first two columns are string (e.g.,
1999-01-02, 01:00:00) among the 50 columns. Other columns store numerical
values (int, or float)
I need to do data analysis for these data. For example, extract the each
month data from each of the cvs files (each csv file stores 1 year data)
and there are 20 year data.
in addition, I want to store the data in disk so that I can retrieve data
quickly, just like save and load in Matlab.
Currently, I use structured array
data = []
i = 0
for s in range(1991, 2011):
fileName = folder +_{_sY}0101_{_sY}1231_725300.csv".format(_sY=s)
data.append(np.genfromtxt(fileName, delimiter=",", dtype=None,
names=True))
i += 1
np.save("alldata", data)
====
However, when I load data "np.load("alldata.npy")", it is becomes 0-d array
which is different from original one.
My question is that
(1) How to store or save the data?
(2) as you can see, I use list to store all the 20 ndarrays, I do not feel
it is a good way. Is there any suggestion for the data structure I should
use?
Thanks in advance.
My 20 csv files has string header, and first two columns are string (e.g.,
1999-01-02, 01:00:00) among the 50 columns. Other columns store numerical
values (int, or float)
I need to do data analysis for these data. For example, extract the each
month data from each of the cvs files (each csv file stores 1 year data)
and there are 20 year data.
in addition, I want to store the data in disk so that I can retrieve data
quickly, just like save and load in Matlab.
Currently, I use structured array
data = []
i = 0
for s in range(1991, 2011):
fileName = folder +_{_sY}0101_{_sY}1231_725300.csv".format(_sY=s)
data.append(np.genfromtxt(fileName, delimiter=",", dtype=None,
names=True))
i += 1
np.save("alldata", data)
====
However, when I load data "np.load("alldata.npy")", it is becomes 0-d array
which is different from original one.
My question is that
(1) How to store or save the data?
(2) as you can see, I use list to store all the 20 ndarrays, I do not feel
it is a good way. Is there any suggestion for the data structure I should
use?
Thanks in advance.