I
Isaac Won
Dear all,
I am trying to calculate correlation coefficients between one time series data and other time series. However,there are some missing values. So, I interploated each time series with 1d interpolation in scipy and got correlation coefficients between them. This code works well for some data sets, but doesn't for some others. Following is actual error I got:
0.0708904109589
0.0801369863014
0.0751141552511
0.0938356164384
0.0769406392694
Traceback (most recent call last):
File "error_removed.py", line 56, in <module>
i2 = interp(indices)
File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 394, in __call__
out_of_bounds = self._check_bounds(x_new)
File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 449, in _check_bounds
raise ValueError("A value in x_new is below the interpolation "
ValueError: A value in x_new is below the interpolation range.
--------------------------------------------------------------------
This time is 'x_new is below the interpolation range", but some times, it shows
"above interpolation range.'
I would like to make some self-contained code, but, I am not sure how to make it to represent my case well.
I just put all of my code here. I apologize for this inconvenience.
-----------------------------------------------------------------------
---------
a = []
c = 4
with open(filin1, 'r') as f1:
arrays = [map(float, line.split()) for line in f1]
newa = [[x[1],x[2]] for x in arrays]
o = newa[58]
f = open(filin, "r")
percent1 = []
for columns in ( raw.strip().split() for raw in f ):
a.append(columns[63])
x = np.array(a, float)
not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i1 = interp(indices)
f.close
h1 = []
p1 = []
while c <278:
c = c + 1
d = c - 5
b = []
f.seek(0,0)
for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])
y = np.array(b, float)
h = haversine.distance(o, newa[d])
n = len(y)
l = b.count('nan')
percent = l/8760.
percent1 = percent1 + [percent]
#print l, percent
if percent < 0.1:
not_nan = np.logical_not(np.isnan(y))
indices = np.arange(len(y))
interp = interp1d(indices[not_nan], y[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i2 = interp(indices)
pearcoef = sp.pearsonr(i1,i2)
p = pearcoef[0]
p1 = p1 + [p]
h1 = h1 + [h]
print percent
print h1
print p1
print len(p1)
plt.plot(h1, p1, 'o')
plt.xlabel('Distance(km)')
plt.ylabel('Correlation coefficient')
plt.grid(True)
plt.show()
---------------------------------------------------------------
For any help or advice, I will really appreciate.
Best regards,
Isaac
I am trying to calculate correlation coefficients between one time series data and other time series. However,there are some missing values. So, I interploated each time series with 1d interpolation in scipy and got correlation coefficients between them. This code works well for some data sets, but doesn't for some others. Following is actual error I got:
0.0708904109589
0.0801369863014
0.0751141552511
0.0938356164384
0.0769406392694
Traceback (most recent call last):
File "error_removed.py", line 56, in <module>
i2 = interp(indices)
File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 394, in __call__
out_of_bounds = self._check_bounds(x_new)
File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 449, in _check_bounds
raise ValueError("A value in x_new is below the interpolation "
ValueError: A value in x_new is below the interpolation range.
--------------------------------------------------------------------
This time is 'x_new is below the interpolation range", but some times, it shows
"above interpolation range.'
I would like to make some self-contained code, but, I am not sure how to make it to represent my case well.
I just put all of my code here. I apologize for this inconvenience.
-----------------------------------------------------------------------
---------
a = []
c = 4
with open(filin1, 'r') as f1:
arrays = [map(float, line.split()) for line in f1]
newa = [[x[1],x[2]] for x in arrays]
o = newa[58]
f = open(filin, "r")
percent1 = []
for columns in ( raw.strip().split() for raw in f ):
a.append(columns[63])
x = np.array(a, float)
not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i1 = interp(indices)
f.close
h1 = []
p1 = []
while c <278:
c = c + 1
d = c - 5
b = []
f.seek(0,0)
for columns in ( raw.strip().split() for raw in f ):
b.append(columns[c])
y = np.array(b, float)
h = haversine.distance(o, newa[d])
n = len(y)
l = b.count('nan')
percent = l/8760.
percent1 = percent1 + [percent]
#print l, percent
if percent < 0.1:
not_nan = np.logical_not(np.isnan(y))
indices = np.arange(len(y))
interp = interp1d(indices[not_nan], y[not_nan])
#interp = np.interp(indices, indices[not_nan], x[not_nan])
i2 = interp(indices)
pearcoef = sp.pearsonr(i1,i2)
p = pearcoef[0]
p1 = p1 + [p]
h1 = h1 + [h]
print percent
print h1
print p1
print len(p1)
plt.plot(h1, p1, 'o')
plt.xlabel('Distance(km)')
plt.ylabel('Correlation coefficient')
plt.grid(True)
plt.show()
---------------------------------------------------------------
For any help or advice, I will really appreciate.
Best regards,
Isaac