genfromtxt and comment identifier

S

simona bellavista

Hi All,

I have a problem with reading data from a file using genfromtxt of
numpy module.

I have prepared a minimal example similar to the ones presented in

http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#splitting-the-lines-into-columns

The script is

import numpy as np
from StringIO import StringIO
file = open('esempio.dat')
for line in file.xreadlines() :
if not line : break
print line
np.genfromtxt(StringIO(line), comments="#", delimiter=",")

I have a data file - esempio.dat - like the following:

#
# Skip me !
# Skip me too !
1, 2
3, 4
5, 6 #This is the third line of the data
7, 8
# And here comes the last line
9, 0
"""

The code is breaking at the first line, it looks like it doesn't
recognize # like comment identifier.

The error in the python interpreter is

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "esempio.py", line 7, in <module>
np.genfromtxt(StringIO(line), comments="#", delimiter=",")
File "/opt/numpy/1.5.1/lib/python2.6/site-packages/numpy/lib/
npyio.py", line 1174, in genfromtxt
raise IOError('End-of-file reached before encountering data.')
IOError: End-of-file reached before encountering data.


It is clear that I haven't understood something, what am I doing
wrong?

Cheers
 
P

Peter Otten

simona said:
Hi All,

I have a problem with reading data from a file using genfromtxt of
numpy module.

I have prepared a minimal example similar to the ones presented in

http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#splitting- the-lines-into-columns

The script is

import numpy as np
from StringIO import StringIO
file = open('esempio.dat')
for line in file.xreadlines() :
if not line : break
print line
np.genfromtxt(StringIO(line), comments="#", delimiter=",")

I have a data file - esempio.dat - like the following:

#
# Skip me !
# Skip me too !
1, 2
3, 4
5, 6 #This is the third line of the data
7, 8
# And here comes the last line
9, 0
"""

The code is breaking at the first line, it looks like it doesn't
recognize # like comment identifier.

The error in the python interpreter is

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "esempio.py", line 7, in <module>
np.genfromtxt(StringIO(line), comments="#", delimiter=",")
File "/opt/numpy/1.5.1/lib/python2.6/site-packages/numpy/lib/
npyio.py", line 1174, in genfromtxt
raise IOError('End-of-file reached before encountering data.')
IOError: End-of-file reached before encountering data.


It is clear that I haven't understood something, what am I doing
wrong?

The examples use StringIO to simulate a file, but you are wrapping every
line of your actual file. The first "simulated file" is then

StringIO("#\n")

i. e. it contains only a comment, no data -- and that's what genfromtxt()
complains about. Read the file in one fell swoop and you should be OK:

import numpy as np

with open('esempio.dat') as instream:
print np.genfromtxt(instream, comments="#", delimiter=",")
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top