readlines() doesn't read entire file

[Jeremy]

I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file. I have the following
syntax:



# some initial stuff
XS = xsdir(Datapath + '/xsdir', options.debug)
# some more stuff

class xsdir(object): #{{{1
"""This class handles all of the data and methods for reading
the xsdir file."""

def __init__(self, Datapath, debug=False):
self.xsdir = file(Datapath, 'r') # File object
self.lines = self.xsdir.readlines()
if debug:
print self.lines
# and then other stuff as well


I can see all the lines in the list self.lines, but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines, but Python cannot. Can someone help me with this one?
Thanks,
Jeremy

 

[Peter Hansen]

I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file. I have the following
syntax:
....
self.xsdir = file(Datapath, 'r') # File object

I can see all the lines in the list self.lines, but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines, but Python cannot. Can someone help me with this one?

What platform? What version of Python?

You're opening the file in "text" mode. If you are on Windows and the
file actually contains a ^Z (byte 26) it is treated as EOF. Is that the
problem?

If not, have you tried cutting parts out of the file, to produce the
smallest file that still shows the problem? At that point you will
likely resolve the issue on your own. Also, does the same thing happen
if you use the interactive interpreter to read the file "manually"?

These are all basic troubleshooting techniques you can use at any time
on any problem...

-Peter

 

[Jeremy]

What platform? What version of Python?

You're opening the file in "text" mode. If you are on Windows and the
file actually contains a ^Z (byte 26) it is treated as EOF. Is that the
problem?

If not, have you tried cutting parts out of the file, to produce the
smallest file that still shows the problem? At that point you will
likely resolve the issue on your own. Also, does the same thing happen
if you use the interactive interpreter to read the file "manually"?

These are all basic troubleshooting techniques you can use at any time
on any problem...

-Peter

Well, now I have to make a humbling retraction. I realized I wasn't
reading the file I thought I was, but one similar to it. Now that I am
reading the correct file, I am getting all the lines I expected to.
Thanks,
Jeremy

 

[John Machin]

I have a most aggravating problem. I don't understand what is causing
readlines() not to read all the lines in the file.

Answer all of Peter Hansen's questions, then read on ...

You are on platform X; did you get the file from platform Y where Y != X?

Where did you get the file from? How was it constructed? What is the
encoding: ascii/utf-8/iso-something/cp125x/dunno/what's-an-encoding ...

I have the following
syntax:



# some initial stuff
XS = xsdir(Datapath + '/xsdir', options.debug)
# some more stuff

class xsdir(object): #{{{1
"""This class handles all of the data and methods for reading
the xsdir file."""

def __init__(self, Datapath, debug=False):
self.xsdir = file(Datapath, 'r') # File object

If you replace 'r' by 'rU', does the behaviour change?

self.lines = self.xsdir.readlines()
if debug:
print self.lines

Add this: print len(self.lines)

# and then other stuff as well


I can see all the lines in the list self.lines,

These are *not* pedantic questions: How many can you "see"? What does
"see" mean? What is your definition of "line"?

but they are not all the
lines in the file. When I look at the file in Vim, I can see all the
lines,

Repeat above questions.

AND where are the extra lines that you can "see" with vim? In one block
at the end/beginning/middle? Randomly interspersed? Are you sure you are
not being confused by a screen-wraparound effect?

but Python cannot. Can someone help me with this one?

Thanks,
Jeremy

What does "wc -l yourfile" tell you?
[If you are on Windows, get a copy of wc.exe from the gnuwin32 site.]