reading in lines from a file -FAST!

[Rajarshi Guha]

Hi
I have a file containing 168092 lines (each line a single word) and when
I use

for line in f:
s = s + line

it takes for ages to read it all in - so long in fact that it makes the
program unusable. Is there any way to do something like C's fread in
Python so that I can just slurp in 1.7MB of data at one go, rather than
reading line by line?

Thanks,
Rajarshi

 

[Adam]

Hi
I have a file containing 168092 lines (each line a single word) and when
I use

for line in f:
s = s + line

it takes for ages to read it all in - so long in fact that it makes the
program unusable. Is there any way to do something like C's fread in
Python so that I can just slurp in 1.7MB of data at one go, rather than
reading line by line?

Thanks,
Rajarshi

Assuming f is an open File Object, you can use:

string = f.read()

Which reads the entire content of f into string. Notice, though, that
this solution isn't scalable: reading an entire file to memory becomes
messier the larger the file, whereas reading it one line at a time works
pretty much the same no matter how big the file is.

Adam

 

[sreekant]

Why not use

s=openfile.read()
s=string.split(s,'\n')

just a thought
sree

 

[Mark Day]

Hi
I have a file containing 168092 lines (each line a single word) and when
I use

for line in f:
s = s + line

it takes for ages to read it all in - so long in fact that it makes the
program unusable. Is there any way to do something like C's fread in
Python so that I can just slurp in 1.7MB of data at one go, rather than
reading line by line?

How about:
s = f.read()
(assuming f is a file object)

-Mark