reading in lines from a file -FAST!

Discussion in 'Python' started by Rajarshi Guha, Jul 30, 2003.

  1. Hi
    I have a file containing 168092 lines (each line a single word) and when
    I use

    for line in f:
    s = s + line

    it takes for ages to read it all in - so long in fact that it makes the
    program unusable. Is there any way to do something like C's fread in
    Python so that I can just slurp in 1.7MB of data at one go, rather than
    reading line by line?

    Thanks,
    Rajarshi
    Rajarshi Guha, Jul 30, 2003
    #1
    1. Advertising

  2. Rajarshi Guha

    Adam Guest

    Rajarshi Guha wrote:

    > Hi
    > I have a file containing 168092 lines (each line a single word) and when
    > I use
    >
    > for line in f:
    > s = s + line
    >
    > it takes for ages to read it all in - so long in fact that it makes the
    > program unusable. Is there any way to do something like C's fread in
    > Python so that I can just slurp in 1.7MB of data at one go, rather than
    > reading line by line?
    >
    > Thanks,
    > Rajarshi


    Assuming f is an open File Object, you can use:

    string = f.read()

    Which reads the entire content of f into string. Notice, though, that
    this solution isn't scalable: reading an entire file to memory becomes
    messier the larger the file, whereas reading it one line at a time works
    pretty much the same no matter how big the file is.

    Adam
    Adam, Jul 30, 2003
    #2
    1. Advertising

  3. Rajarshi Guha

    sreekant Guest

    Why not use

    s=openfile.read()
    s=string.split(s,'\n')

    just a thought
    sree
    sreekant, Jul 30, 2003
    #3
  4. Rajarshi Guha

    Mark Day Guest

    In article <>,
    Rajarshi Guha <> wrote:

    > Hi
    > I have a file containing 168092 lines (each line a single word) and when
    > I use
    >
    > for line in f:
    > s = s + line
    >
    > it takes for ages to read it all in - so long in fact that it makes the
    > program unusable. Is there any way to do something like C's fread in
    > Python so that I can just slurp in 1.7MB of data at one go, rather than
    > reading line by line?


    How about:
    s = f.read()
    (assuming f is a file object)

    -Mark
    Mark Day, Jul 31, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    646
  2. Joe Wright
    Replies:
    0
    Views:
    499
    Joe Wright
    Jul 27, 2003
  3. Michele Simionato

    Python is darn fast (was: How fast is Python)

    Michele Simionato, Aug 23, 2003, in forum: Python
    Replies:
    13
    Views:
    551
  4. Murali
    Replies:
    2
    Views:
    544
    Jerry Coffin
    Mar 9, 2006
  5. Juha Nieminen
    Replies:
    22
    Views:
    996
    Kai-Uwe Bux
    Oct 12, 2007
Loading...

Share This Page