speed up linecache.getline()

Discussion in 'Python' started by bbarbero@inescporto.pt, Oct 13, 2009.

  1. Guest

    Hi Everyone!!


    I am using linecache.getline, to access to a line in a long file. It s
    really fast, appx 4seconds, but I was just wandering if any of you,
    know either another way, or there is something that I can do to speed
    it up... thank you very much for your help!!

    Regards,
    Bea











    Quoting Chris Rebert <>:

    > On Wed, Oct 7, 2009 at 10:21 AM, <> wrote:
    >> Hi again!
    >>
    >> After testing the whole day, I have got my goals from the last email, but as
    >> always, another issues came up! and now that Ive been able to save a list of
    >> list (or multi-arrays) as below :
    >>
    >> ['100.mp3\n' '10008.mp3\n' '10005.mp3\n' '10001.mp3\n' '10006.mp3\n']
    >> ['10001.mp3\n' '10005.mp3\n' '100.mp3\n' '10008.mp3\n' '10006.mp3\n']
    >> ['10005.mp3\n' '10001.mp3\n' '100.mp3\n' '10008.mp3\n' '10006.mp3\n']
    >> ['10006.mp3\n' '10005.mp3\n' '10001.mp3\n' '100.mp3\n' '10008.mp3\n']
    >> ['10008.mp3\n' '100.mp3\n' '10001.mp3\n' '10005.mp3\n' '10006.mp3\n']
    >>
    >> I am not able to manipulate it again! I read it with:
    >> Myfile.read() and all what I get is a str type data, what make my aim very
    >> difficult to reach!  What I want, is just to read one line(one specific
    >> line, so  I wouldnt have to read the whole file) and to get the numbers of
    >> the songs from that line. Maybe I should save the information in another
    >> way... But I just get those lines as lists, and write them in a file. Is
    >> there a better way? I am very receptive to suggestions! Thanks again for
    >> your help!

    >
    > Have you considered using the `json` module
    > (http://docs.python.org/library/json.html) to serialize and
    > deserialize the lists to/from a file in JSON format?
    > The `pickle` module is another option:
    > http://docs.python.org/library/pickle.html
    >
    > Cheers,
    > Chris
    > --
    > http://blog.rebertia.com
    >




    ----------------------------------------------------------------
    This message was sent using IMP, the Internet Messaging Program.
     
    , Oct 13, 2009
    #1
    1. Advertising

  2. Peter Otten Guest

    wrote:

    > I am using linecache.getline, to access to a line in a long file. It s
    > really fast, appx 4seconds, but I was just wandering if any of you,
    > know either another way, or there is something that I can do to speed
    > it up... thank you very much for your help!!


    If it is a single file that never changes just read it into a list:

    with open(filename) as f:
    cached_lines = list(f)

    If that alone takes about 4 seconds the runtime of your script is spent
    reading the file from disk. An SSD might help then ;)

    In some situations you could delay reading line n until that line is
    actually needed. This may speed up your script for cases where all requested
    lines are near the beginning of the cached file.

    Peter
     
    Peter Otten, Oct 13, 2009
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ham

    I need speed Mr .Net....speed

    Ham, Oct 28, 2004, in forum: ASP .Net
    Replies:
    6
    Views:
    2,356
    Antony Baula
    Oct 29, 2004
  2. Ross Hetherington

    linecache and comparison with input

    Ross Hetherington, Jun 30, 2007, in forum: Python
    Replies:
    2
    Views:
    366
    Ross Hetherington
    Jun 30, 2007
  3. jo3c

    linecache and glob

    jo3c, Jan 4, 2008, in forum: Python
    Replies:
    5
    Views:
    498
  4. R. Bernstein
    Replies:
    3
    Views:
    380
    R. Bernstein
    Dec 21, 2008
  5. rocky
    Replies:
    0
    Views:
    109
    rocky
    Oct 23, 2009
Loading...

Share This Page