Re: Python garbage collector/memory manager behaving strangely

Discussion in 'Python' started by Dennis Lee Bieber, Sep 17, 2012.

  1. On Mon, 17 Sep 2012 10:28:34 +0800, "Jadhav, Alok"
    <> declaimed the following in
    gmane.comp.python.general:


    > - As you have seen, the line separator is not '\n' but its '|\n'.
    > Sometimes the data itself has '\n' characters in the middle of the line
    > and only way to find true end of the line is that previous character
    > should be a bar '|'. I was not able specify end of line using
    > readlines() function, but I could do it using split() function.
    > (One hack would be to readlines and combine them until I find '|\n'. is
    > there a cleaner way to do this?)


    Then chunk the file...

    {PSEUDOCODE}

    last = ""
    while True:
    chunk = fin.read(500000) #read 1/2 MB max
    #and attach to left-overs
    if not chunk: #EOF, process any left overs
    processLine(last)
    break #and exit

    lines = (last + chunk).split("|\n")
    last = lines[-1] #save "partial" line

    for ln in lines[:-1]: #loop over rest
    processLine(ln)

    fin.close()
    --
    Wulfraed Dennis Lee Bieber AF6VN
    HTTP://wlfraed.home.netcom.com/
    Dennis Lee Bieber, Sep 17, 2012
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?iso-8859-1?q?Nils Hedstr=f6m

    Stateserver behaving strangely

    =?iso-8859-1?q?Nils Hedstr=f6m, Feb 11, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    1,026
    =?iso-8859-1?q?Nils Hedstr=f6m
    Feb 11, 2005
  2. Josh Close

    re behaving strangely

    Josh Close, Jun 9, 2005, in forum: Python
    Replies:
    2
    Views:
    286
    Josh Close
    Jun 9, 2005
  3. Jadhav, Alok
    Replies:
    9
    Views:
    308
    Thomas Rachel
    Nov 15, 2012
  4. Jadhav, Alok
    Replies:
    0
    Views:
    246
    Jadhav, Alok
    Sep 17, 2012
  5. Robert Miles
    Replies:
    0
    Views:
    268
    Robert Miles
    Nov 1, 2012
Loading...

Share This Page