Force flushing buffers

Discussion in 'Python' started by Madhusudan Singh, Oct 7, 2005.

  1. Hi

    I have a python application that writes a lot of data to a bunch of files
    from inside a loop. Sometimes, the application has to be interrupted and I
    find that a lot of data has not yet been writen (and hence is lost). How do
    I flush the buffer and force python to write the buffers to the files ? I
    intend to put this inside the loop.

    Thanks.
     
    Madhusudan Singh, Oct 7, 2005
    #1
    1. Advertising

  2. Madhusudan Singh

    Guest

    Madhusudan> How do I flush the buffer and force python to write the
    Madhusudan> buffers to the files ? I intend to put this inside the loop.

    f = open("somefile", "w")
    f.write("foo")
    f.flush()

    Skip
     
    , Oct 7, 2005
    #2
    1. Advertising

  3. Madhusudan Singh wrote:
    > I have a python application that writes a lot of data to a bunch of files
    > from inside a loop. Sometimes, the application has to be interrupted and I
    > find that a lot of data has not yet been writen (and hence is lost). How do
    > I flush the buffer and force python to write the buffers to the files ? I
    > intend to put this inside the loop.
    >
    > Thanks.

    ...
    files_to_close = []
    try:
    somefile = open(somename, 'w')
    files_to_close.append(somefile)
    ... calculations, probably in a loop ...
    finally:
    for afile in files_to_close:
    afile.close()
    ...

    --Scott David Daniels
     
    Scott David Daniels, Oct 7, 2005
    #3
  4. Madhusudan Singh schrieb:
    > Hi
    >
    > I have a python application that writes a lot of data to a bunch of files
    > from inside a loop. Sometimes, the application has to be interrupted and I
    > find that a lot of data has not yet been writen (and hence is lost). How do
    > I flush the buffer and force python to write the buffers to the files ? I
    > intend to put this inside the loop.
    >
    > Thanks.

    disable the buffer!

    open( filename[, mode[, bufsize]])

    open takes an optional 3ed argument set bufsize = 0 means unbuffered.
    see the documentation of the in build file() mehtod as open is just
    another name
     
    Robert Wierschke, Oct 8, 2005
    #4
  5. wrote:

    >
    > Madhusudan> How do I flush the buffer and force python to write the
    > Madhusudan> buffers to the files ? I intend to put this inside the
    > loop.
    >
    > f = open("somefile", "w")
    > f.write("foo")
    > f.flush()
    >
    > Skip


    Thanks !!
     
    Madhusudan Singh, Oct 12, 2005
    #5
  6. Robert Wierschke wrote:

    > Madhusudan Singh schrieb:
    >> Hi
    >>
    >> I have a python application that writes a lot of data to a bunch
    >> of files
    >> from inside a loop. Sometimes, the application has to be interrupted and
    >> I find that a lot of data has not yet been writen (and hence is lost).
    >> How do I flush the buffer and force python to write the buffers to the
    >> files ? I intend to put this inside the loop.
    >>
    >> Thanks.

    > disable the buffer!
    >
    > open( filename[, mode[, bufsize]])
    >
    > open takes an optional 3ed argument set bufsize = 0 means unbuffered.
    > see the documentation of the in build file() mehtod as open is just
    > another name


    Though I will not be using this solution (plan to use flush() explicitly)
    for speed reasons, thanks ! I will file this away for future reference :)
     
    Madhusudan Singh, Oct 12, 2005
    #6
  7. On Wed, 12 Oct 2005 15:55:10 -0400, Madhusudan Singh <> wrote:

    >Robert Wierschke wrote:
    >
    >> Madhusudan Singh schrieb:
    >>> Hi
    >>>
    >>> I have a python application that writes a lot of data to a bunch
    >>> of files
    >>> from inside a loop. Sometimes, the application has to be interrupted and
    >>> I find that a lot of data has not yet been writen (and hence is lost).
    >>> How do I flush the buffer and force python to write the buffers to the
    >>> files ? I intend to put this inside the loop.
    >>>
    >>> Thanks.

    >> disable the buffer!
    >>
    >> open( filename[, mode[, bufsize]])
    >>
    >> open takes an optional 3ed argument set bufsize = 0 means unbuffered.
    >> see the documentation of the in build file() mehtod as open is just
    >> another name

    >
    >Though I will not be using this solution (plan to use flush() explicitly)
    >for speed reasons, thanks ! I will file this away for future reference :)

    I suspect Scott's try/finally approach will get you better speed, since it
    avoids unneeded flush calls and the associated buffer management, but
    it is best to measure speed when you are concerned.

    Regards,
    Bengt Richter
     
    Bengt Richter, Oct 16, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eric

    Flushing the Buffer in .Net

    Eric, Apr 25, 2004, in forum: ASP .Net
    Replies:
    3
    Views:
    669
  2. z. f.

    flushing inside a repeaterItem

    z. f., Jan 30, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    333
    z. f.
    Jan 30, 2005
  3. Dave Rudolf

    Flushing a GZIPOutputStream

    Dave Rudolf, Jan 8, 2004, in forum: Java
    Replies:
    3
    Views:
    1,880
    Anton Spaans
    Jan 8, 2004
  4. Hikikomori
    Replies:
    4
    Views:
    6,475
    Thomas 'Skip' Hollowell
    Mar 24, 2005
  5. Hal Vaughan
    Replies:
    7
    Views:
    244
    Tad McClellan
    Aug 30, 2005
Loading...

Share This Page