Force flushing buffers

  • Thread starter Madhusudan Singh
  • Start date
M

Madhusudan Singh

Hi

I have a python application that writes a lot of data to a bunch of files
from inside a loop. Sometimes, the application has to be interrupted and I
find that a lot of data has not yet been writen (and hence is lost). How do
I flush the buffer and force python to write the buffers to the files ? I
intend to put this inside the loop.

Thanks.
 
S

skip

Madhusudan> How do I flush the buffer and force python to write the
Madhusudan> buffers to the files ? I intend to put this inside the loop.

f = open("somefile", "w")
f.write("foo")
f.flush()

Skip
 
S

Scott David Daniels

Madhusudan said:
I have a python application that writes a lot of data to a bunch of files
from inside a loop. Sometimes, the application has to be interrupted and I
find that a lot of data has not yet been writen (and hence is lost). How do
I flush the buffer and force python to write the buffers to the files ? I
intend to put this inside the loop.

Thanks.
...
files_to_close = []
try:
somefile = open(somename, 'w')
files_to_close.append(somefile)
... calculations, probably in a loop ...
finally:
for afile in files_to_close:
afile.close()
...

--Scott David Daniels
(e-mail address removed)
 
R

Robert Wierschke

Madhusudan said:
Hi

I have a python application that writes a lot of data to a bunch of files
from inside a loop. Sometimes, the application has to be interrupted and I
find that a lot of data has not yet been writen (and hence is lost). How do
I flush the buffer and force python to write the buffers to the files ? I
intend to put this inside the loop.

Thanks.
disable the buffer!

open( filename[, mode[, bufsize]])

open takes an optional 3ed argument set bufsize = 0 means unbuffered.
see the documentation of the in build file() mehtod as open is just
another name
 
M

Madhusudan Singh

Madhusudan> How do I flush the buffer and force python to write the
Madhusudan> buffers to the files ? I intend to put this inside the
loop.

f = open("somefile", "w")
f.write("foo")
f.flush()

Skip

Thanks !!
 
M

Madhusudan Singh

Robert said:
Madhusudan said:
Hi

I have a python application that writes a lot of data to a bunch
of files
from inside a loop. Sometimes, the application has to be interrupted and
I find that a lot of data has not yet been writen (and hence is lost).
How do I flush the buffer and force python to write the buffers to the
files ? I intend to put this inside the loop.

Thanks.
disable the buffer!

open( filename[, mode[, bufsize]])

open takes an optional 3ed argument set bufsize = 0 means unbuffered.
see the documentation of the in build file() mehtod as open is just
another name

Though I will not be using this solution (plan to use flush() explicitly)
for speed reasons, thanks ! I will file this away for future reference :)
 
B

Bengt Richter

Robert said:
Madhusudan said:
Hi

I have a python application that writes a lot of data to a bunch
of files
from inside a loop. Sometimes, the application has to be interrupted and
I find that a lot of data has not yet been writen (and hence is lost).
How do I flush the buffer and force python to write the buffers to the
files ? I intend to put this inside the loop.

Thanks.
disable the buffer!

open( filename[, mode[, bufsize]])

open takes an optional 3ed argument set bufsize = 0 means unbuffered.
see the documentation of the in build file() mehtod as open is just
another name

Though I will not be using this solution (plan to use flush() explicitly)
for speed reasons, thanks ! I will file this away for future reference :)
I suspect Scott's try/finally approach will get you better speed, since it
avoids unneeded flush calls and the associated buffer management, but
it is best to measure speed when you are concerned.

Regards,
Bengt Richter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads


Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top