timeout with read operations

[Jim]

Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

 

[P]

Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?

I don't think anything could happen
that would cause a block. That's the
assumption I made when implementing:
http://www.pixelbeat.org/libs/subProcess.py

If you find this is not the case you can
always make the fileno non blocking but
I really don't think this is required.

Pádraig.

 

[G. S. Hayes]

Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?

What's the application? It's easier to make rational suggestions if
we know what you're trying to do.

Disk reads are uninterruptable--things like setting O_NONBLOCK don't
have any effect on files AFAICS (they work fine on sockets/pipes).

In general (for non-disk system calls), investigate alarm() and
setitimer(). Python wraps the former in signal.alarm (check the docu
for the signal module), there is no standard Python wrapper for the
latter that I know of.

Possible solutions for disk reads include:
1. Use async I/O (I don't know of a Python API for this)
2. Use some sort of parallel processing, either threads or processes;
I'd prefer processes for this particular application (unless you're on
a platform like Windows where there is a significant performance
penalty to processes).
3. Read files into memory before entering the time-sensitive portion
of the code

If 3 is infeasible, consider 2; set up a pipe with os.pipe, fork with
os.fork, and have the child process read off disk and write to the
parent. The parent can then select on the pipe and read--the pipe
doesn't select as readable until data is already in the pipe buffer,
so you're guaranteed not to block on THAT (you may get an error
returned immediately). If you're lazy, popen("/bin/cat filename")
will work, though you may more control and efficiency if you do the
fork, pipe, and read calls on your own (popen invokes an unnecessary
shell)

Threads won't be any easier to integrate with a select loop, but if
there's a major performance issue you can avoid a memory copy by
having a thread do the read and using a pipe, socket, or signal to
trigger the select loop. Almost certainly not worth the
implementation pain.

Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

AFAIK, SO_RCVTIMEO and SO_SNDTIMEO are not writeable under Linux, so
I'd guess that settimeout plays games with alarm() or setitimer().

Sumner

 

[G. S. Hayes]

Jim wrote:

I don't think anything could happen
that would cause a block.

Another process could start up and hog the disk.

A cable could be loose.

Actually, I think select() on a regular file always returns readable
(except if you're at EOF maybe?)

In practice, it's unlikely that you'll block for very long, which is
sufficient for many applications.

If you find this is not the case you can
always make the fileno non blocking

O_NONBLOCK has no effect on regular files--see my other reply in this
thread.

Sumner

 

[googleboy]

Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

Could you not use signals to do what you need?

Someone just pointed me in that direction, and it worked great for
me. About 5 lines of code.

http://www.python.org/doc/current/lib/module-signal.html

Check the example out.

I hope I have helped! I am usually the n00b asking the questions
around here.

googleboy

 

[googleboy]

Hi,

I am trying to figure out a way to implement a timeout along with a
read() call on an open file. It only has to work on linux,
for now I am trying:

ret = select.select( [fileno],[],[], timeout )
if ret[0] == [fileno]
# do read

but .. what if between the select and read call something happens and
the read blocks indefinitly?
Is there a way I can have a functions like file.read( timeout ), where
timeout out is a gauranteed lower bound? I know that there is for
sockets, with the settimeout function, but I don't know how they
implemented it.

Could you not use signals to do what you need?

Someone just pointed me in that direction, and it worked great for
me. About 5 lines of code.

http://www.python.org/doc/current/lib/module-signal.html

Check the example out.

I hope I have helped! I am usually the n00b asking the questions
around here.

googleboy