timeout with read operations

Discussion in 'Python' started by Jim, Jul 22, 2004.

  1. Jim

    Jim Guest

    Hi,

    I am trying to figure out a way to implement a timeout along with a
    read() call on an open file. It only has to work on linux,
    for now I am trying:

    ret = select.select( [fileno],[],[], timeout )
    if ret[0] == [fileno]
    # do read

    but .. what if between the select and read call something happens and
    the read blocks indefinitly?
    Is there a way I can have a functions like file.read( timeout ), where
    timeout out is a gauranteed lower bound? I know that there is for
    sockets, with the settimeout function, but I don't know how they
    implemented it.
     
    Jim, Jul 22, 2004
    #1
    1. Advertising

  2. Jim

    Guest

    Jim wrote:
    > Hi,
    >
    > I am trying to figure out a way to implement a timeout along with a
    > read() call on an open file. It only has to work on linux,
    > for now I am trying:
    >
    > ret = select.select( [fileno],[],[], timeout )
    > if ret[0] == [fileno]
    > # do read
    >
    > but .. what if between the select and read call something happens and
    > the read blocks indefinitly?


    I don't think anything could happen
    that would cause a block. That's the
    assumption I made when implementing:
    http://www.pixelbeat.org/libs/subProcess.py

    If you find this is not the case you can
    always make the fileno non blocking but
    I really don't think this is required.

    Pádraig.
     
    , Jul 22, 2004
    #2
    1. Advertising

  3. Jim

    G. S. Hayes Guest

    (Jim) wrote in message news:<>...
    > Hi,
    >
    > I am trying to figure out a way to implement a timeout along with a
    > read() call on an open file. It only has to work on linux,
    > for now I am trying:
    >
    > ret = select.select( [fileno],[],[], timeout )
    > if ret[0] == [fileno]
    > # do read
    >
    > but .. what if between the select and read call something happens and
    > the read blocks indefinitly?


    What's the application? It's easier to make rational suggestions if
    we know what you're trying to do.

    Disk reads are uninterruptable--things like setting O_NONBLOCK don't
    have any effect on files AFAICS (they work fine on sockets/pipes).

    In general (for non-disk system calls), investigate alarm() and
    setitimer(). Python wraps the former in signal.alarm (check the docu
    for the signal module), there is no standard Python wrapper for the
    latter that I know of.

    Possible solutions for disk reads include:
    1. Use async I/O (I don't know of a Python API for this)
    2. Use some sort of parallel processing, either threads or processes;
    I'd prefer processes for this particular application (unless you're on
    a platform like Windows where there is a significant performance
    penalty to processes).
    3. Read files into memory before entering the time-sensitive portion
    of the code

    If 3 is infeasible, consider 2; set up a pipe with os.pipe, fork with
    os.fork, and have the child process read off disk and write to the
    parent. The parent can then select on the pipe and read--the pipe
    doesn't select as readable until data is already in the pipe buffer,
    so you're guaranteed not to block on THAT (you may get an error
    returned immediately). If you're lazy, popen("/bin/cat filename")
    will work, though you may more control and efficiency if you do the
    fork, pipe, and read calls on your own (popen invokes an unnecessary
    shell)

    Threads won't be any easier to integrate with a select loop, but if
    there's a major performance issue you can avoid a memory copy by
    having a thread do the read and using a pipe, socket, or signal to
    trigger the select loop. Almost certainly not worth the
    implementation pain.

    > Is there a way I can have a functions like file.read( timeout ), where
    > timeout out is a gauranteed lower bound? I know that there is for
    > sockets, with the settimeout function, but I don't know how they
    > implemented it.


    AFAIK, SO_RCVTIMEO and SO_SNDTIMEO are not writeable under Linux, so
    I'd guess that settimeout plays games with alarm() or setitimer().

    Sumner
     
    G. S. Hayes, Jul 22, 2004
    #3
  4. Jim

    G. S. Hayes Guest

    wrote in message news:<>...
    > Jim wrote:


    > > but .. what if between the select and read call something happens and
    > > the read blocks indefinitly?

    >
    > I don't think anything could happen
    > that would cause a block.


    Another process could start up and hog the disk.

    A cable could be loose.

    Actually, I think select() on a regular file always returns readable
    (except if you're at EOF maybe?)

    In practice, it's unlikely that you'll block for very long, which is
    sufficient for many applications.

    > If you find this is not the case you can
    > always make the fileno non blocking


    O_NONBLOCK has no effect on regular files--see my other reply in this
    thread.

    Sumner
     
    G. S. Hayes, Jul 22, 2004
    #4
  5. Jim

    googleboy Guest

    (Jim) wrote in message news:<>...
    > Hi,
    >
    > I am trying to figure out a way to implement a timeout along with a
    > read() call on an open file. It only has to work on linux,
    > for now I am trying:
    >
    > ret = select.select( [fileno],[],[], timeout )
    > if ret[0] == [fileno]
    > # do read
    >
    > but .. what if between the select and read call something happens and
    > the read blocks indefinitly?
    > Is there a way I can have a functions like file.read( timeout ), where
    > timeout out is a gauranteed lower bound? I know that there is for
    > sockets, with the settimeout function, but I don't know how they
    > implemented it.



    Could you not use signals to do what you need?

    Someone just pointed me in that direction, and it worked great for
    me. About 5 lines of code.

    http://www.python.org/doc/current/lib/module-signal.html

    Check the example out.

    I hope I have helped! I am usually the n00b asking the questions
    around here.

    googleboy
     
    googleboy, Jul 22, 2004
    #5
  6. Jim

    googleboy Guest

    (Jim) wrote in message news:<>...
    > Hi,
    >
    > I am trying to figure out a way to implement a timeout along with a
    > read() call on an open file. It only has to work on linux,
    > for now I am trying:
    >
    > ret = select.select( [fileno],[],[], timeout )
    > if ret[0] == [fileno]
    > # do read
    >
    > but .. what if between the select and read call something happens and
    > the read blocks indefinitly?
    > Is there a way I can have a functions like file.read( timeout ), where
    > timeout out is a gauranteed lower bound? I know that there is for
    > sockets, with the settimeout function, but I don't know how they
    > implemented it.



    Could you not use signals to do what you need?

    Someone just pointed me in that direction, and it worked great for
    me. About 5 lines of code.

    http://www.python.org/doc/current/lib/module-signal.html

    Check the example out.

    I hope I have helped! I am usually the n00b asking the questions
    around here.

    googleboy
     
    googleboy, Jul 22, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jesus M. Salvo Jr.
    Replies:
    2
    Views:
    4,306
    robert
    Feb 11, 2006
  2. Jason Heyes
    Replies:
    6
    Views:
    432
    Mark Stijnman
    Apr 22, 2005
  3. Smoothy

    File read/write operations

    Smoothy, Nov 26, 2003, in forum: C Programming
    Replies:
    2
    Views:
    452
    Smoothy
    Nov 26, 2003
  4. Replies:
    7
    Views:
    978
    Arne Vajhøj
    Apr 24, 2009
  5. Mark Probert

    Timeout::timeout and Socket timeout

    Mark Probert, Oct 6, 2004, in forum: Ruby
    Replies:
    1
    Views:
    1,334
    Brian Candler
    Oct 6, 2004
Loading...

Share This Page