Threading issue: [Error 9]: bad file descriptor

Discussion in 'Python' started by Kevin, Jul 6, 2003.

  1. Kevin

    Kevin Guest

    Has anyone else run into random IOErrors ([Error 9] bad file descriptor) in
    multi-threaded Python apps?

    I've searched the newsgroups, and the only references I can find to that
    seem to be sporadically related to threading, but nobody seems to have a
    "fix" or even a clear understanding of what the issue is.

    The app I'm running into this with (once in a while... not really repeatable
    predictably) is using 3 threads, and I'm using locks around all shared
    objects. The error seems to be happening mostly when I'm trying to delete a
    temporary file that was originally created by a different thread.

    ANy suggestions? Thanks,

    Kevin.
     
    Kevin, Jul 6, 2003
    #1
    1. Advertising

  2. On Sun, 6 Jul 2003, Kevin wrote:

    > Has anyone else run into random IOErrors ([Error 9] bad file descriptor) in
    > multi-threaded Python apps?


    Not indicating which of the platforms with Python supported threads you're
    experiencing this on doesn't give much scope for others to help...

    --
    Andrew I MacIntyre "These thoughts are mine alone..."
    E-mail: (pref) | Snail: PO Box 370
    (alt) | Belconnen ACT 2616
    Web: http://www.andymac.org/ | Australia
     
    Andrew MacIntyre, Jul 6, 2003
    #2
    1. Advertising

  3. Kevin

    Aahz Guest

    In article <KaQNa.394960$>,
    Kevin <> wrote:
    >
    >Has anyone else run into random IOErrors ([Error 9] bad file
    >descriptor) in multi-threaded Python apps?
    >
    >The app I'm running into this with (once in a while... not really
    >repeatable predictably) is using 3 threads, and I'm using locks around
    >all shared objects. The error seems to be happening mostly when I'm
    >trying to delete a temporary file that was originally created by a
    >different thread.


    Sounds like the problem isn't locking per se, but lack of
    synchronization. The simple answer: never use an external object in
    multiple threads.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Not everything in life has a clue in front of it...." --JMS
     
    Aahz, Jul 6, 2003
    #3
  4. Kevin

    Cliff Wells Guest

    On Sun, 2003-07-06 at 22:37, Kevin wrote:
    > My oversight, thanks!
    >
    > I'm testing on Windows 2000 Professional, but have had other users report
    > the same problem on other Windows flavours (NT, 98).


    I experienced a similar problem a few years ago (on Windows NT Server).
    A long-running threaded application would sporadically raise an
    exception when writing to a log file. Never figured out the cause, but
    "solved" it by catching the exception and reopening the file (IIRC).

    Regards,

    --
    Cliff Wells, Software Engineer
    Logiplex Corporation (www.logiplex.net)
    (503) 978-6726 (800) 735-0555
     
    Cliff Wells, Jul 7, 2003
    #4
  5. Kevin

    David Bolen Guest

    "Kevin" <> writes:

    > My oversight, thanks!
    >
    > I'm testing on Windows 2000 Professional, but have had other users report
    > the same problem on other Windows flavours (NT, 98).


    I can only speak for NT, but I have seen timing issues in the past
    where there is some additional latency imposed by the system between
    when I release all references to a file and when it can be removed
    from the filesystem by the same process. This has affected me in C
    code as well as Python. Perhaps you're running into the same thing,
    although for me it normally showed up as a permission denied error.

    Sometimes the higher level Python I/O exceptions can be a bit vague
    (by their nature, a lot of underlying Win32 error codes eventually get
    translated into a far fewer number of C RTL error codes, which in turn
    bubble up as Python exceptions). One thing you could try, depending
    on the operation in question, is to replace the Python operation (such
    as os.remove) with a matching win32all module operation (such as
    win32file.DeleteFile), which should raise a more specific exception.

    In my past experiences, the most practical, albeit inelegant,
    workaround was to retry a failed removal operation after several
    seconds. Definitely a kluge, but I was never able to isolate any more
    well-defined approach.

    This of course assumes that you don't really have a threading race
    condition under which you still do hold active handles to the file
    that you are trying to remove. The fact that you're getting back an
    invalid handle error certainly makes it seem that you make still be
    looking at a race condition or inter-thread data corruption within
    your application.

    Depending on the structure of the application, if you can wrap (or if
    you have already) all access to your file handles through a common
    class you should be able to instrument it in order to at least get
    some sort of trace as to how they are accessed, and perhaps catch one
    of the failures. But as you probably know, debugging this sort of
    thing can be nasty, particularly if it's truly sporadic in nature.

    -- David
     
    David Bolen, Jul 7, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. drs
    Replies:
    0
    Views:
    386
  2. Replies:
    9
    Views:
    747
    Abhijit Soman
    Jan 6, 2005
  3. Vishnu
    Replies:
    0
    Views:
    524
    Vishnu
    Jan 6, 2005
  4. hg
    Replies:
    1
    Views:
    5,363
  5. lynology

    Bad File Descriptor Error on strcat/strcpy

    lynology, Aug 18, 2004, in forum: C Programming
    Replies:
    4
    Views:
    535
    Chris Torek
    Aug 18, 2004
Loading...

Share This Page