fork after creating temporary file using NamedTemporaryFile

Discussion in 'Python' started by rparimi@gmail.com, Jul 15, 2008.

  1. Guest

    Hello pythoners,

    When I create temporary file using the tempfile module, and forkI)
    later on in my program, I always see errors when the program exits. Is
    this because the child process deletes temp file?
    Here's a stripped down version of my script that exhibits this
    problem:

    #!/usr/bin/python

    import os
    import tempfile
    import sys

    cmd = []
    cmd.append('/bin/ls')
    cmd.append('-l')
    cmd.append('/tmp')

    foo = tempfile.NamedTemporaryFile(mode='w+b')

    pid = os.fork()
    if pid:
    print 'I am parent'
    else:
    print 'I am child'
    sys.exit(0)

    $ python sub.py
    I am child
    I am parent
    Exception exceptions.OSError: (2, 'No such file or directory', '/tmp/
    tmp-mZTPq') in <bound method _TemporaryFileWrapper.__del__ of <closed
    file '<fdopen>', mode 'w+b' at 0xb7d2a578>> ignored


    How can these warnings be avoided? I tried to catch this exception
    using try/except but it didn't work.

    thanks!
     
    , Jul 15, 2008
    #1
    1. Advertising

  2. <>:

    > When I create temporary file using the tempfile module, and forkI)
    > later on in my program, I always see errors when the program exits. Is
    > this because the child process deletes temp file?
    > Here's a stripped down version of my script that exhibits this
    > problem:
    >
    > #!/usr/bin/python
    >
    > import os
    > import tempfile
    > import sys
    >
    > cmd = []
    > cmd.append('/bin/ls')
    > cmd.append('-l')
    > cmd.append('/tmp')
    >
    > foo = tempfile.NamedTemporaryFile(mode='w+b')
    >
    > pid = os.fork()
    > if pid:
    > print 'I am parent'
    > else:
    > print 'I am child'
    > sys.exit(0)
    >
    > $ python sub.py
    > I am child
    > I am parent
    > Exception exceptions.OSError: (2, 'No such file or directory', '/tmp/
    > tmp-mZTPq') in <bound method _TemporaryFileWrapper.__del__ of <closed
    > file '<fdopen>', mode 'w+b' at 0xb7d2a578>> ignored


    NamedTemporaryFile attempts to delete the file, when "close" is invoked
    (which is done by the destructor in this case, because you're senselessly
    not closing that file explicitly). A "fork()" clones the whole process
    memory, after fork() is done, there are *two* NamedTemporaryFile objects,
    pointing to the *same* file, existing in two separate process (one in the
    parent, the other, cloned one, in the child). When the first process
    exists, the file is removed cleanly, the second process can't remove
    anything anymore, since there's nothing left to remove.

    Since the order of process execution is not deterministic and completely up
    to the systems scheduler, you can *never* say, which one will exit first.

    You should replace NamedTempraryFile with "mkstemp" and take care of
    deletion yourself. You could for instance call "os.wait" to wait for the
    child's termination in the parent process and thus delete the temporary
    file, once the child has gone.

    Of course, you could also catch the exception, if you properly close the
    file object (what you should be doing anyway!), but I'd consider this a not
    very robust solution. If one of the two processes exit unexpectedly early,
    the directory entry is gone, though it might still be needed by the parent
    process.

    > How can these warnings be avoided? I tried to catch this exception
    > using try/except but it didn't work.


    If you want to catch that exception, you should properly close the named
    temporary file and wrap the "close" call inside a try-except statement.
    Beginning with Python 2.5 you might also want to use the with-statement
    here. Relying on the destructor is *always* a bad idea, you should always
    close files explicitly!

    --
    Freedom is always the freedom of dissenters.
    (Rosa Luxemburg)
     
    Sebastian \lunar\ Wiesner, Jul 15, 2008
    #2
    1. Advertising

  3. In message <g5ikc7$k52$01$-online.com>, Sebastian "lunar" Wiesner
    wrote:

    > Relying on the destructor is *always* a bad idea, you should always
    > close files explicitly!


    There shouldn't be any problem with files opened read-only.
     
    Lawrence D'Oliveiro, Jul 17, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    0
    Views:
    541
  2. Lee Harr
    Replies:
    5
    Views:
    709
    Tim Peters
    Dec 30, 2005
  3. Jason Lunz

    monkeypatching NamedTemporaryFile

    Jason Lunz, May 26, 2006, in forum: Python
    Replies:
    2
    Views:
    298
    Jason Lunz
    May 27, 2006
  4. Replies:
    7
    Views:
    3,260
    James Kanze
    Feb 12, 2008
  5. Eric Snow

    os.fork and pty.fork

    Eric Snow, Jan 8, 2009, in forum: Python
    Replies:
    0
    Views:
    579
    Eric Snow
    Jan 8, 2009
Loading...

Share This Page