os.rename [Errno 31] Too many links

Discussion in 'Python' started by Richard, Mar 10, 2010.

  1. Richard

    Richard Guest

    I want to atomically write to a file so have been using temporary
    files and renaming:

    temp = tempfile.NamedTemporaryFile(delete=False)
    temp.file.write(data)
    temp.file.close()
    os.rename(temp.name, output_file)

    This worked but after 39567 files os.rename raises an OSError: [Errno
    31] Too many links

    I can still create files in this directory so I doubt it is a platform
    limitation.
    Can you tell what is wrong? Am I not freeing the temporary file
    resources properly?

    thanks,
    Richard
     
    Richard, Mar 10, 2010
    #1
    1. Advertising

  2. Richard wrote:
    > I want to atomically write to a file so have been using temporary
    > files and renaming:
    >
    > temp = tempfile.NamedTemporaryFile(delete=False)
    > temp.file.write(data)
    > temp.file.close()
    > os.rename(temp.name, output_file)
    >
    > This worked but after 39567 files os.rename raises an OSError: [Errno
    > 31] Too many links
    >
    > I can still create files in this directory so I doubt it is a platform
    > limitation.
    > Can you tell what is wrong? Am I not freeing the temporary file
    > resources properly?


    If you are on linux you can have a look at the open file descriptors of
    a running process like this:

    ls -l /proc/PID/fd/

    But I guess it is a limitation of your filesystem. What do you use?

    I once had this problem with ext2. It has a low limit for
    subdirectories.

    With xfs the limits are much greater.

    Thomas


    --
    Thomas Guettler, http://www.thomas-guettler.de/
    E-Mail: guettli (*) thomas-guettler + de
     
    Thomas Guettler, Mar 10, 2010
    #2
    1. Advertising

  3. Richard

    Richard Guest

    On Mar 11, 3:08 am, Thomas Guettler <> wrote:
    > Richard wrote:
    > > I want to atomically write to a file so have been using temporary
    > > files and renaming:

    >
    > > temp = tempfile.NamedTemporaryFile(delete=False)
    > > temp.file.write(data)
    > > temp.file.close()
    > > os.rename(temp.name, output_file)

    >
    > > This worked but after 39567 files os.rename raises an OSError: [Errno
    > > 31] Too many links

    >
    > > I can still create files in this directory so I doubt it is a platform
    > > limitation.
    > > Can you tell what is wrong? Am I not freeing the temporary file
    > > resources properly?

    >
    > If you are on linux you can have a look at the open file descriptors of
    > a running process like this:
    >
    > ls -l /proc/PID/fd/
    >
    > But I guess it is a limitation of your filesystem. What do you use?
    >
    > I once had this problem with ext2. It has a low limit for
    > subdirectories.
    >
    > With xfs the limits are much greater.
    >
    >   Thomas
    >
    > --
    > Thomas Guettler,http://www.thomas-guettler.de/
    > E-Mail: guettli (*) thomas-guettler + de



    I am using Ubuntu 9.10 with ext3, which I believe has a limit for the
    number of subdirectories but not files.

    Thanks for the open file descriptor tip - I will check that out.

    Richard
     
    Richard, Mar 11, 2010
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. viza

    &errno, sizeof errno

    viza, Sep 12, 2008, in forum: C Programming
    Replies:
    20
    Views:
    1,051
    Tim Rentsch
    Sep 14, 2008
  2. Glenn Linderman

    errno 22 instead of errno 2

    Glenn Linderman, Jan 28, 2009, in forum: Python
    Replies:
    0
    Views:
    379
    Glenn Linderman
    Jan 28, 2009
  3. Replies:
    2
    Views:
    561
  4. Replies:
    2
    Views:
    372
  5. Jim Surname
    Replies:
    3
    Views:
    444
    Jim Surname
    Jan 25, 2008
Loading...

Share This Page