os.rename [Errno 31] Too many links

R

Richard

I want to atomically write to a file so have been using temporary
files and renaming:

temp = tempfile.NamedTemporaryFile(delete=False)
temp.file.write(data)
temp.file.close()
os.rename(temp.name, output_file)

This worked but after 39567 files os.rename raises an OSError: [Errno
31] Too many links

I can still create files in this directory so I doubt it is a platform
limitation.
Can you tell what is wrong? Am I not freeing the temporary file
resources properly?

thanks,
Richard
 
T

Thomas Guettler

Richard said:
I want to atomically write to a file so have been using temporary
files and renaming:

temp = tempfile.NamedTemporaryFile(delete=False)
temp.file.write(data)
temp.file.close()
os.rename(temp.name, output_file)

This worked but after 39567 files os.rename raises an OSError: [Errno
31] Too many links

I can still create files in this directory so I doubt it is a platform
limitation.
Can you tell what is wrong? Am I not freeing the temporary file
resources properly?

If you are on linux you can have a look at the open file descriptors of
a running process like this:

ls -l /proc/PID/fd/

But I guess it is a limitation of your filesystem. What do you use?

I once had this problem with ext2. It has a low limit for
subdirectories.

With xfs the limits are much greater.

Thomas
 
R

Richard

Richard said:
I want to atomically write to a file so have been using temporary
files and renaming:
temp = tempfile.NamedTemporaryFile(delete=False)
temp.file.write(data)
temp.file.close()
os.rename(temp.name, output_file)
This worked but after 39567 files os.rename raises an OSError: [Errno
31] Too many links
I can still create files in this directory so I doubt it is a platform
limitation.
Can you tell what is wrong? Am I not freeing the temporary file
resources properly?

If you are on linux you can have a look at the open file descriptors of
a running process like this:

ls -l /proc/PID/fd/

But I guess it is a limitation of your filesystem. What do you use?

I once had this problem with ext2. It has a low limit for
subdirectories.

With xfs the limits are much greater.

  Thomas


I am using Ubuntu 9.10 with ext3, which I believe has a limit for the
number of subdirectories but not files.

Thanks for the open file descriptor tip - I will check that out.

Richard
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top