Intermittent "permission denied" errors when using os.rename and a recently deleted path??

[Russell Warren]

I've been having a hard time tracking down a very intermittent problem
where I get a "permission denied" error when trying to rename a file to
something that has just been deleted (on win32).

The code snippet that gets repeatedly called is here:

...
if os.path.exists(oldPath):
os.remove(oldPath)
os.rename(newPath, oldPath)
...

And I get the permission denied exception on the os.rename line.
Somehow the rename target is still locked? I don't get it.

I found a post that seemed to refer to precisely this problem:
http://groups.google.com/group/comp...gst&q=os.remove+delay&rnum=1#e5c19db11d8b6d4e

However - this post describes a case where there are multiple threads
making use of other os calls. I am running a single threaded
application and still getting this problem. ie: the suggested fix does
not work for me.

I'm trying to see if implementing a "trap the exception and try again,
but not too many times" hack fix will do the trick, but I'm not a big
fan of this "solution", and at this point I'm not entirely certain it
will work because confirming that it *did* work is tough (it is very
difficult to repeatably create the problem).

Does anyone know of a real solution to this problem, or know what
exactly is happening so that I can work out a proper solution?

Thanks,
Russ

 

[Neil Hodgson]

Russell Warren:

I've been having a hard time tracking down a very intermittent problem
where I get a "permission denied" error when trying to rename a file to
something that has just been deleted (on win32).

Are you running a background file accessing tool like Google Desktop
Search or an anti-virus application? If so, try turning them off as a test.

Neil

 

[Russell Warren]

Are you running a background file accessing tool like Google Desktop

Search or an anti-virus application? If so, try turning them off as a test.

I'm actually running both... but I would think that once os.remove
returns that the file is actually gone from the hdd. Why would either
application be blocking access to a non-existent file?

Of course, my thinking is obviously wrong since I do get the permission
problem... I will definitely try disabling those. Now if only I could
reproducably repeat it to make testing easier.

Another thing is that I certainly do want the code to work in the
presence of such tools.

 

[Neil Hodgson]

Russell Warren:

I'm actually running both... but I would think that once os.remove
returns that the file is actually gone from the hdd. Why would either
application be blocking access to a non-existent file?

Does it actually tell you the target is the problem? I see an
"OSError: [Errno 17] File exists" for that case, not a permission error.
A permission error could occur, for example, if GDS has the source open
or locked when you call os.rename.

Neil

 

[Russell Warren]

Does it actually tell you the target is the problem? I see an

"OSError: [Errno 17] File exists" for that case, not a permission error.
A permission error could occur, for example, if GDS has the source open
or locked when you call os.rename.

No it doesn't tell me the target is the issue... you are of course
right that it could be either. I did some looking to see if/why GDS
would lock files at any time while scanning but didn't turn up anything
useful so far. I'd be surprised if it did as that would be one heck of
an annoying design flaw.

Anyway - the retry-on-failure workaround seems to prevent it from
happening, although it still seems very hackish and I don't like it:

...
if os.path.exists(path1): os.remove(path1)
startTime = time.clock()
while 1:
try:
os.rename(self.path2, self.path1)
break
except OSError:
if (time.clock() - startTime) > MAX_RETRY_DURATION_s:
raise
else:
time.sleep(0)
...

It feels very weird to have to verify a simple operation like this, but
if it works it works.

Russ