os.rename copies when old is in-use - is this deliberate?

[Tony Meyer]

On Windows, if I do os.rename(old, new) where old is a file that is
in-use (e.g. python itself, or a dll that is loaded), I would expect
that an error would be raised (e.g. as when os.remove is called with
an in-use file). However, what happens is that a copy of the file is
made, and the old file still exists.

For example:

C:\>c:\python24\python.exe
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
D:\>dir c:\python24\p*.exe
Volume in drive C is ACER
Volume Serial Number is 320D-180E

Directory of c:\python24

28/09/2005 12:41 p.m. 4,608 python.exe
28/09/2005 12:41 p.m. 5,120 pythonw.exe
2 File(s) 9,728 bytes
0 Dir(s) 16,018,685,952 bytes free

C:\>dir d:\p*24.exe
Volume in drive D is DATA
Volume Serial Number is 4019-78E0

Directory of d:\

28/09/2005 12:41 p.m. 4,608 python24.exe
1 File(s) 4,608 bytes
0 Dir(s) 15,362,207,744 bytes free

Is this the intended behaviour? The documentation doesn't give any
indication that it is (so unless I'm missing something, this is at
least a documentation bug).

Any insight appreciated (I do know that I can work around it by
doing a remove after the rename, if the file exists).

=Tony.Meyer

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Is this the intended behaviour?

Sort-of. os.rename invokes the C library's rename, and does whatever
this does. It is expected that most platform's C libraries do what
the documentation says rename does, but platforms may vary in their
implementation of the C library, and from one compiler version to
the other.

Microsoft's documentation of rename/_rename/_wrename is here:

http://msdn.microsoft.com/library/d...en-us/vclib/html/_crt_rename.2c_._wrename.asp

It doesn't say anything about renaming in-use files, either.
Looking at the implementation of _rename, I see that it directly
calls MoveFile, which is documented here:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/movefile.asp

That doesn't mention the in-use case, either.

Regards,
Martin

 

[Tony Meyer]

[Tony Meyer]

[Martin v. Löwis]
Sort-of. os.rename invokes the C library's rename, and does whatever
this does. It is expected that most platform's C libraries do what
the documentation says rename does, but platforms may vary in their
implementation of the C library, and from one compiler version to
the other.

[snip links to Microsoft documentation, which don't cover the in-use
case]

Thanks for that. In your opinion, would a documentation patch that
explained that this would occur on Windows (after the existing note
about the Windows rename not being atomic) be acceptable?

(The Windows platform C library for Python 2.4+ is in msvcrt71.dll,
right? Does that mean that behaviour will be consistent across
Windows versions, or could 9x/NT/XP/etc all behave differently?)

=Tony.Meyer

 

[Bengt Richter]

On Windows, if I do os.rename(old, new) where old is a file that is
in-use (e.g. python itself, or a dll that is loaded), I would expect
that an error would be raised (e.g. as when os.remove is called with
an in-use file). However, what happens is that a copy of the file is
made, and the old file still exists.

For example:

C:\>c:\python24\python.exe
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on wi=
n32
Type "help", "copyright", "credits" or "license" for more information.
D:\>dir c:\python24\p*.exe
Volume in drive C is ACER
Volume Serial Number is 320D-180E

Directory of c:\python24

28/09/2005 12:41 p.m. 4,608 python.exe
28/09/2005 12:41 p.m. 5,120 pythonw.exe
2 File(s) 9,728 bytes
0 Dir(s) 16,018,685,952 bytes free

C:\>dir d:\p*24.exe
Volume in drive D is DATA
Volume Serial Number is 4019-78E0

Directory of d:\

28/09/2005 12:41 p.m. 4,608 python24.exe
1 File(s) 4,608 bytes
0 Dir(s) 15,362,207,744 bytes free

Is this the intended behaviour? The documentation doesn't give any
indication that it is (so unless I'm missing something, this is at
least a documentation bug).

Any insight appreciated (I do know that I can work around it by
doing a remove after the rename, if the file exists).

Seems like os.rename may be trying to be too helpful

(windows NT4)
[ 3:26] C:\pywk\grammar>help ren
Renames a file or files.

RENAME [drive:][path]filename1 filename2.
REN [drive:][path]filename1 filename2.

Note that you cannot specify a new drive or path for your destination file.

[ 3:26] C:\pywk\grammar>py24
Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright", "credits" or "license" for more information. Help on built-in function rename in module nt:

rename(...)
rename(old, new)

Rename a file or directory.

Regards,
Bengt Richter

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Thanks for that. In your opinion, would a documentation patch that
explained that this would occur on Windows (after the existing note
about the Windows rename not being atomic) be acceptable?

In principle, yes. We cannot do that for every platform, of course,
but it probably won't be necessary for any platform, either.

(The Windows platform C library for Python 2.4+ is in msvcrt71.dll,
right? Does that mean that behaviour will be consistent across Windows
versions, or could 9x/NT/XP/etc all behave differently?)

The behaviour could be different in principle, yes. I found a KB article
that says that MoveFile, on Windows CE 4.0, on a FAT file system, would
move the file even if it was open.

I'm actually also surprised by the behaviour - I would have expected
that the failure to delete the file after the copy would cause an
error, or that the attempt to open the file with FILE_SHARE_DELETE
would cause an error right away.

Somebody should investigate this with Sysinternal's filemon, to find
out whether the MoveFile operation really succeeds. Perhaps there is
a bug in the CRT, or in Python, also.

Regards,
Martin

 

[=?iso-8859-1?q?J=E9r=F4me_Laheurte?=]

What happens if you specify a destination file on the same drive as the
source file ?