something similar to shutil.copytree that can overwrite?

[Ben Sizer]

I need to copy directories from one place to another, but it needs to
overwrite individual files and directories rather than just exiting if
a destination file already exists. Previous suggestions have focused
on looking at the source for copytree, but it has several places where
exceptions can be raised, and the documentation for the shutil
functions that copytree is implemented in terms of isn't exactly clear
about which exceptions get raised and when. This makes duplicating a
one-line shell operation a non-trivial task.

Has anybody got any pre-written code that does what I'm looking for?

 

[Justin Ezequiel]

I need to copy directories from one place to another, but it needs to
overwrite individual files and directories rather than just exiting if
a destination file already exists.

What version of Python do you have?
Nothing in the source would make it exit if a target file exists.
(Unless perhaps you have sym-links or the like.)

Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)]

copytree calls copy2 which calls copyfile

from shutil.py:
#--------------------------------------------------------------------
def _samefile(src, dst):
# Macintosh, Unix.
if hasattr(os.path,'samefile'):
try:
return os.path.samefile(src, dst)
except OSError:
return False

# All other platforms: check for same pathname.
return (os.path.normcase(os.path.abspath(src)) ==
os.path.normcase(os.path.abspath(dst)))

def copyfile(src, dst):
"""Copy data from src to dst"""
if _samefile(src, dst):
raise Error, "`%s` and `%s` are the same file" % (src, dst)

fsrc = None
fdst = None
try:
fsrc = open(src, 'rb')
fdst = open(dst, 'wb')
copyfileobj(fsrc, fdst)
finally:
if fdst:
fdst.close()
if fsrc:
fsrc.close()
#--------------------------------------------------------------------

 

[Ben Sizer]

What version of Python do you have?
Nothing in the source would make it exit if a target file exists.
(Unless perhaps you have sym-links or the like.)

I have 2.5, and I believe the behaviour I saw was that it exits if a
directory already exists and it skips any files that already exist. It
certainly wouldn't overwrite anything.

 

[Justin Ezequiel]

def copytree(src, dst, symlinks=False):
"""Recursively copy a directory tree using copy2().

The destination directory must not already exist.

XXX Consider this example code rather than the ultimate tool.

"""
names = os.listdir(src)
if not os.path.exists(dst): os.makedirs(dst) # add check here

 

[Ben Sizer]

def copytree(src, dst, symlinks=False):
"""Recursively copy a directory tree using copy2().

The destination directory must not already exist.

XXX Consider this example code rather than the ultimate tool.

"""
names = os.listdir(src)
if not os.path.exists(dst): os.makedirs(dst) # add check here

That's the easy bit to fix; what about overwriting existing files
instead of copying them? Do I have to explicitly check for them and
delete them? It seems like there are several different copy functions
in the module and it's not clear what each of them do. What's the
difference between copy, copyfile, and copy2? Why do the docs imply
that they overwrite existing files when copytree skips existing
files?

 

[Justin Ezequiel]

That's the easy bit to fix; what about overwriting existing files
instead of copying them? Do I have to explicitly check for them and
delete them? It seems like there are several different copy functions
in the module and it's not clear what each of them do. What's the
difference between copy, copyfile, and copy2? Why do the docs imply
that they overwrite existing files when copytree skips existing
files?

copytree does not skip existing files
if will overwrite said files
I just missed the makedirs call at the start
the failed makedirs call will cause the copytree function to exit
completely
thus you do not get your files updated
but with the if exists check, your files should get overwritten

 

[John J. Lee]

I have 2.5, and I believe the behaviour I saw was that it exits if a
directory already exists and it skips any files that already exist. It
certainly wouldn't overwrite anything.

How about distutils.dir_util.copy_tree? It's a documented API:

http://docs.python.org/dist/module-distutils.dirutil.html


Here's a demo. Note the arguments to distutils.dir_util.copy_tree
have a different meaning to shutil.copytree IIRC (you need to pass the
parent of the directory rather than the directory itself):


from distutils.dir_util import copy_tree
import os

def mkdir(dirname):
os.mkdir(dirname)

def write(filename, data):
f = open(filename, "w")
try:
f.write(data)
finally:
f.close()

def read(filename):
f = open(filename)
try:
return f.read()
finally:
f.close()

def make_tree_1():
mkdir("1")
mkdir(os.path.join("1", "1"))
write(os.path.join("1", "1", "a"), "abc")
return "1"

def make_tree_2():
mkdir("2")
mkdir(os.path.join("2", "1"))
write(os.path.join("2", "1", "a"), "bcd")
return "2"

dirname_1 = make_tree_1()
dirname_2 = make_tree_2()
copy_tree(dirname_1, dirname_2)
result = read(os.path.join(dirname_2, "1", "a"))
assert result == "abc", result



John