Merge two directories together

[Keith Hughitt]

Suppose you have two file-trees with common sub-directories but
different files that you want to merge together, e.g.

/test/
/test/a/
/test/a/file1

/test2/
/test2/a/
/test2/a/file2

You can easily merge the directories in Linux using the "cp" command:

cp -r test/* test2/

While Python provides some helpful methods for moving files and
directories around (shutil.move, shutil.copytree, etc), none of them
seem to be able to merge two directories.

I've looked around some on Google for an alternative to calling cp
directly, but so far no luck.

Any ideas?

 

[Steven Howe]

Think about using the subprocess module. There are calls made just for
your. Notably (using command pydoc subprrocess.call) :
---------------------
subprocess.call = call(*popenargs, **kwargs)
Run command with arguments. Wait for command to complete, then
return the returncode attribute.

The arguments are the same as for the Popen constructor. Example:

retcode = call(["ls", "-l"])

 

[Dave W.]

While Python provides some helpful methods for moving files and

directories around (shutil.move, shutil.copytree, etc), none of
them seem to be able to merge two directories. -snip-
Any ideas?

It's not pretty, but you could hack up the original copytree()
source so it ignores errors from the makedirs() call. Then it
should work more-or-less like 'cp -r'. This isn't ideal, because
'OSError' is thrown, which could mean that the dir already exists
(okay), or it could be a 'real' error, like the current user doesn't
have permission to write to the destination. (See 'XXX' in the code
below.)

Better would be to match on the error string and only ignore the
'directory exists' error. Unfortunately, the error messages do
vary per-platform. Here's what I see under Windows when the
directory already exists:

WindowsError: [Error 183] Cannot create a file when that file
already exists: 'test2'

and under CentOS:

OSError: [Errno 17] File exists: 'test2'

The code below seems to work under both Linux and Windows; but I
didn't really test it much, so handle with care. :-}

----------

def copytree(src, dst, symlinks=False, ignore=None):
import os
from shutil import copy2, copystat, Error

names = os.listdir(src)
if ignore is not None:
ignored_names = ignore(src, names)
else:
ignored_names = set()

try:
os.makedirs(dst)
except OSError, exc:
# XXX - this is pretty ugly
if "file already exists" in exc[1]: # Windows
pass
elif "File exists" in exc[1]: # Linux
pass
else:
raise

errors = []
for name in names:
if name in ignored_names:
continue
srcname = os.path.join(src, name)
dstname = os.path.join(dst, name)
try:
if symlinks and os.path.islink(srcname):
linkto = os.readlink(srcname)
os.symlink(linkto, dstname)
elif os.path.isdir(srcname):
copytree(srcname, dstname, symlinks, ignore)
else:
copy2(srcname, dstname)
# XXX What about devices, sockets etc.?
except (IOError, os.error), why:
errors.append((srcname, dstname, str(why)))
# catch the Error from the recursive copytree so that we can
# continue with other files
except Error, err:
errors.extend(err.args[0])
try:
copystat(src, dst)
except WindowsError:
# can't copy file access times on Windows
pass
except OSError, why:
errors.extend((src, dst, str(why)))
if errors:
raise Error, errors