unicode shutil.copy() changes a file name during copy?

[dave]

i'm on windows, using active python 2.7.1

i've written a script to copy a folder of files to dest folder..

one if the files in this folder has the section symbol (§, '\x15') as
part of the file name
shutil.copy(src_file, dst_file) "can't find the file specified" when
it does the os.chmod() part, can't find dest file,
cuz the file got copied with "_" in place of the section symbol.
gar.

Traceback (most recent call last):
File "../../python/post_build.py", line 159, in <module>
main(proB, debugB)
File "../../python/post_build.py", line 105, in main
paths.copy_folder(srcResFolder + 'Export', destRes + '/codecs/
Export')
File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
\paths.py", line 77, in copy_folder
copy_folder(srcPath, new_dst_obj)
File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
\paths.py", line 80, in copy_folder
copy_file(srcPath, new_dst_obj)
File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
\paths.py", line 37, in copy_file
shutil.copy(src_file, dst_file)
File "C:\Python27\lib\shutil.py", line 117, in copy
copymode(src, dst)
File "C:\Python27\lib\shutil.py", line 90, in copymode
os.chmod(dst, mode)
WindowsError: [Error 2] The system cannot find the file specified:
'build\\kJams Pro Debug.app/Contents/Resources/codecs/Export/Codec
[MooV]/Animation \xa7 AAC.rtm'

if i replace that with shutil.copyfile(src_file, dst_file) it "works",
again by replacing the section symbol with underbar.
but that leaves me to call os.chmod() myself, which of course fails
because of the character substitution.

i'm gathering the list of files to iterate over the folder via:
fileList = os.listdir(src_dir)
for fileName in fileList:

and yes, src_dir is "unicode". but the fileName when i print it shows
the degree symbol (°, '\xa7') in place of the section symbol.

i suspect it's a 'dbcs' or 'latin_1' problem?

this same exact python code works on the mac (it's utf8 down to the
metal) and, get this, it works when running Windows under Parallels
Desktop. It fails only when running Windows natively. I can't see
how that makes a difference.

i can give a whole lot more info, but i just want to know if i'm
trying to do something impossible. If this seems strange to y'all
gladly i'll fill in all the details, but hoping it can be short
circuited by someone going "ah hah! you have to do such and so".

 

[Terry Reedy]

i'm on windows, using active python 2.7.1

i've written a script to copy a folder of files to dest folder..

one if the files in this folder has the section symbol (§, '\x15') as
part of the file name
shutil.copy(src_file, dst_file) "can't find the file specified" when
it does the os.chmod() part, can't find dest file,
cuz the file got copied with "_" in place of the section symbol.
gar.

What happens if you try this with 3.2?

 

[dave]

i don't see an "active python 3.2" i see "3.1" and "Many 3rd-party
modules and extensions that you may depend upon may not yet be
available for Python 3. As a result you may want to continue to use
Python 2 for the time being". and i depend on some 3rd party modules.

no i really think it's something like:

string = string.decode('latin_1').endocde('utf8') or something. i
just don't know what's expected. i've tried various flavors but can't
figure it out. it's very easy to test it yourself, just make a file
with the section symbol in it, then try to copy it with python 2.7.1

 

[Jerry Hill]

figure it out.  it's very easy to test it yourself, just make a file
with the section symbol in it, then try to copy it with python 2.7.1

This works fine for me on Windows XP and python 2.6.4 on an NTFS
formatted drive. Are either your source or destination network
drives? What does sys.getfilesystemencoding() say the encoding of
your filesystem is?

 

[dave]

ah! an interesting point! hmm yes when i'm running parallels, both the
source and destination are sortof "Network Drive"'s, they're actually
my native mac drives (file system reported as "PrlSF"). In this
situation all works well. the reported getfilesystemencoding() is
'mbcs' which is a convenient lie according to the python docs: "On
Windows NT+, file names are Unicode natively, so no conversion is
performed. getfilesystemencoding() still returns 'mbcs', as this is
the encoding that applications should use when they explicitly want to
convert Unicode strings to byte strings that are equivalent when used
as file names".

when i run natively, the drive is NOT a network drive, it is my mac
drive supported by "MacDrive" and the file system is reported as
"HFSJ". it seems the getfilesystemencoding() should take a drive
letter as a parameter cuz the encoding can be different per drive?
maybe? but now i think perhaps the problem is with MacDrive? but it
works flawlessly with all other software. hmmm.

 

[dave]

ah! an interesting point! hmm yes when i'm running parallels, both the
source and destination are sortof "Network Drive"'s, they're actually
my native mac drives (file system reported as "PrlSF"). In this
situation all works well. the reported getfilesystemencoding() is
'mbcs' which is a convenient lie according to the python docs: "On
Windows NT+, file names are Unicode natively, so no conversion is
performed. getfilesystemencoding() still returns 'mbcs', as this is
the encoding that applications should use when they explicitly want to
convert Unicode strings to byte strings that are equivalent when used
as file names".

when i run natively, the drive is NOT a network drive, it is my mac
drive supported by "MacDrive" and the file system is reported as
"HFSJ". it seems the getfilesystemencoding() should take a drive
letter as a parameter cuz the encoding can be different per drive?
maybe? but now i think perhaps the problem is with MacDrive? but it
works flawlessly with all other software. hmmm.

 

[dave]

thanks to your hint about drive format, i contacted MacDrive and they
confirmed it was an incorrect setting, i have since fixed the setting
and all is working!

not a python bug!

thanks for the replies.

-dave