unicode shutil.copy() changes a file name during copy?

Discussion in 'Python' started by dave, Feb 16, 2011.

  1. dave

    dave Guest

    i'm on windows, using active python 2.7.1

    i've written a script to copy a folder of files to dest folder..

    one if the files in this folder has the section symbol (§, '\x15') as
    part of the file name
    shutil.copy(src_file, dst_file) "can't find the file specified" when
    it does the os.chmod() part, can't find dest file,
    cuz the file got copied with "_" in place of the section symbol.
    gar.

    Traceback (most recent call last):
    File "../../python/post_build.py", line 159, in <module>
    main(proB, debugB)
    File "../../python/post_build.py", line 105, in main
    paths.copy_folder(srcResFolder + 'Export', destRes + '/codecs/
    Export')
    File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
    \paths.py", line 77, in copy_folder
    copy_folder(srcPath, new_dst_obj)
    File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
    \paths.py", line 80, in copy_folder
    copy_file(srcPath, new_dst_obj)
    File "F:\Users\davec\Developer\depot\kJams\Development\xplat\python
    \paths.py", line 37, in copy_file
    shutil.copy(src_file, dst_file)
    File "C:\Python27\lib\shutil.py", line 117, in copy
    copymode(src, dst)
    File "C:\Python27\lib\shutil.py", line 90, in copymode
    os.chmod(dst, mode)
    WindowsError: [Error 2] The system cannot find the file specified:
    'build\\kJams Pro Debug.app/Contents/Resources/codecs/Export/Codec
    [MooV]/Animation \xa7 AAC.rtm'

    if i replace that with shutil.copyfile(src_file, dst_file) it "works",
    again by replacing the section symbol with underbar.
    but that leaves me to call os.chmod() myself, which of course fails
    because of the character substitution.

    i'm gathering the list of files to iterate over the folder via:
    fileList = os.listdir(src_dir)
    for fileName in fileList:

    and yes, src_dir is "unicode". but the fileName when i print it shows
    the degree symbol (°, '\xa7') in place of the section symbol.

    i suspect it's a 'dbcs' or 'latin_1' problem?

    this same exact python code works on the mac (it's utf8 down to the
    metal) and, get this, it works when running Windows under Parallels
    Desktop. It fails only when running Windows natively. I can't see
    how that makes a difference.

    i can give a whole lot more info, but i just want to know if i'm
    trying to do something impossible. If this seems strange to y'all
    gladly i'll fill in all the details, but hoping it can be short
    circuited by someone going "ah hah! you have to do such and so".
     
    dave, Feb 16, 2011
    #1
    1. Advertising

  2. dave

    Terry Reedy Guest

    On 2/15/2011 11:50 PM, dave wrote:
    > i'm on windows, using active python 2.7.1
    >
    > i've written a script to copy a folder of files to dest folder..
    >
    > one if the files in this folder has the section symbol (§, '\x15') as
    > part of the file name
    > shutil.copy(src_file, dst_file) "can't find the file specified" when
    > it does the os.chmod() part, can't find dest file,
    > cuz the file got copied with "_" in place of the section symbol.
    > gar.


    What happens if you try this with 3.2?

    --
    Terry Jan Reedy
     
    Terry Reedy, Feb 16, 2011
    #2
    1. Advertising

  3. dave

    dave Guest

    i don't see an "active python 3.2" i see "3.1" and "Many 3rd-party
    modules and extensions that you may depend upon may not yet be
    available for Python 3. As a result you may want to continue to use
    Python 2 for the time being". and i depend on some 3rd party modules.

    no i really think it's something like:

    string = string.decode('latin_1').endocde('utf8') or something. i
    just don't know what's expected. i've tried various flavors but can't
    figure it out. it's very easy to test it yourself, just make a file
    with the section symbol in it, then try to copy it with python 2.7.1
     
    dave, Feb 16, 2011
    #3
  4. dave

    Jerry Hill Guest

    On Wed, Feb 16, 2011 at 1:28 PM, dave <> wrote:
    > figure it out.  it's very easy to test it yourself, just make a file
    > with the section symbol in it, then try to copy it with python 2.7.1


    This works fine for me on Windows XP and python 2.6.4 on an NTFS
    formatted drive. Are either your source or destination network
    drives? What does sys.getfilesystemencoding() say the encoding of
    your filesystem is?

    --
    Jerry
     
    Jerry Hill, Feb 16, 2011
    #4
  5. dave

    dave Guest

    ah! an interesting point! hmm yes when i'm running parallels, both the
    source and destination are sortof "Network Drive"'s, they're actually
    my native mac drives (file system reported as "PrlSF"). In this
    situation all works well. the reported getfilesystemencoding() is
    'mbcs' which is a convenient lie according to the python docs: "On
    Windows NT+, file names are Unicode natively, so no conversion is
    performed. getfilesystemencoding() still returns 'mbcs', as this is
    the encoding that applications should use when they explicitly want to
    convert Unicode strings to byte strings that are equivalent when used
    as file names".

    when i run natively, the drive is NOT a network drive, it is my mac
    drive supported by "MacDrive" and the file system is reported as
    "HFSJ". it seems the getfilesystemencoding() should take a drive
    letter as a parameter cuz the encoding can be different per drive?
    maybe? but now i think perhaps the problem is with MacDrive? but it
    works flawlessly with all other software. hmmm.
     
    dave, Feb 16, 2011
    #5
  6. dave

    dave Guest

    ah! an interesting point! hmm yes when i'm running parallels, both the
    source and destination are sortof "Network Drive"'s, they're actually
    my native mac drives (file system reported as "PrlSF"). In this
    situation all works well. the reported getfilesystemencoding() is
    'mbcs' which is a convenient lie according to the python docs: "On
    Windows NT+, file names are Unicode natively, so no conversion is
    performed. getfilesystemencoding() still returns 'mbcs', as this is
    the encoding that applications should use when they explicitly want to
    convert Unicode strings to byte strings that are equivalent when used
    as file names".

    when i run natively, the drive is NOT a network drive, it is my mac
    drive supported by "MacDrive" and the file system is reported as
    "HFSJ". it seems the getfilesystemencoding() should take a drive
    letter as a parameter cuz the encoding can be different per drive?
    maybe? but now i think perhaps the problem is with MacDrive? but it
    works flawlessly with all other software. hmmm.
     
    dave, Feb 16, 2011
    #6
  7. dave

    dave Guest

    thanks to your hint about drive format, i contacted MacDrive and they
    confirmed it was an incorrect setting, i have since fixed the setting
    and all is working!

    not a python bug!

    thanks for the replies.

    -dave
     
    dave, Feb 16, 2011
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Joe

    file copy shutil

    Joe, Feb 9, 2004, in forum: Python
    Replies:
    1
    Views:
    353
    Michael Hudson
    Feb 9, 2004
  2. BartlebyScrivener

    shutil.copy time stamp errors on win XP

    BartlebyScrivener, Apr 15, 2006, in forum: Python
    Replies:
    2
    Views:
    404
    BartlebyScrivener
    Apr 16, 2006
  3. Russell Warren
    Replies:
    1
    Views:
    833
    Tim Roberts
    Apr 27, 2006
  4. Tim Golden
    Replies:
    0
    Views:
    627
    Tim Golden
    Apr 27, 2006
  5. Fletcher Johnson

    Why is the shutil module called shutil?

    Fletcher Johnson, Sep 24, 2011, in forum: Python
    Replies:
    11
    Views:
    912
    alex23
    Sep 26, 2011
Loading...

Share This Page