os.stat() distorts filenames that end with period (nt.stat())

Discussion in 'Python' started by ruck, Sep 7, 2012.

  1. ruck

    ruck Guest

    (This with Python 2.7.2 on Windows 7)

    os.stat() won't recognize a filename ending in period.
    It will ignore trailing periods.
    If you ask it about file 'goo...' it will report on file 'goo'
    And if 'goo' doesn't exist, os.stat will complain.

    create file goo, then

    >>> os.stat('goo')

    nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)
    >>> os.stat('goo...')

    nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    rename goo to "goo...", then,

    >>> os.stat('goo...')


    Traceback (most recent call last):
    File "<pyshell#16>", line 1, in <module>
    os.stat('goo...')
    WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    Puzzling, to me at least.
    Any comments?
    This with Python 2.7.2 on Windows 7.
    Is there a workaround?
    Thanks,
    John
     
    ruck, Sep 7, 2012
    #1
    1. Advertising

  2. ruck

    Dave Angel Guest

    On 09/06/2012 08:55 PM, ruck wrote:
    > (This with Python 2.7.2 on Windows 7)
    >
    > os.stat() won't recognize a filename ending in period.
    > It will ignore trailing periods.
    > If you ask it about file 'goo...' it will report on file 'goo'
    > And if 'goo' doesn't exist, os.stat will complain.
    >
    > create file goo, then
    >
    > >>> os.stat('goo')

    > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)
    > >>> os.stat('goo...')

    > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)
    >
    > rename goo to "goo...", then,
    >
    > >>> os.stat('goo...')

    >
    > Traceback (most recent call last):
    > File "<pyshell#16>", line 1, in <module>
    > os.stat('goo...')
    > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'
    >
    > Puzzling, to me at least.
    > Any comments?
    > This with Python 2.7.2 on Windows 7.
    > Is there a workaround?
    > Thanks,
    > John


    FWIW, it seems to work okay here in Linux 11.04, both Python 2.7 and 3.2


    --

    DaveA
     
    Dave Angel, Sep 7, 2012
    #2
    1. Advertising

  3. ruck

    alex23 Guest

    On Sep 7, 10:55 am, ruck <> wrote:
    > (This with Python 2.7.2 on Windows 7)
    > rename goo to "goo...", then,


    I'm unable to rename any file to have a '...' suffix, are you certain
    the file exists in the form you think after the rename?
     
    alex23, Sep 7, 2012
    #3
  4. ruck

    ruck Guest

    On Thursday, September 6, 2012 6:19:11 PM UTC-7, Dave Angel wrote:
    > On 09/06/2012 08:55 PM, ruck wrote:
    >
    > > (This with Python 2.7.2 on Windows 7)

    >
    > >

    >
    > > os.stat() won't recognize a filename ending in period.

    >
    > > It will ignore trailing periods.

    >
    > > If you ask it about file 'goo...' it will report on file 'goo'

    >
    > > And if 'goo' doesn't exist, os.stat will complain.

    >
    > >

    >
    > > create file goo, then

    >
    > >

    >
    > > >>> os.stat('goo')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > > >>> os.stat('goo...')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > >

    >
    > > rename goo to "goo...", then,

    >
    > >

    >
    > > >>> os.stat('goo...')

    >
    > >

    >
    > > Traceback (most recent call last):

    >
    > > File "<pyshell#16>", line 1, in <module>

    >
    > > os.stat('goo...')

    >
    > > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    >
    > >

    >
    > > Puzzling, to me at least.

    >
    > > Any comments?

    >
    > > This with Python 2.7.2 on Windows 7.

    >
    > > Is there a workaround?

    >
    > > Thanks,

    >
    > > John

    >
    >
    >
    > FWIW, it seems to work okay here in Linux 11.04, both Python 2.7 and 3.2
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Thanks, I agree, I expect this is Windows 7 or win32 specific.

    Also, in creating a test case to demonstrate, I may have clouded my intent.
    I want to walk() a dir,
    reporting the output of os.stat() for files below the dir.
    One of the existing files happens to be named like "goo..."
    And os.stat('goo...') fails to see the file.
    I am not trying to rename the file, just hoping to stat an existing file.
    John
     
    ruck, Sep 7, 2012
    #4
  5. ruck

    ruck Guest

    On Thursday, September 6, 2012 6:19:11 PM UTC-7, Dave Angel wrote:
    > On 09/06/2012 08:55 PM, ruck wrote:
    >
    > > (This with Python 2.7.2 on Windows 7)

    >
    > >

    >
    > > os.stat() won't recognize a filename ending in period.

    >
    > > It will ignore trailing periods.

    >
    > > If you ask it about file 'goo...' it will report on file 'goo'

    >
    > > And if 'goo' doesn't exist, os.stat will complain.

    >
    > >

    >
    > > create file goo, then

    >
    > >

    >
    > > >>> os.stat('goo')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > > >>> os.stat('goo...')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > >

    >
    > > rename goo to "goo...", then,

    >
    > >

    >
    > > >>> os.stat('goo...')

    >
    > >

    >
    > > Traceback (most recent call last):

    >
    > > File "<pyshell#16>", line 1, in <module>

    >
    > > os.stat('goo...')

    >
    > > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    >
    > >

    >
    > > Puzzling, to me at least.

    >
    > > Any comments?

    >
    > > This with Python 2.7.2 on Windows 7.

    >
    > > Is there a workaround?

    >
    > > Thanks,

    >
    > > John

    >
    >
    >
    > FWIW, it seems to work okay here in Linux 11.04, both Python 2.7 and 3.2
    >
    >
    >
    >
    >
    > --
    >
    >
    >
    > DaveA


    Thanks, I agree, I expect this is Windows 7 or win32 specific.

    Also, in creating a test case to demonstrate, I may have clouded my intent.
    I want to walk() a dir,
    reporting the output of os.stat() for files below the dir.
    One of the existing files happens to be named like "goo..."
    And os.stat('goo...') fails to see the file.
    I am not trying to rename the file, just hoping to stat an existing file.
    John
     
    ruck, Sep 7, 2012
    #5
  6. ruck

    Terry Reedy Guest

    On 9/6/2012 8:55 PM, ruck wrote:
    > (This with Python 2.7.2 on Windows 7)
    >
    > os.stat() won't recognize a filename ending in period.
    > It will ignore trailing periods.
    > If you ask it about file 'goo...' it will report on file 'goo'
    > And if 'goo' doesn't exist, os.stat will complain.
    >
    > create file goo, then
    >
    > >>> os.stat('goo')

    > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)
    > >>> os.stat('goo...')

    > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)
    >
    > rename goo to "goo...", then,
    >
    > >>> os.stat('goo...')

    >
    > Traceback (most recent call last):
    > File "<pyshell#16>", line 1, in <module>
    > os.stat('goo...')
    > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'
    >
    > Puzzling, to me at least.
    > Any comments?


    Windows have restrictions on filenames. The restrictions are not
    consistent in that some parts of Windows will let you make names that
    other parts do not recognize or regard as illegal. I ran into this some
    years ago and there may be a discussion on the tracker, but I have
    forgetten the details except that one of the 'parts' was Windows
    Explorer. This *might* be what you are running into.

    --
    Terry Jan Reedy
     
    Terry Reedy, Sep 7, 2012
    #6
  7. On Thu, 06 Sep 2012 17:55:02 -0700, ruck wrote:

    > (This with Python 2.7.2 on Windows 7)
    >
    > os.stat() won't recognize a filename ending in period. It will ignore
    > trailing periods.
    > If you ask it about file 'goo...' it will report on file 'goo' And if
    > 'goo' doesn't exist, os.stat will complain.



    I don't have access to a Windows machine to test this, but I suspect that
    you have found a side-effect of a Windows bug.

    http://answers.microsoft.com/en-us/...iod-to-a/4329b1f1-746e-4c45-9c32-75622b6ab526

    http://support.microsoft.com/kb/115827



    --
    Steven
     
    Steven D'Aprano, Sep 7, 2012
    #7
  8. ruck

    ruck Guest

    On Thursday, September 6, 2012 6:49:43 PM UTC-7, alex23 wrote:
    > On Sep 7, 10:55 am, ruck <> wrote:
    >
    > > (This with Python 2.7.2 on Windows 7)

    >
    > > rename goo to "goo...", then,

    >
    >
    >
    > I'm unable to rename any file to have a '...' suffix, are you certain
    >
    > the file exists in the form you think after the rename?


    I see your point. Using cygwin bash, I have no problem naming files with '....' suffix. But I see you're right, it seems I cannot do so from Windows Explorer or from cmd shell.
    FWIW, Windows Explorer displays to me that the file is named with '...' suffix.
    John
     
    ruck, Sep 7, 2012
    #8
  9. ruck

    Ian Kelly Guest

    On Thu, Sep 6, 2012 at 6:55 PM, ruck <> wrote:
    > (This with Python 2.7.2 on Windows 7)
    >
    > os.stat() won't recognize a filename ending in period.
    > It will ignore trailing periods.
    > If you ask it about file 'goo...' it will report on file 'goo'
    > And if 'goo' doesn't exist, os.stat will complain.


    Due to the weirdness of Windows filename extensions, these names refer
    to the same file.

    C:\Users\Ian>echo hello > goo

    C:\Users\Ian>type goo
    hello

    C:\Users\Ian>type goo.
    hello

    C:\Users\Ian>type goo..
    hello

    C:\Users\Ian>type goo...
    hello

    C:\Users\Ian>echo world > goo...

    C:\Users\Ian>dir goo*
    Volume in drive C is OS
    Volume Serial Number is 9881-66F0

    Directory of C:\Users\Ian

    09/06/2012 09:10 PM 8 goo
    1 File(s) 8 bytes
    0 Dir(s) 8,884,142,080 bytes free

    C:\Users\Ian>type goo
    world
     
    Ian Kelly, Sep 7, 2012
    #9
  10. ruck

    ruck Guest

    On Thursday, September 6, 2012 7:05:39 PM UTC-7, Terry Reedy wrote:
    > On 9/6/2012 8:55 PM, ruck wrote:
    >
    > > (This with Python 2.7.2 on Windows 7)

    >
    > >

    >
    > > os.stat() won't recognize a filename ending in period.

    >
    > > It will ignore trailing periods.

    >
    > > If you ask it about file 'goo...' it will report on file 'goo'

    >
    > > And if 'goo' doesn't exist, os.stat will complain.

    >
    > >

    >
    > > create file goo, then

    >
    > >

    >
    > > >>> os.stat('goo')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > > >>> os.stat('goo...')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > >

    >
    > > rename goo to "goo...", then,

    >
    > >

    >
    > > >>> os.stat('goo...')

    >
    > >

    >
    > > Traceback (most recent call last):

    >
    > > File "<pyshell#16>", line 1, in <module>

    >
    > > os.stat('goo...')

    >
    > > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    >
    > >

    >
    > > Puzzling, to me at least.

    >
    > > Any comments?

    >
    >
    >
    > Windows have restrictions on filenames. The restrictions are not
    >
    > consistent in that some parts of Windows will let you make names that
    >
    > other parts do not recognize or regard as illegal. I ran into this some
    >
    > years ago and there may be a discussion on the tracker, but I have
    >
    > forgetten the details except that one of the 'parts' was Windows
    >
    > Explorer. This *might* be what you are running into.
    >
    >
    >
    > --
    >
    > Terry Jan Reedy


    Summary:
    I was complaining that
    >>> os.stat('goo...')


    Traceback (most recent call last):
    File "<pyshell#35>", line 1, in <module>
    os.stat('goo...')
    WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    is actually looking for file 'goo' instead of the existing file 'goo...' -- that's why it errors.
    (Python 2.7.2, Windows 7)

    Here's a workaround.
    >>> os.stat('\\\\?\\' + os.getcwd() + '\\' + 'goo...')

    nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0,
    st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L,
    st_ctime=1346978160L)

    In other words, prefix the path (full path?) with \\?\ to disable the Windows API filename interceptions.

    Detail:
    "Naming Files, Paths, and Namespaces" ( http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx )
    says

    Naming Conventions
    The following fundamental rules enable applications to
    create and process valid names for files and directories,
    regardless of the file system:

    * Do not end a file or directory name with a space or a
    period. Although the underlying file system may support
    such names, the Windows shell and user interface does not.

    In my case, the filesystem is NTFS and it does support filenames like 'goo...', and I'm not trying to use Windows shell or UI. So how do I get to the underlying filesystem without interference? Here (from the same reference)

    Namespaces
    There are two main categories of namespace conventions
    used in the Windows APIs, commonly referred to as NT
    namespaces and the Win32 namespaces. ...

    Win32 File Namespaces
    The Win32 namespace prefixing and conventions are
    summarized in this section and the following section,
    with descriptions of how they are used. Note that these
    examples are intended for use with the Windows API
    functions and do not all necessarily work with Windows
    shell applications such as Windows Explorer. For this
    reason there is a wider range of possible paths than is
    usually available from Windows shell applications, and
    Windows applications that take advantage of this can be
    developed using these namespace conventions.

    For file I/O, the "\\?\" prefix to a path string tells
    the Windows APIs to disable all string parsing and to
    send the string that follows it straight to the file
    system. For example, if the file system supports large
    paths and file names, you can exceed the MAX_PATH limits
    that are otherwise enforced by the Windows APIs.

    Thanks all for your comments and pointers.
    John
     
    ruck, Sep 7, 2012
    #10
  11. ruck

    ruck Guest

    On Thursday, September 6, 2012 7:05:39 PM UTC-7, Terry Reedy wrote:
    > On 9/6/2012 8:55 PM, ruck wrote:
    >
    > > (This with Python 2.7.2 on Windows 7)

    >
    > >

    >
    > > os.stat() won't recognize a filename ending in period.

    >
    > > It will ignore trailing periods.

    >
    > > If you ask it about file 'goo...' it will report on file 'goo'

    >
    > > And if 'goo' doesn't exist, os.stat will complain.

    >
    > >

    >
    > > create file goo, then

    >
    > >

    >
    > > >>> os.stat('goo')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > > >>> os.stat('goo...')

    >
    > > nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0, st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L, st_ctime=1346978160L)

    >
    > >

    >
    > > rename goo to "goo...", then,

    >
    > >

    >
    > > >>> os.stat('goo...')

    >
    > >

    >
    > > Traceback (most recent call last):

    >
    > > File "<pyshell#16>", line 1, in <module>

    >
    > > os.stat('goo...')

    >
    > > WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    >
    > >

    >
    > > Puzzling, to me at least.

    >
    > > Any comments?

    >
    >
    >
    > Windows have restrictions on filenames. The restrictions are not
    >
    > consistent in that some parts of Windows will let you make names that
    >
    > other parts do not recognize or regard as illegal. I ran into this some
    >
    > years ago and there may be a discussion on the tracker, but I have
    >
    > forgetten the details except that one of the 'parts' was Windows
    >
    > Explorer. This *might* be what you are running into.
    >
    >
    >
    > --
    >
    > Terry Jan Reedy


    Summary:
    I was complaining that
    >>> os.stat('goo...')


    Traceback (most recent call last):
    File "<pyshell#35>", line 1, in <module>
    os.stat('goo...')
    WindowsError: [Error 2] The system cannot find the file specified: 'goo...'

    is actually looking for file 'goo' instead of the existing file 'goo...' -- that's why it errors.
    (Python 2.7.2, Windows 7)

    Here's a workaround.
    >>> os.stat('\\\\?\\' + os.getcwd() + '\\' + 'goo...')

    nt.stat_result(st_mode=33206, st_ino=0L, st_dev=0, st_nlink=0, st_uid=0,
    st_gid=0, st_size=0L, st_atime=1346978160L, st_mtime=1346978160L,
    st_ctime=1346978160L)

    In other words, prefix the path (full path?) with \\?\ to disable the Windows API filename interceptions.

    Detail:
    "Naming Files, Paths, and Namespaces" ( http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx )
    says

    Naming Conventions
    The following fundamental rules enable applications to
    create and process valid names for files and directories,
    regardless of the file system:

    * Do not end a file or directory name with a space or a
    period. Although the underlying file system may support
    such names, the Windows shell and user interface does not.

    In my case, the filesystem is NTFS and it does support filenames like 'goo...', and I'm not trying to use Windows shell or UI. So how do I get to the underlying filesystem without interference? Here (from the same reference)

    Namespaces
    There are two main categories of namespace conventions
    used in the Windows APIs, commonly referred to as NT
    namespaces and the Win32 namespaces. ...

    Win32 File Namespaces
    The Win32 namespace prefixing and conventions are
    summarized in this section and the following section,
    with descriptions of how they are used. Note that these
    examples are intended for use with the Windows API
    functions and do not all necessarily work with Windows
    shell applications such as Windows Explorer. For this
    reason there is a wider range of possible paths than is
    usually available from Windows shell applications, and
    Windows applications that take advantage of this can be
    developed using these namespace conventions.

    For file I/O, the "\\?\" prefix to a path string tells
    the Windows APIs to disable all string parsing and to
    send the string that follows it straight to the file
    system. For example, if the file system supports large
    paths and file names, you can exceed the MAX_PATH limits
    that are otherwise enforced by the Windows APIs.

    Thanks all for your comments and pointers.
    John
     
    ruck, Sep 7, 2012
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. B.J.
    Replies:
    4
    Views:
    772
    Toby Inkster
    Apr 23, 2005
  2. Steven T. Hatton
    Replies:
    3
    Views:
    9,764
    Pete Becker
    Jun 2, 2005
  3. Patrick Useldinger

    os.stat('<filename>')[stat.ST_INO] on Windows

    Patrick Useldinger, Feb 27, 2005, in forum: Python
    Replies:
    6
    Views:
    1,227
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Mar 3, 2005
  4. Magesh
    Replies:
    3
    Views:
    465
    Gordon Burditt
    Oct 5, 2007
  5. Rolf Krüger
    Replies:
    2
    Views:
    640
    Ian Collins
    Mar 12, 2008
Loading...

Share This Page