os.access() under windows

Discussion in 'Python' started by Yann Leboulanger, Dec 1, 2007.

  1. Hi,

    Under Windows XP os.access has a strange behaviour:

    I create a folder test under e:

    then os.access('e:\\test', os.W_OK) returns True. Everything's ok.

    Now I move My Documents to this e:\test folder

    Then os.access('e:\\test', os.W_OK) returns False !!

    but this works:
    f = open('e:\\test\\test', 'a')
    f.write('abc')
    f.close()

    So why os.access returns False ?

    --
    Yann
    Yann Leboulanger, Dec 1, 2007
    #1
    1. Advertising

  2. Yann Leboulanger

    David Tweet Guest

    To answer indirectly, usually the EAFP (easier to ask forgiveness than
    permission) approach works better for this kind of thing.

    try:
    f = open('e:\\test\\test', 'a')
    f.write('abc')
    f.close()
    except IOError:
    print "couldn't write test file, continuing..."

    On Dec 1, 2007 1:48 AM, Yann Leboulanger <> wrote:
    > Hi,
    >
    > Under Windows XP os.access has a strange behaviour:
    >
    > I create a folder test under e:
    >
    > then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >
    > Now I move My Documents to this e:\test folder
    >
    > Then os.access('e:\\test', os.W_OK) returns False !!
    >
    > but this works:
    > f = open('e:\\test\\test', 'a')
    > f.write('abc')
    > f.close()
    >
    > So why os.access returns False ?
    >
    > --
    > Yann
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >




    --
    -David
    David Tweet, Dec 1, 2007
    #2
    1. Advertising

  3. David Tweet wrote:
    > To answer indirectly, usually the EAFP (easier to ask forgiveness than
    > permission) approach works better for this kind of thing.
    >
    > try:
    > f = open('e:\\test\\test', 'a')
    > f.write('abc')
    > f.close()
    > except IOError:
    > print "couldn't write test file, continuing..."
    >
    > On Dec 1, 2007 1:48 AM, Yann Leboulanger <> wrote:
    >> Hi,
    >>
    >> Under Windows XP os.access has a strange behaviour:
    >>
    >> I create a folder test under e:
    >>
    >> then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >>
    >> Now I move My Documents to this e:\test folder
    >>
    >> Then os.access('e:\\test', os.W_OK) returns False !!
    >>
    >> but this works:
    >> f = open('e:\\test\\test', 'a')
    >> f.write('abc')
    >> f.close()
    >>
    >> So why os.access returns False ?
    >>
    >> --
    >> Yann
    >> --
    >> http://mail.python.org/mailman/listinfo/python-list
    >>

    >



    yep but the test file I try to create to know if I can write there may
    be a folder for exemple, and the open() will fail.

    So this can be considered as a bug in os module?
    Yann Leboulanger, Dec 2, 2007
    #3
  4. > I create a folder test under e:
    >
    > then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >
    > Now I move My Documents to this e:\test folder
    >
    > Then os.access('e:\\test', os.W_OK) returns False !!


    This description is, unfortunately, too imprecise to allow reproducing
    that effect. What precisely do you mean by "I move My Documents to
    this e:\test folder"?

    Regards,
    Martin
    Martin v. Löwis, Dec 3, 2007
    #4
  5. Martin v. Löwis a écrit :
    >> I create a folder test under e:
    >>
    >> then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >>
    >> Now I move My Documents to this e:\test folder
    >>
    >> Then os.access('e:\\test', os.W_OK) returns False !!

    >
    > This description is, unfortunately, too imprecise to allow reproducing
    > that effect. What precisely do you mean by "I move My Documents to
    > this e:\test folder"?
    >



    I Right click on "My Documents" folder, and change the path to e:\test
    there.
    So that "My documents" folder points to e:\test
    Yann Leboulanger, Dec 3, 2007
    #5
  6. Yann Leboulanger

    Tim Golden Guest

    Yann Leboulanger wrote:
    > Martin v. Löwis a écrit :
    >>> I create a folder test under e:
    >>>
    >>> then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >>>
    >>> Now I move My Documents to this e:\test folder
    >>>
    >>> Then os.access('e:\\test', os.W_OK) returns False !!

    >> This description is, unfortunately, too imprecise to allow reproducing
    >> that effect. What precisely do you mean by "I move My Documents to
    >> this e:\test folder"?
    >>

    >
    >
    > I Right click on "My Documents" folder, and change the path to e:\test
    > there.
    > So that "My documents" folder points to e:\test


    Python uses the GetFileAttributesW API call to determine
    access to the path you pass in. It then tests the return
    against the FILE_ATTRIBUTE_READONLY flag and returns False
    if you asked for Write and the Readonly flag is set.

    The problem seems to be twofold: whereas a normal directory
    (such as e:\temp) will not usually have its readonly flag set,
    on a special directory such as the My Documents folder the flag
    seems to be set by the system; also, the READONLY flag on a
    directory refers to its deleteability, not to its writeablility.
    At least, according to:

    http://msdn2.microsoft.com/en-us/library/aa364944.aspx

    I suspect this constitutes a bug in os.access which
    should do some more interpretation of the results. But
    no doubt Martin von L can comment with more authority
    than I on this one.

    TJG
    Tim Golden, Dec 3, 2007
    #6
  7. Yann Leboulanger

    Tim Golden Guest

    Tim Golden wrote:
    > Yann Leboulanger wrote:
    >> Martin v. Löwis a écrit :
    >>>> I create a folder test under e:
    >>>>
    >>>> then os.access('e:\\test', os.W_OK) returns True. Everything's ok.
    >>>>
    >>>> Now I move My Documents to this e:\test folder
    >>>>
    >>>> Then os.access('e:\\test', os.W_OK) returns False !!
    >>> This description is, unfortunately, too imprecise to allow reproducing
    >>> that effect. What precisely do you mean by "I move My Documents to
    >>> this e:\test folder"?
    >>>

    >>
    >> I Right click on "My Documents" folder, and change the path to e:\test
    >> there.
    >> So that "My documents" folder points to e:\test

    >
    > Python uses the GetFileAttributesW API call to determine
    > access to the path you pass in. It then tests the return
    > against the FILE_ATTRIBUTE_READONLY flag and returns False
    > if you asked for Write and the Readonly flag is set.
    >
    > The problem seems to be twofold: whereas a normal directory
    > (such as e:\temp) will not usually have its readonly flag set,
    > on a special directory such as the My Documents folder the flag
    > seems to be set by the system; also, the READONLY flag on a
    > directory refers to its deleteability, not to its writeablility.
    > At least, according to:
    >
    > http://msdn2.microsoft.com/en-us/library/aa364944.aspx
    >
    > I suspect this constitutes a bug in os.access which
    > should do some more interpretation of the results. But
    > no doubt Martin von L can comment with more authority
    > than I on this one.
    >
    > TJG


    And just to complicate matters, it seems that readonly on
    a folder has an entirely special meaning. At least, according
    to this:

    http://support.microsoft.com/kb/326549

    Goodness knows what we're supposed to do with that.
    Part of the issue here is that we're using a Unix-style
    access API against Windows-style filesystem semantics.
    According to the man pages, it looks as though it basically
    checks the standard security bits against whatever you're
    asking for for your user. On Windows, the security semantics
    are quite different and you have to make some kind of
    decision as to what the os.access means.

    I'm happy to contribute a doc patch if I can imagine what
    exactly to write.

    TJG
    Tim Golden, Dec 3, 2007
    #7
  8. Tim Golden a écrit :
    >
    > I'm happy to contribute a doc patch if I can imagine what
    > exactly to write.
    >


    "Don't use it under windows, always consider it's True"?

    Maybe it would be a good idea to remove support for windows for this
    function, or make it always return True?
    Current behaviour is worth that nothing, access() return False but
    open() doesn't fail ...

    --
    Yann
    Yann Leboulanger, Dec 3, 2007
    #8
  9. Tim Golden a écrit :
    >
    > I'm happy to contribute a doc patch if I can imagine what
    > exactly to write.
    >


    "Don't use it under windows, always consider it's True"?

    Maybe it would be a good idea to remove support for windows for this
    function, or make it always return True?
    Current behaviour is worth that nothing, access() return False but
    open() doesn't fail ...

    --
    Yann
    Yann Leboulanger, Dec 3, 2007
    #9
  10. Yann Leboulanger

    Tim Golden Guest

    Yann Leboulanger wrote:
    > Tim Golden a écrit :
    >>
    >> I'm happy to contribute a doc patch if I can imagine what
    >> exactly to write.
    >>

    >
    > "Don't use it under windows, always consider it's True"?


    Well, that's not the case for files: if you set your
    file's readonly attribute to True, then os.access (W_OK)
    will return False and you won't be able to write to the
    file:

    <code>
    import os
    open ("temp.tmp", "w").close ()
    os.access ("temp.tmp", os.W_OK)
    os.system ("attrib +r temp.tmp")
    os.access ("temp.tmp", os.W_OK)
    open ("temp.tmp", "w").close ()
    os.system ("attrib -r temp.tmp")
    os.access ("temp.tmp", os.W_OK)
    </code>

    > Maybe it would be a good idea to remove support for windows for this
    > function, or make it always return True?


    The only issue (at least, the only one we're discussing here) is:
    If os.W_OK on a directory returns True, that won't stop you writing
    into that directory.

    > Current behaviour is worth that nothing, access() return False but
    > open() doesn't fail ...


    To be precise: open () on a file within that directory doesn't fail.

    Personally, I sympathise with you here. Python comes from a Unix
    background and, unsurprisingly, it offers all the major Unix
    system calls. Since Windows historically offered a Posix layer
    which mapped them to *something*[1], the developers simply called
    those under the covers. And the basic policy was: whatever Windows
    passes back to Python, Python passes on to you.

    Later (mid-2006, I think) some or all of these Posix-layer functions
    were replaced by native Win32 APIs. At that point, it arguably became
    Python's responsibility to define semantics. But it's a fuzzy sort of
    area. I think a doc patch which said something like: "Calls
    FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
    might meet the case, although a bit of a cop-out.

    TJG

    [1] http://msdn2.microsoft.com/en-us/library/1w06ktdy(VS.80).aspx
    Tim Golden, Dec 3, 2007
    #10
  11. Tim Golden a écrit :
    > Well, that's not the case for files: if you set your
    > file's readonly attribute to True, then os.access (W_OK)
    > will return False and you won't be able to write to the
    > file:



    > The only issue (at least, the only one we're discussing here) is:
    > If os.W_OK on a directory returns True, that won't stop you writing
    > into that directory.
    >



    >
    > To be precise: open () on a file within that directory doesn't fail.


    Yep sorry I was a bit too expeditive :)

    > Personally, I sympathise with you here. Python comes from a Unix
    > background and, unsurprisingly, it offers all the major Unix
    > system calls. Since Windows historically offered a Posix layer
    > which mapped them to *something*[1], the developers simply called
    > those under the covers. And the basic policy was: whatever Windows
    > passes back to Python, Python passes on to you.
    >
    > Later (mid-2006, I think) some or all of these Posix-layer functions
    > were replaced by native Win32 APIs. At that point, it arguably became
    > Python's responsibility to define semantics. But it's a fuzzy sort of
    > area. I think a doc patch which said something like: "Calls
    > FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
    > might meet the case, although a bit of a cop-out.
    >


    Ok thanks for all those information, I'll remove the call to os.access()
    on folders for windows in my application.

    --
    Yann
    Yann Leboulanger, Dec 3, 2007
    #11
  12. Tim Golden a écrit :
    > Well, that's not the case for files: if you set your
    > file's readonly attribute to True, then os.access (W_OK)
    > will return False and you won't be able to write to the
    > file:



    > The only issue (at least, the only one we're discussing here) is:
    > If os.W_OK on a directory returns True, that won't stop you writing
    > into that directory.
    >



    >
    > To be precise: open () on a file within that directory doesn't fail.


    Yep sorry I was a bit too expeditive :)

    > Personally, I sympathise with you here. Python comes from a Unix
    > background and, unsurprisingly, it offers all the major Unix
    > system calls. Since Windows historically offered a Posix layer
    > which mapped them to *something*[1], the developers simply called
    > those under the covers. And the basic policy was: whatever Windows
    > passes back to Python, Python passes on to you.
    >
    > Later (mid-2006, I think) some or all of these Posix-layer functions
    > were replaced by native Win32 APIs. At that point, it arguably became
    > Python's responsibility to define semantics. But it's a fuzzy sort of
    > area. I think a doc patch which said something like: "Calls
    > FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
    > might meet the case, although a bit of a cop-out.
    >


    Ok thanks for all those information, I'll remove the call to os.access()
    on folders for windows in my application.

    --
    Yann
    Yann Leboulanger, Dec 3, 2007
    #12
  13. Yann Leboulanger

    Tim Golden Guest

    Yann Leboulanger wrote:
    > Ok thanks for all those information, I'll remove the call to os.access()
    > on folders for windows in my application.
    >


    FWIW, I think it's worth bearing in mind what was said
    earlier in this thread: it's easier to ask forgiveness
    than permission. Technically, even if os.access did
    exactly what you expected, there's nothing to stop the
    access changing between os.access (...) and open (...).
    Unless you have reason not to, it's considered perfectly
    good practice in Python to try/except the open ():

    <code>
    try:
    f = open ("e:/test/test.tst", "w")
    except (IOError, WindowsError):
    print "Something went wrong"
    else:
    "Do whatever"
    </code>

    TJG
    Tim Golden, Dec 3, 2007
    #13
  14. > http://support.microsoft.com/kb/326549
    >
    > Goodness knows what we're supposed to do with that.


    Just in case it's not clear what Tim is getting at:

    if a folder is marked read-only on Windows, it doesn't mean
    that you can only read from it. The read-only bit is a legacy
    thing, anyway, since you are supposed to use ACLs to mark
    a folder as read-only (by only granting write access to those
    who are supposed to write)

    As the read-only bit is otherwise unused, Explorer uses it
    to mark folders as being special, such a My Documents.
    So by redirecting My Documents, you set the read-only bit
    on the folder, causing access() to claim that write access
    is not granted.

    > Part of the issue here is that we're using a Unix-style
    > access API against Windows-style filesystem semantics.
    > According to the man pages, it looks as though it basically
    > checks the standard security bits against whatever you're
    > asking for for your user. On Windows, the security semantics
    > are quite different and you have to make some kind of
    > decision as to what the os.access means.
    >
    > I'm happy to contribute a doc patch if I can imagine what
    > exactly to write.


    It would be possible to fix this specific case, by always
    returning True for directories; and perhaps I'll do that for
    2.5.2.

    A more general issue is whether the ACL should also be
    taken into account. This would involve calling things like
    OpenThreadToken, MapGenericMask, and AccessCheck. These are
    all functions from the NT security API, and unavailable
    on Win9x - which is the likely reason why the MS CRT did
    not use them, either. Providing a proper access() implementation
    for NT+ then only becomes possible for 2.6 (where W9x
    is no longer supported).

    Regards,
    Martin

    P.S. I would never guessed that "move My Documents to test"
    doesn't mean "drag-and-drop My Documents into test", but
    "redirect My Documents to the test folder".
    Martin v. Löwis, Dec 3, 2007
    #14
  15. Martin v. Löwis wrote:
    > Just in case it's not clear what Tim is getting at:
    >
    > if a folder is marked read-only on Windows, it doesn't mean
    > that you can only read from it. The read-only bit is a legacy
    > thing, anyway, since you are supposed to use ACLs to mark
    > a folder as read-only (by only granting write access to those
    > who are supposed to write)
    >
    > As the read-only bit is otherwise unused, Explorer uses it
    > to mark folders as being special, such a My Documents.
    > So by redirecting My Documents, you set the read-only bit
    > on the folder, causing access() to claim that write access
    > is not granted.
    >
    > It would be possible to fix this specific case, by always
    > returning True for directories; and perhaps I'll do that for
    > 2.5.2.
    >
    > A more general issue is whether the ACL should also be
    > taken into account. This would involve calling things like
    > OpenThreadToken, MapGenericMask, and AccessCheck. These are
    > all functions from the NT security API, and unavailable
    > on Win9x - which is the likely reason why the MS CRT did
    > not use them, either. Providing a proper access() implementation
    > for NT+ then only becomes possible for 2.6 (where W9x
    > is no longer supported).
    >


    Great, thanks for those information!

    > P.S. I would never guessed that "move My Documents to test"
    > doesn't mean "drag-and-drop My Documents into test", but
    > "redirect My Documents to the test folder".


    Right, sorry :/
    --
    Yann
    Yann Leboulanger, Dec 3, 2007
    #15
  16. Yann Leboulanger

    Tim Golden Guest

    Martin v. Löwis wrote:
    > It would be possible to fix this specific case, by always
    > returning True for directories; and perhaps I'll do that for
    > 2.5.2.


    Martin. Could you confirm that the outline below correctly
    describes the behaviour of the os.access function under
    Windows, please? If you confirm its accuracy, I'll write it
    up as a docs patch against the development docs.

    """
    The os.access function originates on Unix where it straightforwardly
    reflects the permission-bit model of security. On Windows, that
    model does not obtain so the Python function compromises as follows:

    + If the path does not exist or if access if forbidden, return False
    + Requests for access mode R_OK and X_OK will always return True
    if the path can be accessed at all.
    + If the path refers to a directory, return True since the readonly
    attribute on a directory does not prevent writing files.
    + If W_OK access is requested and the file's readonly flag is set,
    return False.
    + If W_OK access is requested and the file's readonly flag is not
    set, return True.

    NB None of the checks explicitly take into account security
    ACLs.
    """

    > A more general issue is whether the ACL should also be
    > taken into account. This would involve calling things like
    > OpenThreadToken, MapGenericMask, and AccessCheck. These are
    > all functions from the NT security API, and unavailable
    > on Win9x - which is the likely reason why the MS CRT did
    > not use them, either. Providing a proper access() implementation
    > for NT+ then only becomes possible for 2.6 (where W9x
    > is no longer supported).


    Agreed. I think I'd like to see that happen, but I have to
    down several strengthening drinks every time I approach the
    Windows Security API!

    TJG
    Tim Golden, Dec 4, 2007
    #16
  17. > Martin. Could you confirm that the outline below correctly
    > describes the behaviour of the os.access function under
    > Windows, please?


    It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
    the test "if directory:return True" was not implemented.

    Notice that the first sentence:

    "If the path does not exist or if access if forbidden, return False"

    is confusing. It seems to say "access() returns false if access is
    forbidden". I think you are referring to the directory (in the sense
    of os.path.dirname) of the path to be tested.

    > Agreed. I think I'd like to see that happen, but I have to
    > down several strengthening drinks every time I approach the
    > Windows Security API!


    There is a good tutorial example in the AccessCheck documentation.
    If you leave out the impersonation part, it should be
    straight-forward to apply this to Python. Contributions are
    welcome.

    Regards,
    Martin
    Martin v. Löwis, Dec 4, 2007
    #17
  18. Yann Leboulanger

    Tim Golden Guest

    Martin v. Löwis wrote:
    >> Martin. Could you confirm that the outline below correctly
    >> describes the behaviour of the os.access function under
    >> Windows, please?

    >
    > It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
    > the test "if directory:return True" was not implemented.


    Thanks. Version dependent stuff noted.

    > Notice that the first sentence:
    >
    > "If the path does not exist or if access if forbidden, return False"
    >
    > is confusing. It seems to say "access() returns false if access is
    > forbidden". I think you are referring to the directory (in the sense
    > of os.path.dirname) of the path to be tested.


    Now, ironically, I'm confused by your recap :) What I meant to say was
    that the os.access function as implemented under Windows returns False
    if the path in question (say, "x:\someones-private-docs\diary.doc") was
    inaccessible to the process invoking os.access by virtue of file
    system permissions. (Or, even, that it simply didn't exist).

    TJG
    Tim Golden, Dec 4, 2007
    #18
  19. > Now, ironically, I'm confused by your recap :) What I meant to say was
    > that the os.access function as implemented under Windows returns False
    > if the path in question (say, "x:\someones-private-docs\diary.doc") was
    > inaccessible to the process invoking os.access by virtue of file
    > system permissions. (Or, even, that it simply didn't exist).


    Isn't that exactly the question that access() is trying to answer?
    (whether the file is inaccessible?)
    Then the reader could say "if it's inaccessible, return False,
    otherwise, return True". It suggests that we already have a mechanism
    to determine accessibility.

    The case you are referring to is when GetFileAttributes fails, with
    a permission error, right?

    I'm not quite sure why it could fail in cases where the file is
    present, in particular, if the user has the privilege to bypass
    traversal checking.

    Regards,
    Martin
    Martin v. Löwis, Dec 4, 2007
    #19
  20. Yann Leboulanger

    Tim Golden Guest

    Martin v. Löwis wrote:
    >> Now, ironically, I'm confused by your recap :) What I meant to say was
    >> that the os.access function as implemented under Windows returns False
    >> if the path in question (say, "x:\someones-private-docs\diary.doc") was
    >> inaccessible to the process invoking os.access by virtue of file
    >> system permissions. (Or, even, that it simply didn't exist).

    >
    > Isn't that exactly the question that access() is trying to answer?
    > (whether the file is inaccessible?)
    > Then the reader could say "if it's inaccessible, return False,
    > otherwise, return True". It suggests that we already have a mechanism
    > to determine accessibility.


    Up to a point: this meets the case where we fail to access
    the file at all (for read or write or whatever). But what
    about where we can read the directory entry, and the
    read-only attribute isn't set? At present, we'll return
    True to a W_OK access check in these circs, but this user
    might in fact be denied write access by the ACLs. (In fact,
    they might even be denied read access, since I imagine we
    only need access to the directory entry to check the
    attributes).

    Have I missed something?

    BTW I can't see a tutorial in the AccessCheck docs here:

    http://msdn2.microsoft.com/en-us/library/aa374815.aspx

    or in the SDK help file. Were you referring to a different
    set of docs? I'm girding my loins to provide a patch if I
    can!

    TJG
    Tim Golden, Dec 4, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. hshdude
    Replies:
    12
    Views:
    1,037
    Dimitri Maziuk
    Nov 4, 2004
  2. Marcel Achim
    Replies:
    2
    Views:
    449
    Ed Suominen
    Apr 9, 2004
  3. Jeff Dege
    Replies:
    2
    Views:
    568
    Jeff Dege
    Jan 31, 2008
  4. Ran
    Replies:
    1
    Views:
    495
    Bob Walton
    Dec 7, 2003
  5. Stefan Ram
    Replies:
    5
    Views:
    129
    Stefan Ram
    Mar 17, 2014
Loading...

Share This Page