os.access() under windows

[Yann Leboulanger]

Hi,

Under Windows XP os.access has a strange behaviour:

I create a folder test under e:

then os.access('e:\\test', os.W_OK) returns True. Everything's ok.

Now I move My Documents to this e:\test folder

Then os.access('e:\\test', os.W_OK) returns False !!

but this works:
f = open('e:\\test\\test', 'a')
f.write('abc')
f.close()

So why os.access returns False ?

 

[David Tweet]

To answer indirectly, usually the EAFP (easier to ask forgiveness than
permission) approach works better for this kind of thing.

try:
f = open('e:\\test\\test', 'a')
f.write('abc')
f.close()
except IOError:
print "couldn't write test file, continuing..."

 

[Yann Leboulanger]

To answer indirectly, usually the EAFP (easier to ask forgiveness than
permission) approach works better for this kind of thing.

try:
f = open('e:\\test\\test', 'a')
f.write('abc')
f.close()
except IOError:
print "couldn't write test file, continuing..."

yep but the test file I try to create to know if I can write there may
be a folder for exemple, and the open() will fail.

So this can be considered as a bug in os module?

 

[Martin v. Löwis]

I create a folder test under e:

then os.access('e:\\test', os.W_OK) returns True. Everything's ok.

Now I move My Documents to this e:\test folder

Then os.access('e:\\test', os.W_OK) returns False !!

This description is, unfortunately, too imprecise to allow reproducing
that effect. What precisely do you mean by "I move My Documents to
this e:\test folder"?

Regards,
Martin

 

[Yann Leboulanger]

Martin v. Löwis a écrit :

This description is, unfortunately, too imprecise to allow reproducing
that effect. What precisely do you mean by "I move My Documents to
this e:\test folder"?

I Right click on "My Documents" folder, and change the path to e:\test
there.
So that "My documents" folder points to e:\test

 

[Tim Golden]

Martin v. Löwis a écrit :


I Right click on "My Documents" folder, and change the path to e:\test
there.
So that "My documents" folder points to e:\test

Python uses the GetFileAttributesW API call to determine
access to the path you pass in. It then tests the return
against the FILE_ATTRIBUTE_READONLY flag and returns False
if you asked for Write and the Readonly flag is set.

The problem seems to be twofold: whereas a normal directory
(such as e:\temp) will not usually have its readonly flag set,
on a special directory such as the My Documents folder the flag
seems to be set by the system; also, the READONLY flag on a
directory refers to its deleteability, not to its writeablility.
At least, according to:

http://msdn2.microsoft.com/en-us/library/aa364944.aspx

I suspect this constitutes a bug in os.access which
should do some more interpretation of the results. But
no doubt Martin von L can comment with more authority
than I on this one.

TJG

 

[Tim Golden]

Python uses the GetFileAttributesW API call to determine
access to the path you pass in. It then tests the return
against the FILE_ATTRIBUTE_READONLY flag and returns False
if you asked for Write and the Readonly flag is set.

The problem seems to be twofold: whereas a normal directory
(such as e:\temp) will not usually have its readonly flag set,
on a special directory such as the My Documents folder the flag
seems to be set by the system; also, the READONLY flag on a
directory refers to its deleteability, not to its writeablility.
At least, according to:

http://msdn2.microsoft.com/en-us/library/aa364944.aspx

I suspect this constitutes a bug in os.access which
should do some more interpretation of the results. But
no doubt Martin von L can comment with more authority
than I on this one.

TJG

And just to complicate matters, it seems that readonly on
a folder has an entirely special meaning. At least, according
to this:

http://support.microsoft.com/kb/326549

Goodness knows what we're supposed to do with that.
Part of the issue here is that we're using a Unix-style
access API against Windows-style filesystem semantics.
According to the man pages, it looks as though it basically
checks the standard security bits against whatever you're
asking for for your user. On Windows, the security semantics
are quite different and you have to make some kind of
decision as to what the os.access means.

I'm happy to contribute a doc patch if I can imagine what
exactly to write.

TJG

 

[Yann Leboulanger]

Tim Golden a écrit :

I'm happy to contribute a doc patch if I can imagine what
exactly to write.

"Don't use it under windows, always consider it's True"?

Maybe it would be a good idea to remove support for windows for this
function, or make it always return True?
Current behaviour is worth that nothing, access() return False but
open() doesn't fail ...

 

[Yann Leboulanger]

Tim Golden a écrit :

I'm happy to contribute a doc patch if I can imagine what
exactly to write.

"Don't use it under windows, always consider it's True"?

Maybe it would be a good idea to remove support for windows for this
function, or make it always return True?
Current behaviour is worth that nothing, access() return False but
open() doesn't fail ...

 

[Tim Golden]

Tim Golden a écrit :

"Don't use it under windows, always consider it's True"?

Well, that's not the case for files: if you set your
file's readonly attribute to True, then os.access (W_OK)
will return False and you won't be able to write to the
file:

<code>
import os
open ("temp.tmp", "w").close ()
os.access ("temp.tmp", os.W_OK)
os.system ("attrib +r temp.tmp")
os.access ("temp.tmp", os.W_OK)
open ("temp.tmp", "w").close ()
os.system ("attrib -r temp.tmp")
os.access ("temp.tmp", os.W_OK)

Maybe it would be a good idea to remove support for windows for this
function, or make it always return True?

The only issue (at least, the only one we're discussing here) is:
If os.W_OK on a directory returns True, that won't stop you writing
into that directory.

Current behaviour is worth that nothing, access() return False but
open() doesn't fail ...

To be precise: open () on a file within that directory doesn't fail.

Personally, I sympathise with you here. Python comes from a Unix
background and, unsurprisingly, it offers all the major Unix
system calls. Since Windows historically offered a Posix layer
which mapped them to *something*[1], the developers simply called
those under the covers. And the basic policy was: whatever Windows
passes back to Python, Python passes on to you.

Later (mid-2006, I think) some or all of these Posix-layer functions
were replaced by native Win32 APIs. At that point, it arguably became
Python's responsibility to define semantics. But it's a fuzzy sort of
area. I think a doc patch which said something like: "Calls
FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
might meet the case, although a bit of a cop-out.

TJG

[1] http://msdn2.microsoft.com/en-us/library/1w06ktdy(VS.80).aspx

 

[Yann Leboulanger]

Tim Golden a écrit :

Well, that's not the case for files: if you set your
file's readonly attribute to True, then os.access (W_OK)
will return False and you won't be able to write to the
file:

The only issue (at least, the only one we're discussing here) is:
If os.W_OK on a directory returns True, that won't stop you writing
into that directory.



To be precise: open () on a file within that directory doesn't fail.

Yep sorry I was a bit too expeditive

Personally, I sympathise with you here. Python comes from a Unix
background and, unsurprisingly, it offers all the major Unix
system calls. Since Windows historically offered a Posix layer
which mapped them to *something*[1], the developers simply called
those under the covers. And the basic policy was: whatever Windows
passes back to Python, Python passes on to you.

Later (mid-2006, I think) some or all of these Posix-layer functions
were replaced by native Win32 APIs. At that point, it arguably became
Python's responsibility to define semantics. But it's a fuzzy sort of
area. I think a doc patch which said something like: "Calls
FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
might meet the case, although a bit of a cop-out.

Ok thanks for all those information, I'll remove the call to os.access()
on folders for windows in my application.

 

[Yann Leboulanger]

Tim Golden a écrit :

Well, that's not the case for files: if you set your
file's readonly attribute to True, then os.access (W_OK)
will return False and you won't be able to write to the
file:

The only issue (at least, the only one we're discussing here) is:
If os.W_OK on a directory returns True, that won't stop you writing
into that directory.



To be precise: open () on a file within that directory doesn't fail.

Yep sorry I was a bit too expeditive

Personally, I sympathise with you here. Python comes from a Unix
background and, unsurprisingly, it offers all the major Unix
system calls. Since Windows historically offered a Posix layer
which mapped them to *something*[1], the developers simply called
those under the covers. And the basic policy was: whatever Windows
passes back to Python, Python passes on to you.

Later (mid-2006, I think) some or all of these Posix-layer functions
were replaced by native Win32 APIs. At that point, it arguably became
Python's responsibility to define semantics. But it's a fuzzy sort of
area. I think a doc patch which said something like: "Calls
FileGetAttribute[A|W] and compares against FILE_READONLY_ATTRIBUTE"
might meet the case, although a bit of a cop-out.

Ok thanks for all those information, I'll remove the call to os.access()
on folders for windows in my application.

 

[Tim Golden]

Ok thanks for all those information, I'll remove the call to os.access()
on folders for windows in my application.

FWIW, I think it's worth bearing in mind what was said
earlier in this thread: it's easier to ask forgiveness
than permission. Technically, even if os.access did
exactly what you expected, there's nothing to stop the
access changing between os.access (...) and open (...).
Unless you have reason not to, it's considered perfectly
good practice in Python to try/except the open ():

<code>
try:
f = open ("e:/test/test.tst", "w")
except (IOError, WindowsError):
print "Something went wrong"
else:
"Do whatever"
</code>

TJG

 

[Martin v. Löwis]

http://support.microsoft.com/kb/326549

Goodness knows what we're supposed to do with that.

Just in case it's not clear what Tim is getting at:

if a folder is marked read-only on Windows, it doesn't mean
that you can only read from it. The read-only bit is a legacy
thing, anyway, since you are supposed to use ACLs to mark
a folder as read-only (by only granting write access to those
who are supposed to write)

As the read-only bit is otherwise unused, Explorer uses it
to mark folders as being special, such a My Documents.
So by redirecting My Documents, you set the read-only bit
on the folder, causing access() to claim that write access
is not granted.

Part of the issue here is that we're using a Unix-style
access API against Windows-style filesystem semantics.
According to the man pages, it looks as though it basically
checks the standard security bits against whatever you're
asking for for your user. On Windows, the security semantics
are quite different and you have to make some kind of
decision as to what the os.access means.

I'm happy to contribute a doc patch if I can imagine what
exactly to write.

It would be possible to fix this specific case, by always
returning True for directories; and perhaps I'll do that for
2.5.2.

A more general issue is whether the ACL should also be
taken into account. This would involve calling things like
OpenThreadToken, MapGenericMask, and AccessCheck. These are
all functions from the NT security API, and unavailable
on Win9x - which is the likely reason why the MS CRT did
not use them, either. Providing a proper access() implementation
for NT+ then only becomes possible for 2.6 (where W9x
is no longer supported).

Regards,
Martin

P.S. I would never guessed that "move My Documents to test"
doesn't mean "drag-and-drop My Documents into test", but
"redirect My Documents to the test folder".

 

[Yann Leboulanger]

Just in case it's not clear what Tim is getting at:

if a folder is marked read-only on Windows, it doesn't mean
that you can only read from it. The read-only bit is a legacy
thing, anyway, since you are supposed to use ACLs to mark
a folder as read-only (by only granting write access to those
who are supposed to write)

As the read-only bit is otherwise unused, Explorer uses it
to mark folders as being special, such a My Documents.
So by redirecting My Documents, you set the read-only bit
on the folder, causing access() to claim that write access
is not granted.

It would be possible to fix this specific case, by always
returning True for directories; and perhaps I'll do that for
2.5.2.

A more general issue is whether the ACL should also be
taken into account. This would involve calling things like
OpenThreadToken, MapGenericMask, and AccessCheck. These are
all functions from the NT security API, and unavailable
on Win9x - which is the likely reason why the MS CRT did
not use them, either. Providing a proper access() implementation
for NT+ then only becomes possible for 2.6 (where W9x
is no longer supported).

Great, thanks for those information!

P.S. I would never guessed that "move My Documents to test"
doesn't mean "drag-and-drop My Documents into test", but
"redirect My Documents to the test folder".

Right, sorry :/

 

[Tim Golden]

It would be possible to fix this specific case, by always
returning True for directories; and perhaps I'll do that for
2.5.2.

Martin. Could you confirm that the outline below correctly
describes the behaviour of the os.access function under
Windows, please? If you confirm its accuracy, I'll write it
up as a docs patch against the development docs.

"""
The os.access function originates on Unix where it straightforwardly
reflects the permission-bit model of security. On Windows, that
model does not obtain so the Python function compromises as follows:

+ If the path does not exist or if access if forbidden, return False
+ Requests for access mode R_OK and X_OK will always return True
if the path can be accessed at all.
+ If the path refers to a directory, return True since the readonly
attribute on a directory does not prevent writing files.
+ If W_OK access is requested and the file's readonly flag is set,
return False.
+ If W_OK access is requested and the file's readonly flag is not
set, return True.

NB None of the checks explicitly take into account security
ACLs.
"""

A more general issue is whether the ACL should also be
taken into account. This would involve calling things like
OpenThreadToken, MapGenericMask, and AccessCheck. These are
all functions from the NT security API, and unavailable
on Win9x - which is the likely reason why the MS CRT did
not use them, either. Providing a proper access() implementation
for NT+ then only becomes possible for 2.6 (where W9x
is no longer supported).

Agreed. I think I'd like to see that happen, but I have to
down several strengthening drinks every time I approach the
Windows Security API!

TJG

 

[Martin v. Löwis]

Martin. Could you confirm that the outline below correctly

describes the behaviour of the os.access function under
Windows, please?

It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
the test "if directory:return True" was not implemented.

Notice that the first sentence:

"If the path does not exist or if access if forbidden, return False"

is confusing. It seems to say "access() returns false if access is
forbidden". I think you are referring to the directory (in the sense
of os.path.dirname) of the path to be tested.

Agreed. I think I'd like to see that happen, but I have to
down several strengthening drinks every time I approach the
Windows Security API!

There is a good tutorial example in the AccessCheck documentation.
If you leave out the impersonation part, it should be
straight-forward to apply this to Python. Contributions are
welcome.

Regards,
Martin

 

[Tim Golden]

It's correct for Python 2.5.2 and 2.6; for 2.5.1 (as discussed)
the test "if directory:return True" was not implemented.

Thanks. Version dependent stuff noted.

Notice that the first sentence:

"If the path does not exist or if access if forbidden, return False"

is confusing. It seems to say "access() returns false if access is
forbidden". I think you are referring to the directory (in the sense
of os.path.dirname) of the path to be tested.

Now, ironically, I'm confused by your recap What I meant to say was
that the os.access function as implemented under Windows returns False
if the path in question (say, "x:\someones-private-docs\diary.doc") was
inaccessible to the process invoking os.access by virtue of file
system permissions. (Or, even, that it simply didn't exist).

TJG

 

[Martin v. Löwis]

Now, ironically, I'm confused by your recap What I meant to say was

that the os.access function as implemented under Windows returns False
if the path in question (say, "x:\someones-private-docs\diary.doc") was
inaccessible to the process invoking os.access by virtue of file
system permissions. (Or, even, that it simply didn't exist).

Isn't that exactly the question that access() is trying to answer?
(whether the file is inaccessible?)
Then the reader could say "if it's inaccessible, return False,
otherwise, return True". It suggests that we already have a mechanism
to determine accessibility.

The case you are referring to is when GetFileAttributes fails, with
a permission error, right?

I'm not quite sure why it could fail in cases where the file is
present, in particular, if the user has the privilege to bypass
traversal checking.

Regards,
Martin

 

[Tim Golden]

Isn't that exactly the question that access() is trying to answer?
(whether the file is inaccessible?)
Then the reader could say "if it's inaccessible, return False,
otherwise, return True". It suggests that we already have a mechanism
to determine accessibility.

Up to a point: this meets the case where we fail to access
the file at all (for read or write or whatever). But what
about where we can read the directory entry, and the
read-only attribute isn't set? At present, we'll return
True to a W_OK access check in these circs, but this user
might in fact be denied write access by the ACLs. (In fact,
they might even be denied read access, since I imagine we
only need access to the directory entry to check the
attributes).

Have I missed something?

BTW I can't see a tutorial in the AccessCheck docs here:

http://msdn2.microsoft.com/en-us/library/aa374815.aspx

or in the SDK help file. Were you referring to a different
set of docs? I'm girding my loins to provide a patch if I
can!

TJG