Python doesn't see the directories I create

[mr_gadget]

When I create a subfolder, python is not seeing it. Can someone please
explain this behaviour ? I just started with python, read the tutorial over
the weekend and am writing my very first script. So I may not be seeing
something. Both os.path and glob.glob seem not to see a folder I created.
Other sibling folders seem to work fine. On a whim I tried paths with \\
double slashes and that worked. But why should single slashes work for some
folders and not for others ??

What I need is to match a bunch of files in a folder with
glob.glob(C:\enhancement\rawfiles\*.bin.gz) and send them to a function that
unzips them. But I always get []. Yes the folder does have my files, and I
checked all permissions etc, everything looks identical to the other folders
which python IS seeing. Same problem on my vista pc too.

Behaviour reproduced below.

C:\Enhancement>dir /ad

Volume in drive C has no label.

Volume Serial Number is 8056-41E7

Directory of C:\Enhancement

08/28/2007 06:15 PM <DIR> .

08/28/2007 06:15 PM <DIR> ..

08/28/2007 06:07 PM <DIR> BSA Documentation

08/28/2007 05:56 PM <DIR> output

08/29/2007 07:08 PM <DIR> rawfiles

08/23/2007 04:38 PM <DIR> SnapCell2.3.2

08/28/2007 06:15 PM <DIR> test

0 File(s) 0 bytes

7 Dir(s) 35,703,283,712 bytes free

C:\>python -V

Python 2.5.1

C:\>python -c "import os; print os.path.exists('C:\enhancement\output')"

True

C:\>python -c "import os; print os.path.exists('C:\enhancement\rawfiles')"

False

C:\>python -c "import os; print os.path.exists('C:\\enhancement\\rawfiles')"

True

C:\>python -c "import glob; print glob.glob('C:\enhancement\rawfiles\*')"

[]

C:\>python -c "import os; print os.path.exists('C:\enhancement\test')"

False

C:\>python -c "import os; print
os.path.exists('C:\enhancement\snapcell2.3.2')"

True

 

[Bruno Desthuilliers]

mr_gadget a écrit :

When I create a subfolder, python is not seeing it. Can someone please
explain this behaviour ? I just started with python, read the tutorial over
the weekend and am writing my very first script. So I may not be seeing
something. Both os.path and glob.glob seem not to see a folder I created.
Other sibling folders seem to work fine. On a whim I tried paths with \\
double slashes and that worked. But why should single slashes work for some
folders and not for others ??

s/slash/antislash/g

It's a very well known gotcha due to MS's choice to use the antislash as
path separator. In most languages - Python included - the antislash is
used for escape sequences (non-printable characters). '\r' is the escape
sequence for CR (carriage return). Doubling the antislash prevents
escaping.

You can avoid all escaping by using raw strings:

mypath = r"C:\enhancement\rawfiles\"

Also and IIRC, using slash instead should also work, ie:

mypath = r"C:/enhancement/rawfiles/"

HTH

 

[Diez B. Roggisch]

When I create a subfolder, python is not seeing it. Can someone please
explain this behaviour ? I just started with python, read the tutorial over
the weekend and am writing my very first script. So I may not be seeing
something. Both os.path and glob.glob seem not to see a folder I created.
Other sibling folders seem to work fine. On a whim I tried paths with \\
double slashes and that worked. But why should single slashes work for some
folders and not for others ??

What I need is to match a bunch of files in a folder with
glob.glob(C:\enhancement\rawfiles\*.bin.gz) and send them to a function that
unzips them. But I always get []. Yes the folder does have my files, and I
checked all permissions etc, everything looks identical to the other folders
which python IS seeing. Same problem on my vista pc too.

Behaviour reproduced below.

C:\Enhancement>dir /ad

Volume in drive C has no label.

Volume Serial Number is 8056-41E7

Directory of C:\Enhancement

08/28/2007 06:15 PM <DIR> .

08/28/2007 06:15 PM <DIR> ..

08/28/2007 06:07 PM <DIR> BSA Documentation

08/28/2007 05:56 PM <DIR> output

08/29/2007 07:08 PM <DIR> rawfiles

08/23/2007 04:38 PM <DIR> SnapCell2.3.2

08/28/2007 06:15 PM <DIR> test

0 File(s) 0 bytes

7 Dir(s) 35,703,283,712 bytes free

C:\>python -V

Python 2.5.1

C:\>python -c "import os; print os.path.exists('C:\enhancement\output')"

True

C:\>python -c "import os; print os.path.exists('C:\enhancement\rawfiles')"

False

C:\>python -c "import os; print os.path.exists('C:\\enhancement\\rawfiles')"

True

C:\>python -c "import glob; print glob.glob('C:\enhancement\rawfiles\*')"

[]

C:\>python -c "import os; print os.path.exists('C:\enhancement\test')"

False

C:\>python -c "import os; print
os.path.exists('C:\enhancement\snapcell2.3.2')"

I'm pretty sure it's a missing backslash escape thingy. Either use \\
for a backslash in paths, or just the forward slash, it works as well.

And case-sensitivity might be an issue as well.
Diez

 

[Marc 'BlackJack' Rintsch]

C:\>python -c "import os; print os.path.exists('C:\enhancement\rawfiles')"

False

C:\>python -c "import os; print os.path.exists('C:\\enhancement\\rawfiles')"

True

The backward slash has a special meaning in string literals. It is used
to escape special character. One such sequence is '\r' which is *one*
character, namely the return character. To insert *one* backslash it has
to be protected by another backslash or the string literal may be prefixed
by an 'r' to tell the compiler that backslashes have no special meaning in
that raw string literal.

In [23]: len('\r')
Out[23]: 1

In [24]: len('\\')
Out[24]: 1

In [25]: len(r'\r')
Out[25]: 2

In [26]: len(r'\\')
Out[26]: 2

Ciao,
Marc 'BlackJack' Rintsch

 

[lupino]

Ciao mr_gadget,

When I create a subfolder, python is not seeing it. Can someone please
explain this behaviour ? I just started with python, read the tutorial over
the weekend and am writing my very first script. So I may not be seeing
something. Both os.path and glob.glob seem not to see a folder I created.
Other sibling folders seem to work fine. On a whim I tried paths with \\
double slashes and that worked. But why should single slashes work for some
folders and not for others ??

Instead of relying on your own error-prone ability of using forward and back
slashes as path separators, try to get as much as possible from python.

You can use os.sep as a cross-platform path separator, instead of forward
slashes or escaped back slashes.

Moreover you can use the os.path.join function to let python take care of your
problem.
'C:/Programs/MySubFolder'

HTH,

 

[Bruno Desthuilliers]

Marc 'BlackJack' Rintsch a écrit :

Even on Unix it shouldn't be a problem on most file systems to create a
directory named 'C:'

bj@s8n:~$ mkdir C:
bj@s8n:~$ touch C:/test.txt
bj@s8n:~$ ls -l C:
total 0
-rw-r--r-- 1 bj bj 0 2007-08-30 14:38 test.txt

Ok, I get it, you guys are trying to make me mad !-)

 

[Steve Holden]

mr_gadget a écrit :

Note you *didn't* try paths with double slashes, you merely correctly
represented the paths with single slashes

s/slash/antislash/g

It's a very well known gotcha due to MS's choice to use the antislash as
path separator. In most languages - Python included - the antislash is
used for escape sequences (non-printable characters). '\r' is the escape
sequence for CR (carriage return). Doubling the antislash prevents
escaping.

You can avoid all escaping by using raw strings:

mypath = r"C:\enhancement\rawfiles\"

Please note that the above is a well-known syntax error. A string
literal cannot end with a single backslash, as it escapes the closing quote.
File "<stdin>", line 1
mypath = r"C:\enhancement\rawfiles\"
^
SyntaxError: EOL while scanning single-quoted string

Also and IIRC, using slash instead should also work, ie:

mypath = r"C:/enhancement/rawfiles/"

That does indeed work in most situations, but ideally (i.e. for maximum
code portability) paths should be constructed using os.path.join(), or
collected from the environment somehow.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Bruno Desthuilliers]

Steve Holden a écrit :

Bruno Desthuilliers wrote: (snip)
Please note that the above is a well-known syntax error. A string
literal cannot end with a single backslash, as it escapes the closing
quote.

oops ! My bad
Thanks for the correction...

(snip)

That does indeed work in most situations, but ideally (i.e. for maximum
code portability) paths should be constructed using os.path.join(), or
collected from the environment somehow.

Indeed. But I doubt a path starting with 'C:' will work fine on a
unix-like environment anyway !-)

 

[Steve Holden]

Steve Holden a écrit :

Bruno Desthuilliers wrote: [...]
That does indeed work in most situations, but ideally (i.e. for maximum
code portability) paths should be constructed using os.path.join(), or
collected from the environment somehow.

Indeed. But I doubt a path starting with 'C:' will work fine on a
unix-like environment anyway !-)

Just to be contrarian:

sholden@bigboy ~/Projects/python.org/build
$ ls C:/Steve/
Apache SANSsecurityOverview.pdf
Apache2 SQLServer.txt
...
Resume.odt todo.txt
Resume.pdf untitled-1.py

sholden@bigboy ~/Projects/python.org/build

That's Cygwin, of course. Is that sufficiently "unix-like"? Though I
have to admit that the different utilities all take different approaches
to the use of Windows paths, and some just won't take them at all.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Marc 'BlackJack' Rintsch]

Just to be contrarian:

sholden@bigboy ~/Projects/python.org/build
$ ls C:/Steve/
Apache SANSsecurityOverview.pdf
Apache2 SQLServer.txt
...
Resume.odt todo.txt
Resume.pdf untitled-1.py

sholden@bigboy ~/Projects/python.org/build

That's Cygwin, of course. Is that sufficiently "unix-like"? Though I
have to admit that the different utilities all take different approaches
to the use of Windows paths, and some just won't take them at all.

Even on Unix it shouldn't be a problem on most file systems to create a
directory named 'C:'

bj@s8n:~$ mkdir C:
bj@s8n:~$ touch C:/test.txt
bj@s8n:~$ ls -l C:
total 0
-rw-r--r-- 1 bj bj 0 2007-08-30 14:38 test.txt



Ciao,
Marc 'BlackJack' Rintsch

 

[Neil Cerutti]

mr_gadget a écrit :

s/slash/antislash/g

It's a very well known gotcha due to MS's choice to use the
antislash as path separator. In most languages - Python
included - the antislash is used for escape sequences
(non-printable characters).

Keeping in mind which came first, isn't it at least as accurate
to attribute this problem to Python's choice of escape character?
There were probably advantages to adopting the same escape
character as other well-known languages/codes, but the choice has
caused some trouble over the years.

To me, Python's collection of special-purpose string literal
notations is one of its little warts.

Of course, I'm not smart enough to have delivered the ONE TRUE
string literal notation either, but I do have a computer and an
internet connection, so there you are.

 

[Steve Holden]

Keeping in mind which came first, isn't it at least as accurate
to attribute this problem to Python's choice of escape character?
There were probably advantages to adopting the same escape
character as other well-known languages/codes, but the choice has
caused some trouble over the years.

To me, Python's collection of special-purpose string literal
notations is one of its little warts.

Of course, I'm not smart enough to have delivered the ONE TRUE
string literal notation either, but I do have a computer and an
internet connection, so there you are.

Well, it's a wart that's shared with many other languages - including,
interestingly enough, Microsoft's very own C#, from whose documentation
the following examples are taken:

string a = "hello, world"; // hello, world
string b = @"hello, world"; // hello, world
string c = "hello \t world"; // hello world
string d = @"hello \t world"; // hello \t world
string e = "Joe said \"Hello\" to me"; // Joe said "Hello" to me
string f = @"Joe said ""Hello"" to me"; // Joe said "Hello" to me
string g = "\\\\server\\share\\file.txt"; // \\server\share\file.txt
string h = @"\\server\share\file.txt"; // \\server\share\file.txt
string i = "one\r\ntwo\r\nthree";

The fact is that some strings are always going to cause trouble.
Unfortunately programming itself is a task that requires a little more
knowledge to be applied to the task. Just learn the rules and move on.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Steve Holden]

Steve Holden wrote:
[...]

The fact is that some strings are always going to cause trouble.
Unfortunately programming itself is a task that requires a little more
knowledge to be applied to the task. Just learn the rules and move on.

As a quick follow-up, I had intended to comment on the usefulness of
"verbatim literals".

string a = "hello, world"; // hello, world
string b = @"hello, world"; // hello, world
string c = "hello \t world"; // hello world
string d = @"hello \t world"; // hello \t world
string e = "Joe said \"Hello\" to me"; // Joe said "Hello" to me
string f = @"Joe said ""Hello"" to me"; // Joe said "Hello" to me
string g = "\\\\server\\share\\file.txt"; // \\server\share\file.txt
string h = @"\\server\share\file.txt"; // \\server\share\file.txt
string i = "one\r\ntwo\r\nthree";

The @ character introduces a verbatim literal, and you can see from the
examples given that they are implement a mixture of raw-string and
triple-quote capabilities (even allowing multi-line string constants). I
haven't fired up Visual Studio to verify, but I imagine you can even end
a verbatim literal with a back-quote.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------

 

[Neil Cerutti]

Well, it's a wart that's shared with many other languages -
including, interestingly enough, Microsoft's very own C#, from
whose documentation the following examples are taken:

string a = "hello, world"; // hello, world
string b = @"hello, world"; // hello, world
string c = "hello \t world"; // hello world
string d = @"hello \t world"; // hello \t world
string e = "Joe said \"Hello\" to me"; // Joe said "Hello" to me
string f = @"Joe said ""Hello"" to me"; // Joe said "Hello" to me
string g = "\\\\server\\share\\file.txt"; // \\server\share\file.txt
string h = @"\\server\share\file.txt"; // \\server\share\file.txt
string i = "one\r\ntwo\r\nthree";

Still, that's only two.

The fact is that some strings are always going to cause
trouble. Unfortunately programming itself is a task that
requires a little more knowledge to be applied to the task.

Or fortunately, depending on the dictates of your temperament.

 

[Lawrence D'Oliveiro]

Keeping in mind which came first, isn't it at least as accurate
to attribute this problem to Python's choice of escape character?

No, it's Microsoft's fault. The use of backslash as an escape character goes
back to Unix systems in the early 1970s--long before Microsoft came on the
scene.

When Microsoft introduced MS-DOS 1.0 in 1981, it didn't have directory
hierarchies. Commands used the slash character to delimit options. Then
when MS-DOS 2.0 introduced subdirectories in 1983, they decided they
couldn't use the slash as the path separator, so they used the backslash
instead. That has been a source of confusion ever since then.

 

[Lawrence D'Oliveiro]

Even on Unix it shouldn't be a problem on most file systems to create a
directory named 'C:'

Except that some commands (e.g. scp, rsync) will interpret the colon as
delimiting a host name.

Also remember that, unless you write it "/C:", it's only going to be
interpreted relative to your current working directory.

 

[Neil Cerutti]

No, it's Microsoft's fault. The use of backslash as an escape
character goes back to Unix systems in the early 1970s--long
before Microsoft came on the scene.

When Microsoft introduced MS-DOS 1.0 in 1981, it didn't have
directory hierarchies. Commands used the slash character to
delimit options. Then when MS-DOS 2.0 introduced subdirectories
in 1983, they decided they couldn't use the slash as the path
separator, so they used the backslash instead. That has been a
source of confusion ever since then.

Going back and checking the Python History page, it seems I was
off by four years with when I thought Python was first released
(I was thinking 1995, when the first release was actually 1991).
That makes Python's choice of escape character way more practical
than I thought, since Microsoft hadn't yet conquered the desktop
computing world when Python was in it's infancy.

That strange whirring sound is me backpedaling furiously. Thanks
very much for all your patiences.