A little assistance with os.walk please.

[KraftDiner]

The os.walk function walks the operating systems directory tree.

This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?

for root, dirs, files in os.walk('/directory/'):
print root
# print dirs
# print files

 

[Pontus Ekberg]

The os.walk function walks the operating systems directory tree.

This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?

for root, dirs, files in os.walk('/directory/'):
print root
# print dirs
# print files

Directory tree generator.

For each directory in the directory tree rooted at top (including top
itself, but excluding '.' and '..'), yields a 3-tuple

dirpath, dirnames, filenames

dirpath is a string, the path to the directory. dirnames is a list of
the names of the subdirectories in dirpath (excluding '.' and '..').
filenames is a list of the names of the non-directory files in dirpath.
Note that the names in the lists are just names, with no path components.
To get a full path (which begins with top) to a file or directory in
dirpath, do os.path.join(dirpath, name).

If optional arg 'topdown' is true or not specified, the triple for a
directory is generated before the triples for any of its subdirectories
(directories are generated top down). If topdown is false, the triple
for a directory is generated after the triples for all of its
subdirectories (directories are generated bottom up).

When topdown is true, the caller can modify the dirnames list in-place
(e.g., via del or slice assignment), and walk will only recurse into the
subdirectories whose names remain in dirnames; this can be used to prune
the search, or to impose a specific order of visiting. Modifying
dirnames when topdown is false is ineffective, since the directories in
dirnames have already been generated by the time dirnames itself is
generated.

By default errors from the os.listdir() call are ignored. If
optional arg 'onerror' is specified, it should be a function; it
will be called with one argument, an os.error instance. It can
report the error to continue with the walk, or raise the exception
to abort the walk. Note that the filename is available as the
filename attribute of the exception object.

Caution: if you pass a relative pathname for top, don't change the
current working directory between resumptions of walk. walk never
changes the current directory, and assumes that the client doesn't
either.

Example:

from os.path import join, getsize
for root, dirs, files in walk('python/Lib/email'):
print root, "consumes",
print sum([getsize(join(root, name)) for name in files]),
print "bytes in", len(files), "non-directory files"
if 'CVS' in dirs:
dirs.remove('CVS') # don't visit CVS directories

 

[Tim Chase]

The os.walk function walks the operating systems directory tree.

Yup.

This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?

for root, dirs, files in os.walk('/directory/'):
print root
# print dirs
# print files

As cheesy as it may sound, try uncommenting the two commented
lines. For a more explicit variant:

print repr(dirs)
print repr(files)

You'll notice that they're lists.

In the directory "root", you'll find the subdirectories given in
the list "dirs" and you'll find the files given in the list "files".

-tkc

 

[Larry Bates]

The os.walk function walks the operating systems directory tree.

This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?

for root, dirs, files in os.walk('/directory/'):
print root
# print dirs
# print files

Actually returns two tuples: dirs and files

root - is the directory branch you are currently walking
dirs - are the directory branches that are subdirectories of this
directory branch
files - are the files that live in this directory branch


To process all the files here you do something like:

for afile in files: # resist the urge to call it 'file'
fullpath=os.path.join(root, afile)
#
# Do something with fullpath
#

Hard to figure out item - If you wish to NOT process some of the
dirs, you can delete them from the dirs list here and they won't
get walked. You MUST delete them in place with del dirs[n] or
dirs.pop or some other function that deletes in-place.

You might want to type: help(os.walk) to get some more info.

-Larry Bates

 

[KraftDiner]

The os.walk function walks the operating systems directory tree.

This seems to work, but I don't quite understand the tupple that is
returned...
Can someone explain please?

for root, dirs, files in os.walk('/directory/'):
print root
# print dirs
# print files

Actually returns two tuples: dirs and files

root - is the directory branch you are currently walking
dirs - are the directory branches that are subdirectories of this
directory branch
files - are the files that live in this directory branch


To process all the files here you do something like:

for afile in files: # resist the urge to call it 'file'
fullpath=os.path.join(root, afile)
#
# Do something with fullpath
#

Hard to figure out item - If you wish to NOT process some of the
dirs, you can delete them from the dirs list here and they won't
get walked. You MUST delete them in place with del dirs[n] or
dirs.pop or some other function that deletes in-place.

You might want to type: help(os.walk) to get some more info.

Yep done that. Thanks.
Two things..
1) there seems to be an optional topdown flag. Is that passed to
os.walk(path, topdownFlag)
2) I only want to process files that match *.txt for example... Does
that mean I need to parse the list of files for the .txt extention or
can I pass a wildcard in the path parameter?

 

[Tim Chase]

1) there seems to be an optional topdown flag. Is that passed to

os.walk(path, topdownFlag)
Yes.

2) I only want to process files that match *.txt for example... Does
that mean I need to parse the list of files for the .txt extention or
can I pass a wildcard in the path parameter?

.... for f in files:
.... if not f.lower().endswith(".txt"): continue
.... print os.path.join(path, f)

If you want to be more complex:

>>> from os.path import splitext
>>> allowed = ['.txt', '.sql']
>>> for path, dirs, files in os.walk("."):

.... for f in files:
.... if splitext(f)[1].lower() not in allowed: continue
.... fn = os.path.join(path, f)
.... print "do something with %s" % fn


Just a few ideas,

-tkc

 

[KraftDiner]

1) there seems to be an optional topdown flag. Is that passed to
os.walk(path, topdownFlag)
Yes.

2) I only want to process files that match *.txt for example... Does
that mean I need to parse the list of files for the .txt extention or
can I pass a wildcard in the path parameter?

... for f in files:
... if not f.lower().endswith(".txt"): continue
... print os.path.join(path, f)

If you want to be more complex:

from os.path import splitext
allowed = ['.txt', '.sql']
for path, dirs, files in os.walk("."):

... for f in files:
... if splitext(f)[1].lower() not in allowed: continue
... fn = os.path.join(path, f)
... print "do something with %s" % fn


Just a few ideas,

-tkc

Many thanks all.
B.

 

[Tim Chase]

allowed = ['.txt', '.sql']

... for f in files:
... if splitext(f)[1].lower() not in allowed: continue

Additionally, if you really do want to specify wildcards:

>>> allowed = ['*.txt', 'README*', '*.xml', '*.htm*']
>>> from glob import fnmatch
>>> import os
>>> for path, dirs, files in os.walk("."):

.... good_files = []
.... for pat in allowed:
.... good_files.extend(fnmatch.filter(files, pat))
.... if good_files: break
.... for f in good_files:
.... print "doing something with %s" %
os.path.join(path, f)

-tkc