os.walk() usage

[rbt]

I'm trying to write very small, modular code as functions to break up a
big monolithic script that does a file system search for particular
strings. The script works well, but it's not easy to maintain or add
features to.

I'd like to have a function that builds a list of files with os.walk()
and then have other functions accept that list as a parameter and modify
it as needed. For example, if the user has specified that certain files
and folders be excluded from the walk, I'd like to have functions like this:

def build_list(path):
fs_list = os.walk(path)
return fs_list

def skip_files(fs_list):
remove files
return fs_list

def skip_dirs(fs_list):
remove dirs
return fs_list

def search(fs_list):
pass

The problem I'm encountering is passing the list to other functions.
It's almost as if each function needs to build the list itself (walk the
filesystem)... which gets in the way of what I was asked to do (break
this thing up into modular, maintainable pieces).

Any tips on this?

Thanks

 

[wittempj]

every object in os.walk() returns a 3-tuple, like below, it seems your
code assumes it returns only a list of files.

for d in os.walk('c:\\temp'):
(dirpath, dirnames, filenames) = d
print dirpath
print dirnames
print filenames

 

[bruno modulix]

I'm trying to write very small, modular code as functions to break up a
big monolithic script that does a file system search for particular
strings. The script works well, but it's not easy to maintain or add
features to.

I'd like to have a function that builds a list of files with os.walk()
and then have other functions accept that list as a parameter and modify
it as needed. For example, if the user has specified that certain files
and folders be excluded from the walk, I'd like to have functions like
this:

def build_list(path):
fs_list = os.walk(path)
return fs_list

def skip_files(fs_list):
remove files
return fs_list

def skip_dirs(fs_list):
remove dirs
return fs_list

def search(fs_list):
pass

The problem I'm encountering is passing the list to other functions.

err...

search(skip_files(skip_dirs(build_list(path)))) ?

What's your problem *exactly* ?-)

BTW, you may want to have a look at the path module :
http://www.jorendorff.com/articles/python/path/

 

[rbt]

every object in os.walk() returns a 3-tuple, like below, it seems your
code assumes it returns only a list of files.

for d in os.walk('c:\\temp'):
(dirpath, dirnames, filenames) = d
print dirpath
print dirnames
print filenames

Thank you, this fixed it. I didn't read the docs well enough