Newbie Question: Getting a list of files

[Brian]

Hello,

I am currently working on putting together a free, open source,
anti-spyware program for the Mac (and eventually for other OS's too.)
However, there's one thing that I am currently trying to figure out:

How do I, in Python, obtain a recursive list of files in a specified
directory, including the subdirectories, etc? For example, in the old
MS-DOS days, we could specify at the command prompt "DIR /S" and this
would provide a listing of all files, including those in subdirectories,
no matter how far down the branch they were. How can this be done with
Python?

P.S. I have been searching Google, but haven't found the answer yet.

Any info appreciated!

Dusty
---

 

[John Machin]

Hello,

I am currently working on putting together a free, open source,
anti-spyware program for the Mac (and eventually for other OS's too.)
However, there's one thing that I am currently trying to figure out:

How do I, in Python, obtain a recursive list of files in a specified
directory, including the subdirectories, etc? For example, in the old
MS-DOS days, we could specify at the command prompt "DIR /S" and this
would provide a listing of all files, including those in subdirectories,
no matter how far down the branch they were. How can this be done with
Python?

P.S. I have been searching Google, but haven't found the answer yet.

Try reading the Python docs instead
In general, anyone doing anything much with files and directories
should know what's in the os and os.path modules.
In particular, os.walk is probably what you are looking for.

HTH,
John

 

[Steven D'Aprano]

How do I, in Python, obtain a recursive list of files in a specified
directory, including the subdirectories, etc? For example, in the old
MS-DOS days, we could specify at the command prompt "DIR /S" and this
would provide a listing of all files, including those in subdirectories,
no matter how far down the branch they were. How can this be done with
Python?

In MS DOS days, you were dealing with a small number of files. Python has
to deal with file systems that could contain billions of files. If each
file name has an average of eight characters, your recursive list of
files could require a gigabyte of memory just to hold the file names. (It
might not, of course -- it depends on the file system in use.)

I suggest you have a look at the os module, in particular os.listdir and
os.walk.

 

[Brian]

Hi Steven,

Thanks for all of the good advice! Much appreciated.

Dusty
---

 

[Gerard Flanagan]

How do I, in Python, obtain a recursive list of files in a specified
directory, including the subdirectories, etc?

import os

def iter_dirs(root, dirs=False):
stack = [root]
while stack:
dir = stack.pop(0)
for f in (os.path.join(dir, basename) for basename in
os.listdir(dir)):
if os.path.isdir(f) and not os.path.islink(f):
stack.append(f)
if dirs:
yield f
else:
yield f

 

[Ant]

import os

def iter_dirs(root, dirs=False):

....

Rather than rolling your own directory walker:

The same iterator using os.walk:

def iter_dirs(root, dirs=False):
for root, directories, files in os.walk(root):
if dirs:
for dir in directories:
yield os.path.join(root, dir)
for file in files:
yield os.path.join(root, file)


os.path.walk requires a different mindset:

def visitor(yield_dirs, dirname, names):
for name in names:
pth = os.path.join(dirname, name)
if os.path.isfile(pth) or yield_dirs:
print pth

os.path.walk(root, visitor, False)

 

[Brian]

Thank you very much for your examples! Much appreciated.

Dusty
---