how to get files in a directory

[Anand K Rayudu]

Hi all,

I am trying to find a way to get the files recursively in a given
directory,

The following code is failing, can some one please suggest what could be
problem here


from os import walk,join

for root,dir,files in os.walk("E:\myDir1\MyDir2"):
for i in dir:
for j in files:
fille = root+i+j
print file

Surprisingly if i give os.walk("E:\myDir1") the above code works, but
not if i have 2 levels of directories.

Thanks & Best Regards,
Anand

 

[Dan Perl]

First of all, Jeremy already pointed out that your code has several errors.
I'll assume though that your real code is more complicated and you just
tried to show us a simplification of your code.

On the other hand, without the real code we cannot tell why it works for 1
level and it does not work for 2 levels. As someone pointed out already, it
migh be because you are using "\" instead of "\\" for path separators.
Python will accept "\c" as if it was with double backslashes but it will
reject "\x" because that is a special character.

Based on the example that you gave, you are also misunderstanding os.walk.
os.walk returns 3 values (I'll use the names in your code):
- root - the parent directory at this point in the traversal of the tree
- dir - the list of children directories in root (right below root)
- files - the list of files in root
So the files in the 'files' list are not under dir but under root. So it
does not make sense to loop over 'files' inside the loop over 'dir' or to
join root+i+j. Both your 'i' and your 'j' are just under 'root'.

I'm not sure either what you are trying to do with "root+i+j". You probably
need os.path.join(root,i) and os.path.join(root,j)

Dan

 

[wes weston]

Hi all,

I am trying to find a way to get the files recursively in a given
directory,

The following code is failing, can some one please suggest what could be
problem here


from os import walk,join

for root,dir,files in os.walk("E:\myDir1\MyDir2"):
for i in dir:
for j in files:
fille = root+i+j
print file

Surprisingly if i give os.walk("E:\myDir1") the above code works, but
not if i have 2 levels of directories.

Thanks & Best Regards,
Anand

Anand,
Does this help?
wes

def GetFilesWithExtRecursively(path,ext_list): #ext_list is like [".h",".cpp",...]
dir_list = []
file_list = []
if not os.path.isdir(path):
print "\a"
return [],[]

filenames = os.listdir( path )

for fn in filenames:
if path[-1] == "/":
xfn = path + fn
else:
xfn = path + "/" + fn
if( os.path.isdir( xfn ) ):
file_list = file_list + GetFilesWithExtRecursively( xfn ,ext_list)
elif( os.path.isfile( xfn ) ):
front,back = os.path.splitext(xfn)
if back in ext_list:
file_list.append( xfn )
return file_list

 

[Elbert Lev]

Hi all,

I am trying to find a way to get the files recursively in a given
directory,

The following code is failing, can some one please suggest what could be
problem here


from os import walk,join

for root,dir,files in os.walk("E:\myDir1\MyDir2"):
for i in dir:
for j in files:
fille = root+i+j
print file

Surprisingly if i give os.walk("E:\myDir1") the above code works, but
not if i have 2 levels of directories.

Thanks & Best Regards,
Anand

All is wrong!
0. Wrong import
1. not "E:\myDir1\MyDir2", but "E:\\myDir1\\MyDir2"
2. dir and files are in the SAME directory. In other words if the
structure is:
myDir1
MyDir2
somefile.py
the first itteration will give an EMPTY dir= [] and files =
['somefile.py']


Here is an example from the manual:

This example displays the number of bytes taken by non-directory files
in each directory under the starting directory, except that it doesn't
look under any CVS subdirectory

import os
from os.path import join, getsize
for root, dirs, files in os.walk('python/Lib/email'):
print root, "consumes",
print sum([getsize(join(root, name)) for name in files]),
print "bytes in", len(files), "non-directory files"
if 'CVS' in dirs:
dirs.remove('CVS') # don't visit CVS directories

so if you want just print full path of the files in E:\myDir1\MyDir2

for root,dir,files in os.walk("E:\myDir1\MyDir2"):
for j in files:
fille = os.path.join(root,j)
print fille

Do not touch dir. It is supposed to be used to restrict the search.
If you do not modify dir it walk will progress into root's
subdirectories.

 

[Michael Kent]

I am trying to find a way to get the files recursively in a given
directory,

The following code is failing, can some one please suggest what could be
problem here

You don't really give enough information, but I'll make some
reasonable guesses. My guess is that you have a directory tree
"E:\myDir1\MyDir2", and that "MyDir2" contains files but no
subdirectories.

for root,dir,files in os.walk("E:\myDir1\MyDir2"):

Here you are iterating over a directory tree. For each directory in
the tree, os.walk returns the path to the directory, a list of the
subdirectories in that directory, and a list of non-directory files in
that directory.

for i in dir:

Here you are examining each of the subdirectories of the current
directory. You are ignoring any files in that directory. So, for the
top-level directory you gave to os.walk, "E:\myDir1\MyDir2", you are
ignoring the actual files in that directory, and examining only its
subdirectories. My guess is that there are no subdirectories, so you
get no results.

Surprisingly if i give os.walk("E:\myDir1") the above code works, but
not if i have 2 levels of directories.

Unsurprisingly. If there are any files in "myDir1", your code would
ignore them, and only find files in its subdirectory, "MyDir2".

 

[Lonnie Princehouse]

Backslash begins an escape sequence in a string literal.
You have a few options:

0. Use a forward slash (which will work in Python for DOS paths):
"E:/myDir1/myDir2""

1. Use the escape sequence for backslash: "E:\\myDir1\\myDir2"

2. Use a raw string: r"E:\myDir1\myDir2"

Further reference:
http://www.python.org/doc/current/ref/strings.html

Surprisingly if i give os.walk("E:\myDir1") the above code works, but
not if i have 2 levels of directories.

Yes, that is kind of surprising. Also, your nested loop...

for i in dir:
for j in files:
fille = root+i+j
print file

.... doesn't make any sense. "dir" is a list of sub-directories in
root, while "files" is a list of non-directory files in root. The
files in "files" are not in the subdirectories.