Walk thru each subdirectory from a top directory

S

silverburgh.meryl

i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirectory = "/home/meryl/saved_pages/data"

dir=open(savedPagesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirectory + file + 'index.html'
print fileName

$ ./scripts/regressionTest.py
Traceback (most recent call last):
File "./scripts/regressionTest.py", line 12, in ?
dir=open(savedPagesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?

Thank you.
 
D

Diez B. Roggisch

i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirectory = "/home/meryl/saved_pages/data"

dir=open(savedPagesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirectory + file + 'index.html'
print fileName

$ ./scripts/regressionTest.py
Traceback (most recent call last):
File "./scripts/regressionTest.py", line 12, in ?
dir=open(savedPagesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?
Click to expand...

You can't open a directory.

Use the function os.walk to do what you want.

Diez
 
A

Adam

i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirectory = "/home/meryl/saved_pages/data"

dir=open(savedPagesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirectory + file + 'index.html'
print fileName

$ ./scripts/regressionTest.py
Traceback (most recent call last):
File "./scripts/regressionTest.py", line 12, in ?
dir=open(savedPagesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?

Thank you.
Click to expand...
From Alan Gaulds Tut.
Click to expand...
.... print t
....
('Root', ['D1', 'D2', 'D3'], ['FA.txt', 'FB.txt'])
('Root/D1', ['D1-1'], ['FC.txt'])
('Root/D1/D1-1', [], ['FF.txt'])
('Root/D2', [], ['FD.txt'])
('Root/D3', ['D3-1'], ['FE.txt'])
('Root/D3/D3-1', [], ['target.txt'])

This bit below is from one of my first programs as I'm currently
learning. It is designed to go form the root down and return the full
paths of everything it finds into a list. (I then check the reults for
files paths that exceed a certain length - but you don't need to know
that.)


def findallfiles(self, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
self.scan = glob.glob("*")
for r in self.scan:
if root[-1] == "\\":
self.results.append(root + r)
else:
self.results.append(root + "\\" + r)
return self.results
 
D

Dennis Lee Bieber

This bit below is from one of my first programs as I'm currently
learning. It is designed to go form the root down and return the full
paths of everything it finds into a list. (I then check the reults for
files paths that exceed a certain length - but you don't need to know
that.)


def findallfiles(self, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
self.scan = glob.glob("*")
Click to expand...

Why the glob invocation -- you already have a list of directories
and files to be found in "root"...
for r in self.scan:
if root[-1] == "\\":
self.results.append(root + r)
else:
self.results.append(root + "\\" + r)
return self.results
Click to expand...

-=-=-=-=-=-=-=-
import os

def findallfiles(base):
for root, dirs, files in os.walk(base):
for f in files:
print os.path.join(root, f)

findallfiles(r"e:\userdata\dennis lee bieber\my documents")
-=-=-=-=-=-=-=-
e:\userdata\dennis lee bieber\my documents\01_Title.jpg
e:\userdata\dennis lee bieber\my documents\405themovie_320x224.mpg
e:\userdata\dennis lee bieber\my documents\73P.gif
e:\userdata\dennis lee bieber\my documents\ana_cm_d_04.mov
e:\userdata\dennis lee bieber\my documents\ana_cm_d_15.mov
e:\userdata\dennis lee bieber\my documents\APRS101.pdf
e:\userdata\dennis lee bieber\my documents\Bestiaria 20060328
1018.sql.zip
e:\userdata\dennis lee bieber\my documents\Blue Ball Machine.htm
e:\userdata\dennis lee bieber\my documents\booklist
e:\userdata\dennis lee bieber\my documents\booklist.html
e:\userdata\dennis lee bieber\my documents\booklist.py
e:\userdata\dennis lee bieber\my documents\BookList.sql
e:\userdata\dennis lee bieber\my documents\booklist.zip
e:\userdata\dennis lee bieber\my documents\counter.py
e:\userdata\dennis lee bieber\my documents\counterthread.py
e:\userdata\dennis lee bieber\my documents\cp.py
e:\userdata\dennis lee bieber\my documents\crash-me.php.htm
e:\userdata\dennis lee bieber\my documents\C__PROGRAM FILES_THE MASTER
GENEALOGIST_RRW_DC_L1P0505.pdf
e:\userdata\dennis lee bieber\my documents\Default.sfvidcap
e:\userdata\dennis lee bieber\my documents\desktop.ini
e:\userdata\dennis lee bieber\my documents\eclipse_code.xml
e:\userdata\dennis lee bieber\my documents\FGS.pdf
e:\userdata\dennis lee bieber\my documents\Glucose.xls
e:\userdata\dennis lee bieber\my documents\GunV1.wmv
e:\userdata\dennis lee bieber\my documents\lasersaber.wmv
e:\userdata\dennis lee bieber\my documents\MySQLdb.html
e:\userdata\dennis lee bieber\my documents\Once upon a time.doc
e:\userdata\dennis lee bieber\my documents\oth_tsr_rm_750.ram
e:\userdata\dennis lee bieber\my documents\pub_ax25.html
e:\userdata\dennis lee bieber\my documents\Script1.py
e:\userdata\dennis lee bieber\my documents\Script11.py
e:\userdata\dennis lee bieber\my documents\t.html
e:\userdata\dennis lee bieber\my documents\t.rx
e:\userdata\dennis lee bieber\my documents\testcat.db
e:\userdata\dennis lee bieber\my documents\Thumbs.db
e:\userdata\dennis lee bieber\my documents\TMG Sources.htm
e:\userdata\dennis lee bieber\my documents\usage-guide.doc
e:\userdata\dennis lee bieber\my documents\.metadata\.lock
e:\userdata\dennis lee bieber\my documents\.metadata\.log
e:\userdata\dennis lee bieber\my documents\.metadata\version.ini
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\gnat-gpl-2005-pentium-mingw32msv-bin.exe
e:\userdata\dennis lee bieber\my documents\2005\x86-windows\README.txt
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\asis\asis-gpl-2005-src.tgz
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\aunit\aunit-1.03-src.zip
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\aunit\README
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\aws\aws-gpl-2.1a.tgz
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\glade\glade-gpl-2005-src.tgz
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\gtkada\gtkada-2.4.0-nt.exe
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\gtkada\README.GTKADA.txt
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\polyorb\polyorb-gpl-1.3r.tar.gz
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\win32ada\win32ada-gpl-2005.exe
e:\userdata\dennis lee bieber\my
documents\2005\x86-windows\tools\xmlada\xmlada-gpl-2.0.1.tgz
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\amateur.txt
e:\userdata\dennis lee bieber\my documents\Ada Progs\Assignment2.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment2.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment2.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment2.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3b.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3b.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3b.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\assignment3b.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\BigInts.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\bigints.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\bigints.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\bigints.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\cpp.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\date_package.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\date_package.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\date_package.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\date_package.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\default.gpr
e:\userdata\dennis lee bieber\my documents\Ada Progs\fortran-data.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\gnat.ago
e:\userdata\dennis lee bieber\my documents\Ada Progs\pascal-data.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\pascal.zip
e:\userdata\dennis lee bieber\my documents\Ada Progs\person_package.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\person_package.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\person_package.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\person_package.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\pop.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\pop.adb version1
e:\userdata\dennis lee bieber\my documents\Ada Progs\pop.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\pop.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\stations.txt
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_date.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_date.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_date.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_date.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_person.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_person.ali
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_person.exe
e:\userdata\dennis lee bieber\my documents\Ada Progs\test_person.o
e:\userdata\dennis lee bieber\my documents\Ada Progs\visual.txt
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\ast2body.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\ast2body.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astdc.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astdc.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astinter.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astinter.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astiod.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astiod.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astmanv.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astmanv.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astmath.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astmath.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astpert.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astpert.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astreduc.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astreduc.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\asttime.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\asttime.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astutil.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\astutil.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\complex.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\complex.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\constants.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\constants.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\dc.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\dcf.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\dcf.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\README
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\sgp4prog.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\sgp4unit.adb
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\sgp4unit.ads
e:\userdata\dennis lee bieber\my documents\Ada Progs\ada\testastf.adb
e:\userdata\dennis lee bieber\my documents\Ada
Progs\adadata\atmoserp.dat.dat
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\geos1.inp
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\geos2.inp
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\geos5.inp
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\nut85.dat
e:\userdata\dennis lee bieber\my documents\Ada
Progs\adadata\nutation.dat
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\sensor.dat
e:\userdata\dennis lee bieber\my documents\Ada
Progs\adadata\testastf.dat
e:\userdata\dennis lee bieber\my documents\Ada Progs\adadata\tstast.dat
e:\userdata\dennis lee bieber\my documents\Agent
Data\#beastie.Undernet.20050824.log
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000001.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000001.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000002.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000002.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000004.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\00000004.IDX

<snip>

e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\0000AFCE.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\0000AFCE.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\0000AFCF.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\0000AFCF.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\AGENT.INI
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\alert.htm
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\borc.c
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\custenu5.tlx
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\errorlog.xml
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\FILTERS.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\FILTERS.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\folders.dat
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\groupdir.dat
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\groupdir.dat_old
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\grpprops.dat
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\MFILTERS.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\MFILTERS.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\mutex.tmp
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\MWORDS.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\MWORDS.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\personal.aab
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\plugmgr10b.exe
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\tasklog.xml
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\urltype.dat
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\wappen.JPG
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\WORDS.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\WORDS.IDX
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\XPOST.DAT
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\alt.fan.furry.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\alt.lifestyle.furry.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\alt.scooter.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\alt.telescopes.meade.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\comp.db.any.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\comp.lang.any.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\dm.admin.any.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\dm.genealogy.any.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\fur.misc.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\Archived\soc.genealogy.any.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\a.txt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.c
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.dsp
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.dsw
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.ncb
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.opt
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\c.plg
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\c.exe
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\c.ilk
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\c.obj
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\c.pch
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\c.pdb
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\vc50.idb
e:\userdata\dennis lee bieber\my documents\Agent Data\Agent
NetComplete\Data\temp\Debug\vc50.pdb
e:\userdata\dennis lee bieber\my documents\Bills of Sale\Bill of Sale -
PlymouthLaser.qfl
e:\userdata\dennis lee bieber\my documents\Bills of Sale\EBay
2080LX2-1.xls
e:\userdata\dennis lee bieber\my documents\Bills of Sale\EBay
2080LX2-2.xls
e:\userdata\dennis lee bieber\my documents\Bills of Sale\EBay
2080LX2-3.xls
e:\userdata\dennis lee bieber\my documents\Bills of Sale\EBay
2080LX2-4.xls
e:\userdata\dennis lee bieber\my documents\Bills of Sale\EBay
Binocular.xls
e:\userdata\dennis lee bieber\my documents\Blue Ball
Machine_files\blank.htm
e:\userdata\dennis lee bieber\my documents\Blue Ball
Machine_files\image.gif
e:\userdata\dennis lee bieber\my documents\Blue Ball Machine_files\js.js
e:\userdata\dennis lee bieber\my documents\Blue Ball
Machine_files\stars_red_45.png
e:\userdata\dennis lee bieber\my documents\Blue Ball
Machine_files\votebar.js
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\diff.png
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\error.gif
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\js.js
e:\userdata\dennis lee bieber\my
documents\crash-me.php_files\mysql_100x52-64.gif
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\no.gif
e:\userdata\dennis lee bieber\my
documents\crash-me.php_files\nswpat80x15.gif
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\ok.gif
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\print.css
e:\userdata\dennis lee bieber\my documents\crash-me.php_files\screen.css
e:\userdata\dennis lee bieber\my
documents\crash-me.php_files\thumbs_up.gif
e:\userdata\dennis lee bieber\my
documents\crash-me.php_files\warning.gif
e:\userdata\dennis lee bieber\my documents\Eudora\- Budweiser (Nude
Ferret).zip
e:\userdata\dennis lee bieber\my documents\Eudora\2006 rate changes.doc
e:\userdata\dennis lee bieber\my documents\Eudora\3Y0X Peter I
DXpedition.pdf
e:\userdata\dennis lee bieber\my documents\Eudora\ada.zip
e:\userdata\dennis lee bieber\my documents\Eudora\ASTPERT.ADB
e:\userdata\dennis lee bieber\my documents\Eudora\Author.txt
e:\userdata\dennis lee bieber\my documents\Eudora\B-Day Listing.xls
e:\userdata\dennis lee bieber\my documents\Eudora\BookList.zip

Just a sample of the start of a very long listing...

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/
 
S

Scott David Daniels

Adam said:
i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script: ....
Click to expand...

This bit below is from one of my first programs as I'm currently
learning. It is designed to go form the root down and return the full
paths of everything it finds into a list. (I then check the reults for
files paths that exceed a certain length - but you don't need to know
that.)


def findallfiles(self, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
Click to expand...
^^^ Mistake here, don't change directories during os.walk ^^^
self.scan = glob.glob("*")
for r in self.scan:
if root[-1] == "\\":
self.results.append(root + r)
else:
self.results.append(root + "\\" + r)
return self.results
Click to expand...

def produce_all_files(base):
for root, dirs, files in os.walk(base):
for r in files:
yield os.path.join(root, r)
### possibly also (but I'd only go for files)
#for r in dirs:
# yield os.path.join(root, r)

def findallfiles(base):
return list(produce_all_files(base))
 
S

Steven D'Aprano

def findallfiles(self, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
Click to expand...
^^^ Mistake here, don't change directories during os.walk ^^^
Click to expand...

I came across this problem some time ago. I had to walk a directory tree,
calling an external program on each file. Unfortunately, that external
program wrote directly to the current working directory, which caused all
sorts of havoc. This is how I dealt with it:


def unbin(where):
"""Walk through a directory tree, calling macunpack to extract the
contents of MacBinary files.
"""
def _unbin(data, dirname, files):
for oldname in files:
fullname = os.path.normpath(os.path.join(dirname, oldname))
if os.path.isfile(fullname):
# Dammit, macunpack writes directly to the current
# working directory. Changing the cwd breaks the file
# tree walker, so we have to remember the current
# directory, change it to where we want to be, then
# change it back.
wd = os.getcwd()
os.chdir(dirname)
result = os.system('macunpack -f "%s"' % oldname)
if result == 0:
# Unpacking worked, so delete the original.
os.remove(oldname)
os.chdir(wd) # sigh...

os.path.walk(where, _unbin, None)


Is there another (better) way of dealing with this sort of situation?
 
S

Steven D'Aprano

On Tue, 27 Feb 2007 06:22:51 +0000, Dennis Lee Bieber wrote:

[snip 350-odd file names]
Just a sample of the start of a very long listing...
Click to expand...

Thanks for sharing. What's with the nude ferret?
 
P

Peter Otten

Steven said:
def findallfiles(self, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
Click to expand...
^^^ Mistake here, don't change directories during os.walk ^^^
Click to expand...

I came across this problem some time ago. I had to walk a directory tree,
calling an external program on each file. Unfortunately, that external
program wrote directly to the current working directory, which caused all
sorts of havoc. This is how I dealt with it:


def unbin(where):
"""Walk through a directory tree, calling macunpack to extract the
contents of MacBinary files.
"""
def _unbin(data, dirname, files):
for oldname in files:
fullname = os.path.normpath(os.path.join(dirname, oldname))
if os.path.isfile(fullname):
# Dammit, macunpack writes directly to the current
# working directory. Changing the cwd breaks the file
# tree walker, so we have to remember the current
# directory, change it to where we want to be, then
# change it back.
wd = os.getcwd()
os.chdir(dirname)
result = os.system('macunpack -f "%s"' % oldname)
if result == 0:
# Unpacking worked, so delete the original.
os.remove(oldname)
os.chdir(wd) # sigh...

os.path.walk(where, _unbin, None)


Is there another (better) way of dealing with this sort of situation?
Click to expand...

Does the problem occur if you pass an absolute path to
os.walk()/os.path.walk()?

Peter
 
S

Steven D'Aprano

Does the problem occur if you pass an absolute path to
os.walk()/os.path.walk()?
Click to expand...

Well, arguably the problem is that macunpack insists on writing to the
current working directory.

Or the problem is that os.walk sets the working directory to the initial
argument when you start, and then keeps it until it finishes. (Arguably
that's the right behaviour.)

You can play around with this to check:

def walker(where):
"""Walk through a directory tree, printing the current working directory."
def _walk(data, dirname, files):
print "dirname = '%s'; wd = '%s'" % (dirname, os.getcwd())
os.path.walk(where, _walk, None)


For those times when os.walk's behaviour doesn't mesh well with that of
the external program you are calling (like macunpack) is there an
alternative to:

- save the cwd;
- change directories;
- call the program;
- return to the saved directory

?
 
P

Peter Otten

Steven said:
For those times when os.walk's behaviour doesn't mesh well with that of
the external program you are calling (like macunpack) is there an
alternative to:

- save the cwd;
- change directories;
- call the program;
- return to the saved directory

?
Click to expand...

os.walk() creates paths from the root you provide and the files/direcotories
it finds via os.listdir(), and uses that to determine. If that root is an
absolute path any such paths are absolute, too, and undoing the directory
change is not necessary.

cwd = os.getcwd()
for path, dirs, files in os.walk(os.path.abspath(root)):
os.chdir(path)
for file in files:
invoke external program
os.chdir(cwd)

That said, undoing the change immediately is better style, and probably a
good usecase for a with statement like

with current_working_directory(path):
invoke external program

Last minute idea:

for path, dirs, files in os.walk(root):
for file in files:
os.system("cd '%s' && macunpack '%s'" % (path, file))

Peter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

How to connect to shared folder and access files from it using python 1
Uniquely identifying each & every html template 58
How do I encode and decode this data to write to a file? 11
Why 'files.py' does not print the filenames into a table format? 32
files.py (weird encoding error) 0
files.py (encoding error) 0
Guessing the encoding from a BOM 7
Recursively making a copy of a subdirectory.. 2

Members online

No members online now.
Total: 43 (members: 2, guests: 41)
Robots: 415

Forum statistics

Threads
473,769
Messages
2,569,577
Members
45,054
Latest member
LucyCarper

Latest Threads

Top