Chronological Processing of Files

[yoda]

This feels like a stupid question but I'll ask it anyway.

How can I process files chronologically (newest last) when using
os.walk()?

 

[Peter Hansen]

This feels like a stupid question but I'll ask it anyway.

How can I process files chronologically (newest last) when using
os.walk()?

Do you want the ordering to apply just to files within each directory,
or to all the files found (recursively) during the entire walk? Define
"newest" (most recent modified date or something else?). Is there any
reason why sorting with the results of os.access().st_mtime as the key
is not possible or sufficient?

-Peter

 

[Paul]

untested, ugly, but something like this would sort all the files in the
directory on os.path.getctime (not using os.walk() though). I'm sure
there is probably better ways to do it

filelist = []

def walkdir(currdir):
for files in os.listdir(currdir):
path = os.path.join(currdir, files)
if not os.path.isdir(path):
filelist.append([os.path.getctime(path), path])
else:
walkdir(path)

walkdir(r'c:\somedirectory')

filelist.sort()

for item in filelist:
dosomething(item[1])

dosomething is whatever function to process the files

 

[Jeremy Jones]

This feels like a stupid question but I'll ask it anyway.

Definitely not a stupid question.

How can I process files chronologically (newest last) when using
os.walk()?

Try this:

In [16]: file_list = [(os.stat(f)[8], f) for f in [os.path.join(i[0],
j) for i in os.walk("/home/jmjones/public_html") for j in i[2]]]

In [17]: file_list.sort()

In [18]: sorted_file_list = [f[1] for f in file_list]


I *think* os.stat()[8] is the modification time element...but this
should probably work for you. That first list comprehension looks like
a booger if you're not familiar with them. If you have any trouble with
it, just shoot a message back to the list and I'll decypher it for you.

- JMJ

 

[George Sakkis]

Do you want the ordering to apply just to files within each directory,
or to all the files found (recursively) during the entire walk? Define
"newest" (most recent modified date or something else?). Is there any
reason why sorting with the results of os.access().st_mtime as the key
is not possible or sufficient?

-Peter

You can make your life easier using the non-standard (yet ?) path module:

from path import path

top = path('.')
sort_kwds = dict(key=path.mtime.__get__, reverse=True)

# sort all files together
sorted_all = sorted(top.walkfiles(), **sort_kwds)

# sort files by directory

sorted_by_dir = sorted(top.files(), **sort_kwds) \
+ sum((sorted(dir.files(), **sort_kwds)
for dir in path(top).walkdirs()), [])

George

 

[yoda]

Just to clarify:

Newest== modified last

The processing\sorting should apply to all the files found recursively
during the entire walk.

That being said, thanks for all the responses. I'll test the code
shortly and get back to everyone.

ps. This is why comp.lang.python is truly the greatest list ever.
(better than comp.lang.lisp?) Everyone is so helpful. Thanks again guys.

 

[yoda]

I've tried using the path module and it works like a *charm*.. plus my
code is cleaner and clearer..

The list comprehension using os.stat() works well though I had to call
an additional reverse() on the resultant list so that I could get the
list in order of "newest first".

So, in conclusion, I'll use the path module.

Thanks again guys. You've been a great help.