os.listdir

Discussion in 'Python' started by Hari, Sep 8, 2003.

  1. Hari

    Hari Guest

    Hi,

    I have question regarding os.listdir()


    I wanted to know if this is a reliable way to check for files added to
    a directory??

    files = os.lisdir(path)

    currentlen = len(files)

    // directory modified

    files = os.listdor(path)

    newlen = len(files)

    then the files in the range (currentlen -1) to (newlen -1) are all the
    files added to the directory.

    What I wanted to know was, is it guaranteed that between 2 calls
    os.listdir any files added to the directory are appended and the
    earlier order is maintained?

    thanks,

    Hari
    Hari, Sep 8, 2003
    #1
    1. Advertising

  2. Hari wrote:

    >Hi,
    >
    >I have question regarding os.listdir()
    >
    >
    >I wanted to know if this is a reliable way to check for files added to
    >a directory??
    >
    >files = os.lisdir(path)
    >
    >currentlen = len(files)
    >
    >// directory modified
    >
    >files = os.listdor(path)
    >
    >newlen = len(files)
    >
    >then the files in the range (currentlen -1) to (newlen -1) are all the
    >files added to the directory.
    >
    >What I wanted to know was, is it guaranteed that between 2 calls
    >os.listdir any files added to the directory are appended and the
    >earlier order is maintained?
    >
    >


    Unless the documentation for os.listdir specifies such behaviour
    explicitly, you should assume the answer is /no/. There are many
    Pythons, running on many platforms; don't assume that they all share
    identical unspecified behaviour.

    There's no need for such sequence guarantees, though. It's probably not
    the most efficient but here's a way to do it:

    import os
    import time

    somedir = '/tmp'
    snapshot1 = os.listdir(somedir)
    time.sleep(...)
    snapshot2 = os.listdir(somedir)

    newfiles = [f for f in snapshot2 if not f in snapshot1]

    Pretty efficient, really: that's an O(n) comparison if I remember my
    Python internals correctly. Which I don't, so don't trust my word for
    it. ;-)

    >thanks,
    >
    >Hari
    >
    >

    -- Graham
    Graham Fawcett, Sep 9, 2003
    #2
    1. Advertising

  3. Hari

    Jeremy Jones Guest

    * Hari () wrote:
    <snip>
    >
    > What I wanted to know was, is it guaranteed that between 2 calls
    > os.listdir any files added to the directory are appended and the
    > earlier order is maintained?


    Even if the module does guarantee this to you, you probably don't want to
    just rely on the length of the list (nor the order of it). You probably
    want to build yourself a dictionary and keep track of it that way. One
    reason is that you may not be able to guarantee that a file didn't get
    deleted out of the directory. Even if you think you are in total control
    of that filesystem, I still wouldn't trust that no files would be deleted.
    I dunno, maybe I'm just paranoid. Further, as the value of the dictionary
    (didn't mention this above, but I'd let the filename serve as the key), I'd
    do os.stat and use the last modified date. Again, maybe I'm just paranoid.

    Jeremy Jones

    >
    > thanks,
    >
    > Hari
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    Jeremy Jones, Sep 9, 2003
    #3
  4. On 8 Sep 2003 15:41:58 -0700, rumours say that
    (Hari) might have written:

    >What I wanted to know was, is it guaranteed that between 2 calls
    >os.listdir any files added to the directory are appended and the
    >earlier order is maintained?


    No, the order of filenames in os.listdir depends on underlying
    (file|operating) system.

    The easiest way to find differences would be to use sets. Example:
    check the /tmp directory for changes in 30 seconds time:

    >>> import os, time
    >>> import sets
    >>> os.chdir("/tmp")
    >>> files0 = sets.Set(os.listdir("."))
    >>> time.sleep(30)

    files1 = sets.Set(os.listdir("."))
    >>> >>> print files1 - files0

    Set(['new-file'])

    ie shows only the file I created in another session for the purpose of
    this example.
    --
    TZOTZIOY, I speak England very best,
    Microsoft Security Alert: the Matrix began as open source.
    Christos TZOTZIOY Georgiou, Sep 9, 2003
    #4
  5. Hari

    Peter Otten Guest

    Hari wrote:

    > I wanted to know if this is a reliable way to check for files added to
    > a directory??


    Short answer: NO!

    How about:

    >>> from sets import Set
    >>> s1 = Set("abc")
    >>> s2 = Set("abd")
    >>> print s2 - s1

    Set(['d'])
    >>>


    Just replace "abc" and "abd" with os.listdir(path), and Python will show you
    all new files, or rather files with new names. This is so simple that
    there's no need to be paranoid (and I don't think Jeremy Jones is) to use
    this approach, that does not rely on both OS specifics (stable file name
    order) and user bahaviour (I promise I won't delete anything).

    Peter
    Peter Otten, Sep 9, 2003
    #5
  6. "Graham Fawcett" <>
    > Hari wrote:


    > >What I wanted to know was, is it guaranteed that between 2 calls
    > >os.listdir any files added to the directory are appended and the
    > >earlier order is maintained?


    As others pinted out, this is jighly improbably, espacially under windows


    > There's no need for such sequence guarantees, though. It's probably not
    > the most efficient but here's a way to do it:
    >
    > import os
    > import time
    >
    > somedir = '/tmp'
    > snapshot1 = os.listdir(somedir)
    > time.sleep(...)
    > snapshot2 = os.listdir(somedir)
    >
    > newfiles = [f for f in snapshot2 if not f in snapshot1]
    >
    > Pretty efficient, really: that's an O(n) comparison if I remember my
    > Python internals correctly. Which I don't, so don't trust my word for
    > it. ;-)



    You are wright to mistrust it ;-)
    Though 'f in list' looks harmless it itself is o(n) because there seems to
    be a linear search (This is different whith 'f in dict' of course). So your
    list comprehension is of o(n*n).

    Kindly
    Michael P

    > -- Graham
    Michael Peuser, Sep 9, 2003
    #6
  7. "Michael Peuser" <> wrote in message news:<bjk10v$ue0$04$-online.com>...
    > "Graham Fawcett" <>
    > > Hari wrote:

    >
    > > >What I wanted to know was, is it guaranteed that between 2 calls
    > > >os.listdir any files added to the directory are appended and the
    > > >earlier order is maintained?

    >
    > As others pinted out, this is jighly improbably, espacially under windows
    >
    >
    > > There's no need for such sequence guarantees, though. It's probably not
    > > the most efficient but here's a way to do it:
    > >
    > > import os
    > > import time
    > >
    > > somedir = '/tmp'
    > > snapshot1 = os.listdir(somedir)
    > > time.sleep(...)
    > > snapshot2 = os.listdir(somedir)
    > >
    > > newfiles = [f for f in snapshot2 if not f in snapshot1]
    > >
    > > Pretty efficient, really: that's an O(n) comparison if I remember my
    > > Python internals correctly. Which I don't, so don't trust my word for
    > > it. ;-)

    >
    >
    > You are wright to mistrust it ;-)
    > Though 'f in list' looks harmless it itself is o(n) because there seems to
    > be a linear search (This is different whith 'f in dict' of course). So your
    > list comprehension is of o(n*n).
    >
    > Kindly
    > Michael P


    D'oh! Thanks, Michael, I *knew* I'd messed that up...

    I'll repeat "list searches linear, dict lookups constant..." a hundred
    times before bed tonight.

    -- Graham
    Graham Fawcett, Sep 9, 2003
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jason Kratz

    stupid question about os.listdir

    Jason Kratz, Jun 27, 2003, in forum: Python
    Replies:
    11
    Views:
    1,095
    Erik Max Francis
    Jun 27, 2003
  2. kai
    Replies:
    1
    Views:
    357
  3. kai
    Replies:
    0
    Views:
    358
  4. kai
    Replies:
    0
    Views:
    305
  5. kai
    Replies:
    0
    Views:
    301
Loading...

Share This Page