Re: remove a list from a list

Discussion in 'Python' started by Rares Vernica, Nov 17, 2006.

  1. Sorry for not being clear from the beginning and for not using clear
    variable names.

    Problem context:

    import os
    dirs_exclude = set(('a', 'b', 'e'))
    for root, dirs, files in os.walk('python/Lib/email'):
    # Task:
    # delete from "dirs" the directory names from "dirs_exclude"
    # case-insensitive

    The solution so far is:

    for i in xrange(len(dirs), 0, -1):
    if dirs[i-1].lower() in dirs_exclude:
    del dirs[i-1]

    I am looking for a nicer solution.

    Thanks a lot,
    Ray

    Tim Chase wrote:
    >> Yeah, I ended up doing a similar kind of loop. That is pretty messy.
    >>
    >> Is there any other way?

    >
    > I've already provided 2 (or 3 depending on how one counts)
    > solutions, each of which solve an interpretation of your original
    > problem, neither of which involve more than 3 lines of fairly
    > clean code. Perhaps a little more context regarding what you
    > *want* to do would help. However, I suspect that answer is
    > "there is no *cleaner* way to do it".
    >
    > Unless you're modifying an existing list that is referenced
    > elsewhere, the reassignment (l = [x for x in l ...]) solution
    > should work just fine. Thus, unless you have a situation akin to:
    >
    > g = l
    > l = [x for x in l if x.lower() not in s]
    > assert(thing_from_s not in g)
    >
    > then just reassign "l". If not, use the loop. It's that easy
    > and clean. Don't try to opaquify it by collapsing it further.
    > Perhaps, if your loop is messy, use my clean loop suggestion.
    >
    > -tkc
    >
    >
    >
     
    Rares Vernica, Nov 17, 2006
    #1
    1. Advertising

  2. Rares Vernica

    Matimus Guest

    > The solution so far is:
    >
    > for i in xrange(len(dirs), 0, -1):
    > if dirs[i-1].lower() in dirs_exclude:
    > del dirs[i-1]


    This won't affect much, but uses better style I think.

    Change:

    for i in xrange(len(dirs),0,-1):

    To:

    for i in reversed(xrange(len(dirs))):

    and then just use 'i' instead of 'i-1'
     
    Matimus, Nov 17, 2006
    #2
    1. Advertising

  3. Rares Vernica

    Neil Cerutti Guest

    On 2006-11-17, Rares Vernica <> wrote:
    > Sorry for not being clear from the beginning and for not using
    > clear variable names.
    >
    > Problem context:
    >
    > import os
    > dirs_exclude = set(('a', 'b', 'e'))
    > for root, dirs, files in os.walk('python/Lib/email'):
    > # Task:
    > # delete from "dirs" the directory names from "dirs_exclude"
    > # case-insensitive
    >
    > The solution so far is:
    >
    > for i in xrange(len(dirs), 0, -1):
    > if dirs[i-1].lower() in dirs_exclude:
    > del dirs[i-1]
    >
    > I am looking for a nicer solution.


    I'd probably just skip over those dirs as I came them instead of
    troubling about mutating the list. Unless the list is needed in
    more than one place.

    --
    Neil Cerutti
     
    Neil Cerutti, Nov 17, 2006
    #3
  4. On Fri, 17 Nov 2006 12:00:46 -0800, Rares Vernica wrote:

    > Problem context:
    >
    > import os
    > dirs_exclude = set(('a', 'b', 'e'))
    > for root, dirs, files in os.walk('python/Lib/email'):
    > # Task:
    > # delete from "dirs" the directory names from "dirs_exclude"
    > # case-insensitive
    >
    > The solution so far is:
    >
    > for i in xrange(len(dirs), 0, -1):
    > if dirs[i-1].lower() in dirs_exclude:
    > del dirs[i-1]
    >
    > I am looking for a nicer solution.


    Define "nicer".

    First thing I'd do is change the loop:

    for i in xrange(len(dirs)-1, -1, -1):
    if dirs.lower() in dirs_exclude:
    del dirs

    Second thing I'd do is encapsulate it in a function instead of calling it
    in place:

    def remove_in_place(source, target):
    for i in xrange(len(source)-1, -1, -1):
    if source.lower() in target:
    del source

    Third thing I'd do is replace the delete-in-place code away, and build a
    new list using the set idiom, finally using list slicing to change the
    source in place:

    def remove_in_place2(source, target):
    target = set(s.lower() for s in target)
    source[:] = [x for x in source if x.lower() not in target]
    # note the assignment to a slice

    And finally, I would test the two versions remove_in_place and
    remove_in_place2 to see which is faster.


    import timeit

    setup = """from __main__ import remove_in_place
    target = list("aEIOu")
    source = list("AbcdEfghIjklmnOpqrstUvwxyz")
    """

    tester = """tmplist = source[:] # make a copy of the list!
    remove_in_place(tmplist, target)
    """

    timeit.Timer(tester, setup).timer()

    You have to make a copy of the list on every iteration because you are
    changing it in place; otherwise you change the values you are testing
    against, and the second iteration onwards doesn't have to remove anything.


    (All code above untested. Use at own risk.)

    --
    Steven.
     
    Steven D'Aprano, Nov 17, 2006
    #4
  5. The problem with skipping over them is that "walk" would still walk them
    and their content. If they have a lot of other dirs and files inside
    then this might end up being time consuming.

    Thanks,
    Ray

    Neil Cerutti wrote:
    > On 2006-11-17, Rares Vernica <> wrote:
    >> Sorry for not being clear from the beginning and for not using
    >> clear variable names.
    >>
    >> Problem context:
    >>
    >> import os
    >> dirs_exclude = set(('a', 'b', 'e'))
    >> for root, dirs, files in os.walk('python/Lib/email'):
    >> # Task:
    >> # delete from "dirs" the directory names from "dirs_exclude"
    >> # case-insensitive
    >>
    >> The solution so far is:
    >>
    >> for i in xrange(len(dirs), 0, -1):
    >> if dirs[i-1].lower() in dirs_exclude:
    >> del dirs[i-1]
    >>
    >> I am looking for a nicer solution.

    >
    > I'd probably just skip over those dirs as I came them instead of
    > troubling about mutating the list. Unless the list is needed in
    > more than one place.
    >
     
    Rares Vernica, Nov 17, 2006
    #5
  6. This solution I think is pretty nice:

    source[:] = [x for x in source if x.lower() not in target]

    Thanks a lot for all the answers,
    Ray

    Steven D'Aprano wrote:
    > On Fri, 17 Nov 2006 12:00:46 -0800, Rares Vernica wrote:
    >
    >> Problem context:
    >>
    >> import os
    >> dirs_exclude = set(('a', 'b', 'e'))
    >> for root, dirs, files in os.walk('python/Lib/email'):
    >> # Task:
    >> # delete from "dirs" the directory names from "dirs_exclude"
    >> # case-insensitive
    >>
    >> The solution so far is:
    >>
    >> for i in xrange(len(dirs), 0, -1):
    >> if dirs[i-1].lower() in dirs_exclude:
    >> del dirs[i-1]
    >>
    >> I am looking for a nicer solution.

    >
    > Define "nicer".
    >
    > First thing I'd do is change the loop:
    >
    > for i in xrange(len(dirs)-1, -1, -1):
    > if dirs.lower() in dirs_exclude:
    > del dirs
    >
    > Second thing I'd do is encapsulate it in a function instead of calling it
    > in place:
    >
    > def remove_in_place(source, target):
    > for i in xrange(len(source)-1, -1, -1):
    > if source.lower() in target:
    > del source
    >
    > Third thing I'd do is replace the delete-in-place code away, and build a
    > new list using the set idiom, finally using list slicing to change the
    > source in place:
    >
    > def remove_in_place2(source, target):
    > target = set(s.lower() for s in target)
    > source[:] = [x for x in source if x.lower() not in target]
    > # note the assignment to a slice
    >
    > And finally, I would test the two versions remove_in_place and
    > remove_in_place2 to see which is faster.
    >
    >
    > import timeit
    >
    > setup = """from __main__ import remove_in_place
    > target = list("aEIOu")
    > source = list("AbcdEfghIjklmnOpqrstUvwxyz")
    > """
    >
    > tester = """tmplist = source[:] # make a copy of the list!
    > remove_in_place(tmplist, target)
    > """
    >
    > timeit.Timer(tester, setup).timer()
    >
    > You have to make a copy of the list on every iteration because you are
    > changing it in place; otherwise you change the values you are testing
    > against, and the second iteration onwards doesn't have to remove anything.
    >
    >
    > (All code above untested. Use at own risk.)
    >
     
    Rares Vernica, Nov 17, 2006
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Simon-Pierre  Jarry
    Replies:
    2
    Views:
    2,412
    Henrik
    Aug 10, 2005
  2. tshad
    Replies:
    6
    Views:
    21,534
    tshad
    Aug 8, 2006
  3. Rares Vernica

    remove a list from a list

    Rares Vernica, Nov 17, 2006, in forum: Python
    Replies:
    1
    Views:
    265
    J. Clifford Dyer
    Nov 20, 2006
  4. Rares Vernica

    Re: remove a list from a list

    Rares Vernica, Nov 17, 2006, in forum: Python
    Replies:
    4
    Views:
    315
    John Henry
    Nov 17, 2006
  5. Andy
    Replies:
    3
    Views:
    499
    James Kanze
    Jun 8, 2007
Loading...

Share This Page