RE: delete duplicates in list

Discussion in 'Python' started by Martin Chilvers, Oct 31, 2003.

  1. Maybe I'm just old-school, and it won't win any prizes for speed but...

    >>> a = [1, 3, 2, 3, 4, 2, 1, 4, 3, 2, 1, 2, 4]
    >>> reduce(lambda l, x: x not in l and l.append(x) or l, a, [])

    [1, 3, 2, 4]

    ;^)

    Martin

    > -----Original Message-----
    > From: python-list-bounces+martin=
    > [mailto:python-list-bounces+martin=]
    > On Behalf Of Bengt Richter
    > Sent: Thursday, October 30, 2003 5:26 PM
    > To:
    > Subject: Re: delete duplicates in list
    >
    >
    > On Wed, 29 Oct 2003 22:02:08 +0100, christof hoeke
    > <> wrote:
    >
    > >hello,
    > >this must have come up before, so i am already sorry for asking but a
    > >quick googling did not give me any answer.
    > >
    > >i have a list from which i want a simpler list without the

    > duplicates
    > >an easy but somehow contrived solution would be
    > >
    > > >>> a = [1, 2, 2, 3]
    > > >>> d = {}.fromkeys(a)
    > > >>> b = d.keys()
    > > >>> print b

    > >[1, 2, 3]
    > >
    > >there should be an easier or more intuitive solution, maybe

    > with a list
    > >comprehension=
    > >
    > >somthing like
    > >
    > > >>> b = [x for x in a if x not in b]
    > > >>> print b

    > >[]
    > >
    > >does not work though.
    > >

    > If you want to replace the original list without a temporary
    > new list, and your original is sorted (or you don't mind
    > having it sorted), then you could do the following (not
    > tested beyond what you see ;-), which as an extra benefit
    > doesn't require hashability:
    >
    > >>> def elimdups(thelist):

    > ... thelist.sort() # remove if you just want to
    > eliminate adjacent duplicates
    > ... i = 0
    > ... for item in thelist:
    > ... if item==thelist: continue
    > ... i += 1
    > ... thelist = item
    > ... del thelist[i+1:]
    > ...
    > >>> a = [1, 2, 2, 3]
    > >>> elimdups(a)
    > >>> a

    > [1, 2, 3]
    > >>> a=[]
    > >>> elimdups(a)
    > >>> a

    > []
    > >>> a = [123]
    > >>> elimdups(a)
    > >>> a

    > [123]
    > >>> a = ['a', ['b', 2], ['c',3], ['b',2], 'd']
    > >>> a

    > ['a', ['b', 2], ['c', 3], ['b', 2], 'd']
    > >>> elimdups(a)
    > >>> a

    > [['b', 2], ['c', 3], 'a', 'd']
    > >>> a = list('deaacbb')
    > >>> elimdups(a)
    > >>> a

    > ['a', 'b', 'c', 'd', 'e']
    >
    > Not sure how this was decided, but that's the way it works:
    > >>> 'a' > ['b', 2]

    > True
    >
    > Hm, it would have been nicer to have an optional sort flag as
    > a second parameter. Oh, well, another time...
    >
    > Regards,
    > Bengt Richter
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Martin Chilvers, Oct 31, 2003
    #1
    1. Advertising

  2. In article <>, Martin Chilvers wrote:
    > Maybe I'm just old-school, and it won't win any prizes for speed but...
    >
    >>>> a = [1, 3, 2, 3, 4, 2, 1, 4, 3, 2, 1, 2, 4]
    >>>> reduce(lambda l, x: x not in l and l.append(x) or l, a, [])

    > [1, 3, 2, 4]
    >
    > ;^)


    Hey, I wanna play this game too:

    [d.setdefault(x, x) for d in [{}] for x in a if x not in d]

    Now, try replacing that with "old-school" functionals. I'm stumped. =)

    --
    ..:[ dave benjamin (ramenboy) -:- www.ramenfest.com -:- www.3dex.com ]:.
    : d r i n k i n g l i f e o u t o f t h e c o n t a i n e r :
     
    Dave Benjamin, Nov 3, 2003
    #2
    1. Advertising

  3. In article <>, Dave Benjamin wrote:
    > In article <>, Martin Chilvers wrote:
    >> Maybe I'm just old-school, and it won't win any prizes for speed but...
    >>
    >>>>> a = [1, 3, 2, 3, 4, 2, 1, 4, 3, 2, 1, 2, 4]
    >>>>> reduce(lambda l, x: x not in l and l.append(x) or l, a, [])

    >> [1, 3, 2, 4]
    >>
    >> ;^)

    >
    > Hey, I wanna play this game too:
    >
    > [d.setdefault(x, x) for d in [{}] for x in a if x not in d]
    >
    > Now, try replacing that with "old-school" functionals. I'm stumped. =)


    Nevermind, I'm so there... ;)

    >>> filter(lambda x, d={}: x not in d and d.setdefault(x, True), a)

    [1, 3, 2, 4]

    Corrupting Python since '02,
    Dave
     
    Dave Benjamin, Nov 3, 2003
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. christof hoeke

    delete duplicates in list

    christof hoeke, Oct 29, 2003, in forum: Python
    Replies:
    22
    Views:
    835
    Michael Hudson
    Oct 31, 2003
  2. Paul
    Replies:
    9
    Views:
    511
    Paul Rubin
    Jul 7, 2004
  3. Steven Bethard
    Replies:
    11
    Views:
    1,069
    Alex Martelli
    Feb 7, 2005
  4. basi
    Replies:
    4
    Views:
    184
    Wayne Vucenic
    Aug 1, 2005
  5. Amit Kulkarni

    Delete Duplicates

    Amit Kulkarni, Jun 11, 2008, in forum: Ruby
    Replies:
    3
    Views:
    103
    Amit Kulkarni
    Jun 12, 2008
Loading...

Share This Page