newbie/ merging lists of lists with items in common

Discussion in 'Python' started by ardief, Feb 2, 2007.

  1. ardief

    ardief Guest

    Hi everyone
    Here is my problem:
    I have a list that looks like this -
    [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]

    and I would like to end up with something like this, i.e. with the
    only one list per letter:

    [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    ['e', ['11', '5', '16', '7']]]

    I have the feeling it's trivial, and I've scoured the group archives -
    sets might be a possibility, but I'm not sure how to operate on a list
    of lists with sets.

    This function also gives me what I want, more or less, but I don't
    know how to make it run until it's covered all the possibilities, if
    that makes sense...

    def sigh(list):
    for a in list:
    i = list.index(a)
    if a != list[-1]: ##if a is not the last one, i.e. there is a
    next one
    n = alist[i+1]
    if a[0] == n[0]:
    a.append(n[1:])
    del alist[i+1]

    Sorry about the lengthy message and thanks for your suggestions - I'm
    trying to learn...
     
    ardief, Feb 2, 2007
    #1
    1. Advertising

  2. ardief

    Miki Guest

    Hello,
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]

    I'd use a dictionary to store value for a given key:
    >>> def aggregate(lst):

    items = {} # key -> values
    for key, value in lst:
    values = items.get(key)
    if values:
    if type(values) == list:
    values.append(value)
    else:
    items[key] = [values, value]
    else:
    items[key] = value
    return [list(pair) for pair in items.items()]

    >>> aggregate(lst)

    [['a', ['13', '3']], ['c', ['12', '15', '4']], ['b', '6'], ['e',
    ['11', '5', '16', '7']], ['d', '2']]
    >>>


    HTH,
    --
    Miki <>
    http://pythonwise.blogspot.com
     
    Miki, Feb 2, 2007
    #2
    1. Advertising

  3. ardief

    Larry Bates Guest

    ardief wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >

    One solution:

    l=[['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c','4'],
    ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    d={}
    for k, v in l:
    if d.has_key(k): d[k].append(v)
    else: d[k]=[v]
    print "d=", d

    l=[x for x in d.items()]
    print l


    -Larry
     
    Larry Bates, Feb 2, 2007
    #3
  4. ardief

    Paddy Guest

    On Feb 2, 1:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...


    : python
    Python 2.5 (r25:51908, Nov 28 2006, 16:10:01)
    [GCC 3.4.3 (TWW)] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from pprint import pprint as pp
    >>> from collections import defaultdict
    >>> data = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >>> d = defaultdict(list)
    >>> _ = [d[x0].append(x1) for x0,x1 in data]
    >>> pp(d)

    defaultdict(<type 'list'>, {'a': ['13', '3'], 'c': ['12', '15', '4'],
    'b': ['6'], 'e': ['11', '5', '16', '7'], 'd': ['2']})
    >>> pp(sorted(d.items()))

    [('a', ['13', '3']),
    ('b', ['6']),
    ('c', ['12', '15', '4']),
    ('d', ['2']),
    ('e', ['11', '5', '16', '7'])]
    >>>


    - Paddy
     
    Paddy, Feb 2, 2007
    #4
  5. ardief

    Neil Cerutti Guest

    On 2007-02-02, ardief <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with
    > the only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group
    > archives - sets might be a possibility, but I'm not sure how to
    > operate on a list of lists with sets.


    This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!

    Er... I mean itertools.groupby.

    I took the liberty of not assuming the alist was sorted already.
    This could be a one-liner if you wanted to be evil.

    def bundle_alist(seq):
    """ Bundle together some alist tails.

    >>> seq = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >>> bundle_alist(seq)

    [['a', ['13', '3']], ['b', ['6']], ['c', ['12', '15', '4']], ['d', ['2']], ['e', ['11', '5', '16', '7']]]

    """
    from itertools import groupby
    def key_func(t):
    return t[0]
    groups = groupby(sorted(seq, key=key_func), key_func)
    seq = []
    for item in groups:
    seq.append([item[0], [a[1] for a in item[1]]])
    return seq


    --
    Neil Cerutti
    Music gets more chromatic and heavy towards the end of the century. One of the
    popular questions of the day was, "Why?" --Music Lit Essay
     
    Neil Cerutti, Feb 2, 2007
    #5
  6. ardief a écrit :
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >


    You may take a look to the groupby iterator in the standard itertools
    module.

    http://docs.python.org/lib/itertools-functions.html
     
    Laurent Pointal, Feb 2, 2007
    #6
  7. ardief

    Paddy Guest

    On Feb 2, 2:39 pm, "Paddy" <> wrote:
    > On Feb 2, 1:55 pm, "ardief" <> wrote:
    >
    >
    >
    > > Hi everyone
    > > Here is my problem:
    > > I have a list that looks like this -
    > > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]

    >
    > > and I would like to end up with something like this, i.e. with the
    > > only one list per letter:

    >
    > > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > > ['e', ['11', '5', '16', '7']]]

    >
    > > I have the feeling it's trivial, and I've scoured the group archives -
    > > sets might be a possibility, but I'm not sure how to operate on a list
    > > of lists with sets.

    >
    > > This function also gives me what I want, more or less, but I don't
    > > know how to make it run until it's covered all the possibilities, if
    > > that makes sense...

    >
    > > def sigh(list):
    > > for a in list:
    > > i = list.index(a)
    > > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > > next one
    > > n = alist[i+1]
    > > if a[0] == n[0]:
    > > a.append(n[1:])
    > > del alist[i+1]

    >
    > > Sorry about the lengthy message and thanks for your suggestions - I'm
    > > trying to learn...

    >
    > : python
    > Python 2.5 (r25:51908, Nov 28 2006, 16:10:01)
    > [GCC 3.4.3 (TWW)] on sunos5
    > Type "help", "copyright", "credits" or "license" for more information.>>> from pprint import pprint as pp
    > >>> from collections import defaultdict
    > >>> data = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    > >>> d = defaultdict(list)
    > >>> _ = [d[x0].append(x1) for x0,x1 in data]
    > >>> pp(d)

    >
    > defaultdict(<type 'list'>, {'a': ['13', '3'], 'c': ['12', '15', '4'],
    > 'b': ['6'], 'e': ['11', '5', '16', '7'], 'd': ['2']})>>> pp(sorted(d.items()))
    >
    > [('a', ['13', '3']),
    > ('b', ['6']),
    > ('c', ['12', '15', '4']),
    > ('d', ['2']),
    > ('e', ['11', '5', '16', '7'])]
    >
    >
    >
    > - Paddy



    Use defaultdict(set) and d[x0].add(x1) if you also want to remove
    duplicates.

    - Paddy.
     
    Paddy, Feb 2, 2007
    #7
  8. On Feb 2, 2:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]


    your_list = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c',
    '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'],
    ['e', '7']]
    d = {}
    for k,v in your_list: d.setdefault(k,[]).append(v)
    result = [list(x) for x in d.items()] # if you really need it as list
    of lists
    # if list of tuples is ok, then it´d be just:
    # result = d.items()
    result.sort() # if you need this list sorted by keys
     
    Bart Ogryczak, Feb 2, 2007
    #8
  9. Neil Cerutti a écrit :
    > On 2007-02-02, ardief <> wrote:

    <zip>

    > This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!
    >
    > Er... I mean itertools.groupby.
    >

    <zip>
    > def key_func(t):
    > return t[0]


    Not needed: --> from operator import itemgetter

    See in the example:
    http://docs.python.org/lib/itertools-example.html

    So much stuff in libraries, so few we know. Thanks to doc writers,
    Usenet contributors & Google search engines.
     
    Laurent Pointal, Feb 2, 2007
    #9
  10. On Feb 2, 3:19 pm, Larry Bates <> wrote:

    > l=[x for x in d.items()]


    d.items() is not an iterator, you don´t need this. This code is
    equivalent to l = d.items().
     
    Bart Ogryczak, Feb 2, 2007
    #10
  11. ardief

    Neil Cerutti Guest

    On 2007-02-02, Laurent Pointal <> wrote:
    > Neil Cerutti a écrit :
    >> On 2007-02-02, ardief <> wrote:

    ><zip>
    >
    >> This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!
    >>
    >> Er... I mean itertools.groupby.
    >>

    ><zip>
    >> def key_func(t):
    >> return t[0]

    >
    > Not needed: --> from operator import itemgetter


    I agree. But I used it anyway, to make it easier to see that the
    sort and the groupby must be and are using the same key function.

    In this case I admit it's a not a huge readability win, but I was
    also following the "Do Not Repeat Yourself Rule", which makes the
    key function easier to refactor.

    > See in the example:
    > http://docs.python.org/lib/itertools-example.html
    >
    > So much stuff in libraries, so few we know. Thanks to doc
    > writers, Usenet contributors & Google search engines.


    Yup.

    --
    Neil Cerutti
     
    Neil Cerutti, Feb 2, 2007
    #11
  12. ardief

    ardief Guest

    On Feb 2, 1:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]

    .....

    thanks to everyone for the help, and the speed of it! It's really
    useful and I will spend some time working on understanding the code
    you posted. I'd be so lost without this group...

    r
     
    ardief, Feb 2, 2007
    #12
  13. ardief

    James Stroud Guest

    ardief wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >



    Did someone suggest this? I couldn't find it and it seems simplest to me
    (no imports, etc.):

    keys = sorted(set(i[0] for i in alist))
    d = [(k,[i[1] for i in alist if i[0]==k]) for k in keys]

    E.g.:
    py> alist = [['a', '13'],
    .... ['a', '3'],
    .... ['b', '6'],
    .... ['c', '12'],
    .... ['c', '15'],
    .... ['c', '4'],
    .... ['d', '2'],
    .... ['e', '11'],
    .... ['e', '5'],
    .... ['e', '16'],
    .... ['e', '7']]
    py> keys = sorted(set(i[0] for i in alist))
    py> d = [(k,[i[1] for i in alist if i[0]==k]) for k in keys]
    py> d

    [('a', ['13', '3']),
    ('b', ['6']),
    ('c', ['12', '15', '4']),
    ('d', ['2']),
    ('e', ['11', '5', '16', '7'])]



    James
     
    James Stroud, Feb 2, 2007
    #13
  14. ardief

    Guest

    Paddy:
    >>> _ = [d[x0].append(x1) for x0,x1 in data]


    I think that you probably want:

    for text, num in data: d[text].append(num)


    ardief:
    > thanks to everyone for the help, and the speed of it! It's really
    > useful and I will spend some time working on understanding the code
    > you posted. I'd be so lost without this group...


    If you have Python 2.5, and if you don't have particular requirements,
    I think the best solution is Paddy's.

    Bye,
    bearophile
     
    , Feb 2, 2007
    #14
  15. ardief

    Paddy Guest

    On Feb 2, 10:34 pm, wrote:
    > Paddy:
    >
    > >>> _ = [d[x0].append(x1) for x0,x1 in data]

    Yep, definitely a case of overdoing the list comprehensions when you
    throw away the list.
    I'll watch out for that in the future,

    Ta.

    - Paddy.

    >
    > I think that you probably want:
    >
    > for text, num in data: d[text].append(num)
    >
    > ardief:
    >
    > > thanks to everyone for the help, and the speed of it! It's really
    > > useful and I will spend some time working on understanding the code
    > > you posted. I'd be so lost without this group...

    >
    > If you have Python 2.5, and if you don't have particular requirements,
    > I think the best solution is Paddy's.
    >
    > Bye,
    > bearophile
     
    Paddy, Feb 3, 2007
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Henrik Goldman

    Merging a list of similar items

    Henrik Goldman, Jun 13, 2007, in forum: C++
    Replies:
    6
    Views:
    304
  2. Denny
    Replies:
    1
    Views:
    781
  3. antar2
    Replies:
    2
    Views:
    393
    Bighead
    Jul 17, 2008
  4. John Reye
    Replies:
    28
    Views:
    1,371
    Tim Rentsch
    May 8, 2012
  5. John Mathew

    Merging xls into a common one

    John Mathew, Oct 17, 2013, in forum: Python
    Replies:
    0
    Views:
    86
    John Mathew
    Oct 17, 2013
Loading...

Share This Page