newbie/ merging lists of lists with items in common

Discussion in 'Python' started by ardief, Feb 2, 2007.

  1. ardief

    ardief Guest

    Hi everyone
    Here is my problem:
    I have a list that looks like this -
    [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]

    and I would like to end up with something like this, i.e. with the
    only one list per letter:

    [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    ['e', ['11', '5', '16', '7']]]

    I have the feeling it's trivial, and I've scoured the group archives -
    sets might be a possibility, but I'm not sure how to operate on a list
    of lists with sets.

    This function also gives me what I want, more or less, but I don't
    know how to make it run until it's covered all the possibilities, if
    that makes sense...

    def sigh(list):
    for a in list:
    i = list.index(a)
    if a != list[-1]: ##if a is not the last one, i.e. there is a
    next one
    n = alist[i+1]
    if a[0] == n[0]:
    a.append(n[1:])
    del alist[i+1]

    Sorry about the lengthy message and thanks for your suggestions - I'm
    trying to learn...
     
    ardief, Feb 2, 2007
    #1
    1. Advertisements

  2. ardief

    Miki Guest

    Hello,
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]

    I'd use a dictionary to store value for a given key:
    >>> def aggregate(lst):

    items = {} # key -> values
    for key, value in lst:
    values = items.get(key)
    if values:
    if type(values) == list:
    values.append(value)
    else:
    items[key] = [values, value]
    else:
    items[key] = value
    return [list(pair) for pair in items.items()]

    >>> aggregate(lst)

    [['a', ['13', '3']], ['c', ['12', '15', '4']], ['b', '6'], ['e',
    ['11', '5', '16', '7']], ['d', '2']]
    >>>


    HTH,
    --
    Miki <>
    http://pythonwise.blogspot.com
     
    Miki, Feb 2, 2007
    #2
    1. Advertisements

  3. ardief

    Larry Bates Guest

    ardief wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >

    One solution:

    l=[['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c','4'],
    ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    d={}
    for k, v in l:
    if d.has_key(k): d[k].append(v)
    else: d[k]=[v]
    print "d=", d

    l=[x for x in d.items()]
    print l


    -Larry
     
    Larry Bates, Feb 2, 2007
    #3
  4. ardief

    Paddy Guest

    On Feb 2, 1:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...


    : python
    Python 2.5 (r25:51908, Nov 28 2006, 16:10:01)
    [GCC 3.4.3 (TWW)] on sunos5
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from pprint import pprint as pp
    >>> from collections import defaultdict
    >>> data = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >>> d = defaultdict(list)
    >>> _ = [d[x0].append(x1) for x0,x1 in data]
    >>> pp(d)

    defaultdict(<type 'list'>, {'a': ['13', '3'], 'c': ['12', '15', '4'],
    'b': ['6'], 'e': ['11', '5', '16', '7'], 'd': ['2']})
    >>> pp(sorted(d.items()))

    [('a', ['13', '3']),
    ('b', ['6']),
    ('c', ['12', '15', '4']),
    ('d', ['2']),
    ('e', ['11', '5', '16', '7'])]
    >>>


    - Paddy
     
    Paddy, Feb 2, 2007
    #4
  5. ardief

    Neil Cerutti Guest

    On 2007-02-02, ardief <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with
    > the only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group
    > archives - sets might be a possibility, but I'm not sure how to
    > operate on a list of lists with sets.


    This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!

    Er... I mean itertools.groupby.

    I took the liberty of not assuming the alist was sorted already.
    This could be a one-liner if you wanted to be evil.

    def bundle_alist(seq):
    """ Bundle together some alist tails.

    >>> seq = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >>> bundle_alist(seq)

    [['a', ['13', '3']], ['b', ['6']], ['c', ['12', '15', '4']], ['d', ['2']], ['e', ['11', '5', '16', '7']]]

    """
    from itertools import groupby
    def key_func(t):
    return t[0]
    groups = groupby(sorted(seq, key=key_func), key_func)
    seq = []
    for item in groups:
    seq.append([item[0], [a[1] for a in item[1]]])
    return seq


    --
    Neil Cerutti
    Music gets more chromatic and heavy towards the end of the century. One of the
    popular questions of the day was, "Why?" --Music Lit Essay
     
    Neil Cerutti, Feb 2, 2007
    #5
  6. ardief a écrit :
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >


    You may take a look to the groupby iterator in the standard itertools
    module.

    http://docs.python.org/lib/itertools-functions.html
     
    Laurent Pointal, Feb 2, 2007
    #6
  7. ardief

    Paddy Guest

    On Feb 2, 2:39 pm, "Paddy" <> wrote:
    > On Feb 2, 1:55 pm, "ardief" <> wrote:
    >
    >
    >
    > > Hi everyone
    > > Here is my problem:
    > > I have a list that looks like this -
    > > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]

    >
    > > and I would like to end up with something like this, i.e. with the
    > > only one list per letter:

    >
    > > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > > ['e', ['11', '5', '16', '7']]]

    >
    > > I have the feeling it's trivial, and I've scoured the group archives -
    > > sets might be a possibility, but I'm not sure how to operate on a list
    > > of lists with sets.

    >
    > > This function also gives me what I want, more or less, but I don't
    > > know how to make it run until it's covered all the possibilities, if
    > > that makes sense...

    >
    > > def sigh(list):
    > > for a in list:
    > > i = list.index(a)
    > > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > > next one
    > > n = alist[i+1]
    > > if a[0] == n[0]:
    > > a.append(n[1:])
    > > del alist[i+1]

    >
    > > Sorry about the lengthy message and thanks for your suggestions - I'm
    > > trying to learn...

    >
    > : python
    > Python 2.5 (r25:51908, Nov 28 2006, 16:10:01)
    > [GCC 3.4.3 (TWW)] on sunos5
    > Type "help", "copyright", "credits" or "license" for more information.>>> from pprint import pprint as pp
    > >>> from collections import defaultdict
    > >>> data = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    > >>> d = defaultdict(list)
    > >>> _ = [d[x0].append(x1) for x0,x1 in data]
    > >>> pp(d)

    >
    > defaultdict(<type 'list'>, {'a': ['13', '3'], 'c': ['12', '15', '4'],
    > 'b': ['6'], 'e': ['11', '5', '16', '7'], 'd': ['2']})>>> pp(sorted(d.items()))
    >
    > [('a', ['13', '3']),
    > ('b', ['6']),
    > ('c', ['12', '15', '4']),
    > ('d', ['2']),
    > ('e', ['11', '5', '16', '7'])]
    >
    >
    >
    > - Paddy



    Use defaultdict(set) and d[x0].add(x1) if you also want to remove
    duplicates.

    - Paddy.
     
    Paddy, Feb 2, 2007
    #7
  8. On Feb 2, 2:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]


    your_list = [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c',
    '15'], ['c', '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'],
    ['e', '7']]
    d = {}
    for k,v in your_list: d.setdefault(k,[]).append(v)
    result = [list(x) for x in d.items()] # if you really need it as list
    of lists
    # if list of tuples is ok, then it´d be just:
    # result = d.items()
    result.sort() # if you need this list sorted by keys
     
    Bart Ogryczak, Feb 2, 2007
    #8
  9. Neil Cerutti a écrit :
    > On 2007-02-02, ardief <> wrote:

    <zip>

    > This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!
    >
    > Er... I mean itertools.groupby.
    >

    <zip>
    > def key_func(t):
    > return t[0]


    Not needed: --> from operator import itemgetter

    See in the example:
    http://docs.python.org/lib/itertools-example.html

    So much stuff in libraries, so few we know. Thanks to doc writers,
    Usenet contributors & Google search engines.
     
    Laurent Pointal, Feb 2, 2007
    #9
  10. On Feb 2, 3:19 pm, Larry Bates <> wrote:

    > l=[x for x in d.items()]


    d.items() is not an iterator, you don´t need this. This code is
    equivalent to l = d.items().
     
    Bart Ogryczak, Feb 2, 2007
    #10
  11. ardief

    Neil Cerutti Guest

    On 2007-02-02, Laurent Pointal <> wrote:
    > Neil Cerutti a écrit :
    >> On 2007-02-02, ardief <> wrote:

    ><zip>
    >
    >> This is a job for... duhn-duhn-DAAAAH! Captain CHAOS!
    >>
    >> Er... I mean itertools.groupby.
    >>

    ><zip>
    >> def key_func(t):
    >> return t[0]

    >
    > Not needed: --> from operator import itemgetter


    I agree. But I used it anyway, to make it easier to see that the
    sort and the groupby must be and are using the same key function.

    In this case I admit it's a not a huge readability win, but I was
    also following the "Do Not Repeat Yourself Rule", which makes the
    key function easier to refactor.

    > See in the example:
    > http://docs.python.org/lib/itertools-example.html
    >
    > So much stuff in libraries, so few we know. Thanks to doc
    > writers, Usenet contributors & Google search engines.


    Yup.

    --
    Neil Cerutti
     
    Neil Cerutti, Feb 2, 2007
    #11
  12. ardief

    ardief Guest

    On Feb 2, 1:55 pm, "ardief" <> wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]

    .....

    thanks to everyone for the help, and the speed of it! It's really
    useful and I will spend some time working on understanding the code
    you posted. I'd be so lost without this group...

    r
     
    ardief, Feb 2, 2007
    #12
  13. ardief

    James Stroud Guest

    ardief wrote:
    > Hi everyone
    > Here is my problem:
    > I have a list that looks like this -
    > [['a', '13'], ['a', '3'], ['b', '6'], ['c', '12'], ['c', '15'], ['c',
    > '4'], ['d', '2'], ['e', '11'], ['e', '5'], ['e', '16'], ['e', '7']]
    >
    > and I would like to end up with something like this, i.e. with the
    > only one list per letter:
    >
    > [['a', ['13' '3']], ['b', '6'], ['c', ['12', '15', '4']], ['d', '2'],
    > ['e', ['11', '5', '16', '7']]]
    >
    > I have the feeling it's trivial, and I've scoured the group archives -
    > sets might be a possibility, but I'm not sure how to operate on a list
    > of lists with sets.
    >
    > This function also gives me what I want, more or less, but I don't
    > know how to make it run until it's covered all the possibilities, if
    > that makes sense...
    >
    > def sigh(list):
    > for a in list:
    > i = list.index(a)
    > if a != list[-1]: ##if a is not the last one, i.e. there is a
    > next one
    > n = alist[i+1]
    > if a[0] == n[0]:
    > a.append(n[1:])
    > del alist[i+1]
    >
    > Sorry about the lengthy message and thanks for your suggestions - I'm
    > trying to learn...
    >



    Did someone suggest this? I couldn't find it and it seems simplest to me
    (no imports, etc.):

    keys = sorted(set(i[0] for i in alist))
    d = [(k,[i[1] for i in alist if i[0]==k]) for k in keys]

    E.g.:
    py> alist = [['a', '13'],
    .... ['a', '3'],
    .... ['b', '6'],
    .... ['c', '12'],
    .... ['c', '15'],
    .... ['c', '4'],
    .... ['d', '2'],
    .... ['e', '11'],
    .... ['e', '5'],
    .... ['e', '16'],
    .... ['e', '7']]
    py> keys = sorted(set(i[0] for i in alist))
    py> d = [(k,[i[1] for i in alist if i[0]==k]) for k in keys]
    py> d

    [('a', ['13', '3']),
    ('b', ['6']),
    ('c', ['12', '15', '4']),
    ('d', ['2']),
    ('e', ['11', '5', '16', '7'])]



    James
     
    James Stroud, Feb 2, 2007
    #13
  14. ardief

    Guest

    Paddy:
    >>> _ = [d[x0].append(x1) for x0,x1 in data]


    I think that you probably want:

    for text, num in data: d[text].append(num)


    ardief:
    > thanks to everyone for the help, and the speed of it! It's really
    > useful and I will spend some time working on understanding the code
    > you posted. I'd be so lost without this group...


    If you have Python 2.5, and if you don't have particular requirements,
    I think the best solution is Paddy's.

    Bye,
    bearophile
     
    , Feb 2, 2007
    #14
  15. ardief

    Paddy Guest

    On Feb 2, 10:34 pm, wrote:
    > Paddy:
    >
    > >>> _ = [d[x0].append(x1) for x0,x1 in data]

    Yep, definitely a case of overdoing the list comprehensions when you
    throw away the list.
    I'll watch out for that in the future,

    Ta.

    - Paddy.

    >
    > I think that you probably want:
    >
    > for text, num in data: d[text].append(num)
    >
    > ardief:
    >
    > > thanks to everyone for the help, and the speed of it! It's really
    > > useful and I will spend some time working on understanding the code
    > > you posted. I'd be so lost without this group...

    >
    > If you have Python 2.5, and if you don't have particular requirements,
    > I think the best solution is Paddy's.
    >
    > Bye,
    > bearophile
     
    Paddy, Feb 3, 2007
    #15
    1. Advertisements

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Charlotte Henkle

    Counter for items in lists in lists?

    Charlotte Henkle, Sep 25, 2004, in forum: Python
    Replies:
    8
    Views:
    589
    Charlotte Henkle
    Sep 26, 2004
  2. Henrik Goldman

    Merging a list of similar items

    Henrik Goldman, Jun 13, 2007, in forum: C++
    Replies:
    6
    Views:
    427
  3. Denny
    Replies:
    1
    Views:
    1,056
  4. antar2
    Replies:
    2
    Views:
    557
    Bighead
    Jul 17, 2008
  5. Alexzive
    Replies:
    6
    Views:
    802
    Chris Rebert
    Mar 20, 2009
  6. John Reye
    Replies:
    28
    Views:
    1,691
    Tim Rentsch
    May 8, 2012
  7. John Mathew

    Merging xls into a common one

    John Mathew, Oct 17, 2013, in forum: Python
    Replies:
    0
    Views:
    146
    John Mathew
    Oct 17, 2013
  8. Jason Mellone
    Replies:
    3
    Views:
    306
    Jurko Gospodnetić
    May 7, 2014
Loading...