Question regarding Higher-Order-Programming in Python

Discussion in 'Python' started by Mark Fink, Dec 22, 2010.

  1. Mark Fink

    Mark Fink Guest

    I am about to learn Higher-Order-Programming with Lisp, Haskell, and
    Python. Some people who have gone completely out of their mind call
    this FP.

    In Haskell I learned that when I use map on a list it starts nesting
    as soon as I start adding elements. If I do not like the nesting I use

    In Python I have a similar scenario. I have a generator which creates
    some combinatorics of a input dictionary. The list starts nesting. Now
    I could use reduce(concat, ...) which would be the correct thing from
    a functional perspective. But from a resource utilization perspective
    it is not a good idea since the reduce is executing the generator.

    I tried to illustrate this using a small example (the often in
    combinatorics the real thing would be much bigger that is why I need
    to use a generator):
    .... # helper to get products from keys in the following form:
    .... # [('bold', True), ('color', 'black')]
    .... values = itemgetter(*keys)(dims)
    .... product = it.product(*values)
    .... return map(partial(zip, keys), product)
    [[[('special', '+'), ('number', 1)], [('special', '+'), ('number',
    2)], [('special', '+'), ('number', 3)], [('special', '-'), ('number',
    1)], [('special', '-'), ('number', 2)], [('special', '-'), ('number',
    3)]], [[('special', '+'), ('letter', 'a')], [('special', '+'),
    ('letter', 'b')], [('special', '-'), ('letter', 'a')], [('special',
    '-'), ('letter', 'b')]], [[('number', 1), ('letter', 'a')],
    [('number', 1), ('letter', 'b')], [('number', 2), ('letter', 'a')],
    [('number', 2), ('letter', 'b')], [('number', 3), ('letter', 'a')],
    [('number', 3), ('letter', 'b')]]] # the resulting list is nested one
    level to deep caused by the map(get_products, ..
    My problem is that I want to get single elements from the generator
    like [('special', '+'), ('number', 1)]. But this does not work because
    the list is now nested to deep.

    That is what I expect: (I could get something like that with the
    following >>> res = reduce(concat, res)
    [[('special', '+'), ('number', 1)], [('special', '+'), ('number', 2)],
    [('special', '+'), ('number', 3)], [('special', '-'), ('number', 1)],
    [('special', '-'), ('number', 2)], [('special', '-'), ('number', 3)],
    [('special', '+'), ('letter', 'a')], [('special', '+'), ('letter',
    'b')], [('special', '-'), ('letter', 'a')], [('special', '-'),
    ('letter', 'b')], [('number', 1), ('letter', 'a')], [('number', 1),
    ('letter', 'b')], [('number', 2), ('letter', 'a')], [('number', 2),
    ('letter', 'b')], [('number', 3), ('letter', 'a')], [('number', 3),
    ('letter', 'b')]]

    I have seen the problem many times but so far I could not google a
    solution on the web.

    By the way do you know any substantial example using FP in Python
    (generators, imap, ifilter, ...)?
    Mark Fink, Dec 22, 2010
    1. Advertisements

  2. Mark Fink

    Peter Otten Guest

    Like this?
    [('special', '+'), ('special', '-'), ('number', 1), ('number', 2),
    ('number', 3), ('letter', 'a'), ('letter', 'b')]

    But these are just glorified for-loops, so:
    [('special', '+'), ('special', '-'), ('number', 1), ('number', 2),
    ('number', 3), ('letter', 'a'), ('letter', 'b')]

    Peter Otten, Dec 22, 2010
    1. Advertisements

  3. Mark Fink

    Mark Fink Guest

    list(chain.from_iterable(starmap(product, izip(izip(dims.iterkeys()),
    so far I have never noticed chain.from_iterable, but many thanks to
    you Peter, I have now a beautiful solution to this problem.
    [('special', '+'), ('number', 2)]
    Mark Fink, Dec 22, 2010
  4. You can also write this as:

    l = (p for c in comb for p in get_products(c))
    Also in your original post you define get_products:
    You could define it as e.g.

    def get_products(keys):
    key_values = [[(k, v) for v in values] for k in keys]
    return it.product(*key_values)

    But maybe for some reason you are trying to avoid list comprehensions
    and generator expressions?
    Arnaud Delobelle, Dec 22, 2010
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.