Need help porting Perl function

Discussion in 'Python' started by kj, Jun 7, 2008.

  1. kj

    kj Guest

    Hi. I'd like to port a Perl function that does something I don't
    know how to do in Python. (In fact, it may even be something that
    is distinctly un-Pythonic!)

    The original Perl function takes a reference to an array, removes
    from this array all the elements that satisfy a particular criterion,
    and returns the list consisting of the removed elements. Hence
    this function returns a value *and* has a major side effect, namely
    the target array of the original argument will be modified (this
    is the part I suspect may be un-Pythonic).

    Can a Python function achieve the same effect? If not, how would
    one code a similar functionality in Python? Basically the problem
    is to split one list into two according to some criterion.

    TIA!

    Kynn

    --
    NOTE: In my address everything before the first period is backwards;
    and the last period, and everything after it, should be discarded.
     
    kj, Jun 7, 2008
    #1
    1. Advertising

  2. > Hi. I'd like to port a Perl function that does something I don't
    > know how to do in Python. (In fact, it may even be something that
    > is distinctly un-Pythonic!)
    >
    > The original Perl function takes a reference to an array, removes
    > from this array all the elements that satisfy a particular criterion,
    > and returns the list consisting of the removed elements. Hence
    > this function returns a value *and* has a major side effect, namely
    > the target array of the original argument will be modified (this
    > is the part I suspect may be un-Pythonic).
    >
    > Can a Python function achieve the same effect? If not, how would
    > one code a similar functionality in Python? Basically the problem
    > is to split one list into two according to some criterion.


    This function will take a list of integers and modify it in place such
    that it removes even integers. The removed integers are returned as a
    new list (disclaimer: I'm 100% sure it can be done better, more
    optimized, etc, etc):

    def mod( alist ):
    old = alist[:]
    ret = [ ]
    for i in old:
    if i % 2 == 0:
    ret.append( alist.pop( alist.index( i ) ) )

    return ret

    x = range(10,20)

    print x
    r = mod( x )
    print r
    print x

    HTH,
    Daniel
    --
    Psss, psss, put it down! - http://www.cafepress.com/putitdown
     
    Daniel Fetchinson, Jun 7, 2008
    #2
    1. Advertising

  3. kj

    kj Guest

    >This function will take a list of integers and modify it in place such
    >that it removes even integers. The removed integers are returned as a
    >new list

    <snip>

    Great!

    Thanks!

    kynn

    --
    NOTE: In my address everything before the first period is backwards;
    and the last period, and everything after it, should be discarded.
     
    kj, Jun 7, 2008
    #3
  4. kj

    Guest

    On Jun 7, 2:42 pm, "Daniel Fetchinson" <>
    wrote:
    > > Hi.  I'd like to port a Perl function that does something I don't
    > > know how to do in Python.  (In fact, it may even be something that
    > > is distinctly un-Pythonic!)

    >
    > > The original Perl function takes a reference to an array, removes
    > > from this array all the elements that satisfy a particular criterion,
    > > and returns the list consisting of the removed elements.  Hence
    > > this function returns a value *and* has a major side effect, namely
    > > the target array of the original argument will be modified (this
    > > is the part I suspect may be un-Pythonic).

    >
    > > Can a Python function achieve the same effect?  If not, how would
    > > one code a similar functionality in Python?  Basically the problem
    > > is to split one list into two according to some criterion.

    >
    > This function will take a list of integers and modify it in place such
    > that it removes even integers. The removed integers are returned as a
    > new list (disclaimer: I'm 100% sure it can be done better, more
    > optimized, etc, etc):
    >
    > def mod( alist ):
    >     old = alist[:]
    >     ret = [ ]
    >     for i in old:
    >         if i % 2 == 0:
    >             ret.append( alist.pop( alist.index( i ) ) )
    >
    >     return ret
    >
    > x = range(10,20)
    >
    > print x
    > r = mod( x )
    > print r
    > print x
    >
    > HTH,
    > Daniel
    > --
    > Psss, psss, put it down! -http://www.cafepress.com/putitdown


    def mod( alist ):
    return [ alist.pop( alist.index( x ) ) for x in alist if x % 2 ==
    0 ]

    alist = range(10,20)
    blist = mod( alist )

    print alist
    print blist

    The same thing with list comprehensions.
     
    , Jun 7, 2008
    #4
  5. kj

    Sam Denton Guest

    kj wrote:
    > Hi. I'd like to port a Perl function that does something I don't
    > know how to do in Python. (In fact, it may even be something that
    > is distinctly un-Pythonic!)
    >
    > The original Perl function takes a reference to an array, removes
    > from this array all the elements that satisfy a particular criterion,
    > and returns the list consisting of the removed elements. Hence
    > this function returns a value *and* has a major side effect, namely
    > the target array of the original argument will be modified (this
    > is the part I suspect may be un-Pythonic).


    The two solutions thus far use the .index() method, which itself runs in
    O(n) time. This means that the provided functions run in O(n^2) time,
    which can be a problem if the list is big. I'd go with this:

    def partition(alist, criteria):
    list1, list2 = [], []
    for item in alist:
    if criteria(item):
    list1.append(item)
    else:
    list2.append(item)
    return (list1, list2)

    def mod(alist, criteria=lambda x: x % 2 == 0):
    alist[:], blist = partition(alist, criteria)
    return blist


    >>> partition(range(10), lambda x: x % 2 == 0)

    ([0, 2, 4, 6, 8], [1, 3, 5, 7, 9])
    >>> l=range(10)
    >>> mod(l)

    [1, 3, 5, 7, 9]
    >>> l

    [0, 2, 4, 6, 8]
     
    Sam Denton, Jun 7, 2008
    #5
  6. kj

    Paul McGuire Guest

    On Jun 7, 1:24 pm, kj <> wrote:
    > The original Perl function takes a reference to an array, removes
    > from this array all the elements that satisfy a particular criterion,
    > and returns the list consisting of the removed elements.  Hence
    > this function returns a value *and* has a major side effect, namely
    > the target array of the original argument will be modified (this
    > is the part I suspect may be un-Pythonic).
    >
    > Can a Python function achieve the same effect?  If not, how would
    > one code a similar functionality in Python?  Basically the problem
    > is to split one list into two according to some criterion.
    >


    If you want to avoid side-effects completely, return two lists, the
    list of matches and the list of non-matches. In this example,
    partition creates two lists, and stores all matches in the first list,
    and mismatches in the second. (partition assigns to the associated
    element of retlists based on False evaluating to 0 and True evaluating
    to 1.)

    def partition(lst, ifcond):
    retlists = ([],[])
    for i in lst:
    retlists[ifcond(i)].append(i)
    return retlists[True],retlists[False]

    hasLeadingVowel = lambda x: x[0].upper() in "AEIOU"
    matched,unmatched = partition("The quick brown fox jumps over the lazy
    indolent dog".split(),
    hasLeadingVowel)

    print matched
    print unmatched

    prints:
    ['over', 'indolent']
    ['The', 'quick', 'brown', 'fox', 'jumps', 'the', 'lazy', 'dog']

    -- Paul
     
    Paul McGuire, Jun 7, 2008
    #6
  7. kj

    John Machin Guest

    On Jun 8, 6:05 am, wrote:
    > On Jun 7, 2:42 pm, "Daniel Fetchinson" <>
    > wrote:
    >
    >
    >
    > > > Hi. I'd like to port a Perl function that does something I don't
    > > > know how to do in Python. (In fact, it may even be something that
    > > > is distinctly un-Pythonic!)

    >
    > > > The original Perl function takes a reference to an array, removes
    > > > from this array all the elements that satisfy a particular criterion,
    > > > and returns the list consisting of the removed elements. Hence
    > > > this function returns a value *and* has a major side effect, namely
    > > > the target array of the original argument will be modified (this
    > > > is the part I suspect may be un-Pythonic).

    >
    > > > Can a Python function achieve the same effect? If not, how would
    > > > one code a similar functionality in Python? Basically the problem
    > > > is to split one list into two according to some criterion.

    >
    > > This function will take a list of integers and modify it in place such
    > > that it removes even integers. The removed integers are returned as a
    > > new list (disclaimer: I'm 100% sure it can be done better, more
    > > optimized, etc, etc):

    >
    > > def mod( alist ):
    > > old = alist[:]
    > > ret = [ ]
    > > for i in old:
    > > if i % 2 == 0:
    > > ret.append( alist.pop( alist.index( i ) ) )

    >
    > > return ret

    >
    > > x = range(10,20)

    >
    > > print x
    > > r = mod( x )
    > > print r
    > > print x

    >
    > > HTH,
    > > Daniel
    > > --
    > > Psss, psss, put it down! -http://www.cafepress.com/putitdown

    >
    > def mod( alist ):
    > return [ alist.pop( alist.index( x ) ) for x in alist if x % 2 ==
    > 0 ]
    >
    > alist = range(10,20)
    > blist = mod( alist )
    >
    > print alist
    > print blist
    >
    > The same thing with list comprehensions.


    Not the same. The original responder was careful not to iterate over
    the list which he was mutating.

    >>> def mod(alist):

    .... return [alist.pop(alist.index(x)) for x in alist if x % 2 == 0]
    ....
    >>> a = range(10)
    >>> print mod(a), a

    [0, 2, 4, 6, 8] [1, 3, 5, 7, 9]
    >>> a = [2,2,2,2,2,2,2,2]
    >>> print mod(a), a

    [2, 2, 2, 2] [2, 2, 2, 2]
    # should be [2, 2, 2, 2, 2, 2, 2, 2] []
     
    John Machin, Jun 7, 2008
    #7
  8. kj

    Guest

    On Jun 7, 5:56 pm, John Machin <> wrote:
    > On Jun 8, 6:05 am, wrote:
    >
    >
    >
    > > On Jun 7, 2:42 pm, "Daniel Fetchinson" <>
    > > wrote:

    >
    > > > > Hi.  I'd like to port a Perl function that does something I don't
    > > > > know how to do in Python.  (In fact, it may even be something that
    > > > > is distinctly un-Pythonic!)

    >
    > > > > The original Perl function takes a reference to an array, removes
    > > > > from this array all the elements that satisfy a particular criterion,
    > > > > and returns the list consisting of the removed elements.  Hence
    > > > > this function returns a value *and* has a major side effect, namely
    > > > > the target array of the original argument will be modified (this
    > > > > is the part I suspect may be un-Pythonic).

    >
    > > > > Can a Python function achieve the same effect?  If not, how would
    > > > > one code a similar functionality in Python?  Basically the problem
    > > > > is to split one list into two according to some criterion.

    >
    > > > This function will take a list of integers and modify it in place such
    > > > that it removes even integers. The removed integers are returned as a
    > > > new list (disclaimer: I'm 100% sure it can be done better, more
    > > > optimized, etc, etc):

    >
    > > > def mod( alist ):
    > > >     old = alist[:]
    > > >     ret = [ ]
    > > >     for i in old:
    > > >         if i % 2 == 0:
    > > >             ret.append( alist.pop( alist.index( i ) ) )

    >
    > > >     return ret

    >
    > > > x = range(10,20)

    >
    > > > print x
    > > > r = mod( x )
    > > > print r
    > > > print x

    >
    > > > HTH,
    > > > Daniel
    > > > --
    > > > Psss, psss, put it down! -http://www.cafepress.com/putitdown

    >
    > > def mod( alist ):
    > >     return [ alist.pop( alist.index( x ) ) for x in alist if x % 2 ==
    > > 0 ]

    >
    > > alist = range(10,20)
    > > blist = mod( alist )

    >
    > > print alist
    > > print blist

    >
    > > The same thing with list comprehensions.

    >
    > Not the same. The original responder was careful not to iterate over
    > the list which he was mutating.
    >
    > >>> def mod(alist):

    >
    > ...    return [alist.pop(alist.index(x)) for x in alist if x % 2 == 0]
    > ...>>> a = range(10)
    > >>> print mod(a), a

    >
    > [0, 2, 4, 6, 8] [1, 3, 5, 7, 9]>>> a = [2,2,2,2,2,2,2,2]
    > >>> print mod(a), a

    >
    > [2, 2, 2, 2] [2, 2, 2, 2]
    > # should be [2, 2, 2, 2, 2, 2, 2, 2] []


    Alas, it appears my understanding of list comprehensions is
    significantly less comprehensive than I thought =)
     
    , Jun 7, 2008
    #8
  9. kj

    John Machin Guest

    On Jun 8, 8:17 am, wrote:
    > On Jun 7, 5:56 pm, John Machin <> wrote:
    >
    >
    >
    > > On Jun 8, 6:05 am, wrote:

    >
    > > > On Jun 7, 2:42 pm, "Daniel Fetchinson" <>
    > > > wrote:

    >
    > > > > > Hi. I'd like to port a Perl function that does something I don't
    > > > > > know how to do in Python. (In fact, it may even be something that
    > > > > > is distinctly un-Pythonic!)

    >
    > > > > > The original Perl function takes a reference to an array, removes
    > > > > > from this array all the elements that satisfy a particular criterion,
    > > > > > and returns the list consisting of the removed elements. Hence
    > > > > > this function returns a value *and* has a major side effect, namely
    > > > > > the target array of the original argument will be modified (this
    > > > > > is the part I suspect may be un-Pythonic).

    >
    > > > > > Can a Python function achieve the same effect? If not, how would
    > > > > > one code a similar functionality in Python? Basically the problem
    > > > > > is to split one list into two according to some criterion.

    >
    > > > > This function will take a list of integers and modify it in place such
    > > > > that it removes even integers. The removed integers are returned as a
    > > > > new list (disclaimer: I'm 100% sure it can be done better, more
    > > > > optimized, etc, etc):

    >
    > > > > def mod( alist ):
    > > > > old = alist[:]
    > > > > ret = [ ]
    > > > > for i in old:
    > > > > if i % 2 == 0:
    > > > > ret.append( alist.pop( alist.index( i ) ) )

    >
    > > > > return ret

    >
    > > > > x = range(10,20)

    >
    > > > > print x
    > > > > r = mod( x )
    > > > > print r
    > > > > print x

    >
    > > > > HTH,
    > > > > Daniel
    > > > > --
    > > > > Psss, psss, put it down! -http://www.cafepress.com/putitdown

    >
    > > > def mod( alist ):
    > > > return [ alist.pop( alist.index( x ) ) for x in alist if x % 2 ==
    > > > 0 ]

    >
    > > > alist = range(10,20)
    > > > blist = mod( alist )

    >
    > > > print alist
    > > > print blist

    >
    > > > The same thing with list comprehensions.

    >
    > > Not the same. The original responder was careful not to iterate over
    > > the list which he was mutating.

    >
    > > >>> def mod(alist):

    >
    > > ... return [alist.pop(alist.index(x)) for x in alist if x % 2 == 0]
    > > ...>>> a = range(10)
    > > >>> print mod(a), a

    >
    > > [0, 2, 4, 6, 8] [1, 3, 5, 7, 9]>>> a = [2,2,2,2,2,2,2,2]
    > > >>> print mod(a), a

    >
    > > [2, 2, 2, 2] [2, 2, 2, 2]
    > > # should be [2, 2, 2, 2, 2, 2, 2, 2] []

    >
    > Alas, it appears my understanding of list comprehensions is
    > significantly less comprehensive than I thought =)


    It's nothing to do with list comprehensions, which are syntactical
    sugar for traditional loops. You could rewrite your list comprehension
    in the traditional manner:

    def mod(alist):
    ret = []
    for x in alist:
    if x % 2 == 0:
    ret.append(alist.pop(alist.index(x))
    return ret

    and it would still fail for the same reason: mutating the list over
    which you are iterating.

    At the expense of even more time and memory you can do an easy fix:
    change 'for x in alist' to 'for x in alist[:]' so that you are
    iterating over a copy.

    Alternatively, go back to basics:
    >>> def modr(alist):

    .... ret = []
    .... for i in xrange(len(alist) - 1, -1, -1):
    .... if alist % 2 == 0:
    .... ret.append(alist)
    .... del alist
    .... return ret
    ....
    >>> a = [2,2,2,2,2,2,2,2,2]
    >>> print modr(a), a

    [2, 2, 2, 2, 2, 2, 2, 2, 2] []
    >>>


    HTH,
    John
     
    John Machin, Jun 8, 2008
    #9
  10. kj

    kj Guest

    In <> John Machin <> writes:

    >It's nothing to do with list comprehensions, which are syntactical
    >sugar for traditional loops. You could rewrite your list comprehension
    >in the traditional manner...
    >and it would still fail for the same reason: mutating the list over
    >which you are iterating.


    I normally deal with this problem by iterating backwards over the
    indices. Here's how I coded the function (in "Python-greenhorn
    style"):

    def cull(list):
    culled = []
    for i in range(len(list) - 1, -1, -1):
    if not_wanted(list):
    culled.append(list.pop(i))
    return culled

    ....where not_wanted() is defined elsewhere. (For my purposes at the
    moment, the greater generality provided by making not_wanted() a
    parameter to cull() was not necessary.)

    The specification of the indices at the beginning of the for-loop
    looks pretty ignorant, but aside from that I'm happy with it.

    Kynn

    --
    NOTE: In my address everything before the first period is backwards;
    and the last period, and everything after it, should be discarded.
     
    kj, Jun 8, 2008
    #10
  11. kj

    kj Guest

    In <g2h2kb$43g$> kj <> writes:

    >In <> John Machin <> writes:


    >>It's nothing to do with list comprehensions, which are syntactical
    >>sugar for traditional loops. You could rewrite your list comprehension
    >>in the traditional manner...
    >>and it would still fail for the same reason: mutating the list over
    >>which you are iterating.


    >I normally deal with this problem by iterating backwards over the
    >indices. Here's how I coded the function (in "Python-greenhorn
    >style"):


    ***BLUSH***

    Somehow I missed that John Machin had posted almost the exact same
    implementation that I posted in my reply to him.

    Reading too fast. Reading too fast.

    Kynn

    --
    NOTE: In my address everything before the first period is backwards;
    and the last period, and everything after it, should be discarded.
     
    kj, Jun 8, 2008
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andy
    Replies:
    2
    Views:
    295
    Arne Vajhøj
    Jan 17, 2008
  2. Andy
    Replies:
    4
    Views:
    347
  3. jpw
    Replies:
    1
    Views:
    385
    Gabriel Genellina
    Feb 5, 2008
  4. Aqua
    Replies:
    3
    Views:
    160
  5. Mentifex
    Replies:
    2
    Views:
    354
    Mentifex
    Dec 28, 2012
Loading...

Share This Page