Re: Elementwise -//- first release -//- Element-wise (vectorized)function, method and operator suppo

Discussion in 'Python' started by Nathan Rice, Dec 21, 2011.

  1. Nathan Rice

    Nathan Rice Guest

    On Tue, Dec 20, 2011 at 7:03 PM, Ian Kelly <> wrote:
    > On Tue, Dec 20, 2011 at 12:45 PM, Nathan Rice
    > <> wrote:
    >> There are still some issues with proper support of things like bool()
    >> and int(), which refuse to return things that are not of the correct
    >> type.

    >
    > And that's a good thing.  As type conversion functions, bool(x) and
    > int(x) should *always* return bools and ints respectively (or raise an
    > exception), no matter what you pass in for x.


    I was hoping to have the proxy be completely transparent. I don't
    disagree with your statement though.

    > If I do "list(efoo)", where efoo is an ElementwiseProxy object, should
    > I expect to get the efoo collection converted to a list, or another
    > ElementwiseProxy where each element has been converted to a list?  I
    > would hope the former.


    Iterators are how you go from an ElementwiseProxy back to a regular
    collection. Thus list/set/etc or anything that takes an iterator will
    work.

    >> This was developed as a proof of concept for expanding the role of
    >> element-wise syntax in python, and to that end I welcome comments.

    >
    > The examples you gave are all numerical in nature.  If I might
    > inquire, what might I use this for that I can't already do with numpy?


    efoo2 = ElementwiseProxy(["one", "two", "three", "four"])

    efoo_res = ((efoo2.capitalize() + " little indian").split("
    ").apply(reversed) * 2).apply("_".join) # note that you could do
    reversed(...) instead, I just like to read left to right
    efoo_res.parent.parent.parent # same as ((efoo2.capitalize() + "
    little indian").split(" ") in case you need to debug something and
    want to look at intermediate values

    The idea is to provide a syntax that lets you do very complex things
    on collections in a more readable manner, without having 5 or 6 lines
    of generator expressions.

    Nathan
    Nathan Rice, Dec 21, 2011
    #1
    1. Advertising

  2. Nathan Rice

    alex23 Guest

    On Dec 21, 10:24 am, Nathan Rice <>
    wrote:
    > The idea is to provide a syntax that lets you do very complex things
    > on collections in a more readable manner, without having 5 or 6 lines
    > of generator expressions.


    Have you seen PyLINQ? It has a similar approach to operating on
    collections, returning a PyLINQ object after each call to facilitate
    chaining. https://github.com/kalessin/PyLINQ/blob/master/pylinq/linq.py

    This is a personal opinion on the code, but I'd move instantiating the
    new ElementwiseProxy out of each method and into its own decorator:

    # declare this outside of the class
    def chainable(fn):
    def _(self, *args, **kwargs):
    return ElementwiseProxy(fn(self, *args, **kwargs), self)
    return _

    This way, each method that is chainable is a little more obvious
    without inspecting the code, and the method body itself is only doing
    what the method says it does:

    @chainable
    def __add__(self, other):
    return (e + other for e in object.__getattribute__(self,
    "iterable"))

    Incidentally, displaying an ElementwiseProxy instance doesn't go down
    well with iPython:

    In [1]: from elementwise import *

    In [2]: e = ElementwiseProxy(['one','two','three'])

    In [3]: e
    Out[3]: ERROR: An unexpected error occurred while tokenizing input
    The following traceback may be corrupted or invalid
    The error message is: ('EOF in multi-line statement', (6, 0))
    alex23, Dec 21, 2011
    #2
    1. Advertising

  3. Nathan Rice

    Nathan Rice Guest

    > Have you seen PyLINQ? It has a similar approach to operating on
    > collections, returning a PyLINQ object after each call to facilitate
    > chaining. https://github.com/kalessin/PyLINQ/blob/master/pylinq/linq.py


    I hadn't seen it previously. I am a VERY heavy user of SQL Alchemy
    though, and I am sure chaining generative ClauseElements/Queries/etc,
    has burned some patterns into my subconscious at this point.

    > This is a personal opinion on the code, but I'd move instantiating the
    > new ElementwiseProxy out of each method and into its own decorator:
    >
    >    # declare this outside of the class
    >    def chainable(fn):
    >        def _(self, *args, **kwargs):
    >            return ElementwiseProxy(fn(self, *args, **kwargs),self)
    >        return _
    >
    > This way, each method that is chainable is a little more obvious
    > without inspecting the code, and the method body itself is only doing
    > what the method says it does:
    >
    >    @chainable
    >    def __add__(self, other):
    >        return (e + other for e in object.__getattribute__(self,
    > "iterable"))


    This is a reasonable suggestion and I will play with something along
    those lines soon.

    > Incidentally, displaying an ElementwiseProxy instance doesn't go down
    > well with iPython:
    >
    > In [1]: from elementwise import *
    >
    > In [2]: e = ElementwiseProxy(['one','two','three'])
    >
    > In [3]: e
    > Out[3]: ERROR: An unexpected error occurred while tokenizing input
    > The following traceback may be corrupted or invalid
    > The error message is: ('EOF in multi-line statement', (6, 0))


    I love IPython, but it has had known problems with iterators for
    years. A long time ago, I agonized over what I thought was a bug in
    my code where itertools.count would skip numbers in IPython, but my
    unit tests all passed. Everything should work fine if you tuple() it
    first.

    Nathan
    Nathan Rice, Dec 21, 2011
    #3
  4. Nathan Rice

    Robert Kern Guest

    On 12/21/11 3:15 PM, Nathan Rice wrote:

    >> Incidentally, displaying an ElementwiseProxy instance doesn't go down
    >> well with iPython:
    >>
    >> In [1]: from elementwise import *
    >>
    >> In [2]: e = ElementwiseProxy(['one','two','three'])
    >>
    >> In [3]: e
    >> Out[3]: ERROR: An unexpected error occurred while tokenizing input
    >> The following traceback may be corrupted or invalid
    >> The error message is: ('EOF in multi-line statement', (6, 0))

    >
    > I love IPython, but it has had known problems with iterators for
    > years. A long time ago, I agonized over what I thought was a bug in
    > my code where itertools.count would skip numbers in IPython, but my
    > unit tests all passed. Everything should work fine if you tuple() it
    > first.


    This is a different problem, actually. The problem is that the recently added
    (by me, actually) pretty-printing system tries to dispatch based on the type. In
    order to handle old-style classes, it checks for the type using .__class__
    first. ElementProxy's __getattribute__() gets in the way here by returning a
    generator instead of the ElementProxy class.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Dec 21, 2011
    #4
  5. Nathan Rice

    Nathan Rice Guest

    On Wed, Dec 21, 2011 at 11:29 AM, Robert Kern <> wrote:
    > On 12/21/11 3:15 PM, Nathan Rice wrote:
    >
    >>> Incidentally, displaying an ElementwiseProxy instance doesn't go down
    >>> well with iPython:
    >>>
    >>> In [1]: from elementwise import *
    >>>
    >>> In [2]: e = ElementwiseProxy(['one','two','three'])
    >>>
    >>> In [3]: e
    >>> Out[3]: ERROR: An unexpected error occurred while tokenizing input
    >>> The following traceback may be corrupted or invalid
    >>> The error message is: ('EOF in multi-line statement', (6, 0))

    >>
    >>
    >> I love IPython, but it has had known problems with iterators for
    >> years.  A long time ago, I agonized over what I thought was a bug in
    >> my code where itertools.count would skip numbers in IPython, but my
    >> unit tests all passed.  Everything should work fine if you tuple() it
    >> first.

    >
    >
    > This is a different problem, actually. The problem is that the recently
    > added (by me, actually) pretty-printing system tries to dispatch based on
    > the type. In order to handle old-style classes, it checks for the type using
    > .__class__ first. ElementProxy's __getattribute__() gets in the way here by
    > returning a generator instead of the ElementProxy class.


    Thanks for the heads up Robert. Given that IPython is awesome and
    people should probably be doing an '.apply(type)' in that instance
    anyhow, I will corner case that on __getattribute__.

    Nathan
    Nathan Rice, Dec 21, 2011
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ago
    Replies:
    3
    Views:
    321
  2. Enrique Cruiz

    Row-wise vs. column-wise image processing

    Enrique Cruiz, Jan 25, 2007, in forum: C Programming
    Replies:
    10
    Views:
    702
    christian.bau
    Jan 26, 2007
  3. Enrique Cruiz
    Replies:
    5
    Views:
    399
    Jim Langston
    Jan 25, 2007
  4. Nathan Rice
    Replies:
    10
    Views:
    344
    Ethan Furman
    Dec 21, 2011
  5. Nathan Rice
    Replies:
    0
    Views:
    147
    Nathan Rice
    Jan 16, 2012
Loading...

Share This Page