Is __mul__ sufficient for operator '*'?

Discussion in 'Python' started by Muhammad Alkarouri, Oct 20, 2009.

  1. Hi everyone,

    I was having a go at a simple implementation of Maybe in Python when I
    stumbled on a case where x.__mul__(y) is defined while x*y is not.

    The class defining x is:

    class Maybe(object):
    def __init__(self, obj):
    self.o = obj
    def __repr__(self):
    return 'Maybe(%s)' % object.__getattribute__(self, "o")
    def __getattribute__(self, name):
    try:
    o = object.__getattribute__(self, "o")
    r = getattr(o,name)
    if callable(r):
    f = lambda *x:Maybe(r(*x))
    return f
    else:
    return Maybe(r)
    except:
    return Maybe(None)

    The code exercising this class is:

    >>> x=Maybe(9)
    >>> x.__mul__(7)

    Maybe(63)
    >>> x*7


    Traceback (most recent call last):
    File "<pyshell#83>", line 1, in <module>
    x*7
    TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'

    The farthest I can go in this is that I presume that __mul__ (as
    called by operator *) is supposed to be a bound method while I am
    returning a lambda function. Is this correct? And How can I make the
    implementation support such operators?

    Cheers,

    Muhammad Alkarouri
     
    Muhammad Alkarouri, Oct 20, 2009
    #1
    1. Advertising

  2. Muhammad Alkarouri

    Gary Herron Guest

    Muhammad Alkarouri wrote:
    > Hi everyone,
    >
    > I was having a go at a simple implementation of Maybe in Python when I
    > stumbled on a case where x.__mul__(y) is defined while x*y is not.
    >
    > The class defining x is:
    >
    > class Maybe(object):
    > def __init__(self, obj):
    > self.o = obj
    > def __repr__(self):
    > return 'Maybe(%s)' % object.__getattribute__(self, "o")
    > def __getattribute__(self, name):
    > try:
    > o = object.__getattribute__(self, "o")
    > r = getattr(o,name)
    > if callable(r):
    > f = lambda *x:Maybe(r(*x))
    > return f
    > else:
    > return Maybe(r)
    > except:
    > return Maybe(None)
    >
    > The code exercising this class is:
    >
    >
    >>>> x=Maybe(9)
    >>>> x.__mul__(7)
    >>>>

    > Maybe(63)
    >
    >>>> x*7
    >>>>

    >
    > Traceback (most recent call last):
    > File "<pyshell#83>", line 1, in <module>
    > x*7
    > TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'
    >


    The product 7*x will execute __mul__, but to catch x*7, you need to
    define __rmul__ (the 'r' stands for reverse or some such).

    However, in fact, you have *not* defined __mul__ in your class. Your
    __getattribute__ is catching __mul__ as an undefined reference, and
    doing something with it -- not sure what though.

    As proof, continue testing: x.__mul__ and x.__rmul__ both return
    values (lambdas defined within __getattribute__) and neither x*7 or 7*x
    work.

    If you want both x* and 7*x to be defined, try
    def __mul__(self,r):
    ...
    def __rmul__(self,r):
    ....

    Or the operation is commutative, perhaps you can get away with reusing
    __mul__ for both.
    def __mul__(self,r):
    ...
    __rmul__ = __mull__


    Gary Herron


    > The farthest I can go in this is that I presume that __mul__ (as
    > called by operator *) is supposed to be a bound method while I am
    > returning a lambda function. Is this correct? And How can I make the
    > implementation support such operators?
    >
    > Cheers,
    >
    > Muhammad Alkarouri
    >
     
    Gary Herron, Oct 20, 2009
    #2
    1. Advertising

  3. Muhammad Alkarouri schrieb:
    > Traceback (most recent call last):
    > File "<pyshell#83>", line 1, in <module>
    > x*7
    > TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'
    >
    > The farthest I can go in this is that I presume that __mul__ (as
    > called by operator *) is supposed to be a bound method while I am
    > returning a lambda function. Is this correct? And How can I make the
    > implementation support such operators?


    It does not so much depend on the function being bound, but on the fact
    that Python expects Maybe.__mul__ to be present already when it tries to
    execute '*'. You could always add this, though:

    def __mul__(self, other):
    return Maybe.__getattribute__(self, "__mul__")(other)

    but of course you'd have to do it for every operator you want to use on
    your Maybe objects. Your use of __getattribute__ OTH suggests you're
    trying to avoid exactly that. I'd rather go for a monadic implementation
    with unit, bind and lift operations, e.g:

    ----8<--------8<--------8<--------8<--------8<----

    class Monad(object):
    # unit:
    def __init__(self, value):
    self.value = value
    # bind:
    def __rshift__(self, function):
    return function(self.__class__)(self.value)
    def __str__(self):
    return "%s(%s)" % (self.__class__.__name__, self.value)

    def lift(f):
    def lift_unit(m):
    def lift_bind(x):
    return m(f(x))
    return lift_bind
    return lift_unit

    class idem(Monad):
    def __call__(self, value):
    return value

    class Nothing(object):
    def __rshift__(self, function):
    return Nothing
    def __str__(self):
    return "Nothing"
    Nothing = Nothing()

    class Maybe(Monad):
    def __new__(cls, value=Nothing):
    if value is Nothing:
    return Nothing
    return super(Monad, cls).__new__(cls)

    if __name__ == "__main__":

    x = Maybe(9)
    print x >> lift(lambda v: v * 7)
    print x >> lift(lambda v: v * 7) >> idem

    y = Maybe(Nothing)
    print y >> lift(lambda v: v * 7)
    print y >> lift(lambda v: v * 7) >> idem

    ----8<--------8<--------8<--------8<--------8<----

    While I can see how this monadic stuff is usefull in Haskell et al.,
    I'm still not sure how to apply it to Python. And for the impenetrable
    mathematical language in which Monads are usually presented, well...


    HTH,
    Mick.
     
    Mick Krippendorf, Oct 20, 2009
    #3
  4. En Mon, 19 Oct 2009 21:31:44 -0300, Muhammad Alkarouri
    <> escribió:

    > I was having a go at a simple implementation of Maybe in Python when I
    > stumbled on a case where x.__mul__(y) is defined while x*y is not.


    __special__ methods are searched in the type, not in the instance
    directly. x*y looks for type(x).__mul__ (among other things)


    --
    Gabriel Genellina
     
    Gabriel Genellina, Oct 20, 2009
    #4
  5. Gabriel Genellina schrieb:
    > __special__ methods are searched in the type, not in the instance
    > directly. x*y looks for type(x).__mul__ (among other things)


    So I thought too, but:

    class meta(type):
    def __mul__(*args):
    return 123

    class boo(object):
    __metaclass__ = meta

    print boo.__mul__

    b = boo()
    print b * 7

    also explodes. Or am I misinterpreting the word "type" here?


    Mick.
     
    Mick Krippendorf, Oct 20, 2009
    #5
  6. En Tue, 20 Oct 2009 00:59:12 -0300, Mick Krippendorf <>
    escribió:
    > Gabriel Genellina schrieb:


    >> __special__ methods are searched in the type, not in the instance
    >> directly. x*y looks for type(x).__mul__ (among other things)

    >
    > So I thought too, but:
    >
    > class meta(type):
    > def __mul__(*args):
    > return 123
    >
    > class boo(object):
    > __metaclass__ = meta
    >
    > print boo.__mul__
    >
    > b = boo()
    > print b * 7
    >
    > also explodes. Or am I misinterpreting the word "type" here?


    This is by design; see
    http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes

    Methods defined on the meta-type aren't considered methods of the type;
    otherwise, every object would have the methods defined in `type` itself,
    because this is the metatype of every other object.

    When I said "x*y looks for type(x).__mul__" the search for '__mul__' is
    done in type(x) itself, and all its base types along the MRO - NOT on the
    metatype, and not using getattr. There is no way to express this exact
    search in Python code (that I know of), but it's more or less like
    searching for '__mul__' in dir(type(x)).

    In particular, __mul__ is stored in two slots (a slot is a field in a
    structure holding function pointers: nb_multiply in the tp_as_number
    structure, *and* sq_repeat in tp_as_sequence); defining or assigning to
    __mul__ "magically" updates those internal pointers. When executing x*y,
    __mul__ is retrieved directly from those pointers -- it is *not* searched
    by name.

    In short, you have to define the __mul__ method on the type itself or any
    of its bases.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Oct 20, 2009
    #6
  7. Mick Krippendorf, Oct 20, 2009
    #7
  8. Muhammad Alkarouri schrieb:
    > I was having a go at a simple implementation of Maybe in Python when I
    > stumbled on a case where x.__mul__(y) is defined while x*y is not.
    >
    > class Maybe(object):
    > def __init__(self, obj):
    > self.o = obj
    > def __repr__(self):
    > return 'Maybe(%s)' % object.__getattribute__(self, "o")
    > def __getattribute__(self, name):
    > try:
    > o = object.__getattribute__(self, "o")
    > r = getattr(o,name)
    > if callable(r):
    > f = lambda *x:Maybe(r(*x))
    > return f
    > else:
    > return Maybe(r)
    > except:
    > return Maybe(None)
    >
    >>>> x=Maybe(9)
    >>>> x.__mul__(7)

    > Maybe(63)
    >>>> x*7

    >
    > Traceback (most recent call last):
    > File "<pyshell#83>", line 1, in <module>
    > x*7
    > TypeError: unsupported operand type(s) for *: 'Maybe' and 'int'


    Here's how I'd do it. It will not win a beauty contest any time soon,
    but at least it's a workaround:

    ----8<--------8<--------8<--------8<--------8<--------8<--------8<----

    def meta(lift, lifted, not_lifted=[]):
    not_lifted = list(not_lifted) + object.__dict__.keys()
    class MetaMaybe(type):
    def __new__(meta, mcls, bases, dct):
    dct.update(
    (name, lift(name))
    for name in set(lifted) - set(not_lifted)
    )
    return type(mcls, bases, dct)
    return MetaMaybe

    class Nothing(object):
    __metaclass__ = meta(lambda name: lambda self, *a, **k: self, (
    "__add__", "__sub__", "__mul__", "__div__", "__truediv__",
    "__floordiv__", "__divmod__", "__radd__", "__rsub__",
    "__rmul__", "__rdiv__", "__rtruediv__", "__rfloordiv__",
    "__rdivmod__", "__rshift__", "__lshift__", "__call__",
    # and so on, for every special method that Nothing knows
    ))
    def __new__(cls, value=None):
    try: # singleton
    return cls.value
    except AttributeError:
    cls.value = super(Nothing, cls).__new__(cls)
    return cls.value
    def __str__(self):
    return "Nothing"
    __repr__ = __str__

    Nothing = Nothing()

    def just(vcls):
    def lifter(name):
    attr = getattr(vcls, name)
    def lifted(self, *ms):
    try:
    return self.lift(attr)(self, *ms)
    except:
    return Nothing
    return lifted
    class Just(object):
    __metaclass__ = meta(lifter, vcls.__dict__.keys())
    def __new__(cls, value):
    if value in (Nothing, NotImplemented):
    return Nothing
    return super(Just, cls).__new__(cls)
    def __init__(self, value):
    self.value = value
    def __str__(self):
    return "Just(%s)" % self.value
    @classmethod
    def lift(c, f):
    return lambda *ms:c(f(*(m.value for m in ms)))
    return Just

    from collections import defaultdict

    class TypeDict(defaultdict):
    def __missing__(self, key):
    if self.default_factory is None:
    raise KeyError(key)
    return self.default_factory(key)

    class Maybe(object):
    typemap = TypeDict(just)
    def __new__(cls, value):
    return Maybe.typemap[value.__class__](value)

    def foo(x, y):
    return x * 2 + y * 3

    if __name__ == "__main__":

    print Maybe(Nothing)
    print Maybe(1) / Maybe(0)
    print Maybe(10.) * Maybe(5) / Maybe(2) ** Maybe(3)
    print Maybe(foo)(Maybe(6), Maybe(10))
    print Maybe("hello").upper()
    print Maybe("hello").startswith(Maybe("h"))
    print getattr(Maybe("hello"), "startswith")(Maybe("h"))
    print Maybe(foo)(Maybe("hello! "), Maybe("what? "))
    print Maybe(foo)(Maybe("hello! "), Nothing)

    ----8<--------8<--------8<--------8<--------8<--------8<--------8<----

    I haven't tested it very thoroughly, so it's quite possible there are
    lots of bugs in it, but it is only intended as a demo.

    As Gabriel Genellina pointed out, the search for special methods is done
    in the type, so we have to put our own versions there, during type
    creation in the metaclass' __new__ method. The above code does this for
    all methods of the wrapped object's type, not just special ones, and
    "lifts" them to expect Maybe objects instead of "normal" objects. They
    also wrap their return values into Maybe objects.

    Maybe is an algebraic data type. The call Maybe(some_value) returns
    either Nothing, if some_value happens to be Nothing, or else an object
    of type Just that wraps some_value. More precisely, there is not one
    type Just, but as many as types of Just-wrapped objects (Just<int>,
    Just<string>, ...). Just is therefore a kind of parameterized type. It's
    similar to inheriting from a C++ template parameter type:

    template <class T>
    class MyType : public T {...}

    but not quite, since in C++ the inherited member functions' signatures
    are unchanged, whereas in the above code "inherited" methods are changed
    to expect and return Maybe objects.

    Some things don't work, though, e.g. slicing. But this could be
    implemented via specialized lifting functions for the __XXXitem__
    methods. One thing that does work though, is that ordinary functions can
    be wrapped as Maybe objects which then "do the same thing" on other
    Maybe objects that the normal functions do on normal objects, like in
    the foo-example. So it's quite close to a monadic version, in that it
    "lifts" objects and functions from one type space into another one, the
    Maybe space. But compared to a real Monads it's much more pythonic, IMO.


    HTH,
    Mick.
     
    Mick Krippendorf, Oct 26, 2009
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ali
    Replies:
    0
    Views:
    4,432
  2. Replies:
    11
    Views:
    696
  3. Thomas Philips

    Using __mul__

    Thomas Philips, Jul 3, 2004, in forum: Python
    Replies:
    2
    Views:
    382
    Peter Otten
    Jul 3, 2004
  4. Gerard Flanagan

    Using '__mul__' within a class

    Gerard Flanagan, Sep 24, 2005, in forum: Python
    Replies:
    12
    Views:
    712
    John J. Lee
    Sep 25, 2005
  5. dmitrey
    Replies:
    1
    Views:
    946
    Terry Reedy
    Dec 11, 2009
Loading...

Share This Page