Bug or Feature?

Discussion in 'Python' started by Stephan Diehl, Nov 24, 2003.

  1. I was playing around with defining new types and have seen the
    following behaviour:

    Python 2.3.1
    <type 'int'>

    Basicly: Int is not closed under it's defined operations. :-(

    by contrast:
    <class '__main__.myset'>

    subclasses of Set are closed under the set operations. :)

    Should I consider this as a bug or something I have to live with?

    Example, why this can be problematic:
    Consider, I want to define an integer subtype that holds integers modulo N.

    The straight forward way would be:

    class mod7(int):
    def __new__(cls,value):
    return int.__new__(cls,value%7)

    Since addition, substraction, etc. doesn't work as expected, one must
    redefine all relevant operators.
    This is not what I'd consider object oriented behaviour.


    P.S.: for the mind police out there: Other than that, Python is great.
    Stephan Diehl, Nov 24, 2003
    1. Advertisements

  2. You want my "innermethods" module. Originally I wrote it as a recipe
    for the cookbook, but then I forgot to post it ;)

    A method of a class ``C`` is said to be an ``inner method`` of ``C``
    if it returns an instance of ``C``. For instance, the ``.replace``
    method of the string class is an inner method of ``str``, since
    invoked by a string, it returns a string:
    r <type 'str'>

    In general, the property of being an inner method is not preserved
    by inheritance. For instance, if I define a subclass of the string class,
    .... pass

    then ``Str`` has still a ``.replace`` method (inherited from ``str``),
    but it is not an inner method of ``Str``:
    r <type 'str'>

    In many circumstances, this is not a problem. However, sometimes
    it is important to keep the inner methods safe under inheritance.
    If we are writing the methods from scratch, this can be done
    by returning ``self.__class__(returnvalue)`` instead of the
    simple ``returnvalue``. However, if the methods have been written
    by others and we are simply inhering them, we need a more sophisticated

    This module provides a ``protect_inner_methods`` metaclass-like function
    which guarantees that the inner methods of the class invoking it are
    safe under inheritance. As an exemplication and a commodoty, the
    module also provides a ``Str`` class with protected versions of
    all the inner methods of the builtin ``str`` class. The only
    exception are ``__str__`` and ``__repr__``: it is better than
    they stay unprotected, so the string representation of subclasses
    of ``str`` is a simple plain string, not an enhanced one. In
    this way ``str(x)`` continues to convert ``x`` instances in ``str``
    instances. If ``Str`` instances are wanted, use ``Str()``!

    Here is an example, showing that ``Str.replace`` is an inner method
    of ``Str``:
    r <class 'innermethods.Str'>

    Let me show that the inner methods of ``Str`` are protected under
    inheritance, i.e. they continues to be inner methods of the
    .... pass
    r <class 'MyStr'>


    def protect_inner__method(cls,methname):
    """Given a class and an inner method name, returns a wrapped
    version of the inner method. Raise an error if the method is not
    defined in any ancestors of the class. In the case of duplicated
    names, the first method in the MRO is taken."""
    def _(self,*args,**kw):
    return self.__class__(supermethod(*args,**kw))
    return _

    def protect_inner_methods(name,bases,dic):
    """Metaclass-like function. Returns a new class with inner methods
    protected under inheritance. The calling class must provide a
    ``__innermethods__`` attribute containing the list of the
    method names to be protected, otherwise nothing will be done."""
    cls=type(name,bases,dic) # generate the new class
    # look for the list of methods to wrap and wrap them all
    for methname in innermethods:
    return cls

    class Str(str):
    """A string class with wrapped inner methods."""
    __innermethods__="""__add__ __mod__ __mul__ __rmod__ __rmul__ capitalize
    center expandtabs join ljust lower lstrip replace rjust rstrip strip
    swapcase title translate upper zfill""".split()
    def __radd__(self,other): # inner method
    """Guarantees ``'a'+Str('b')`` returns a ``Str`` instance.
    Also ``s='s'; s+=Str('b')`` makes ``s`` a ``Str`` instance."""
    return self.__class__(other.__add__(self))
    Michele Simionato, Nov 24, 2003
    1. Advertisements

  3. Yes, thanks, this is most helpfull.

    Although, the question was more along the line, if (in the light of the
    principle of least surprise) this behaviour makes sense.
    My private opinion would be, that the usefullness of builtin types as real
    types would be much better, if they were safe under inheritance.
    Stephan Diehl, Nov 24, 2003
  4. The problem is that it's impossible for the int type to know about the
    requirements of your subclass' constructor. Explicit is better than
    implicit, and all that.

    For the example you posted, I don't see much of an advantage to
    inheriting from int.

    Michael Hudson, Nov 24, 2003
  5. Stephan Diehl

    Duncan Booth Guest

    There are some tradeoffs here.

    I think that the majority of users would say that the speed of operations
    using 'int' is more important than the ease of inheriting from them. If you
    can make the 'int' operations work this way, but keep them at least as fast
    as they are today, then there may be a case for changing the behaviour.

    The other problem though is that this would break backwards compatibility.
    For example if any code were to currently add booleans, it would suddenly
    get a boolean result. The (untested) code below, currently counts how many
    items in a list are greater than a specified value, but under your scheme
    it would return a boolean.

    def HowManyGreaterThan(aList, aValue):
    return reduce(operator.add, [ x > aValue for x in aList ])

    I think what would be most useful would be new subclasses of int etc. which
    have the properties you desire, but you can write them yourself easily

    What rules, BTW, are you proposing for operations between two separate
    subclasses of int? e.g. given your mod7 class, and a corresponding mod9
    class should this print 3, 6, 24 or throw an exception?

    x = mod7(3)
    y = mod9(8)
    print x*y

    and does this do the same?

    print y*x
    Duncan Booth, Nov 24, 2003
  6. O.K., since the stuff I'm concerned about here is of no relevance for
    99.9 % of all python programmers, execution speed is a very valid reason for
    this (and I'll consider this behaviour as a feature :)
    Which, of course, would bring us to the discussion if a bool should be a
    subtype of int (and since this was decided long ago, it should not be
    discussed, of course)
    tough question. I'd probably throw an exception.

    thanks for your input

    Stephan Diehl, Nov 24, 2003
  7. This is probably more a theoretical question, but if I decide to have an
    operation within one specific domain, I want the result in that domain too.
    Otherwise, there would be no point at all to define such a numeric class at

    Stephan Diehl, Nov 24, 2003
  8. On Mon, 24 Nov 2003 18:13:34 +0100, Stephan Diehl

    I understand what you're saying, but there's no universal rule that
    says that the result of operations between members of a set are also
    members of the set. Obvious examples include division on the set of
    integers, and division over the set of real numbers, i.e.division by
    zero is not a real number.
    Dang Griffith, Nov 24, 2003
  9. Um, did you actually read what I said? Again, with emphasis:
    Yes, but now you're constrainting future subclasses' constructors.
    There would be ways to avoid that, I think.
    I think someone has written such a UserInt somewhere...

    It seems to be generally good advice to only subclass builtin types in
    Python if you want to ADD behaviour, not change it. Not 100% true,
    but close.

    Michael Hudson, Nov 24, 2003
  10. If we have a look at the mathematical definition of numbers, the interesting
    thing is not the set of these numbers, but the set combined with some
    usefull operators (like addition and multiplication).
    It is (mathematicaly speaking) not possible to have a result under these
    operations that are outside the definition.

    In that sense, there IS this universal rule, you were talking about. (and
    division by zero is just not allowed by definition).

    Stephan Diehl, Nov 24, 2003
  11. And for the set of whole numbers, some divisions are allowed, but any
    division that results in a fraction would not be allowed. By
    definition, i.e. a combination of operators and operands that result
    in a value not in the same domain is not a valid combination. It
    works, but seems circular. :)
    Dang Griffith, Nov 25, 2003
  12. You might have noticed, I was talking about Mathematics, not the imperfect
    implementation of it in computers :)
    To have division as a meaningfull operation, the set in question must be a
    group under the '*' operation (See for example
    "http://en2.wikipedia.org/wiki/Group_(mathematics)" )
    The point is, that division is the reverse operation to multiplication.

    Cheers, Stephan
    Stephan Diehl, Nov 25, 2003
  13. On Tue, 25 Nov 2003 16:04:57 +0100, Stephan Diehl

    But when talking of "numbers" a group is not enough, you want a field
    - a set with two operations (in some situations something weaker, like
    integral domains, is enough). In fields, such as real numbers, complex
    numbers, p mod integers, p-adic fields, etc. every number has a
    multiplicative inverse *except* the zero.

    Pedantically yours,
    G. Rodrigues
    Gonçalo Rodrigues, Nov 25, 2003
  14. There have been a couple of UserInts floating around.
    I think the most recent was mine.

    The bad news is that my UserInt module actually returns
    ints for all operations.

    The good news is that, being a lazy programmer, I didn't
    actually write my "UserInt" module -- I let my computer
    do it for me, by writing a script which would generate
    the module.

    The main reason I posted the script is because it can
    easily be adapted to create any sort of UserXXX classes,
    as well as to add any sort of behavior mods to those
    classes. I think it would be trivial to create a UserInt
    class which returns other instances of itself for arithmetic
    operations by using the script:


    Patrick Maupin, Nov 25, 2003
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.