The curious behavior of integer objects

Discussion in 'Python' started by Jim B. Wilson, Jan 15, 2007.

  1. Am I nuts? Or only profoundly confused? I expected the this little script
    to print "0":

    class foo(int):
    def __init__(self, value):
    self = value & 0xF

    print foo(0x10)

    Instead, it prints "16" (at least on python 2.4.4 (Linux) and 2.5 (Wine).

    Jim Wilson
    GNV, FL
     
    Jim B. Wilson, Jan 15, 2007
    #1
    1. Advertisements

  2. As it turns out, this has little to do with integers and the
    operations you are trying to do on them. I'll explain in more detail.

    Integers are immutable, which you may already know. This presents a
    problem with subclassing them and using the usual special method
    __init__, because the int object has already been created by this
    point and can not change. Another special method, __new__, is called
    passing the class object itself (foo, in this case) for the first
    argument (traditionally named cls, instead of self). The return of
    this should be an integer which will be the value of your new foo
    int-subclass.

    The following will do as you expected your own example to do.

    class foo(int):
    def __new__(cls, value):
    return value & 0xF

    assert foo(0x10) == 0 # Assertions are much better tests than prints :)
     
    Calvin Spealman, Jan 15, 2007
    #2
    1. Advertisements

  3. Jim B. Wilson

    Robert Kern Guest

    That statement only rebinds the local name self to something else. It does not
    modify the object at all or change the result of instantiating foo(). You need
    to override __new__ to get the behavior that you want.

    http://www.python.org/download/releases/2.2.3/descrintro/#__new__

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Jan 15, 2007
    #3
  4. Integers are immutable. Insert a print statement and you'll see your
    __init__ is never called. Anyway, what would you expect from "self =
    something"?
    Use __new__ instead:

    py> class foo(int):
    .... def __new__(cls, value):
    .... return int(value & 0xF)
    ....
    py> foo(1)
    1
    py> foo(0x10)
    0

    See "Special method names" inside the Python Reference Manual.


    --
    Gabriel Genellina
    Softlab SRL






    __________________________________________________
    Preguntá. Respondé. Descubrí.
    Todo lo que querías saber, y lo que ni imaginabas,
    está en Yahoo! Respuestas (Beta).
    ¡Probalo ya!
    http://www.yahoo.com.ar/respuestas
     
    Gabriel Genellina, Jan 15, 2007
    #4
  5. I dispute that assertion (pun intended).

    Firstly, print statements work even if you pass the -O (optimize) flag
    to Python. Your asserts don't.

    Secondly, a bare assertion like that gives you very little information: it
    just tells you that it failed:
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    AssertionError

    To provide the same information that print provides, you need something
    like this:

    assert x == 0, "x == %s not 0" % x

    Thirdly, and most importantly, assert and print aren't alternatives,
    but complementary tools. Assertions are good for automated testing and
    simple data validation, where you already know what values you should
    have. Printing is good for interactively exploring your data when you're
    uncertain about the values you might get: sometimes it is hard to know
    what you should be asserting until you've seen what results your function
    returns.
     
    Steven D'Aprano, Jan 16, 2007
    #5
  6. This is true, but the concept can be adapted to a things like an
    assert_() function.
    What information would I need to know if my test passed? Nothing, I
    say. I only want to know when the tests fail, especially as I add more
    of them. Having no output is a great way to know nothing puked.
    True, but I intended my statement as a nudge towards more proper
    testing. Even when testing things out, I often use an assert rather
    than a print, to verify some condition.
     
    Calvin Spealman, Jan 16, 2007
    #6
  7. Jim B. Wilson

    Ron Adam Guest

    There have been times where I would like assert to be a little more assertive
    than it is. :)

    ie.. not being able to turn them off with the -0/-00 switches, and having them
    generate a more verbose traceback.

    Maybe have an alternative 'warn' as an alternative debugging version that could
    be set to ignore, soft (print to log), and hard (raise an error).

    An assert statement is a little clearer than an if...raise... in cases where you
    want to raise a value exception of some type. But I'm probably in the minority
    on this one.

    Ron
     
    Ron Adam, Jan 16, 2007
    #7
  8. Hence defeating the optimization :)

    So what you're saying is, some custom function that you write yourself is
    better than print? I suppose I can't argue with that :)

    Maybe. But then it is hard to tell the difference between "my test
    function printed nothing because it is broken, and my test function
    printed nothing because it succeeded".

    There are good arguments both for and against successful tests printing a
    result. Keep in mind that a successful assert doesn't actually print
    "nothing" in the interactive interpreter: when it completes, you do get
    feedback because Python prints a prompt. Getting feedback that the test
    completed successfully is vital. The only question is whether that
    feedback should be minimal or verbose.

    But in any case, you have misunderstood the assertion. The error message
    is printed *if the assert fails*, not if it passes.

    Sure -- for unit testing. For interactive exploration, seeing the value of
    a variable is better than guessing. That's why a bare object reference in
    the interactive interpreter prints itself, but in a script does nothing.
     
    Steven D'Aprano, Jan 16, 2007
    #8
  9. 1) I was trying to nudge the OP towards unit testing
    2) if you look at the printed verson of an object you are as much
    guessing as you would with an assert and no output
    3) It was just an arbitrary statement and although I stand by it, is
    it worth this much discussion?
     
    Calvin Spealman, Jan 16, 2007
    #9
  10. If you want something more verbose, you have it:
    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    AssertionError: verbose traceback

    If you want something that won't be turned off by -O, you maybe need to
    write your own:

    def assert_(condition, msg=""):
    if not condition:
    raise AssertionError(msg)
    Check out the warnings module.
     
    Steven D'Aprano, Jan 16, 2007
    #10
  11. Jim B. Wilson

    Ron Adam Guest

    Yes, I know it's there. It just seems like assert and warn() overlap currently.
    Assert doesn't quite fill warn()'s shoes, and warn() isn't as unassertive as
    assert when it's turned off.

    A warn statement could be completely ignored and not even cost a function call
    when it's turned off like assert does now. I don't think it does that
    currently. Isn't there always some overhead when using the warn() function. Or
    is there some magic there?

    An if-raise is just as fast as assert when evaluating as True, so there's no
    speed advantage there unless you have a large percentage of raises. But it does
    looks a lot nicer when you want a short positive test, verses a longer negative
    test with a raise stuck on it.

    But I think the developers would not consider this to be a big enough matter to
    be worth changing. So I'll continue to ignore warn(), and also continue to be
    un-assertive in my python code.

    Cheers,
    Ron
     
    Ron Adam, Jan 16, 2007
    #11
  12. Jim B. Wilson

    Carl Banks Guest

    Personally, I'd rather see it get less assertive, i.e., having it only
    work in a special debugging mode. That way people who haven't RTFM
    don't use it to make sure their input is correct.


    Carl Banks
     
    Carl Banks, Jan 16, 2007
    #12
  13. Jim B. Wilson

    Ron Adam Guest

    Well, the manual could be improved in this area quite a bit. There also really
    need to be easier to find examples for both assert and warnings use.


    But it does only work in a special debugging mode. Didn't you RTFM? ;-)

    http://docs.python.org/ref/assert.html

    It just happens this mode is turned on by default. So you would like this to be
    turned off by default. I agree. I think there may be a way to change pythons
    default startup behavior for this.


    Warnings generated by warn() on the other hand can be silenced, but not
    completely ignored. But I also think they could be a more useful and flexible
    tool for debugging purposes.

    http://docs.python.org/lib/warning-functions.html

    I have to admit that part of why assert seems wrong to me is the meaning of the
    word implies something you shouldn't be able to ignore. While warnings seem
    like something that can be disregarded.

    I think maybe the best way to use both may be to combine them...

    assert <condition> warn(...)

    But I'm not sure that doesn't have problems of it's own. <shrug>

    Cheers,
    Ron
     
    Ron Adam, Jan 16, 2007
    #13
  14. Jim B. Wilson

    Neil Cerutti Guest

    Experienced C coders expect assert to behave like that.

    The only reason (I know of) to turn off error checking is to
    optimize. However, removing tests won't usually make a big enough
    speed difference to be worth the burthen of testing two different
    versions of the same source code.

    So to me the assert statement is either dubious syntax-sugar or
    dangerous, depending on Python's command line arguments.

    The warning module would seem to have limited applications.
    Searching my Python distribution shows that it's used for
    deprecation alerts, and elsewhere for turning those selfsame
    alerts off. How copacetic! It is the null module. ;-)
     
    Neil Cerutti, Jan 16, 2007
    #14
  15. Jim B. Wilson

    Ron Adam Guest

    Ah... but that's the irony. The whole purpose of the existence of the second
    version you refer to, is to better check the first version. These checks should
    not change the flow of code that is executed!

    The problem with assert as a debugging tool, is it *can* change the execution
    flow by raising a catchable exception. So you do have *two* versions that may
    not behave the same.

    Like I suggested earlier, assert should not be turned off *ever*, but should be
    considered a nicer way to generate exceptions to do value checks.

    And warnings should never change code execution order, but we should be able to
    completely turn them off on the byte code level like asserts are when the -O
    command line option is given.

    I believe both of these features would be used a lot more if they where like this.

    Yep.. unless you use them in a very limited way.

    I think you understand the point I'm trying to make. :)

    Ron
     
    Ron Adam, Jan 16, 2007
    #15
  16. Jim B. Wilson

    Carl Banks Guest

    I don't know about you, but I tend put very expensive checks in assert
    statements (or inside an if __debug__). Stuff like checking one-to-one
    synchronicity between two large trees, checking whether lists are
    sorted, etc. If all you're doing is asserting that x==0, yeah, who
    cares. If you're asserting issorted(list), I think you might want to
    shut that off in production code.

    If used as intended (i.e., to claim that a correct program should
    always meet the condition), it shouldn't be any more dangerous than
    omitting the test altogether.

    The danger comes from using it to check for things that aren't
    indicative of a bug in the program; for instance, to verify input.
    Then the program can crash and burn if optimization is turned on.


    Carl Banks
     
    Carl Banks, Jan 16, 2007
    #16
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.