@property decorator doesn't raise exceptions

Discussion in 'Python' started by Rafe, Oct 24, 2008.

  1. Rafe

    Rafe Guest

    Hi,

    I've encountered a problem which is making debugging less obvious than
    it should be. The @property decorator doesn't always raise exceptions.
    It seems like it is bound to the class but ignored when called. I can
    see the attribute using dir(self.__class__) on an instance, but when
    called, python enters __getattr__. If I correct the bug, the attribute
    calls work as expected and do not call __getattr__.

    I can't seem to make a simple repro. Can anyone offer any clues as to
    what might cause this so I can try to prove it?


    Cheers,

    - Rafe
     
    Rafe, Oct 24, 2008
    #1
    1. Advertising

  2. Rafe

    Rafe Guest

    On Oct 24, 2:21 am, Christian Heimes <> wrote:
    > Rafewrote:
    > > Hi,

    >
    > > I've encountered a problem which is making debugging less obvious than
    > > it should be. The @property decorator doesn't always raise exceptions.
    > > It seems like it is bound to the class but ignored when called. I can
    > > see the attribute using dir(self.__class__) on an instance, but when
    > > called, python enters __getattr__. If I correct the bug, the attribute
    > > calls work as expected and do not call __getattr__.

    >
    > > I can't seem to make a simple repro. Can anyone offer any clues as to
    > > what might cause this so I can try to prove it?

    >
    > You must subclass from "object" to get a new style class. properties
    > don't work correctly on old style classes.
    >
    > Christian


    All classes are a sub-class of object. Any other ideas?

    - Rafe
     
    Rafe, Oct 24, 2008
    #2
    1. Advertising

  3. Rafe

    Peter Otten Guest

    Rafe wrote:

    > On Oct 24, 2:21 am, Christian Heimes <> wrote:
    >> Rafewrote:
    >> > Hi,

    >>
    >> > I've encountered a problem which is making debugging less obvious than
    >> > it should be. The @property decorator doesn't always raise exceptions.
    >> > It seems like it is bound to the class but ignored when called. I can
    >> > see the attribute using dir(self.__class__) on an instance, but when
    >> > called, python enters __getattr__. If I correct the bug, the attribute
    >> > calls work as expected and do not call __getattr__.

    >>
    >> > I can't seem to make a simple repro. Can anyone offer any clues as to
    >> > what might cause this so I can try to prove it?

    >>
    >> You must subclass from "object" to get a new style class. properties
    >> don't work correctly on old style classes.
    >>
    >> Christian

    >
    > All classes are a sub-class of object. Any other ideas?


    Hard to tell when you don't give any code.

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError
    .... def __getattr__(self, name):
    .... return "nobody expects the spanish inquisition"
    ....
    >>> A().attribute

    'nobody expects the spanish inquisition'

    Do you mean something like this? I don't think the __getattr__() call can be
    avoided here.

    Peter
     
    Peter Otten, Oct 24, 2008
    #3
  4. On Fri, 24 Oct 2008 09:34:36 -0700, Rafe wrote:

    >> You must subclass from "object" to get a new style class. properties
    >> don't work correctly on old style classes.
    >>
    >> Christian

    >
    > All classes are a sub-class of object. Any other ideas?


    Only in Python 3. If you are relying on that to be true in Python 2.x,
    you're going to disappointed:

    >>> class A():

    .... pass
    ....
    >>> issubclass(A, object)

    False


    There are a lot of classic classes in the standard library. If you
    inherit from them, your class will also be a classic class and properties
    will fail to work correctly.


    Earlier, you wrote:

    "I've encountered a problem which is making debugging less obvious than
    it should be. The @property decorator doesn't always raise exceptions."

    Are you expecting it to raise an exception when the class is defined, or
    when the property is called? e.g.

    class Fail(object):
    @property
    "this should raise an exception"

    Works for me -- I get a syntax error, as expected.


    "It seems like it is bound to the class but ignored when called. I can
    see the attribute using dir(self.__class__) on an instance, but when
    called, python enters __getattr__. If I correct the bug, the attribute
    calls work as expected and do not call __getattr__."


    Perhaps you can give us a minimal example showing the code with the bug,
    then how you corrected the bug? That might give as a hint as to what is
    going on. I know that's hard, when you can't reproduce the problem, but
    it's just as hard for us to solve your problem without more information.





    --
    Steven
     
    Steven D'Aprano, Oct 25, 2008
    #4
  5. On Fri, 24 Oct 2008 01:47:10 -0700, Rafe wrote:

    > Hi,
    >
    > I've encountered a problem which is making debugging less obvious than
    > it should be. The @property decorator doesn't always raise exceptions.


    I don't think that's the problem. I think properties do correctly raise
    all exceptions that occur inside them, at least built-in exceptions.

    I wrote a test that raises exceptions inside properties and they (almost)
    all are raised normally.

    Whatever your problem is, it isn't that properties don't raise exceptions.

    For those who are interested, the test program I used follows.


    # test module

    def get_exceptions():
    import __builtin__
    list_of_exceptions = []
    for key in dir(__builtin__):
    # do not use __builtins__ -- note the 's'.
    obj = getattr(__builtin__, key)
    if not isinstance(obj, type):
    continue
    if issubclass(obj, Exception):
    list_of_exceptions.append(obj)
    return list_of_exceptions

    def builder(list_of_exceptions):
    class PropertyTester(object):
    """Do properties raise exceptions? Let's find out."""
    for exception in list_of_exceptions:
    name = exception.__name__
    @property
    def func(self, exception=exception):
    # Make sure we bind a local copy to exception.
    raise exception
    setattr(PropertyTester, name, func)
    return PropertyTester()

    def testRaises(obj, exception_type, verbose):
    """Test that appropriate exception is raised when accessing an
    attribute."""
    name = exception_type.__name__
    try:
    getattr(obj, name)
    except exception_type:
    if verbose:
    print "Passed: expected exception %s raised correctly" \
    % exception_type
    except Exception, e:
    print "Failed: expected %s but got %r" % (exception_type, e)
    else:
    print "Failed: expected %s but got no exception at all" \
    % exception_type

    if __name__ == '__main__':
    import sys
    if sys.argv[1:] == ['verbose']:
    verbose = True
    else:
    verbose = False
    exceptions = get_exceptions()
    tester = builder(exceptions)
    for exception in exceptions:
    testRaises(tester, exception, verbose)



    --
    Steven
     
    Steven D'Aprano, Oct 25, 2008
    #5
  6. Rafe

    Rafe Guest

    On Oct 24, 9:58 am, Peter Otten <> wrote:
    > Rafe wrote:
    > > On Oct 24, 2:21 am, Christian Heimes <> wrote:
    > >> Rafewrote:
    > >> > Hi,

    >
    > >> > I've encountered a problem which is making debugging less obvious than
    > >> > it should be. The @property decorator doesn't always raise exceptions.
    > >> > It seems like it is bound to the class but ignored when called. I can
    > >> > see the attribute using dir(self.__class__) on an instance, but when
    > >> > called, python enters __getattr__. If I correct the bug, the attribute
    > >> > calls work as expected and do not call __getattr__.

    >
    > >> > I can't seem to make a simple repro. Can anyone offer any clues as to
    > >> > what might cause this so I can try to prove it?

    >
    > >> You must subclass from "object" to get a new style class. properties
    > >> don't work correctly on old style classes.

    >
    > >> Christian

    >
    > > All classes are a sub-class of object. Any other ideas?

    >
    > Hard to tell when you don't give any code.
    >
    > >>> class A(object):

    >
    > ...     @property
    > ...     def attribute(self):
    > ...             raise AttributeError
    > ...     def __getattr__(self, name):
    > ...             return "nobody expects the spanish inquisition"
    > ...>>> A().attribute
    >
    > 'nobody expects the spanish inquisition'
    >
    > Do you mean something like this? I don't think the __getattr__() call can be
    > avoided here.
    >
    > Peter


    You nailed it Peter! I thought __getattr__ was a symptom, not the
    cause of the misleading errors. Here is the repro (pretty much
    regurgitated):

    The expected behavior...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    >>> A().attribute

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in attribute
    AttributeError: Correct Error.


    The unexpected and misleading exception...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    .... def __getattr__(self, name):
    .... cls_name = self.__class__.__name__
    .... msg = "%s has no attribute '%s'." % (cls_name, name)
    .... raise AttributeError(msg)
    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in __getattr__
    AttributeError: A has no attribute 'attribute'.


    The docs state:
    "Called when an attribute lookup has not found the attribute in the
    usual places (i.e. it is not an instance attribute nor is it found in
    the class tree for self). name is the attribute name. This method
    should return the (computed) attribute value or raise an
    AttributeError exception."

    Can anyone explain why this is happening? I can hack a work-around,
    but even then I could use some tips on how to raise the 'real'
    exception so debugging isn't guesswork.


    Cheers,

    - Rafe
     
    Rafe, Oct 25, 2008
    #6
  7. Rafe

    Rafe Guest

    On Oct 24, 9:58 am, Peter Otten <> wrote:
    > Rafe wrote:
    > > On Oct 24, 2:21 am, Christian Heimes <> wrote:
    > >> Rafewrote:
    > >> > Hi,

    >
    > >> > I've encountered a problem which is making debugging less obvious than
    > >> > it should be. The @property decorator doesn't always raise exceptions.
    > >> > It seems like it is bound to the class but ignored when called. I can
    > >> > see the attribute using dir(self.__class__) on an instance, but when
    > >> > called, python enters __getattr__. If I correct the bug, the attribute
    > >> > calls work as expected and do not call __getattr__.

    >
    > >> > I can't seem to make a simple repro. Can anyone offer any clues as to
    > >> > what might cause this so I can try to prove it?

    >
    > >> You must subclass from "object" to get a new style class. properties
    > >> don't work correctly on old style classes.

    >
    > >> Christian

    >
    > > All classes are a sub-class of object. Any other ideas?

    >
    > Hard to tell when you don't give any code.
    >
    > >>> class A(object):

    >
    > ...     @property
    > ...     def attribute(self):
    > ...             raise AttributeError
    > ...     def __getattr__(self, name):
    > ...             return "nobody expects the spanish inquisition"
    > ...>>> A().attribute
    >
    > 'nobody expects the spanish inquisition'
    >
    > Do you mean something like this? I don't think the __getattr__() call can be
    > avoided here.
    >
    > Peter



    Peter nailed it, thanks! I thought __getattr__ was a symptom, not a
    cause of the misleading exceptions. Here is a complete repro:


    The expected behavior...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    >>> A().attribute

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in attribute
    AttributeError: Correct Error.


    The misleading/unexpected behavior...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    .... def __getattr__(self, name):
    .... cls_name = self.__class__.__name__
    .... msg = "%s has no attribute '%s'." % (cls_name, name)
    .... raise AttributeError(msg)
    >>> A().attribute

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in __getattr__
    AttributeError: A has no attribute 'attribute'.


    Removing @property works as expected...

    >>> class A(object):

    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    .... def __getattr__(self, name):
    .... cls_name = self.__class__.__name__
    .... msg = "%s has no attribute '%s'." % (cls_name, name)
    .... raise AttributeError(msg)
    >>> A().attribute() # Note the '()'

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in attribute
    AttributeError: Correct Error.


    The docs seem to suggest this is impossible:
    "Called when an attribute lookup has not found the attribute in the
    usual places (i.e. it is not an instance attribute nor is it found in
    the class tree for self). name is the attribute name. This method
    should return the (computed) attribute value or raise an
    AttributeError exception."

    Can anyone explain why this is happening? Is it a bug? I can write a
    workaround to detect this by comparing the attribute name passed
    __getattr__ with dir(self.__class__) = self.__dict__.keys(), but how
    can I raise the expected exception?


    Thanks,

    - Rafe
     
    Rafe, Oct 25, 2008
    #7
  8. Rafe

    greg Guest

    Rafe wrote:

    > The docs seem to suggest this is impossible:
    > "Called when an attribute lookup has not found the attribute in the
    > usual places (i.e. it is not an instance attribute nor is it found in
    > the class tree for self).


    Getting an AttributeError is the way that the interpreter
    machinery tells that the attribute wasn't found. So when
    your property raises an AttributeError, this is
    indistinguishable from the case where the property wasn't
    there at all.

    To avoid this you would have to raise some exception
    that doesn't derive from AttributeError.

    --
    Greg
     
    greg, Oct 26, 2008
    #8
  9. Rafe

    Rafe Guest

    On Oct 24, 1:47 am, Rafe <> wrote:
    > Hi,
    >
    > I've encountered a problem which is making debugging less obvious than
    > it should be. The @property decorator doesn't always raise exceptions.
    > It seems like it is bound to the class but ignored when called. I can
    > see the attribute using dir(self.__class__) on an instance, but when
    > called, python enters __getattr__. If I correct the bug, the attribute
    > calls work as expected and do not call __getattr__.
    >
    > I can't seem to make a simple repro. Can anyone offer any clues as to
    > what might cause this so I can try to prove it?
    >
    > Cheers,
    >
    > - Rafe



    Peter Oten pointed me in the right direction. I tried to reply to his
    post 2 times and in spite of GoogleGroups reporting the post was
    successful, it never showed up. Here is the repro:

    The expected behavior...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    >>> A().attribute

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in attribute
    AttributeError: Correct Error.


    The misleading/unexpected behavior...

    >>> class A(object):

    .... @property
    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    .... def __getattr__(self, name):
    .... cls_name = self.__class__.__name__
    .... msg = "%s has no attribute '%s'." % (cls_name, name)
    .... raise AttributeError(msg)
    >>> A().attribute

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in __getattr__
    AttributeError: A has no attribute 'attribute'.


    Removing @property works as expected...

    >>> class A(object):

    .... def attribute(self):
    .... raise AttributeError("Correct Error.")
    .... def __getattr__(self, name):
    .... cls_name = self.__class__.__name__
    .... msg = "%s has no attribute '%s'." % (cls_name, name)
    .... raise AttributeError(msg)
    >>> A().attribute() # Note the '()'

    Traceback (most recent call last):
    File "<console>", line 0, in <module>
    File "<console>", line 0, in attribute
    AttributeError: Correct Error.


    I never suspected __getattr__ was the cause and not just a symptom.
    The docs seem to indicate __gettattr__ should never be called when the
    attribute exisits in the class:
    "Called when an attribute lookup has not found the attribute in the
    usual places (i.e. it is not an instance attribute nor is it found in
    the class tree for self). name is the attribute name. This method
    should return the (computed) attribute value or raise an
    AttributeError exception."

    Is this a bug? Any idea why this happens? I can write a hack in to
    __getattr__ in my class which will detect this, but I'm not sure how
    to raise the expected exception.


    Cheers,

    - Rafe
     
    Rafe, Oct 26, 2008
    #9
  10. Rafe

    Peter Otten Guest

    Rafe wrote:

    > Can anyone explain why this is happening?


    When an attribute error is raised that is an indication that the requested
    attribute doesn't exist, and __getattr__() must be called as a fallback.

    > I can hack a work-around,
    > but even then I could use some tips on how to raise the 'real'
    > exception so debugging isn't guesswork.


    Look at the problem again, maybe you can find a solution without
    __getattr__() and use only properties.

    Otherwise you have to wrap your getter with something like

    try:
    ...
    except AttributeError:
    raise BuggyProperty, None, original_traceback


    If you put that functionality into a decorator you get:

    import sys

    class BuggyProperty(Exception):
    pass

    def safe_getter(get):
    def safe_get(self):
    try:
    return get(self)
    except AttributeError:
    t, e, tb = sys.exc_info()
    raise BuggyProperty("AttributeError in getter %s(); "
    "giving original traceback"
    % get.__name__), None, tb
    return property(safe_get)

    class A(object):
    @safe_getter
    def attr(self):
    return self.m(3)

    def m(self, n):
    if n > 0:
    return self.m(n-1)
    raise AttributeError("it's a bug")

    def __getattr__(self, name):
    return "<%s>" % name

    A().attr

    Peter
     
    Peter Otten, Oct 27, 2008
    #10
  11. Rafe

    Rafe Guest

    OT... Sorry about he spam.
    Thanks for taking the time to post this Duncan.

    I had the same thought. I have posted to this list before but never
    experienced anything like this wait. I figured it was possible that I
    hit "Reply to Author" the first time so I sent it again. I waited
    about 8 hours before sending the third time (and I posted to
    GoogleGroups support as well). Even then I didn't see my original
    post. I'm surprised it took so long to update. Next time I'll just
    make sure the post was made successfully and wait...as long as it
    takes.


    Cheers,

    - Rafe


    On Oct 26, 4:23 pm, Duncan Booth <> wrote:
    > Rafe<> wrote:
    > > Peter Oten pointed me in the right direction. I tried to reply to his
    > > post 2 times and in spite of GoogleGroups reporting the post was
    > > successful, it never showed up.

    >
    > This is the third variant on your message that has shown up in the
    > newsgroup.
    >
    > Please be aware that messages take time to propogate through usenet: don't
    > repost just because Google groups hasn't yet got around to displaying your
    > message. If it says the post was successful then the post was successful.
    > Just be patient a bit longer for it to become visible to you.
     
    Rafe, Oct 29, 2008
    #11
  12. Rafe

    Rafe Guest

    On Oct 27, 2:47 pm, Peter Otten <> wrote:
    > Rafewrote:
    > > Can anyone explain why this is happening?

    >
    > When an attribute error is raised that is an indication that the requested
    > attribute doesn't exist, and __getattr__() must be called as a fallback.
    >
    > > I can hack a work-around,
    > > but even then I could use some tips on how to raise the 'real'
    > > exception so debugging isn't guesswork.

    >
    > Look at the problem again, maybe you can find a solution without
    > __getattr__() and use only properties.
    >
    > Otherwise you have to wrap your getter with something like
    >
    > try:
    >     ...
    > except AttributeError:
    >     raise BuggyProperty, None, original_traceback
    >
    > If you put that functionality into a decorator you get:
    >
    > import sys
    >
    > class BuggyProperty(Exception):
    >     pass
    >
    > def safe_getter(get):
    >     def safe_get(self):
    >         try:
    >             return get(self)
    >         except AttributeError:
    >             t, e, tb = sys.exc_info()
    >             raise BuggyProperty("AttributeError in getter %s(); "
    >                             "giving original traceback"
    >                             % get.__name__), None, tb
    >     return property(safe_get)
    >
    > class A(object):
    >     @safe_getter
    >     def attr(self):
    >         return self.m(3)
    >
    >     def m(self, n):
    >         if n > 0:
    >             return self.m(n-1)
    >         raise AttributeError("it's a bug")
    >
    >     def __getattr__(self, name):
    >         return "<%s>" % name
    >
    > A().attr
    >
    > Peter



    Thanks for the idea Peter. What confuses me is why this only happens
    to @Property (and I assume other decorator related bindings?). Does it
    have something to do with the way the class gets built? 'Normal'
    attributes will raise AttributeErrors as expected, without triggering
    __getattr__(). Considering this is a built-in decorator, it would be
    nice if this behavior was fixed if possible.

    Unfortunately, I need __getattr__() because my class is a wrapper (it
    is delegating calls to another object when attributes aren't found in
    the class). As a hack, I am testing the passed attr name against the
    instance, class and super-class attributes. If there is a match, I
    assume it is an error and raise an exception which points the
    developer in the right direction. It isn't ideal, but it helps.

    You're code is better, as it displays the 'real' traceback, but I need
    to know more about the effects of using an exception which is not an
    AttrbiuteError. Which brings me to my next question...

    In case it isn't obvious, I'm fairly new to Python (and this level of
    programming in general). I've been wondering about the optimal use of
    custom exceptions. Up until now, I've been sticking to the built-in
    exceptions, which seem to work in 90% of situations. Are you aware of
    any resources which talk about this aspect of programming (more about
    theory than code)?


    Thanks again,

    - Rafe
     
    Rafe, Oct 29, 2008
    #12
  13. Rafe

    Peter Otten Guest

    Rafe wrote:

    > Thanks for the idea Peter. What confuses me is why this only happens
    > to @Property (and I assume other decorator related bindings?). Does it
    > have something to do with the way the class gets built? 'Normal'
    > attributes will raise AttributeErrors as expected, without triggering
    > __getattr__(). Considering this is a built-in decorator, it would be
    > nice if this behavior was fixed if possible.


    Normal attributes either exist, and then they don't raise an AttributeError,
    or they don't exist, and then __getattr__() *is* triggered. The problem has
    nothing to do with decorators. It is just that properties invoke custom
    code, and Python currently has no way of finding out whether an
    AttributeError was raised accidentally by a buggy getter or whether it is
    meant to signal that the attribute wasn't found in the class hierarchy and
    now should be calculated by __getattr__().

    I guess (without looking into the C source) that it could be changed to meet
    your intuition but that it would complicate the implementation.

    Peter
     
    Peter Otten, Oct 29, 2008
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jacol

    raise or not to raise [Newbie]

    Jacol, Feb 3, 2007, in forum: Python
    Replies:
    5
    Views:
    407
    Gabriel Genellina
    Feb 5, 2007
  2. Phlip
    Replies:
    2
    Views:
    291
    Phlip
    Oct 23, 2010
  3. ernest
    Replies:
    2
    Views:
    292
    Roy Smith
    Nov 14, 2010
  4. Jack Bates
    Replies:
    0
    Views:
    279
    Jack Bates
    May 2, 2011
  5. bvdp

    Raise X or Raise X()?

    bvdp, Mar 11, 2012, in forum: Python
    Replies:
    10
    Views:
    370
    Stefan Behnel
    Mar 12, 2012
Loading...

Share This Page