@property decorator doesn't raise exceptions

[Rafe]

Hi,

I've encountered a problem which is making debugging less obvious than
it should be. The @property decorator doesn't always raise exceptions.
It seems like it is bound to the class but ignored when called. I can
see the attribute using dir(self.__class__) on an instance, but when
called, python enters __getattr__. If I correct the bug, the attribute
calls work as expected and do not call __getattr__.

I can't seem to make a simple repro. Can anyone offer any clues as to
what might cause this so I can try to prove it?


Cheers,

- Rafe

 

[Rafe]

Rafewrote:



You must subclass from "object" to get a new style class. properties
don't work correctly on old style classes.

Christian

All classes are a sub-class of object. Any other ideas?

- Rafe

 

[Peter Otten]

All classes are a sub-class of object. Any other ideas?

Hard to tell when you don't give any code.
.... @property
.... def attribute(self):
.... raise AttributeError
.... def __getattr__(self, name):
.... return "nobody expects the spanish inquisition"
....'nobody expects the spanish inquisition'

Do you mean something like this? I don't think the __getattr__() call can be
avoided here.

Peter

 

[Steven D'Aprano]

All classes are a sub-class of object. Any other ideas?

Only in Python 3. If you are relying on that to be true in Python 2.x,
you're going to disappointed:
.... pass
....False


There are a lot of classic classes in the standard library. If you
inherit from them, your class will also be a classic class and properties
will fail to work correctly.


Earlier, you wrote:

"I've encountered a problem which is making debugging less obvious than
it should be. The @property decorator doesn't always raise exceptions."

Are you expecting it to raise an exception when the class is defined, or
when the property is called? e.g.

class Fail(object):
@property
"this should raise an exception"

Works for me -- I get a syntax error, as expected.


"It seems like it is bound to the class but ignored when called. I can
see the attribute using dir(self.__class__) on an instance, but when
called, python enters __getattr__. If I correct the bug, the attribute
calls work as expected and do not call __getattr__."


Perhaps you can give us a minimal example showing the code with the bug,
then how you corrected the bug? That might give as a hint as to what is
going on. I know that's hard, when you can't reproduce the problem, but
it's just as hard for us to solve your problem without more information.

 

[Steven D'Aprano]

Hi,

I've encountered a problem which is making debugging less obvious than
it should be. The @property decorator doesn't always raise exceptions.

I don't think that's the problem. I think properties do correctly raise
all exceptions that occur inside them, at least built-in exceptions.

I wrote a test that raises exceptions inside properties and they (almost)
all are raised normally.

Whatever your problem is, it isn't that properties don't raise exceptions.

For those who are interested, the test program I used follows.


# test module

def get_exceptions():
import __builtin__
list_of_exceptions = []
for key in dir(__builtin__):
# do not use __builtins__ -- note the 's'.
obj = getattr(__builtin__, key)
if not isinstance(obj, type):
continue
if issubclass(obj, Exception):
list_of_exceptions.append(obj)
return list_of_exceptions

def builder(list_of_exceptions):
class PropertyTester(object):
"""Do properties raise exceptions? Let's find out."""
for exception in list_of_exceptions:
name = exception.__name__
@property
def func(self, exception=exception):
# Make sure we bind a local copy to exception.
raise exception
setattr(PropertyTester, name, func)
return PropertyTester()

def testRaises(obj, exception_type, verbose):
"""Test that appropriate exception is raised when accessing an
attribute."""
name = exception_type.__name__
try:
getattr(obj, name)
except exception_type:
if verbose:
print "Passed: expected exception %s raised correctly" \
% exception_type
except Exception, e:
print "Failed: expected %s but got %r" % (exception_type, e)
else:
print "Failed: expected %s but got no exception at all" \
% exception_type

if __name__ == '__main__':
import sys
if sys.argv[1:] == ['verbose']:
verbose = True
else:
verbose = False
exceptions = get_exceptions()
tester = builder(exceptions)
for exception in exceptions:
testRaises(tester, exception, verbose)

 

[Rafe]

Hard to tell when you don't give any code.


...     @property
...     def attribute(self):
...             raise AttributeError
...     def __getattr__(self, name):
...             return "nobody expects the spanish inquisition"
...>>> A().attribute

'nobody expects the spanish inquisition'

Do you mean something like this? I don't think the __getattr__() call can be
avoided here.

Peter

You nailed it Peter! I thought __getattr__ was a symptom, not the
cause of the misleading errors. Here is the repro (pretty much
regurgitated):

The expected behavior...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in attribute
AttributeError: Correct Error.


The unexpected and misleading exception...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")
.... def __getattr__(self, name):
.... cls_name = self.__class__.__name__
.... msg = "%s has no attribute '%s'." % (cls_name, name)
.... raise AttributeError(msg)
Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in __getattr__
AttributeError: A has no attribute 'attribute'.


The docs state:
"Called when an attribute lookup has not found the attribute in the
usual places (i.e. it is not an instance attribute nor is it found in
the class tree for self). name is the attribute name. This method
should return the (computed) attribute value or raise an
AttributeError exception."

Can anyone explain why this is happening? I can hack a work-around,
but even then I could use some tips on how to raise the 'real'
exception so debugging isn't guesswork.


Cheers,

- Rafe

 

[Rafe]

Hard to tell when you don't give any code.


...     @property
...     def attribute(self):
...             raise AttributeError
...     def __getattr__(self, name):
...             return "nobody expects the spanish inquisition"
...>>> A().attribute

'nobody expects the spanish inquisition'

Do you mean something like this? I don't think the __getattr__() call can be
avoided here.

Peter

Peter nailed it, thanks! I thought __getattr__ was a symptom, not a
cause of the misleading exceptions. Here is a complete repro:


The expected behavior...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in attribute
AttributeError: Correct Error.


The misleading/unexpected behavior...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")
.... def __getattr__(self, name):
.... cls_name = self.__class__.__name__
.... msg = "%s has no attribute '%s'." % (cls_name, name)
.... raise AttributeError(msg)Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in __getattr__
AttributeError: A has no attribute 'attribute'.


Removing @property works as expected...
.... def attribute(self):
.... raise AttributeError("Correct Error.")
.... def __getattr__(self, name):
.... cls_name = self.__class__.__name__
.... msg = "%s has no attribute '%s'." % (cls_name, name)
.... raise AttributeError(msg)Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in attribute
AttributeError: Correct Error.


The docs seem to suggest this is impossible:
"Called when an attribute lookup has not found the attribute in the
usual places (i.e. it is not an instance attribute nor is it found in
the class tree for self). name is the attribute name. This method
should return the (computed) attribute value or raise an
AttributeError exception."

Can anyone explain why this is happening? Is it a bug? I can write a
workaround to detect this by comparing the attribute name passed
__getattr__ with dir(self.__class__) = self.__dict__.keys(), but how
can I raise the expected exception?


Thanks,

- Rafe

 

[greg]

The docs seem to suggest this is impossible:
"Called when an attribute lookup has not found the attribute in the
usual places (i.e. it is not an instance attribute nor is it found in
the class tree for self).

Getting an AttributeError is the way that the interpreter
machinery tells that the attribute wasn't found. So when
your property raises an AttributeError, this is
indistinguishable from the case where the property wasn't
there at all.

To avoid this you would have to raise some exception
that doesn't derive from AttributeError.

 

[Rafe]

Hi,

I've encountered a problem which is making debugging less obvious than
it should be. The @property decorator doesn't always raise exceptions.
It seems like it is bound to the class but ignored when called. I can
see the attribute using dir(self.__class__) on an instance, but when
called, python enters __getattr__. If I correct the bug, the attribute
calls work as expected and do not call __getattr__.

I can't seem to make a simple repro. Can anyone offer any clues as to
what might cause this so I can try to prove it?

Cheers,

- Rafe

Peter Oten pointed me in the right direction. I tried to reply to his
post 2 times and in spite of GoogleGroups reporting the post was
successful, it never showed up. Here is the repro:

The expected behavior...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in attribute
AttributeError: Correct Error.


The misleading/unexpected behavior...
.... @property
.... def attribute(self):
.... raise AttributeError("Correct Error.")
.... def __getattr__(self, name):
.... cls_name = self.__class__.__name__
.... msg = "%s has no attribute '%s'." % (cls_name, name)
.... raise AttributeError(msg)Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in __getattr__
AttributeError: A has no attribute 'attribute'.


Removing @property works as expected...
.... def attribute(self):
.... raise AttributeError("Correct Error.")
.... def __getattr__(self, name):
.... cls_name = self.__class__.__name__
.... msg = "%s has no attribute '%s'." % (cls_name, name)
.... raise AttributeError(msg)Traceback (most recent call last):
File "<console>", line 0, in <module>
File "<console>", line 0, in attribute
AttributeError: Correct Error.


I never suspected __getattr__ was the cause and not just a symptom.
The docs seem to indicate __gettattr__ should never be called when the
attribute exisits in the class:
"Called when an attribute lookup has not found the attribute in the
usual places (i.e. it is not an instance attribute nor is it found in
the class tree for self). name is the attribute name. This method
should return the (computed) attribute value or raise an
AttributeError exception."

Is this a bug? Any idea why this happens? I can write a hack in to
__getattr__ in my class which will detect this, but I'm not sure how
to raise the expected exception.


Cheers,

- Rafe

 

[Peter Otten]

Can anyone explain why this is happening?

When an attribute error is raised that is an indication that the requested
attribute doesn't exist, and __getattr__() must be called as a fallback.

I can hack a work-around,
but even then I could use some tips on how to raise the 'real'
exception so debugging isn't guesswork.

Look at the problem again, maybe you can find a solution without
__getattr__() and use only properties.

Otherwise you have to wrap your getter with something like

try:
...
except AttributeError:
raise BuggyProperty, None, original_traceback


If you put that functionality into a decorator you get:

import sys

class BuggyProperty(Exception):
pass

def safe_getter(get):
def safe_get(self):
try:
return get(self)
except AttributeError:
t, e, tb = sys.exc_info()
raise BuggyProperty("AttributeError in getter %s(); "
"giving original traceback"
% get.__name__), None, tb
return property(safe_get)

class A(object):
@safe_getter
def attr(self):
return self.m(3)

def m(self, n):
if n > 0:
return self.m(n-1)
raise AttributeError("it's a bug")

def __getattr__(self, name):
return "<%s>" % name

A().attr

Peter

 

[Rafe]

OT... Sorry about he spam.
Thanks for taking the time to post this Duncan.

I had the same thought. I have posted to this list before but never
experienced anything like this wait. I figured it was possible that I
hit "Reply to Author" the first time so I sent it again. I waited
about 8 hours before sending the third time (and I posted to
GoogleGroups support as well). Even then I didn't see my original
post. I'm surprised it took so long to update. Next time I'll just
make sure the post was made successfully and wait...as long as it
takes.


Cheers,

- Rafe

 

[Rafe]

Rafewrote:

When an attribute error is raised that is an indication that the requested
attribute doesn't exist, and __getattr__() must be called as a fallback.


Look at the problem again, maybe you can find a solution without
__getattr__() and use only properties.

Otherwise you have to wrap your getter with something like

try:
    ...
except AttributeError:
    raise BuggyProperty, None, original_traceback

If you put that functionality into a decorator you get:

import sys

class BuggyProperty(Exception):
    pass

def safe_getter(get):
    def safe_get(self):
        try:
            return get(self)
        except AttributeError:
            t, e, tb = sys.exc_info()
            raise BuggyProperty("AttributeError in getter %s(); "
                            "giving original traceback"
                            % get.__name__), None, tb
    return property(safe_get)

class A(object):
    @safe_getter
    def attr(self):
        return self.m(3)

    def m(self, n):
        if n > 0:
            return self.m(n-1)
        raise AttributeError("it's a bug")

    def __getattr__(self, name):
        return "<%s>" % name

A().attr

Peter

Thanks for the idea Peter. What confuses me is why this only happens
to @Property (and I assume other decorator related bindings?). Does it
have something to do with the way the class gets built? 'Normal'
attributes will raise AttributeErrors as expected, without triggering
__getattr__(). Considering this is a built-in decorator, it would be
nice if this behavior was fixed if possible.

Unfortunately, I need __getattr__() because my class is a wrapper (it
is delegating calls to another object when attributes aren't found in
the class). As a hack, I am testing the passed attr name against the
instance, class and super-class attributes. If there is a match, I
assume it is an error and raise an exception which points the
developer in the right direction. It isn't ideal, but it helps.

You're code is better, as it displays the 'real' traceback, but I need
to know more about the effects of using an exception which is not an
AttrbiuteError. Which brings me to my next question...

In case it isn't obvious, I'm fairly new to Python (and this level of
programming in general). I've been wondering about the optimal use of
custom exceptions. Up until now, I've been sticking to the built-in
exceptions, which seem to work in 90% of situations. Are you aware of
any resources which talk about this aspect of programming (more about
theory than code)?


Thanks again,

- Rafe

 

[Peter Otten]

Thanks for the idea Peter. What confuses me is why this only happens
to @Property (and I assume other decorator related bindings?). Does it
have something to do with the way the class gets built? 'Normal'
attributes will raise AttributeErrors as expected, without triggering
__getattr__(). Considering this is a built-in decorator, it would be
nice if this behavior was fixed if possible.

Normal attributes either exist, and then they don't raise an AttributeError,
or they don't exist, and then __getattr__() *is* triggered. The problem has
nothing to do with decorators. It is just that properties invoke custom
code, and Python currently has no way of finding out whether an
AttributeError was raised accidentally by a buggy getter or whether it is
meant to signal that the attribute wasn't found in the class hierarchy and
now should be calculated by __getattr__().

I guess (without looking into the C source) that it could be changed to meet
your intuition but that it would complicate the implementation.

Peter