Debugging difficulty in python with __getattr__, decorated propertiesand AttributeError.

[Mr. Joe]

Is there any way to raise the original exception that made the call to
__getattr__? I seem to stumble upon a problem where multi-layered attribute
failure gets obscured due to use of __getattr__. Here's a dummy code to
demonstrate my problems:
"""
import traceback


class BackupAlphabet(object):
pass


class Alphabet(object):
@property
def a(self):
return backupalphabet.a


def __getattr__(self, name):
if name == "b":
return "banana"

raise AttributeError(
"'{0} object has no attribute '{1}'"
.format(self.__class__.__name__, name))


alphabet = Alphabet()
backupalphabet = BackupAlphabet()

print(alphabet.a)
print(alphabet.b)
"""

Running the above code produces this:
"""
Traceback (most recent call last):
File "example.py", line 26, in <module>
print(alphabet.a)
File "example.py", line 20, in __getattr__
.format(self.__class__.__name__, name))
AttributeError: 'Alphabet object has no attribute 'a'
"""

While it's easy enough to identify the problem here, the traceback is
rather unhelpful in complex situations. Any comments?

Regards,
TB

 

[Steven D'Aprano]

Is there any way to raise the original exception that made the call to
__getattr__?

No. There is some discussion on the Python-Dev mailing list about adding
better error reporting to AttributeError, but that may not go anywhere,
and even if it does, it won't help you until you have dropped all support
for versions below Python 3.4 or 3.5.

class BackupAlphabet(object):
pass


class Alphabet(object):
@property
def a(self):
return backupalphabet.a

This raises AttributeError. Since __getattr__ uses AttributeError to
decide whether or not the name exists, the behaviour shown is correct:
Alphabet.a does not exist, as far as the Python semantics of attribute
access are concerned.

Either write better code *wink* that doesn't raise AttributeError from
inside properties, or wrap them with something like this:

class Alphabet(object):
@property
def a(self):
try:
return backupalphabet.a
except AttributeError:
raise MyCustomError

where MyCustomError does NOT inherit from AttributeError.

If you're doing this a lot, you can create a decorator to do the wrapping.

 

[Mr. Joe]

Thanks for clearing up. Developers of python should address this issue, in
my opinion. 3.4/3.5 maybe, but better late than never.

Recently, I've been beaten back for using some exotic features of python.
One is this[ Took me hours to get to the bottom ]. The other one is
'property' decorator. I was using it extensively until I needed to make a
child class. Then I came to know that 'property' does not play well
with polymorphic code. I resorted to some lambda hacks learned from
stackoverflow.com to solve the problem. I know that it's the correct way
for decorators to work, but still, it would be nice to have a language
level solution.

 

[dieter]

...
Then I came to know that 'property' does not play well
with polymorphic code.

Can you elaborate?

I like "polymorphic code" and decorators (such a "property")
never met a problem with the two working nicely together.

I resorted to some lambda hacks learned from
stackoverflow.com to solve the problem. I know that it's the correct way
for decorators to work, but still, it would be nice to have a language
level solution.

There are two approaches: change the language or learn how the language works.

I am very happy that the Python developers try hard to retain backward
compatible and, consequently, are very reluctant towards language changes.
I would hate should the decorator behavior change in an incompatible
way.

On the other hand, decorators are very easy to understand: they are
just syntactic sugar:

@<dec_expression>
def f(...): ...

is equivalent to:

def f(...): ...
f = <dec_expression>(f)

The "property" decorator uses an additional concept: "descriptor"s.
A "descriptor" allows you to customize attribute access.

These two concepts graped, the "property" decorator should no longer
cause surprises -- even in "polymorphic code".

 

[Steven D'Aprano]

Thanks for clearing up. Developers of python should address this issue,
in my opinion. 3.4/3.5 maybe, but better late than never.

Recently, I've been beaten back for using some exotic features of
python.

What do you consider "exotic"? Neither properties nor __getattr__ are
exotic, although of course like all features of a language they have
their own quirks.

One is this[ Took me hours to get to the bottom ].

Well, of course hindsight is 20:20, and I don't know how complicated your
actual code is, but "hours" seems a bit poor. I could believe half an
hour. Maybe an hour. But of course everything is easy to the guy who
doesn't have to do it.

The other one
is 'property' decorator. I was using it extensively until I needed to
make a child class. Then I came to know that 'property' does not play
well with polymorphic code.

I don't understand what you are trying to say. Properties can be as
polymorphic as any other Python function.

I resorted to some lambda hacks learned
from stackoverflow.com to solve the problem.

That's not a good sign. Any sentence containing "hacks" and
"stackoverflow" is a warning that you're probably doing something wrong.

 

[Mr. Joe]

Sorry for digging this old topic back. I see that my "'property' does not
play well with polymorphic code" comment generated some controversy. So
here's something in my defense:

Here's the link to stackoveflow topic I am talking about:

http://stackoverflow.com/questions/237432/python-properties-and-inheritance

The solution that fits my taste:
http://stackoverflow.com/a/14349742

A related blogpost:

http://requires-thinking.blogspot.com/2006/03/note-to-self-python-properties-are-non.html

Yes, I like decorators and descriptors. I also like the ability to create a
"virtual property" in a python class by binding a bunch of methods as
setter/getter. But I find the implementation of this "virtual property
feature" a bit awkward sometimes - every time I need to override a
getter/setter in a child class, I need to decorate them again. Some of you
may like this "explicitness", but I don't.

To Steven D'Aprano: Seriously, what's all the bashing in your last reply
about? You dissected my "thank-you reply" more strictly than the python
interpreter checking for syntax errors. Not in a mood for fight, but I find
your opinions about "bug finding time", "hacks" and "stackoverflow" quite
silly.

 

[dieter]

...
Sorry for digging this old topic back. I see that my "'property' does not
play well with polymorphic code" comment generated some controversy. So
here's something in my defense:

I did not intend to "attack" you.

...
Yes, I like decorators and descriptors. I also like the ability to create a
"virtual property" in a python class by binding a bunch of methods as
setter/getter. But I find the implementation of this "virtual property
feature" a bit awkward sometimes - every time I need to override a
getter/setter in a child class, I need to decorate them again. Some of you
may like this "explicitness", but I don't.

True, I have been hit by this, too - not with "property" but with
other decorators (those for caching).
But, after reflection, I came to the conclusion that I should be
happy with this feature:

If Python would automatically redecorate overridden methods in a derived
class, I would have no control over the process. What if I need
the undecorated method or a differently decorated method (an
uncached or differently cached method, in my case)?

Your getter/setter use case can quite easily be solved
with a class "DelayedMethodAccessor":

class DelayedMethodAccess(object):
"""
def __init__(self, name): self.__name = name
def __call__(self, inst, *args, **kw):
return getattr(inst, self.__name)(*args, **kw)

You can then use:

prop = property(DelayedMethodAccess("<getter_name>"), ...)

Or define a new decorator "property_by_name" and then
use

prop = property_by_name("<getter_name>", ...)

Of course, this requires that the property name and the getter/setter names
differ, but this should be naturally enough.


Python's current behavior is very natural once you know that
decorators are just syntactic sugar and

@<decorator_expression>
def <f>...

simply means:

def <f>...
f = <decorator_expressen>(f)

True, this is no perfect fit for all use cases - but
such a fit (if it existed at all) would have to be so complex
that only experts could understand it.

 

[Mr. Joe]

If Python would automatically redecorate overridden methods in a derived
class, I would have no control over the process. What if I need
the undecorated method or a differently decorated method (an
uncached or differently cached method, in my case)?

On a second thought, I am convinced by your argument.

Your getter/setter use case can quite easily be solved
with a class "DelayedMethodAccessor":

class DelayedMethodAccess(object):
"""
def __init__(self, name): self.__name = name
def __call__(self, inst, *args, **kw):
return getattr(inst, self.__name)(*args, **kw)

You can then use:

prop = property(DelayedMethodAccess("<getter_name>"), ...)

Or define a new decorator "property_by_name" and then
use

prop = property_by_name("<getter_name>", ...)

Of course, this requires that the property name and the getter/setter names
differ, but this should be naturally enough.


Python's current behavior is very natural once you know that
decorators are just syntactic sugar and

@<decorator_expression>
def <f>...

simply means:

def <f>...
f = <decorator_expressen>(f)

True, this is no perfect fit for all use cases - but
such a fit (if it existed at all) would have to be so complex
that only experts could understand it.

Thanks for these really nice patterns. They fits my problem very well.