Debugging difficulty in python with __getattr__, decorated propertiesand AttributeError.

Discussion in 'Python' started by Mr. Joe, May 3, 2013.

  1. Mr. Joe

    Mr. Joe Guest

    Is there any way to raise the original exception that made the call to
    __getattr__? I seem to stumble upon a problem where multi-layered attribute
    failure gets obscured due to use of __getattr__. Here's a dummy code to
    demonstrate my problems:
    """
    import traceback


    class BackupAlphabet(object):
    pass


    class Alphabet(object):
    @property
    def a(self):
    return backupalphabet.a


    def __getattr__(self, name):
    if name == "b":
    return "banana"

    raise AttributeError(
    "'{0} object has no attribute '{1}'"
    .format(self.__class__.__name__, name))


    alphabet = Alphabet()
    backupalphabet = BackupAlphabet()

    print(alphabet.a)
    print(alphabet.b)
    """

    Running the above code produces this:
    """
    Traceback (most recent call last):
    File "example.py", line 26, in <module>
    print(alphabet.a)
    File "example.py", line 20, in __getattr__
    .format(self.__class__.__name__, name))
    AttributeError: 'Alphabet object has no attribute 'a'
    """

    While it's easy enough to identify the problem here, the traceback is
    rather unhelpful in complex situations. Any comments?

    Regards,
    TB
    Mr. Joe, May 3, 2013
    #1
    1. Advertising

  2. Re: Debugging difficulty in python with __getattr__, decoratedpropertiesand AttributeError.

    On Fri, 03 May 2013 05:34:40 +0600, Mr. Joe wrote:

    > Is there any way to raise the original exception that made the call to
    > __getattr__?


    No. There is some discussion on the Python-Dev mailing list about adding
    better error reporting to AttributeError, but that may not go anywhere,
    and even if it does, it won't help you until you have dropped all support
    for versions below Python 3.4 or 3.5.



    > class BackupAlphabet(object):
    > pass
    >
    >
    > class Alphabet(object):
    > @property
    > def a(self):
    > return backupalphabet.a


    This raises AttributeError. Since __getattr__ uses AttributeError to
    decide whether or not the name exists, the behaviour shown is correct:
    Alphabet.a does not exist, as far as the Python semantics of attribute
    access are concerned.

    Either write better code *wink* that doesn't raise AttributeError from
    inside properties, or wrap them with something like this:

    class Alphabet(object):
    @property
    def a(self):
    try:
    return backupalphabet.a
    except AttributeError:
    raise MyCustomError

    where MyCustomError does NOT inherit from AttributeError.

    If you're doing this a lot, you can create a decorator to do the wrapping.



    --
    Steven
    Steven D'Aprano, May 3, 2013
    #2
    1. Advertising

  3. Mr. Joe

    Mr. Joe Guest

    Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.

    Thanks for clearing up. Developers of python should address this issue, in
    my opinion. 3.4/3.5 maybe, but better late than never.

    Recently, I've been beaten back for using some exotic features of python.
    One is this[ Took me hours to get to the bottom ]. The other one is
    'property' decorator. I was using it extensively until I needed to make a
    child class. Then I came to know that 'property' does not play well
    with polymorphic code. :( I resorted to some lambda hacks learned from
    stackoverflow.com to solve the problem. I know that it's the correct way
    for decorators to work, but still, it would be nice to have a language
    level solution.
    Mr. Joe, May 3, 2013
    #3
  4. Mr. Joe

    dieter Guest

    Re: Debugging difficulty in python with __getattr__,decorated properties and AttributeError.

    "Mr. Joe" <> writes:

    > ...
    > Then I came to know that 'property' does not play well
    > with polymorphic code. :(


    Can you elaborate?

    I like "polymorphic code" and decorators (such a "property")
    never met a problem with the two working nicely together.

    > I resorted to some lambda hacks learned from
    > stackoverflow.com to solve the problem. I know that it's the correct way
    > for decorators to work, but still, it would be nice to have a language
    > level solution.


    There are two approaches: change the language or learn how the language works.

    I am very happy that the Python developers try hard to retain backward
    compatible and, consequently, are very reluctant towards language changes.
    I would hate should the decorator behavior change in an incompatible
    way.

    On the other hand, decorators are very easy to understand: they are
    just syntactic sugar:

    @<dec_expression>
    def f(...): ...

    is equivalent to:

    def f(...): ...
    f = <dec_expression>(f)

    The "property" decorator uses an additional concept: "descriptor"s.
    A "descriptor" allows you to customize attribute access.

    These two concepts graped, the "property" decorator should no longer
    cause surprises -- even in "polymorphic code".
    dieter, May 4, 2013
    #4
  5. Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.

    On Fri, 03 May 2013 13:52:23 +0600, Mr. Joe wrote:

    > Thanks for clearing up. Developers of python should address this issue,
    > in my opinion. 3.4/3.5 maybe, but better late than never.
    >
    > Recently, I've been beaten back for using some exotic features of
    > python.


    What do you consider "exotic"? Neither properties nor __getattr__ are
    exotic, although of course like all features of a language they have
    their own quirks.


    > One is this[ Took me hours to get to the bottom ].


    Well, of course hindsight is 20:20, and I don't know how complicated your
    actual code is, but "hours" seems a bit poor. I could believe half an
    hour. Maybe an hour. But of course everything is easy to the guy who
    doesn't have to do it.


    > The other one
    > is 'property' decorator. I was using it extensively until I needed to
    > make a child class. Then I came to know that 'property' does not play
    > well with polymorphic code. :(


    I don't understand what you are trying to say. Properties can be as
    polymorphic as any other Python function.


    > I resorted to some lambda hacks learned
    > from stackoverflow.com to solve the problem.


    That's not a good sign. Any sentence containing "hacks" and
    "stackoverflow" is a warning that you're probably doing something wrong.



    --
    Steven
    Steven D'Aprano, May 4, 2013
    #5
  6. Mr. Joe

    Mr. Joe Guest

    Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.

    Sorry for digging this old topic back. I see that my "'property' does not
    play well with polymorphic code" comment generated some controversy. So
    here's something in my defense:

    Here's the link to stackoveflow topic I am talking about:

    http://stackoverflow.com/questions/237432/python-properties-and-inheritance

    The solution that fits my taste:
    http://stackoverflow.com/a/14349742

    A related blogpost:

    http://requires-thinking.blogspot.com/2006/03/note-to-self-python-properties-are-non.html

    Yes, I like decorators and descriptors. I also like the ability to create a
    "virtual property" in a python class by binding a bunch of methods as
    setter/getter. But I find the implementation of this "virtual property
    feature" a bit awkward sometimes - every time I need to override a
    getter/setter in a child class, I need to decorate them again. Some of you
    may like this "explicitness", but I don't.

    To Steven D'Aprano: Seriously, what's all the bashing in your last reply
    about? You dissected my "thank-you reply" more strictly than the python
    interpreter checking for syntax errors. Not in a mood for fight, but I find
    your opinions about "bug finding time", "hacks" and "stackoverflow" quite
    silly.
    Mr. Joe, May 14, 2013
    #6
  7. Mr. Joe

    dieter Guest

    Re: Debugging difficulty in python with __getattr__,decorated properties and AttributeError.

    "Mr. Joe" <> writes:
    > ...
    > Sorry for digging this old topic back. I see that my "'property' does not
    > play well with polymorphic code" comment generated some controversy. So
    > here's something in my defense:


    I did not intend to "attack" you.

    > ...
    > Yes, I like decorators and descriptors. I also like the ability to create a
    > "virtual property" in a python class by binding a bunch of methods as
    > setter/getter. But I find the implementation of this "virtual property
    > feature" a bit awkward sometimes - every time I need to override a
    > getter/setter in a child class, I need to decorate them again. Some of you
    > may like this "explicitness", but I don't.


    True, I have been hit by this, too - not with "property" but with
    other decorators (those for caching).
    But, after reflection, I came to the conclusion that I should be
    happy with this feature:

    If Python would automatically redecorate overridden methods in a derived
    class, I would have no control over the process. What if I need
    the undecorated method or a differently decorated method (an
    uncached or differently cached method, in my case)?

    Your getter/setter use case can quite easily be solved
    with a class "DelayedMethodAccessor":

    class DelayedMethodAccess(object):
    """
    def __init__(self, name): self.__name = name
    def __call__(self, inst, *args, **kw):
    return getattr(inst, self.__name)(*args, **kw)

    You can then use:

    prop = property(DelayedMethodAccess("<getter_name>"), ...)

    Or define a new decorator "property_by_name" and then
    use

    prop = property_by_name("<getter_name>", ...)

    Of course, this requires that the property name and the getter/setter names
    differ, but this should be naturally enough.


    Python's current behavior is very natural once you know that
    decorators are just syntactic sugar and

    @<decorator_expression>
    def <f>...

    simply means:

    def <f>...
    f = <decorator_expressen>(f)

    True, this is no perfect fit for all use cases - but
    such a fit (if it existed at all) would have to be so complex
    that only experts could understand it.
    dieter, May 15, 2013
    #7
  8. Mr. Joe

    Mr. Joe Guest

    Re: Debugging difficulty in python with __getattr__, decoratedproperties and AttributeError.

    On Wed, May 15, 2013 at 12:15 PM, dieter <> wrote:
    >
    > If Python would automatically redecorate overridden methods in a derived
    > class, I would have no control over the process. What if I need
    > the undecorated method or a differently decorated method (an
    > uncached or differently cached method, in my case)?
    >


    On a second thought, I am convinced by your argument.

    > Your getter/setter use case can quite easily be solved
    > with a class "DelayedMethodAccessor":
    >
    > class DelayedMethodAccess(object):
    > """
    > def __init__(self, name): self.__name = name
    > def __call__(self, inst, *args, **kw):
    > return getattr(inst, self.__name)(*args, **kw)
    >
    > You can then use:
    >
    > prop = property(DelayedMethodAccess("<getter_name>"), ...)
    >
    > Or define a new decorator "property_by_name" and then
    > use
    >
    > prop = property_by_name("<getter_name>", ...)
    >
    > Of course, this requires that the property name and the getter/setter

    names
    > differ, but this should be naturally enough.
    >
    >
    > Python's current behavior is very natural once you know that
    > decorators are just syntactic sugar and
    >
    > @<decorator_expression>
    > def <f>...
    >
    > simply means:
    >
    > def <f>...
    > f = <decorator_expressen>(f)
    >
    > True, this is no perfect fit for all use cases - but
    > such a fit (if it existed at all) would have to be so complex
    > that only experts could understand it.


    Thanks for these really nice patterns. They fits my problem very well.
    Mr. Joe, May 15, 2013
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    1
    Views:
    1,843
    Roedy Green
    Sep 17, 2005
  2. Wolfgang Draxinger

    What are decorated functions?

    Wolfgang Draxinger, Aug 22, 2006, in forum: Python
    Replies:
    4
    Views:
    326
    Richard Jones
    Aug 23, 2006
  3. Zaza
    Replies:
    0
    Views:
    612
  4. Victor Khangulov
    Replies:
    0
    Views:
    232
    Victor Khangulov
    Jul 28, 2011
  5. Beth35

    Difficulty debugging WCF service

    Beth35, Nov 18, 2008, in forum: ASP .Net Web Services
    Replies:
    0
    Views:
    834
    Beth35
    Nov 18, 2008
Loading...

Share This Page