Why doesn't __call__ lead to infinite recursion?

Discussion in 'Python' started by Patrick Lioi, Aug 15, 2003.

  1. Patrick Lioi

    Patrick Lioi Guest

    def foo(): pass

    foo is a function
    foo is a callable object
    foo has method __call__ defined

    foo.__call__ is a function
    foo.__call__ is a callable object
    foo.__call__ has method __call__ defined

    foo.__call__.__call__ is a function...


    This seems to go on forever. How does calling foo() not lead to
    an infinite loop while trying to execute?
     
    Patrick Lioi, Aug 15, 2003
    #1
    1. Advertising

  2. Patrick Lioi

    Aahz Guest

    In article <bhjg3v$q3m$>,
    bromden <> wrote:
    >> foo is a function
    >> foo is a callable object
    >> foo has method __call__ defined

    >
    >true
    >true
    >false


    Wrong. See my followup.
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
     
    Aahz, Aug 16, 2003
    #2
    1. Advertising

  3. Patrick Lioi

    Aahz Guest

    In article <>,
    Patrick Lioi <> wrote:
    >
    >def foo(): pass
    >
    >foo is a function
    >foo is a callable object
    >foo has method __call__ defined
    >
    >foo.__call__ is a function
    >foo.__call__ is a callable object
    >foo.__call__ has method __call__ defined


    You're mixing up the distinction between objects and types. If you do

    print foo.__dict__

    you'll see that __call__ isn't there. However, if you try

    print type(foo).__dict__

    you'll see __call__ there. When you do foo(), Python actually does
    type(foo).__call__(foo). Because type(foo).__call__ is manipulating
    foo, you don't get the circular reference.
    --
    Aahz () <*> http://www.pythoncraft.com/

    This is Python. We don't care much about theory, except where it intersects
    with useful practice. --Aahz
     
    Aahz, Aug 16, 2003
    #3
  4. Patrick Lioi

    John J. Lee Guest

    (Aahz) writes:
    [...]
    > you'll see __call__ there. When you do foo(), Python actually does
    > type(foo).__call__(foo). Because type(foo).__call__ is manipulating
    > foo, you don't get the circular reference.


    Still seems weird that type(foo).__call__.__call__ (etc.) is defined.


    John
     
    John J. Lee, Aug 16, 2003
    #4
  5. Patrick Lioi

    Andrew Dalke Guest

    Aahz:
    > you'll see __call__ there. When you do foo(), Python actually does
    > type(foo).__call__(foo). Because type(foo).__call__ is manipulating
    > foo, you don't get the circular reference.


    Not quite, but I don't understand how everything works so what I
    say may also need corrections.

    The call syntax "foo()" does two things. The first is to
    get the 'thing' used for the call and the second is to actually
    call it. The latter is not done recursively - if the returned
    thing can't be called, the attempt at making the call fails.

    If 'foo' is an instance, then the implementation code is
    something like

    thing_to_call = getattr(foo, "__call__")
    if thing_to_call is not None:
    DO_CALL(thing_to_call, args, kwargs)

    The 'DO_CALL' is not a Python function, it's part of
    how the implementation works.

    The getattr implementation for an instance first tries to
    find "__call__" in foo's instance __dict__. If that fails, it
    looks in the parent class, and returns a bound method,
    that is, a new 'thing' with references to the class method
    and to the instance itself. The DO_CALL does the
    actual call to this thing with the new arguments. The
    bound method thing prepends the self parameter and
    does the actual call to the underlying code.

    Pretty complicated, and I don't think I was very clear
    on that. Here's an example though to show that

    foo() != foo.__class__.__call__(foo)

    >>> class SPAM:

    .... def __init__(self):
    .... def callme(x):
    .... print "Hello", x
    .... self.__call__ = callme
    .... def __call__(self, x):
    .... print "Hej", x
    ....
    >>> spam = SPAM()
    >>> spam("world")

    Hello world
    >>>
    >>> spam.__class__.__call__(spam, "world")

    Hej world
    >>>


    >>> getattr(spam, "__call__")

    <function callme at 0x014DA9B0>
    >>> getattr(spam, "__init__")

    <bound method SPAM.__init__ of <__main__.SPAM instance at 0x013CF148>>
    >>> getattr(SPAM, "__call__")

    <unbound method SPAM.__call__>
    >>>


    I'm also missing something because I don't know how
    functions work. I thought it was always 'use
    getattr(obj, "__call__") to get the thing to call then do
    the call machinery on that thing", but it doesn't seem
    to do that for functions.

    >>> def f(x):

    .... print "f(%s)" % x
    ....
    >>> def g(x):

    .... print "g(%s)" % x
    ....
    >>> f(5)

    f(5)
    >>> f(7)

    f(7)
    >>> f.__call__ = g.__call__
    >>> f(4)

    f(4)
    >>> f.__call__

    <method-wrapper object at 0x014AD6F0>
    >>> f.__call__

    <method-wrapper object at 0x014AD6F0>
    >>> g.__call__

    <method-wrapper object at 0x014A02D0>
    >>> g.__call__

    <method-wrapper object at 0x014AD5B0>
    >>>


    I expected the 'f(4)' call to return 'g(4)' since I replaced
    the function's __call__ with g's __call__.

    What's throwing me off is that g.__call__ returns
    a new wrapper object each time, while once I
    assigned f.__call__, it persistently stayed that way.
    So there's some getattr shenanigans with functions
    I don't understand. To make it worse, there's also

    >>> import types
    >>> types.FunctionType.__call__

    <slot wrapper '__call__' of 'function' objects>
    >>>


    I'll leave the clarification to someone else.

    Andrew
     
    Andrew Dalke, Aug 16, 2003
    #5
  6. Patrick Lioi

    Andrew Dalke Guest

    Aahz:
    > No time to investigate further, but all your examples used classic
    > classes instead of new-style classes; I'm pretty sure that new-style
    > classes will more closely emulate the way functions work. There's also
    > the wrinkle I didn't mention that functions use a dict proxy IIRC.


    Interesting. Very interesting.

    >>> class XYZ(object):

    .... def __init__(self):
    .... def abc(x):
    .... print "Hello", x
    .... self.__call__ = abc
    .... def __call__(self, x):
    .... print "Yo", x
    ....
    >>> xyz = XYZ()
    >>> xyz("fred")

    Yo fred
    >>>
    >>> getattr(xyz, "__call__")

    <function abc at 0x0168CB70>
    >>>

    I wonder if this will affect any of my code.

    It does explain the observed differences better, since FunctionType
    in 2.3 is derived from object while my class was not.

    Andrew
     
    Andrew Dalke, Aug 16, 2003
    #6
  7. "Andrew Dalke" <> writes:

    > Aahz:
    > > No time to investigate further, but all your examples used classic
    > > classes instead of new-style classes; I'm pretty sure that new-style
    > > classes will more closely emulate the way functions work. There's also
    > > the wrinkle I didn't mention that functions use a dict proxy IIRC.

    >
    > Interesting. Very interesting.


    Yes :)

    You have to have something like this when you do things like 'print
    type(foo)'. This should call the *types* *bound* __str__ method, not
    try to call the *instances* *unbound* __str__ method...

    Cheers,
    mwh

    --
    Its unmanageable complexity has spawned more fear-preventing tools
    than any other language, but the solution _should_ have been to
    create and use a language that does not overload the whole goddamn
    human brain with irrelevant details. -- Erik Naggum, comp.lang.lisp
     
    Michael Hudson, Aug 18, 2003
    #7
  8. In article <>, John J. Lee wrote:
    > (Aahz) writes:
    >[...]
    >> you'll see __call__ there. When you do foo(), Python actually does
    >> type(foo).__call__(foo). Because type(foo).__call__ is manipulating
    >> foo, you don't get the circular reference.

    >
    >Still seems weird that type(foo).__call__.__call__ (etc.) is defined.
    >

    [I'm afraid I sent this 2 or 3 already in private by mistake; resending in
    public. I apologize, it's late...]

    The point is that down there, sits the C level which doesn't go
    through the Python definition. When you call ``foo()``, ``type(foo)``
    is asked at the *C* level how to call it (read: the type struct is
    accessed and the call-behavior slot, if non-NULL, is called by plain
    C function call - which can't be intercepted in C so there is no futher
    recursion).

    It so happens that classes, when asked this, go back into the Python
    level and look for the `__call__` attribute. They do this for all
    operations, giving you the dynamism and flexibility we all love.

    OTOH, functions, when asked this, simply execute their code objects with the
    given arguments. The attribute `__call__` on function is a "proxy" attribute.
    It is not used by function objects for the call, it only exposes the C-level
    call machinery at Python level. The same convention is used for other
    built-in types and operations, so that from Python you see Python-level and
    C-level methods in the same way. You only have to be aware of the distinction
    when working with bizzare extension types that don't respect this convention,
    or asking questions like this one about how it all works... (Another one: why
    __getattribute__ is not infinitely recusive? Same explanation.)

    --
    Beni Cherniavsky <>

    Look, Mom, no viruses! [Hint: I use GNU/Linux]
     
    Beni Cherniavsky, Aug 19, 2003
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mr. SweatyFinger
    Replies:
    2
    Views:
    2,090
    Smokey Grindel
    Dec 2, 2006
  2. Replies:
    0
    Views:
    460
  3. Ann
    Replies:
    0
    Views:
    426
  4. Irmen de Jong
    Replies:
    2
    Views:
    446
    Irmen de Jong
    Jun 14, 2011
  5. Ann
    Replies:
    1
    Views:
    229
    Anthony Levensalor
    Jan 2, 2008
Loading...

Share This Page