Why doesn't __call__ lead to infinite recursion?

P

Patrick Lioi

def foo(): pass

foo is a function
foo is a callable object
foo has method __call__ defined

foo.__call__ is a function
foo.__call__ is a callable object
foo.__call__ has method __call__ defined

foo.__call__.__call__ is a function...


This seems to go on forever. How does calling foo() not lead to
an infinite loop while trying to execute?
 
A

Aahz

def foo(): pass

foo is a function
foo is a callable object
foo has method __call__ defined

foo.__call__ is a function
foo.__call__ is a callable object
foo.__call__ has method __call__ defined

You're mixing up the distinction between objects and types. If you do

print foo.__dict__

you'll see that __call__ isn't there. However, if you try

print type(foo).__dict__

you'll see __call__ there. When you do foo(), Python actually does
type(foo).__call__(foo). Because type(foo).__call__ is manipulating
foo, you don't get the circular reference.
 
J

John J. Lee

you'll see __call__ there. When you do foo(), Python actually does
type(foo).__call__(foo). Because type(foo).__call__ is manipulating
foo, you don't get the circular reference.

Still seems weird that type(foo).__call__.__call__ (etc.) is defined.


John
 
A

Andrew Dalke

Aahz:
you'll see __call__ there. When you do foo(), Python actually does
type(foo).__call__(foo). Because type(foo).__call__ is manipulating
foo, you don't get the circular reference.

Not quite, but I don't understand how everything works so what I
say may also need corrections.

The call syntax "foo()" does two things. The first is to
get the 'thing' used for the call and the second is to actually
call it. The latter is not done recursively - if the returned
thing can't be called, the attempt at making the call fails.

If 'foo' is an instance, then the implementation code is
something like

thing_to_call = getattr(foo, "__call__")
if thing_to_call is not None:
DO_CALL(thing_to_call, args, kwargs)

The 'DO_CALL' is not a Python function, it's part of
how the implementation works.

The getattr implementation for an instance first tries to
find "__call__" in foo's instance __dict__. If that fails, it
looks in the parent class, and returns a bound method,
that is, a new 'thing' with references to the class method
and to the instance itself. The DO_CALL does the
actual call to this thing with the new arguments. The
bound method thing prepends the self parameter and
does the actual call to the underlying code.

Pretty complicated, and I don't think I was very clear
on that. Here's an example though to show that

foo() != foo.__class__.__call__(foo)
.... def __init__(self):
.... def callme(x):
.... print "Hello", x
.... self.__call__ = callme
.... def __call__(self, x):
.... print "Hej", x
....
I'm also missing something because I don't know how
functions work. I thought it was always 'use
getattr(obj, "__call__") to get the thing to call then do
the call machinery on that thing", but it doesn't seem
to do that for functions.
.... print "f(%s)" % x
........ print "g(%s)" % x
....
I expected the 'f(4)' call to return 'g(4)' since I replaced
the function's __call__ with g's __call__.

What's throwing me off is that g.__call__ returns
a new wrapper object each time, while once I
assigned f.__call__, it persistently stayed that way.
So there's some getattr shenanigans with functions
I don't understand. To make it worse, there's also

I'll leave the clarification to someone else.

Andrew
(e-mail address removed)
 
A

Andrew Dalke

Aahz:
No time to investigate further, but all your examples used classic
classes instead of new-style classes; I'm pretty sure that new-style
classes will more closely emulate the way functions work. There's also
the wrinkle I didn't mention that functions use a dict proxy IIRC.

Interesting. Very interesting.
.... def __init__(self):
.... def abc(x):
.... print "Hello", x
.... self.__call__ = abc
.... def __call__(self, x):
.... print "Yo", x
....I wonder if this will affect any of my code.

It does explain the observed differences better, since FunctionType
in 2.3 is derived from object while my class was not.

Andrew
(e-mail address removed)
 
M

Michael Hudson

Andrew Dalke said:
Aahz:

Interesting. Very interesting.

Yes :)

You have to have something like this when you do things like 'print
type(foo)'. This should call the *types* *bound* __str__ method, not
try to call the *instances* *unbound* __str__ method...

Cheers,
mwh
 
B

Beni Cherniavsky

(e-mail address removed) (Aahz) writes:
[...]
you'll see __call__ there. When you do foo(), Python actually does
type(foo).__call__(foo). Because type(foo).__call__ is manipulating
foo, you don't get the circular reference.

Still seems weird that type(foo).__call__.__call__ (etc.) is defined.
[I'm afraid I sent this 2 or 3 already in private by mistake; resending in
public. I apologize, it's late...]

The point is that down there, sits the C level which doesn't go
through the Python definition. When you call ``foo()``, ``type(foo)``
is asked at the *C* level how to call it (read: the type struct is
accessed and the call-behavior slot, if non-NULL, is called by plain
C function call - which can't be intercepted in C so there is no futher
recursion).

It so happens that classes, when asked this, go back into the Python
level and look for the `__call__` attribute. They do this for all
operations, giving you the dynamism and flexibility we all love.

OTOH, functions, when asked this, simply execute their code objects with the
given arguments. The attribute `__call__` on function is a "proxy" attribute.
It is not used by function objects for the call, it only exposes the C-level
call machinery at Python level. The same convention is used for other
built-in types and operations, so that from Python you see Python-level and
C-level methods in the same way. You only have to be aware of the distinction
when working with bizzare extension types that don't respect this convention,
or asking questions like this one about how it all works... (Another one: why
__getattribute__ is not infinitely recusive? Same explanation.)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top