getattr(foo, 'foobar') not the same as foo.foobar?

[Dave Kuhlman]

The following code has me mystified:

In [4]: class A(object):
...: def show(self):
...: print 'hello'
...:
...:
In [5]: a = A()
In [6]:
In [7]: x = a.show
In [8]: y = getattr(a, 'show')
In [9]: x
Out[9]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [10]: y
Out[10]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [11]:
In [12]: id(x)
Out[12]: 12419552
In [13]: id(y)
Out[13]: 12419872
In [14]:
In [15]: x is y
Out[15]: False
In [16]:
In [17]: x()
hello
In [18]: y()
hello

Basically, the above code is saying that foo.foobar is not the same as
getattr(foo, 'foobar').

But the documentation at
http://docs.python.org/lib/built-in-funcs.html#l2h-33
says that they are equivalent.

And, the following seems even worse:
False

What gives? This breaks my understanding of id(), the is operator, and
getattr().

Can someone help me make sense of this?

I'm using Python 2.5.2.

- Dave

 

[Imri Goldberg]

The following code has me mystified:

In [4]: class A(object):
...: def show(self):
...: print 'hello'
...:
...:
In [5]: a = A()
In [6]:
In [7]: x = a.show
In [8]: y = getattr(a, 'show')
In [9]: x
Out[9]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [10]: y
Out[10]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [11]:
In [12]: id(x)
Out[12]: 12419552
In [13]: id(y)
Out[13]: 12419872
In [14]:
In [15]: x is y
Out[15]: False
In [16]:
In [17]: x()
hello
In [18]: y()
hello

Basically, the above code is saying that foo.foobar is not the same as
getattr(foo, 'foobar').

But the documentation at
http://docs.python.org/lib/built-in-funcs.html#l2h-33
says that they are equivalent.

And, the following seems even worse:
False

Actually, while I don't know about the basic problem, this doesn't mean
id() is broken.
In the comparison, an object is created, then id() is computed, and the
object is garbage collected.
The same happens to the second object. Since by the time it is created
the first object was garbage collected,
it can have the same id().
As you have already shown in your previous example, when the first
object is not discarded, the id() is different.
Here's some code to illustrate this:

In [17]: bla = [a.foo for x in range(5)]
In [18]: bar = [id(z) for z in bla]
In [19]: bar
Out[19]: [138262924, 137884252, 137884212, 137884452, 137884572]

(This result is equivalent for getattr(a,'foo').)

What gives? This breaks my understanding of id(), the is operator, and
getattr().

Can someone help me make sense of this?

I'm using Python 2.5.2.

- Dave

Cheers,
Imri

 

[Arnaud Delobelle]

The following code has me mystified:

In [4]: class A(object):
   ...:     def show(self):
   ...:         print 'hello'
   ...:
   ...:
In [5]: a = A()
In [6]:
In [7]: x = a.show
In [8]: y = getattr(a, 'show')
In [9]: x
Out[9]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [10]: y
Out[10]: <bound method A.show of <__main__.A object at 0xb557d0>>
In [11]:
In [12]: id(x)
Out[12]: 12419552
In [13]: id(y)
Out[13]: 12419872
In [14]:
In [15]: x is y
Out[15]: False
In [16]:
In [17]: x()
hello
In [18]: y()
hello

Basically, the above code is saying that foo.foobar is not the same as
getattr(foo, 'foobar').

But the documentation athttp://docs.python.org/lib/built-in-funcs.html#l2h-33
says that they are equivalent.

And, the following seems even worse:

  >>> id(getattr(a, 'show')) == id(a.show)
  True
  >>> getattr(a, 'show') is a.show
  False

What gives?  This breaks my understanding of id(), the is operator, and
getattr().

Can someone help me make sense of this?

There are several misconceptions that contribute to your confusion I
think.

1. This has nothing to do with getattr(). If you run the same code
as above, replacing getattr(a, 'show') with a.show you will get the
same results. E.g.
... def bar(self): pass
...
2. The above is because two objects can have the same id if their
lifetimes don't overlap. In (A) by the time the second foo.bar is
created, the first one is already dead. So The second one takes its
place in memory, hence their ids are equal

3. In (B) the first foo.bar is kept alive for comparing with the
second, hence they have a different id.

4. Both points above follow from the fact that foo.bar is really a
function call that returns a (potentially) new object: in fact what
really happens is something like

Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

HTH

 

[Dave Kuhlman]

4. Both points above follow from the fact that foo.bar is really a
function call that returns a (potentially) new object: in fact what
really happens is something like

Arnaud and Imri, too -

No. foo.bar is *not* really a function/method call.

Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

HTH

I appreciate the help, but ...

Actually, it does not help, because ...

My understanding is that foo.bar does *not* create a new object. All it
does is return the value of the bar attribute of object foo. What new
object is being created?

If I have:

class Foo(object):
def bar(self): pass


And I do:

foo = SomeClass()

then:

foo.bar

should return the same (identical) object everytime, no? yes?

I'm still confused.

- Dave

 

[Erik Max Francis]

Basically, the above code is saying that foo.foobar is not the same as
getattr(foo, 'foobar').

Python promises that the behavior is the same. It does not promise that
the _objects_ will be the same, which is what `is` determines. That is,
you're not doing a useful test here.

In Python, bound methods are dynamically generated.

 

[Diez B. Roggisch]

My understanding is that foo.bar does *not* create a new object.

Your understanding is not correct.

All it
does is return the value of the bar attribute of object foo. What new
object is being created?

A bound method. This happens through the descriptor-protocol. Please see
this example:


class Foo(object):
def bar(self):
pass


f = Foo()
a = Foo.bar
b = f.bar
c = f.bar

print a, b, c
print id(b), id(c)


The result is this:


<unbound method Foo.bar> <bound method Foo.bar of <__main__.Foo object
at 0xbf650>> <bound method Foo.bar of <__main__.Foo object at 0xbf650>>
315560 788960

So b and c really are different objects - "a is not b == True"

Diez

 

[castironpi]

Basically, the above code is saying that foo.foobar is not the same as

getattr(foo, 'foobar').

What gives?  This breaks my understanding of id(), the is operator, and
getattr().

4.  Both points above follow from the fact that foo.bar is really a
function call that returns a (potentially) new object: in fact what
really happens is something like

    Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

When is it? Why is the 'owner' argument there? Do you ever use
'__set__'?

 

[Arnaud Delobelle]

Arnaud and Imri, too -

No.  foo.bar is *not* really a function/method call.

It is. The keyword here is 'descriptor'. Maybe reading this will
help:

http://users.rcn.com/python/download/Descriptor.htm

    Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

HTH

I appreciate the help, but ...

Actually, it does not help, because ...

My understanding is that foo.bar does *not* create a new object.  All it
does is return the value of the bar attribute of object foo.  What new
object is being created?

A bound method.

Compare the following:

foo.bar

Foo.__dict__['bar'].__get__(foo, Foo)

If I have:

    class Foo(object):
        def bar(self): pass

And I do:

    foo = SomeClass()

then:

    foo.bar

should return the same (identical) object everytime, no?  yes?

No. This is what I explained in my original reply.

I'm still confused.

That's because you need to adjust your understanding.

 

[Mel]

Your understanding is not correct.


A bound method. This happens through the descriptor-protocol. Please see
this example:


class Foo(object):
def bar(self):
pass


f = Foo()
a = Foo.bar
b = f.bar
c = f.bar

print a, b, c
print id(b), id(c)

(What Diez said.) From what I've seen, f.bar creates a bound method
object by taking the unbound method Foo.bar and binding its first
parameter with f. This is a run-time operation because it's easy to
re-assign some other function to the name Foo.bar, and if you do, the
behaviour of f.bar() will change accordingly.

You can get some very useful effects from these kinds of games. You
can make f into a file-like object, for example, with

import sys
f.write = sys.stdout.write

Here, f.write *is* a straight attribute of f, although it's a built-in
method of the file class. It's still bound, in a way, to sys.stdout.
I'm assuming that a different example could create an attribute of f
that's a bound method of some other object entirely. I've verified
that f.write('howdy') prints 'howdy' on standard output.

Mel.

 

[castironpi]

(What Diez said.)  From what I've seen, f.bar creates a bound method
object by taking the unbound method Foo.bar and binding its first
parameter with f.  This is a run-time operation because it's easy to
re-assign some other function to the name Foo.bar, and if you do, the
behaviour of f.bar() will change accordingly.

You can get some very useful effects from these kinds of games.  You
can make f into a file-like object, for example, with

import sys
f.write = sys.stdout.write

Here, f.write *is* a straight attribute of f, although it's a built-in
method of the file class.  It's still bound, in a way, to sys.stdout.
  I'm assuming that a different example could create an attribute of f
that's a bound method of some other object entirely.  I've verified
that f.write('howdy') prints 'howdy' on standard output.

Accordingly,

f.write= types.MethodType( sys.stdout.__class__.write, sys.stdout ).

It depends on what you want the implicit first (self) to be-- f or
sys.stdout.

But how come this works?
... write= sys.stdout.write
... def g( self ):
... self.write( 'was in \'g\'\n' )
...was in 'g'

Shouldn't 'write' be getting extra parameters? Sounds fishy, not to
mix metaphors.

 

[castironpi]

Accordingly,

f.write= types.MethodType( sys.stdout.__class__.write, sys.stdout ).

It depends on what you want the implicit first (self) to be-- f or
sys.stdout.

But how come this works?


...     write= sys.stdout.write
...     def g( self ):
...             self.write( 'was in \'g\'\n' )
...>>> c= C()

was in 'g'

Shouldn't 'write' be getting extra parameters?  Sounds fishy, not to
mix metaphors.

Ah. Because this doesn't.
... write= sys.stdout.__class__.write #<--
... def g( self ):
... self.write( 'was in \'g\'\n' )
...Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in g
File "c:\programs\python\lib\io.py", line 1236, in write
if self.closed:
AttributeError: 'C' object has no attribute 'closed'
That is, because sys.stdout.write is -not- a user-defined function.
What it is, is a bound member function, and only the former is
converted/wrapped/bound*, as it is in the subsequent example.

*/ whatever.

 

[castironpi]

Ah.  Because this doesn't.
That is, because sys.stdout.write is -not- a user-defined function.
What it is, is a bound member function, and only the former is
converted/wrapped/bound*, as it is in the subsequent example.

Last thing, sorry: Does MethodType.__get__ just return self, or is it
not called, due to some type checking?

 

[Peter Otten]

4. Both points above follow from the fact that foo.bar is really a
function call that returns a (potentially) new object: in fact what
really happens is something like

Arnaud and Imri, too -

No. foo.bar is *not* really a function/method call.

Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

HTH

I appreciate the help, but ...

Actually, it does not help, because ...

My understanding is that foo.bar does *not* create a new object. All it
does is return the value of the bar attribute of object foo. What new
object is being created?

If the attribute has a __get__() method that's completely under the
attribute's control:
.... def __get__(self, *args):
.... print "__get__%s" % (args,)
.... return self.next()
.... def next(self):
.... self.count += 1
.... return self.count
.... count = -1
........ bar = Bar()
.... def __repr__(self): return "foo"
....__get__(foo, <class '__main__.Foo'>)
3

Peter

 

[Bruno Desthuilliers]

Dave Kuhlman a écrit :

4. Both points above follow from the fact that foo.bar is really a
function call that returns a (potentially) new object: in fact what
really happens is something like

Arnaud and Imri, too -

No. foo.bar is *not* really a function/method call.

Foo.__dict__['bar'].__get__(foo, Foo).

So every time foo.bar is executed an object is (or may be) created,
with a new id.

HTH

I appreciate the help, but ...

Actually, it does not help, because ...

My understanding is that foo.bar does *not* create a new object.

Given your implementation, foo.bar *does* create a new object on each
lookup. This object is an instancemethod instance, that is, a thin
wrapper around Foo, foo and Foo.__dict__['bar']. Foo.__dict__['bar'] is
a function instance, it's an attribute of class Foo, and the function
class implements the descriptor protocol. So when bar is looked on a Foo
instance, bar.__get__ is called with foo and Foo as arguments, and
returns an instancemethod instance that has .im_self bound to foo,
..im_class bound to Foo, and .im_func bound to Foo.__dict__['bar'].

All it
does is return the value of the bar attribute of object foo.

Yes. But since function bar is a descriptor, foo.bar is a computed
attribute, which value is a newly created instancemethod object.

What new
object is being created?

An instancemethod.

If I have:

class Foo(object):
def bar(self): pass


And I do:

foo = SomeClass()

then:

foo.bar

should return the same (identical) object everytime, no? yes?
No.

I'm still confused.

Then you have to learn how attribute lookup works in Python.

 

[Bruno Desthuilliers]

Mel a écrit :
(snip)

(What Diez said.) From what I've seen, f.bar creates a bound method
object by taking the unbound method Foo.bar and binding its first
parameter with f.

Nope. it's Foo.__dict__['bar'] (that is, the function bar defined in the
namespace of class Foo) that creates a bound instancemethod object when
looked up on a Foo instance - or an unbound instancemethod object when
looked up on Foo. FWIW, types.UnboundMethodType and types.MethodType are
both aliases to instancemethod type.

 

[Bruno Desthuilliers]

Erik Max Francis a écrit :

Python promises that the behavior is the same. It does not promise that
the _objects_ will be the same, which is what `is` determines. That is,
you're not doing a useful test here.

FWIW, two methods are "the same" if they have identical (function, obj,
cls) attributes. Looks like the equality test is correctly implemented:
True

HTH