AttributeError in "with" statement (3.2.2)

[Steven D'Aprano]

]

After reading your post, I think I have worked out where our disagreement
lines: you think that bound methods and instance methods are not the same
thing, and that a function defined inside a class is different from a
function outside of a class.

For example, you say:

If so, the output is a *bound method*. In your example
above, func is not an instance method and obj is not a bound method. It
is simply an partially evaluated curried function or if you prefer, a
bound function. Take you pick, or make up your own term, but it is NOT
an instance method,

and later on:

These are bound methods. The instance methods are the functions wrapped.

I am afraid you are mistaken. What you say may very well apply to other
languages, but in Python, def creates functions no matter where you
execute it. Always and without exception.

I admit to an earlier mistake: I thought that conversion from function to
method occurred once, when the class statement was executed, but I was
mistaken. Re-reading Raymond Hettinger's excellent article on the
descriptor protocol reminded me that methods are created on an as-needed
basis, at runtime.

Back to methods and def. So let's see what happens in Python 3.1:
.... pass
........ def ham(self): # Allegedly an instance method
.... pass
....

According to your various statements, spam is a function and ham is an
instance method. Merely storing a outside function inside a class doesn't
create an instance method, it creates what you call a bound method
(although I'm unclear what you think the difference is). So let's see
what happens when we compare an alleged "bound method that is not an
instance method" with an actual instance method:
True

Their types are identical. We get the same thing when we compare an
actual function with a function object created inside a class, which you
claim is actually an instance method:

type(spam) is type(K.__dict__['ham']) # Bypass descriptor protocol

True

Nor is there any difference in type between bound and unbound methods:
they are both instance methods differing only in whether or not they have
the first argument "self" available. Bound is an adjective, not part of
the type: small method, obfuscated method, buggy method, or bound method.

Python 3 no longer uses unbound methods, since they are functionally
identical to the unwrapped function, so this snippet is from Python 2.6:

py> k = K()
py> type(k.ham) is type(K().ham) # bound vs bound
True
py> type(k.ham) is type(K.ham) # bound vs unbound
True


In Python, instance methods are wrappers around function objects; they
are created on call, and generally do not exist *anywhere* until needed,
or if you store a reference to them.
False

Under normal circumstances, the only persistent object is the function,
which you can extract from the (class or instance) __dict__ or the method
wrapper:

a.__func__ is K.__dict__['ham']

True

Methods are created by the descriptor protocol: when you use the normal
a.b syntax to access K.__dict__['ham'], the metaclass calls the __get__
method of function ham. If you call it from an instance, you get a method
object bound to that instance. If you call it from the class, in Python 2
you get a method object not bound to an instance, in Python 3 you get the
function without a wrapper.

[Aside: class methods and static methods also work the same way, via
__get__ and the descriptor protocol, but behave differently. Class
methods reuse the instance method type, which is somewhat confusing, and
static methods just return the function.]

So, what the hell does all this mean in practice?

Most of the time, absolutely nothing. That's why I started this
discussion with the disclaimer that I didn't think it was worth changing
the (not quite right) definition of "method" you originally quoted. Most
of the time, we only access methods via the instance.method syntax, which
gives us an instance method. By the principle that if it quacks like a
duck and swims like a duck, it is good enough to call it a duck, I'm
happy to agree that ham here is a method:

class K:
def ham(self): pass

That covers 99% of all use-cases and is good enough for most situations.

But the OP ran into one of those edge cases in the 1%, where things are
not so simple. He was attempting to create an instance method called
"__exit__" and store it in the instance __dict__ instead of the class
__dict__. This is a perfectly reasonable thing to do, but there are two
gotchas to it:

* The descriptor protocol doesn't get used for instance
lookups. That's why you can't stick properties in an
instance, only in the class.

* CPython, and I believe Jython, don't look up special
dunder methods like __exit__ on the instance, only on
the class.

So even if the OP manually created an instance method rather than relying
on the descriptor protocol, his per-instance __exit__ would not be called
without extra work.

 

[Terry Reedy]

]

After reading your post, I think I have worked out where our disagreement
lines: you think that bound methods and instance methods are not the same
thing,

Do you agree that an unbound method and a bound method are different? In
Python, as indicated by the glossary entry, an unspecified 'method' is
usually meant to be an unbound method. It is am important distinction
and I do not see why you insist on confusing the two.

and that a function defined inside a class is different from a
function outside of a class.

That, and your repetition of the same claim further on, is a insulting lie.

Def statements always create functions. I have known that for 14 years
since the first day I started with Python. I have never thought
differently. If you actually think that I have, you are wrong.

What I have said from my first response is that a function that is an
attribute of a class, *whether defined in or outside the class*, gets
special treatment when accessed via an instance of the class. And that
is the truth.

If you are ever interested in learning anything from me on this subject,
re=read what I already wrote with a more open mind than you have so far.
Otherwise, I am done.

 

[Ethan Furman]

]

After reading your post, I think I have worked out where our disagreement
lies: you think that bound methods and instance methods are not the same
thing,

Do you agree that an unbound method and a bound method are different? In
Python, as indicated by the glossary entry, an unspecified 'method' is
usually meant to be an unbound method.

I think you two are in violent agreement as far as how Python is
functioning, and the conflict is in the names given to the various
pieces... I think a glossary would help (please correct me):

function: callable code suite

method: function that lives in a class

unbound method: function that lives in a class

bound method: callable wrapper around function that has been extracted
from class that will supply the instance object to the function (note:
Python does not save these, they are recreated at each lookup)


and here is where I think you two diverge:

instance method (Steven): a bound method that has been saved into the
instance __dict__ (no matter how created)

instance method (Terry): a function that must be looked up in the class


Have I missed anything?

Honestly-trying-learn-the-distinctions-ly yours,

~Ethan~

 

[Ethan Furman]

]

After reading your post, I think I have worked out where our
disagreement
lies: you think that bound methods and instance methods are not the same
thing,

Do you agree that an unbound method and a bound method are different?
In Python, as indicated by the glossary entry, an unspecified 'method'
is usually meant to be an unbound method.

I think you two are in violent agreement as far as how Python is
functioning, and the conflict is in the names given to the various
pieces... I think a glossary would help (please correct me):

function: callable code suite

method: function that lives in a class

unbound method: function that lives in a class

bound method: callable wrapper around function that has been extracted
from class that will supply the instance object to the function (note:
Python does not save these, they are recreated at each lookup)

I think the above 'bound method' definition should be attributed to
Terry, and Steven's follows:

bound method: callable wrapper around any function that will accept an
instance object as the first parameter, and the wrapper will supply said
instance object when calling the function (and where/how function was
created is irrelevent, as is where the wrapper is stored)

 

[Steven D'Aprano]

On Thu, 15 Dec 2011 19:39:17 -0500, Terry Reedy wrote: [...]

After reading your post, I think I have worked out where our
disagreement lines: you think that bound methods and instance methods
are not the same thing,

Do you agree that an unbound method and a bound method are different?

"Are different" or "are different types"?

Bound and unbound methods in Python 2 are the same type. They are
different in the sense that one is effectively a curry of the other, but
they are implemented as a single type. So presumably the difference is a
bound method has a slot filled and an unbound method doesn't, or some
other implementation detail.

In Python 3, unbound methods don't exist.

In
Python, as indicated by the glossary entry, an unspecified 'method' is
usually meant to be an unbound method.

I think you are badly confused here. The glossary talks about calling
methods as an attribute of an instance, e.g. instance.method(), and the
method receiving the instance as the first argument. That's a bound
method, not unbound. How do you conclude that this indicates that an
unspecified "method" is unbound?

Your conclusion is especially nonsensical because unbound methods don't
exist in Python 3 -- you're alleging that the usual meaning of "method"
is something which no longer exists in the language!

method
A function which is defined inside a class body. If called as
an attribute of an instance of that class, the method will get
the instance object as its first argument (which is usually
called self). See function and nested scope.

http://docs.python.org/glossary.html

It is am important distinction
and I do not see why you insist on confusing the two.

It is not an important distinction, and I am not confusing the two. Bound
or unbound, it is still an instance method.

That, and your repetition of the same claim further on, is a insulting
lie.

If you can't assume I'm carrying on this discussion in good faith, then
we have a serious problem. This is not like you Terry -- I've been
reading your posts for many years, and you're not usually so obstreperous.

Def statements always create functions. I have known that for 14 years
since the first day I started with Python. I have never thought
differently. If you actually think that I have, you are wrong.

I'm glad to hear it. But nevertheless you have made statements (which I
quoted, and you deleted from your reply) that suggest the opposite. If I
have misinterpreted them, or if you had worded them badly, there's no
need to attribute malice to me. Calling me a liar for something said in
good faith and which I prefixed with "I think" is simply not cool.

 

[Steven D'Aprano]

On 12/16/2011 4:22 AM, Steven D'Aprano wrote:

[...]

I think you two are in violent agreement as far as how Python is
functioning, and the conflict is in the names given to the various
pieces... I think a glossary would help (please correct me):

function: callable code suite

Sure, why not?

Informally: a function is some object which can be called and returns a
result. Hence we sometimes call str() and int() functions even though
they are actually implemented as types. "Callable" would be better for
such informal use, but newbies may not recognise the term.

To be more precise: a function is a particular type of callable object
created by the def and lambda statements.

method: function that lives in a class

Informally, I'm happy with that definition, and use it myself. Under 99%
of circumstances, it would be overly pedantic to describe spam in the
following as anything other than a method:

class C:
def spam(self): pass

Call it the duck-type principle: since spam quacks like a duck, we might
as well call it a duck. If you only care about the easy cases, then spam
is a method and we're done.

But as a formal definition, it fails on two fronts: functions that live
inside classes are not actually methods, they are functions, and while
methods are commonly found in classes, they can also be found in
instances, or anywhere else for that matter.


Functions inside classes are still functions
--------------------------------------------

I said that since spam quacks like a method, we might as well call it a
method. But it doesn't swim like a method: it's actually still a function
with a remarkable power of mimicry.

There are places and times where the fact that spam is not actually a
method comes through. The three most important examples in my opinion are:

* inside the class suite, during class definition time, if you use
or inspect spam, you will see it is a function and not a method;

* if you bypass the getattr mechanism for looking up attributes, e.g.
use C.__dict__['spam'] instead of C().spam, you will see that spam
is stored in the dict as a function object, not a method object;

* (Python 3 only) if use the standard getattr mechanism, but call it
on the class instead of an instance, you will get the function
object instead of a method.

So, to be precise: spam is actually a function. It is stored as a
function inside C.__dict__, but when you retrieve it using C().spam
Python automagically builds a method object wrapping spam and gives you
that instead. This distinction is important, well, hardly ever, but on
the rare occasion it is important, you need to know it.

One example of when it may be important: because methods are created as
needed, you can't assume that identity checks will necessarily pass for
methods: `inst.spam is inst.spam` may not be true.


Methods can be created outside of classes
-----------------------------------------

The normal way to get a method is by defining it inside a class. In Java,
I believe that is the *only* way to get methods, but this is Python, and
we have other options. Methods are first class objects, which means they
have a type and a constructor, so if you can get hold of their type, you
can make them by hand.

You can retrieve the method type two ways:

* call type on a method you made the usual way: type(C().spam);

* or import types and use types.MethodType.

Confusingly, the method type calls itself <instancemethod> but is
exported via the types module as MethodType. Oh well.

MethodType requires two arguments, In Python 2, it accepts an optional
third argument. Since Python 2 is the past, I'll only talk about the
Python 3 version, where the signature might look something like this:

MethodType(callable, instance) => instancemethod object

This creates a wrapper around callable (usually, but not necessarily a
function) and binds instance to that wrapper so the first argument
(conventionally called self) can be automatically provided.

Another way to create methods is by calling the function object __get__
method by hand. That's what gets used in the normal C().spam case.

So you can create methods without putting them inside a class. One trick
is that you can customise behaviour on a per-instance basis by overriding
the normal method living inside the class with one which lives inside the
instance:

class C:
def spam(self):
return "Spam spam spam, lovely SPAM!!!"

c = C()
c.spam = types.MethodType(lambda self: "Bloody vikings!", c)

Instance c now has a custom method which overrides C.spam.

[Aside #1: I think this was the original point of contention between
Terry and I. For reasons I still don't understand, it seems to me that he
denies that the per-instance method spam is actually a method, since it
doesn't live inside the class.]

[Aside #2: this per-instance method trick may not work for double-
underscore methods like __exit__, since CPython and Jython at least skip
the instance __dict__ when using dunder methods. In Python 2, it will
work for classic classes, but they're gone in Python 3. If you go all the
way back to the initial post in this thread, that was the OP's problem.]

unbound method: function that lives in a class

An unbound method is merely a method that doesn't know what instance to
use, that is, it cannot automatically supply "self".

In Python 2, the easiest way to get an unbound method is by retrieving
the method from the class instead of an instance:

C.spam => returns method which is not bound ("unbound method")
C().spam => returns method which is bound ("bound method")

Since unbound methods are more or less functionally identical to the
underlying function (the only difference is they do a type-check on the
argument), Python 3 gets rid of them and and just uses the function
object itself.

So all the reasons for why methods are not *strictly* functions living
inside classes apply equally to unbound methods, since unbound methods
are merely a particular form of method.

bound method: callable wrapper around function that has been extracted
from class that will supply the instance object to the function (note:
Python does not save these, they are recreated at each lookup)

Bound methods are not *necessarily* extracted from a class, although they
*usually* are. But wherever it comes from, a bound method knows which
instance to provide as first argument ("self") to the function; otherwise
we'd call it an unbound method.

and here is where I think you two diverge:

instance method (Steven): a bound method that has been saved into the
instance __dict__ (no matter how created)

Certainly not! If I gave that impression, I apologize.

Every thing I have said above refers to *instance* methods, or just
"methods" for lazy people. Instance methods get their name from the fact
that they supply the instance as first argument to the function ("self").
If they instead supply the class, we call them a class method; if they
don't supply anything, we call them a static method (much to the
confusion of Java programmers, who use the same names for different
things).

You can even create your own method types:

http://code.activestate.com/recipes/577030-dualmethod-descriptor/

 

[Terry Reedy]

It is am important distinction [unbound versus bound]

It is not an important distinction, and I am not confusing the two.

So we agree on the distinction but disagree on its importance.
Let us leave it at that.

Bound or unbound, it is still an instance method.

OK. So 'instance method' is a bit ambiguous (more than I thought, or
would prefer) in that it can refer to unbound methods, bounds methods,
or both. So be it.

If you can't assume I'm carrying on this discussion in good faith,

If you can't assume that I am intelligent and experienced enough to know
the meaning of def, one on the most basic aspects of Python, and you are
unwilling to give me the benefit of any doubt you might have on that
score, but instead go on to 'correct' me publicly, then no, I can't.

I'm glad to hear it. But nevertheless you have made statements (which I
quoted, and you deleted from your reply) that suggest the opposite.

OK, let us look at the my statement and your 'repetition of the same
claim further on' that I previously deleted. I wrote

As I indicated in response to Ethan, I would now revised the second
sentence now to "The unbound methods are the function wrapped" or "The
instance-requiring methods are the functions wrapped." But that is not
important here.

In my opinion, there is no way that anyone reading that in good faith
could conclude that I do not know the meaning of def statements. They
are not the subject of discussion in that sentence or the rest of this
thread. But in response you wrote.

I am afraid you are mistaken.

About what? You go on to explain.

What you say may very well apply to other languages,
but in Python, def creates functions no matter where you
execute it. Always and without exception.

So that makes twice that you said or implied that I think the location
of a def statement changes what it creates, even though I explicitly
said the opposite when I suggested that the glossary entry might be
revised. What am I to think at such a tactic. You are normally much more
careful in what you write.

If I have misinterpreted them, or if you had worded them badly,
there's no need to attribute malice to me.

I did not do that. I gave my opinion of your statement, just as you have
given your opinions of mine. I really did not and do not know why you
misrepresented my knowledge of Python. I actually consider overt
intentional malice much less likely than other possibilities.

Calling me a liar

I did not do that, any more than you have been calling me things.

I believe you are asking for the same 'benefit of the doubt' that I
believe you denied to me.