Operator Overloading

[Sebastien Boisgerault]

I wonder if the following quotation from the Python Reference Manual
(release 2.3.3) about operator overloading is true :

"For example, if a class defines a method named __getitem__(), and x
is
an instance of this class, then x is equivalent to
x.__getitem__(i)"

Consider the following code:

from Numeric import *
a = array([0.5])
a array([ 0.5])
from Numeric import *
a = array([0.5])
a[0]

0.5

but

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __getitem__

I probably understand why the call to __getitem__: there is no
__dict__ attribute in the variable a and not even a __class__
attribute to find what
the class of the variable a is:
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __class__

I didn't know that you could have an instance without a __class__
attribute ... Anyway, if the __class__ attribute was defined, I guess
that the call to a.__getitem__(0) would succeed because "__getitem__"
belongs to the __dict__
of the type of a.
True

But then, why does the call to a[0] succeed ? It should be exactly
equivalent
to a.__getitem__[0], right ?

 

[Peter Maas]

I wonder if the following quotation from the Python Reference Manual
(release 2.3.3) about operator overloading is true :

"For example, if a class defines a method named __getitem__(), and x
is
an instance of this class, then x is equivalent to
x.__getitem__(i)" [...]

from Numeric import *
a = array([0.5])
a[0]

0.5

but

a.__getitem__(0)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __getitem__

The quotation above is true. Short form:

IF __getitem__ in dict THEN [] works.

What you are wondering about is the opposite direction

IF [] works THEN __getitem__ in dict.

but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

Mit freundlichen Gruessen,

Peter Maas

 

[Pierre Barbier de Reuille]

Peter Maas a écrit :

I wonder if the following quotation from the Python Reference Manual
(release 2.3.3) about operator overloading is true :

"For example, if a class defines a method named __getitem__(), and x
is
an instance of this class, then x is equivalent to
x.__getitem__(i)"
[...]

from Numeric import *
a = array([0.5])
a[0]

0.5

but


a.__getitem__(0)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __getitem__

The quotation above is true. Short form:

IF __getitem__ in dict THEN [] works.

That's not true !!! I's true only for classes defined in Python. If you
try to define a class in C, defining the __getitem__ method does not
lead to the existence of "[]". But when you define "[]" it creates the
"__getitem__" method. This is part of the reason why you cannot dervie
from two types if they derives from different classes written in another
language and imported in Python (just try to create a class deriving
from list and dict ...).

Pierre

 

[Nick Coghlan]

What you are wondering about is the opposite direction

IF [] works THEN __getitem__ in dict.

but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

This is correct, and true of any C extension - classes implemented in C only
need to define the appropriate function pointers in their type structures in
order for Python to find the relevant methods.

Classes that are being *nice* about it put in the actual magic method names as
well (e.g. try "list.__getitem__"), but it is by no means required.

Cheers,
Nick.

 

[Sebastien Boisgerault]

I wonder if the following quotation from the Python Reference Manual
(release 2.3.3) about operator overloading is true :

"For example, if a class defines a method named __getitem__(), and x
is an instance of this class, then x is equivalent to
x.__getitem__(i)" [...]

from Numeric import *
a = array([0.5])
a[0]
0.5

but


a.__getitem__(0)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __getitem__

The quotation above is true. Short form:

IF __getitem__ in dict THEN [] works.

Not exactly the same assertion:
replace "__getitem__ in dict" by "__getitem__ in the class dict"
and more importantly "[] works" by "[] and __getitem__" are *equivalent*.

Here, "__getitem__" does belongs to type(a).__dict__,
so "[]" and "__getitem__" should work exactly the same
according to the reference, but they don't.

[...]
but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

I would buy your argument if I couldn't find the "__getitem__" method.
But it does exist ! Except that it is hidden is the class __dict__ and
apparently cannot be recovered from the instance.__getitem__ call ...

Thanks for your help,

SB

 

[Sebastien Boisgerault]

What you are wondering about is the opposite direction

IF [] works THEN __getitem__ in dict.

but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

This is correct, and true of any C extension - classes implemented in C only
need to define the appropriate function pointers in their type structures in
order for Python to find the relevant methods.

Classes that are being *nice* about it put in the actual magic method names as
well (e.g. try "list.__getitem__"), but it is by no means required.

Nick, Pierre, Peter,

Thanks for your answers. I guess that in the case of the Numeric
package, there was at least the *intent* to support __getitem__
because it is provided at the class level (which is not required,
right ?):

from Numeric import *
a = array([3.14])
Array = type(a)
a.__getitem__(0)

Traceback (most recent call last):
3.14

I guess that everything would work as expected if __class__ was defined at
the instance level ...

Regards,

SB

 

[Peter Maas]

Sebastien Boisgerault schrieb: [...]
but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

I would buy your argument if I couldn't find the "__getitem__" method.
But it does exist ! Except that it is hidden is the class __dict__ and
apparently cannot be recovered from the instance.__getitem__ call ...

This is strange because it's not the same behaviour as in a pure
Python class. If you really need __getitem__ you could write a
wrapper (or derive, not sure if it is possible):

class sbArray(object):
def __init__(self, plainarray):
self.data = plainarray
def __getitem__(self, idx):
return self.data[idx]

 

[Nick Coghlan]

from Numeric import *
a = array([3.14])
Array = type(a)
a.__getitem__(0)

Traceback (most recent call last):

3.14

That's just. . . odd. It's possible that array has a custom __getattr__ or
__getattribute__ implementation that is not falling back to the class dictionary
correctly. The direct class access doesn't use the custom handlers, and gets the
correct answer. However, you'd need someone more familiar with Numeric than I am
to say exactly what is going on.

If Numeric has its own mailing list, I'd try asking there.

Cheers,
Nick.

 

[Bengt Richter]

I wonder if the following quotation from the Python Reference Manual
(release 2.3.3) about operator overloading is true :

"For example, if a class defines a method named __getitem__(), and x
is an instance of this class, then x is equivalent to
x.__getitem__(i)" [...]
from Numeric import *
a = array([0.5])
a[0]

0.5

but


a.__getitem__(0)

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __getitem__

The quotation above is true. Short form:

IF __getitem__ in dict THEN [] works.

Not exactly the same assertion:
replace "__getitem__ in dict" by "__getitem__ in the class dict"
and more importantly "[] works" by "[] and __getitem__" are *equivalent*.

Here, "__getitem__" does belongs to type(a).__dict__,
so "[]" and "__getitem__" should work exactly the same
according to the reference, but they don't.

[...]
but this is not what the Python Reference Manual says. Im not a
Numeric expert but AFAIK Numeric arrays are basically C arrays
having [] intrinsically so there's no need no deliver it via
__getitem__.

I would buy your argument if I couldn't find the "__getitem__" method.
But it does exist ! Except that it is hidden is the class __dict__ and
apparently cannot be recovered from the instance.__getitem__ call ...

Thanks for your help,

SB

I believe the new style classes require looking for a descriptor (which
includes functions, which become bound methods via their descriptor nature)
with the attribute name given, before grabbing something from the instance dict.
Otherwise instance attributes would always shadow corresponding method or property
names, and those things wouldn't work, or would work as in the old style classes.

Therefore looking for __getitem__ is a little trickier than it might seem.
It has to work like any other name, so a.__getitem__ can't be treated differently from a.foo.

So as you noticed, the first place to look is in type(a).__dict__ (which is
also an attribute lookup BTW, with name '__dict__' which could be a descriptor
too, but we'll ignore that for the moment. See further below for that).

Consider that given

>>> import Numeric
>>> a = Numeric.array([0.5])
>>> a

array([ 0.5])

this

>>> a[0]

0.5

produces the same result as this

>>> type(a).__dict__['__getitem__'].__get__(a, type(a)).__call__(0)

0.5

So what happens when we look for type(a).__dict__? '__dict__' is just a name,
so we have to look for a method or property in the chain of base classes.
The buck presumably stops at some base class descriptor named __dict__, if any,
and that descriptor, if present, determines what you get. The chain of search
for type(a).__dict__ presumably starts looking in type(type(a)).__dict__, but
<type 'type'>

is already at the end of the chain.
<dictproxy object at 0x0090B4D0>

Remember, we're going to look _in_ the __dict__, not _for_ it here ;-)

But this has already been processed through the attribute magic, so to see
what '__dict__' is without that processing, we use the proxy to look it up:

>>> type(type(a)).__dict__['__dict__']

<attribute '__dict__' of 'type' objects>

Which is a descriptor if it has a __get__ method:

>>> type(type(a)).__dict__['__dict__'].__get__

<method-wrapper object at 0x0090B4D0>

Sure enough, so we pass type(a) and its type to that, and get back type(a).__dict__ the long way:

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))

<dictproxy object at 0x009015B0>

now we can look for __getitem__ in that:

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))['__getitem__']

<slot wrapper '__getitem__' of 'array' objects>

Which being the function of a method, should have a descriptor's __get__ method, by which
to become a bound method:

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))['__getitem__'].__get__

<method-wrapper object at 0x0090B4D0>

So we pass it the instance and its type:

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))['__getitem__'].__get__(a, type(a))

<method-wrapper object at 0x009015B0>

Which should have a __call__ method if it's callable:

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))['__getitem__'].__get__(

a, type(a)).__call__
<method-wrapper object at 0x0090B4D0>

Which we can call with the index

>>> type(type(a)).__dict__['__dict__'].__get__(type(a), type(type(a)))['__getitem__'].__get__(

a, type(a)).__call__(0)
0.5

Fortunately, we don't normally have to think about all that when we write

>>> a[0]

0.5

;-)

Caveat: this is not based on reading the code internals, so I could be misinterpreting surface
appearances, but at least it ought to be clear that a[0] involves a lot of dynamic decisions that
might ordinarlily not be taken, but which must be allowed for in looking for an innocent method
like __getitem__ ;-)

[Hm, just looking in oubox: this apparently didn't go out the other day.]

Regards,
Bengt Richter