__getitem__ method on (meta)classes

[ron]

Why doesn't this work?
.... class baz(object):
.... def __getitem__(cls, idx): return cls.lst[idx]
.... __getitem__=classmethod(__getitem__)
.... baz.lst = lst
.... return baz
....

f = foo([1,2,3])
f[0]

Traceback (most recent call last):


I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

Thanks,
rg

 

[Simon Percivall]

Well, they're not synonymous. At least not in that context. If you
haven't already tried it, what you're doing will fail for instances as
well. Look in typeobject.c to see why. The gist of it is that the
special methods are looked up on the type rather than the instance (on
the metaclass rather than on the class, in your case), so the internal
lookup mechanism ignore instance attributes completely in this case.

 

[Bengt Richter]

Why doesn't this work?
... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

f = foo([1,2,3])
f[0]

Traceback (most recent call last):


I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

Yes, but what was your "x"?
Note:
... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__ = classmethod(__getitem__)
... baz.lst = lst
... return baz
...

>>> f = foo([1,2,3])
>>> f

<class '__main__.baz'>

Your "x" was the baz *class*, and the baz *class* is not subscriptable *itself*,
even though it defines subscripting for its instances.
To be ordinarily subscriptable, type(f).__getitem__ would have to succeed, which it doesn't:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: type object 'type' has no attribute '__getitem__'

However, a baz instance is different:
<__main__.baz object at 0x02EF168C>

Its type does have a __getitem__ attribute:
<bound method type.__getitem__ of <class '__main__.baz'>>

I think "bound method" is a bit of a misnomer for a classmethod, though
it is a method, and it is bound, just to the class instead of the instance.
(see below)
1

Or straightforward indexing syntax:

>>> finst[0], finst[1], finst[2]

(1, 2, 3)

Contrast with the way an ordinary method repr's when you access it
via class and instance:
... def ordinary_method(self): return 'Hi'
... 'Hi'

Note the difference between ordinary and classmethod:
<bound method type.__getitem__ of <class '__main__.baz'>>

BTW, I see a class factory, but no "(meta)"class per se.

Regards,
Bengt Richter

 

[Leif K-Brooks]

Why doesn't this work?


... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

No, with new-style classes, x[y] and type(x).__getitem__(y) are
synonymous. This works:
.... class bar(type):
.... def __getitem__(self, index):
.... return self.lst[index]
.... class baz(object):
.... __metaclass__ = bar
.... baz.lst = lst
.... return baz
....

>>> foo([1,2,3])[0]

1

 

[Michele Simionato]

Leif Brooks:

with new-style classes, x[y] and type(x).__getitem__(y) are
synonymous.

Yes, but check the discussion around SF789262. The call ``x[y]`` is
converted to ``type(x).__getitem__(x,y)``
*only if* ``__getitem__`` is explicitely defined in ``type(x)``. If
``type(x)`` does not define ``__getitem__`` directly, but only
indirectly via delegation (i.e. overriding ``__getattribute__``),
then the second form (i.e. ``type(x).__getitem__(x,y)``) works but
not the first one (i.e. ``x[y]`` raises an error).

Michele Simionato

 

[Ron Garret]

Why doesn't this work?

def foo(lst):

... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

No, with new-style classes, x[y] and type(x).__getitem__(y) are
synonymous.

Ah.

Did you mean type(x).__getitem__(x,y)?

And where is this documented?

rg

 

[Ron Garret]

Why doesn't this work?

def foo(lst):

... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

f = foo([1,2,3])
f[0]

Traceback (most recent call last):

f.__getitem__(0) 1

I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

Yes, but what was your "x"?
Note:
... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__ = classmethod(__getitem__)
... baz.lst = lst
... return baz
...

f = foo([1,2,3])
f

<class '__main__.baz'>

Your "x" was the baz *class*, and the baz *class* is not subscriptable
*itself*,

But why not? __getitem__ is a class method, not an instance method.

(You can treat that as a rhetorical question. I know the answer already
from other postings in this thread.)

BTW, I see a class factory, but no "(meta)"class per se.

That's because I took out extraneous code to not distract from the
problem I was having, and in the process took out all the useful stuff


What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

In other words, I want to map values onto their representations using
the () operator, and representations onto their values using the []
operator. That requires me to define the [] operator on the enumerated
classes themselves. So my actual code was:

def enum(vals):
class enum(object):
def __init__(self, val):
try:
self.val = type(self).vals.index(val)
except:
raise TypeError, "%s is not a valid %s" % (val, type(self))
def __getitem__(self, index): return self.vals.index(index)
__getitem__=classmethod(__getitem__)
enum.vals = vals
return enum


(Actually, what I'm really trying to do is create a whole hierarchy of
static type descriptors and automatically generate database schema from
those descriptors, but that's a story for another day.)

Thanks for all the responses.

rg

 

[Steven Bethard]

What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

Use a class with __call__ and __getitem__:

py> class enum(object):
.... def __init__(self, vals):
.... self.vals = vals
.... def __call__(self, val):
.... return self.vals.index(val)
.... def __getitem__(self, index):
.... return self.vals[index]
....
py> lst = 'abcd'
py> x = 'b'
py> e1 = enum(lst)
py> v = e1(x)
py> (x in lst) and (e1[v] == x)
True

STeVe

 

[Bengt Richter]

Why doesn't this work?


def foo(lst):

... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

I thought x[y] and x.__getitem__(y) were supposed to always be
synonymous.

No, with new-style classes, x[y] and type(x).__getitem__(y) are
synonymous.

Ah.

Did you mean type(x).__getitem__(x,y)?

Not if x is a classmethod, since type(x).__getitem__ gets you a bound-to-the-class method
instead of the usual unbound method, which would want the x instance as the first argument.
... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... __getitem__=classmethod(__getitem__)
... baz.lst = lst
... return baz
...

>>> f = foo([1,2,3])()
>>> type(f).__getitem__

>>> type(f).__getitem__(0)

1

Leaving out the classmethod:
... class baz(object):
... def __getitem__(cls, idx): return cls.lst[idx]
... baz.lst = lst
... return baz
...

>>> f = foo([1,2,3])()
>>> type(f).__getitem__

>>> type(f).__getitem__(f, 0)

1

And where is this documented?

Between the lines in my previous post ;-)


Regards,
Bengt Richter

 

[Ron Garret]

Wow, this is really cool:

What I'm really trying to do is to create enumerated types...

And the code I ended up with is:

# Inheriting from type, not object, is the key:
class enum_metaclass(type):
def __getitem__(self, index):
return self.vals[index]

def enum(vals):
class enum(object):
__metaclass__ = enum_metaclass
def __init__(self, val):
try:
self.val = type(self).vals.index(val)
except:
raise TypeError, "%s is not a valid %s" % (val, type(self))
enum.vals = vals
return enum


So now I can do, e.g.:

e1 = enum(['a','b','c'])
e1('a')

e1('x')

Traceback (most recent call last):
File "<stdin>", line 1, in ?

But I can also do this:
....
a
b
c
Note that there's no __iter__ method anywhere! It makes an interesting
little puzzle to figure out why this works. (It totally blew me away
when I first tried it. Took me about five minutes of head scratching to
figure it out.)

rg

 

[Steven Bethard]

And the code I ended up with is:

# Inheriting from type, not object, is the key:
class enum_metaclass(type):
def __getitem__(self, index):
return self.vals[index]

def enum(vals):
class enum(object):
__metaclass__ = enum_metaclass
def __init__(self, val):
try:
self.val = type(self).vals.index(val)
except:
raise TypeError, "%s is not a valid %s" % (val, type(self))
enum.vals = vals
return enum

A good example of why 99.9% of people don't need metaclasses. See my
solution using __call__ in the other post.

Note that there's no __iter__ method anywhere! It makes an interesting
little puzzle to figure out why this works. (It totally blew me away
when I first tried it. Took me about five minutes of head scratching to
figure it out.)

Best to add an __iter__ if you want one. The __getitem__ protocol is
basically deprecated though it works for backwards compatibility.

STeVe

 

[Ron Garret]

Not if x is a classmethod,

Oh yeah, right. Duh!

Between the lines in my previous post ;-)

I see. I guess I wasn't asking a stupid question then

rg

 

[Ron Garret]

What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

Use a class with __call__ and __getitem__:

py> class enum(object):
... def __init__(self, vals):
... self.vals = vals
... def __call__(self, val):
... return self.vals.index(val)
... def __getitem__(self, index):
... return self.vals[index]
...
py> lst = 'abcd'
py> x = 'b'
py> e1 = enum(lst)
py> v = e1(x)
py> (x in lst) and (e1[v] == x)
True

Yeah, except I actually left out one thing: I also want type(v)==e1.

rg

 

[Bengt Richter]

What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

Use a class with __call__ and __getitem__:

py> class enum(object):
... def __init__(self, vals):
... self.vals = vals
... def __call__(self, val):
... return self.vals.index(val)
... def __getitem__(self, index):
... return self.vals[index]
...
py> lst = 'abcd'
py> x = 'b'
py> e1 = enum(lst)
py> v = e1(x)
py> (x in lst) and (e1[v] == x)
True

For that, why not just

class enum(list):
def __call__(self, val): return self.index(val)

Regards,
Bengt Richter

 

[Ron Garret]

What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

Use a class with __call__ and __getitem__:

py> class enum(object):
... def __init__(self, vals):
... self.vals = vals
... def __call__(self, val):
... return self.vals.index(val)
... def __getitem__(self, index):
... return self.vals[index]
...
py> lst = 'abcd'
py> x = 'b'
py> e1 = enum(lst)
py> v = e1(x)
py> (x in lst) and (e1[v] == x)
True

For that, why not just

class enum(list):
def __call__(self, val): return self.index(val)

Because I forgot to mention that I also want type(v)==e1. (Enum is a
small part of a static typing system for automatically generating
database schema from data models.)

rg

 

[Steven Bethard]

What I'm really trying to do is to create enumerated types such that if:

e1 = enum(lst) and v = e1(x)

then

(x in lst) and (e1[v] == x)

Use a class with __call__ and __getitem__:

py> class enum(object):
... def __init__(self, vals):
... self.vals = vals
... def __call__(self, val):
... return self.vals.index(val)
... def __getitem__(self, index):
... return self.vals[index]
...
py> lst = 'abcd'
py> x = 'b'
py> e1 = enum(lst)
py> v = e1(x)
py> (x in lst) and (e1[v] == x)
True

Yeah, except I actually left out one thing: I also want type(v)==e1.

Why? In Python usually you rely on duck-typing and not explicit type
checks. What is it that you're trying to gain by asserting type(v) == e1?

STeVe

 

[Bengt Richter]

Not if x is a classmethod,

[/QUOTE]
D'oh. I meant "not if __getitem__ is a classmethod" ;-P

Oh yeah, right. Duh!


I see. I guess I wasn't asking a stupid question then

Seriously, no ;-) It's hard to find good documentation on deep nitty-gritties.
The most reliable documentation of actual software behavior is inevitably
the code that implements it, though that often doesn't give a clue as to the
whys of the whats and hows shown.

As Raymond Hettinger says in his nice doc (which chew slowly for best nutrition ;-)
http://users.rcn.com/python/download/Descriptor.htm
"""
For objects, the machinery is in object.__getattribute__ which transforms
b.x into type(b).__dict__['x'].__get__(b, type(b)).
The implementation works through a precedence chain that gives data descriptors
priority over instance variables, instance variables priority over non-data descriptors,
and assigns lowest priority to __getattr__ if provided.
The full C implementation can be found in PyObject_GenericGetAttr() in Objects/object.c.
"""

The code does show what really happens ;-)
(UIAM, for types the analogous stuff is in Objects/typeobject.c)


Most documentation is probably reachable via
http://www.python.org/
and
http://www.python.org/doc/

but sometimes specific stuff is hard to find. Descriptors are
discussed in various links of

http://www.python.org/doc/newstyle.html

and IMO Raymond's

http://users.rcn.com/python/download/Descriptor.htm

is the most readable.

If you google for descrintro you will find a lot of discussion ;-)

Regards,
Bengt Richter

 

[Ron Garret]

Yeah, except I actually left out one thing: I also want type(v)==e1.

Why? In Python usually you rely on duck-typing and not explicit type
checks. What is it that you're trying to gain by asserting type(v) == e1?[/QUOTE]

Clarity. I want to be able to distinguish a member of an enumeration
from a string or an integer for the same reason one would want to
distinguish 123 from "123".

rg

 

[Steven Bethard]

Why? In Python usually you rely on duck-typing and not explicit type
checks. What is it that you're trying to gain by asserting type(v) == e1?

Clarity. I want to be able to distinguish a member of an enumeration
from a string or an integer for the same reason one would want to
distinguish 123 from "123".[/QUOTE]

So then you don't necessarily need that type(v) == e1, you just need
type(v) != str. How about:

py> class enum(object):
.... class item(object):
.... def __init__(self, val):
.... self.val = val
.... def __repr__(self):
.... return 'enum.item(%r)' % self.val
.... def __init__(self, vals):
.... self.items = [type(self).item(val) for val in vals]
.... self._val_item_map = dict(zip(vals, self.items))
.... self._index_item_map = dict(enumerate(self.items))
.... def __call__(self, val):
.... try:
.... return self._val_item_map[val]
.... except KeyError:
.... raise TypeError("%s is not a valid %s" % (val, type(self)))
.... def __getitem__(self, index):
.... return self._index_item_map[index]
.... def __iter__(self):
.... return iter(self.items)
....
py> e1 = enum(['a', 'b', 'c'])
py> e1('a')
enum.item('a')
py> e1('x')
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 15, in __call__
TypeError: x is not a valid <class '__main__.enum'>
py> e1[2]
enum.item('c')
py> list(e1)
[enum.item('a'), enum.item('b'), enum.item('c')]

Note that unlike your posted solution, the __getitem__ here returns
enum.item objects intead of the strings. Not sure which you intended
here, but it should be easy enough to switch it the other way. I've
also added to auxiliary dicts to make lookup quicker. (They also give
you some more flexibility if you need to support other operations.)

STeVe

 

[Steven Bethard]

py> class enum(object):
... class item(object):
... def __init__(self, val):
... self.val = val
... def __repr__(self):
... return 'enum.item(%r)' % self.val
... def __init__(self, vals):
... self.items = [type(self).item(val) for val in vals]
... self._val_item_map = dict(zip(vals, self.items))
... self._index_item_map = dict(enumerate(self.items))

Sorry, this was before I decided I should save items as an instance
variable. Drop this map, it's unnecessary.

... def __call__(self, val):
... try:
... return self._val_item_map[val]
... except KeyError:
... raise TypeError("%s is not a valid %s" % (val, type(self)))
... def __getitem__(self, index):
... return self._index_item_map[index]

And replace this with self.items[index]

STeVe