slicing, mapping types, ellipsis etc.

[Jerry Sievers]

Fellow Pythonists;

I am totally puzzled on the use of slicing on mapping types and
especially unsure on use of the Ellipsis... and slicing syntax that
has two or more groups seperated by comma.

I am referring to (from the manual);

Slice objects
Slice objects are used to represent slices when extended
slice syntax is used. This is a slice using two colons, or multiple
slices or ellipses separated by commas, e.g., a[i:j:step], a[i:j,
k:l], or a[..., i:j]. They are also created by the built-in slice()
function.

A quick example on how this is used and/or pointer to more reading is
greatly appreciated.

Thanks

 

[Jerry Sievers]

I'm going to assume the following is self-explanatory - type the commands
in yourself.

[snip]

Yes, it certainly is self-explanatory. And, I appreciate your
response.

However, my question pertains to advanced use of the slicing features
which I have found to be not totally explained in the docs.

The use of [start:stop:step] on sequence types is NOT unclear to me at
all.

I'd like to know;

1. what the Ellipsis object or ... syntax is used for
2. what a slice [..., j:k:l] does
3. how slices are applied to object of mapping types

Thank you

 

[Steven Bethard]

1. what the Ellipsis object or ... syntax is used for
2. what a slice [..., j:k:l] does

My understanding is that the Ellipsis object is intended primarily for
Numeric/numarray use. Take a look at:

http://stsdas.stsci.edu/numarray/numarray-1.1.html/node26.html

3. how slices are applied to object of mapping types

The short answer is that they aren't. Slices aren't hashable, so dicts
can't handle them:

>>> d = {}
>>> d = {1:'a', 2:'b'}
>>> d[1:2]

Traceback (most recent call last):
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: unhashable type

You can, however handle them if you write your own mapping type:
.... def __getitem__(self, x):
.... if isinstance(x, slice):
.... return x.start, x.stop, x.step
.... else:
.... return x
....

>>>
>>> m = M()
>>> m[1] 1
>>> m[1:2]

(1, 2, None)

Hope this helps!

Steve

 

[Scott David Daniels]

I'd like to know;
1. what the Ellipsis object or ... syntax is used for

Use the Numeric (or numarray) package to really see these in use.
Otherwise, you mostly get to implement it yourself.

> 2. what a slice [..., j:k:l] does

Again, see numeric. Essentially, select parts of a matrix (or higher
dimensional array) based on a stride in the last dimension.

3. how slices are applied to object of mapping types

One way to investigate this interactively:

class CheckOut(object):
def __init__(self, name=None):
self.name = name or str(self)
def __getitem__(self, *args, **kwargs):
print '%s._gi_(*%r, **%r):' % (self.name, args, kwargs),
return input()

>>> c = CheckOut('a')
>>> b = c[1, ...] c.gi(*(1, Ellipsis), **{}): self.name * 7
>>> b

'aaaaaaa'

--Scott David Daniels
(e-mail address removed)

 

[Bengt Richter]

I'm going to assume the following is self-explanatory - type the commands
in yourself.

[snip]

Yes, it certainly is self-explanatory. And, I appreciate your
response.

However, my question pertains to advanced use of the slicing features
which I have found to be not totally explained in the docs.

The use of [start:stop:step] on sequence types is NOT unclear to me at
all.

I'd like to know;

1. what the Ellipsis object or ... syntax is used for
2. what a slice [..., j:k:l] does
3. how slices are applied to object of mapping types

... def __getitem__(self, i): print i; return None
...

>>> c=C()
>>> c[1] 1
>>> c[1:2] slice(1, 2, None)
>>> c[1:2:3] slice(1, 2, 3)
>>> c[:2:3] slice(None, 2, 3)
>>> c[::3] slice(None, None, 3)
>>> c[::] slice(None, None, None)
>>> c[1,::] (1, slice(None, None, None))
>>> c[1,...,::] (1, Ellipsis, slice(None, None, None))
>>> c[1, 2, ..., 3:4] (1, 2, Ellipsis, slice(3, 4, None))
>>> c[...]

Ellipsis

An object implementing __getitem__ (and/or __setitem__) will be passed arguments
as above and is free to use them in any way that makes sense to it's author.
Making sense to the users of the author's software is another thing ;-)

AFAIK Ellipsis is just a special builtin object that can be used as a handy
standard place holder in the get/setitem context. But any other builtin
could be passed also, e.g.,

>>> c[1,2,abs,3]

(1, 2, <built-in function abs>, 3)

so AFAIK it's just that an Ellipsis reference is generated when you use ... in the
getitem context. The compiler generates a reference to the built-in constant, so
you can make a test for it for your object's logic, e.g.
... def __getitem__(self, i):
... if i is Ellipsis or isinstance(i, tuple) and Ellipsis in i:
... print 'got ... in the index arg %r'%(i,)
... else:
... print 'did not get ... in the index arg %r'%(i,)
...

>>> c=C()
>>> c[1,2] did not get ... in the index arg (1, 2)
>>> c[::] did not get ... in the index arg slice(None, None, None)
>>> c[::, ...] got ... in the index arg (slice(None, None, None), Ellipsis)
>>> c[...] got ... in the index arg Ellipsis
>>> c[Ellipsis]

got ... in the index arg Ellipsis

But interestingly, compare the appearance of Ellipsis in the disassembly
of foo and bar (LOAD_GLOBAL vs LOAD_CONST). The global you can shadow or
rebind, but the const is apparently known by the compiler, and it is arranged
that they both refer to the same object by default, so the 'is' test succeeds.
... def __getitem__(self, i): return i
... def foo(self): return self[...]
... def bar(self): return self.foo() is Ellipsis
...

>>> c= C()
>>> c[...] is Ellipsis True
>>> c.foo() is Ellipsis True
>>> c.bar() True
>>> import dis
>>> dis.dis(C)

Disassembly of __getitem__:
2 0 LOAD_FAST 1 (i)
3 RETURN_VALUE

Disassembly of bar:
4 0 LOAD_FAST 0 (self)
3 LOAD_ATTR 1 (foo)
6 CALL_FUNCTION 0
9 LOAD_GLOBAL 2 (Ellipsis)
12 COMPARE_OP 8 (is)
15 RETURN_VALUE

Disassembly of foo:
3 0 LOAD_FAST 0 (self)
3 LOAD_CONST 1 (Ellipsis)
6 BINARY_SUBSCR
7 RETURN_VALUE

HTH

Regards,
Bengt Richter

 

[Caleb Hattingh]

I'm going to assume the following is self-explanatory - type the commands
in yourself.

'>>> a = 'hello my name is caleb'
'>>> b = a.split()
'>>> b
['hello', 'my', 'name', 'is', 'caleb']
'>>> b[0]
'hello'
'>>> b[1]
'my'
'>>> b[0:1]
['hello']
'>>> b[0:2]
['hello', 'my']
'>>> b[-1]
'caleb'
'>>> b[:-2]
['hello', 'my', 'name']
'>>> b[2:-2]
['name']
'>>>

thx
Caleb

 

[Nick Coghlan]

Fellow Pythonists;

I am totally puzzled on the use of slicing on mapping types and

It's generally not supported (since slices are designed to work with the
numerical indices of a sequence, not the arbitrary keys of a mapping).

The results of d.keys(), d.values() & d.items() can all be sliced like any other
sequence, though.

especially unsure on use of the Ellipsis... and slicing syntax that
has two or more groups seperated by comma.

Others have explained these - one point that may not have been entirely clear is
that none of the classes in the standard library make use of these (although the
point may have been implied by the references to PyNumeric as the place to look
for more exact documentation. Python's own documentation of the slice() builtins
makes reference to Numeric!).

Cheers,
Nick.

 

[Kent Johnson]

It's generally not supported (since slices are designed to work with the
numerical indices of a sequence, not the arbitrary keys of a mapping).

Section 5.3.3 of the Language Reference seems to say that with extended
slicing, the slice elements are used to create a key that indexes a
mapping. "The semantics for an extended slicing are as follows. The
primary must evaluate to a mapping object, and it is indexed with a key
that is constructed from the slice list, as follows."

From my understanding of this thread so far, extended slicing is used
as a form of indexing in Numeric. Are Numeric arrays considered
mappings? Or is this paragraph in 5.3.3 off the mark?

Maybe instead of referring to mappings it should say "The primary must
implement __getitem__(), which is called with a value that is
constructed from the slice list, as follows."

Kent

 

[Nick Coghlan]

Maybe instead of referring to mappings it should say "The primary must
implement __getitem__(), which is called with a value that is
constructed from the slice list, as follows."

In the section you mention, 'mapping' is equivalent to 'has __getitem__
defined', and I'd be surprised if it's an isolated usage. Python actually has
trouble distinguishing between sequences and mappings, as anyone who as tried to
use the 'isMapping' API would know (the isMapping API uses much the same
definition as the reference manual does - so all sequences show up as mappings,
as they map indices and slices to objects).

Section 3 of the reference manual is actually more use for anyone developing
custom types that override special methods. E.g. proper handling of slice
objects is described here under container emulation:
http://www.python.org/dev/doc/devel/ref/sequence-types.html

Regards,
Nick.

 

[Kent Johnson]

In the section you mention, 'mapping' is equivalent to 'has __getitem__
defined', and I'd be surprised if it's an isolated usage. Python
actually has trouble distinguishing between sequences and mappings, as
anyone who as tried to use the 'isMapping' API would know (the isMapping
API uses much the same definition as the reference manual does - so all
sequences show up as mappings, as they map indices and slices to objects).

Section 3 of the reference manual is actually more use for anyone
developing custom types that override special methods. E.g. proper
handling of slice objects is described here under container emulation:
http://www.python.org/dev/doc/devel/ref/sequence-types.html

I understand that the distinction between sequences and mappings is
fuzzy, as both use __getitem__() for access. But the usage of 'mapping'
in 5.3.3 is inconsistent with the section you refer to. That page says
"It is also recommended that mappings provide the methods keys(),
values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
itervalues(), iteritems(), pop(), popitem(), copy(), and update()
behaving similar to those for Python's standard dictionary objects."

and under __getitem__(): "For *sequence* types, the accepted keys should
be integers and slice objects."

I just think 5.3.3 is unnecessarily opaque. Particularly since the only
built-in mapping (dict) doesn't even accept simple slices as indices.

Kent