# Seek support for new slice syntax PEP.

Discussion in 'Python' started by Dave, Dec 14, 2009.

1. ### DaveGuest

Just as sets may now be written as {3,'hi'}, I propose that slices
should be available using [start:end] syntax. Following example comes
from projecteuler.net problem 166. The Numeric community would also
like this, as would the general python user. The slice notation would
require one ":" between the brackets to differentiate it from a list,
which is similar to the set notation requirement that disambiguates it
from a dictionary.

Several times now I've wanted python slice notation. Perhaps I'll
write a Python Enhancement Proposal. I stored slices of vector array

edge = 4
indexes = []
n = edge
nn = n**2
for i in range(edge):
indexes.extend([
slice(i*n,(i+1)*n,1), # rows
slice(i,nn,n), # cols
])

row_slices = indexes[0::2]
col_slices = indexes[1::2]
slash = slice(n-1,n*(n-1)+1,n-1)
backslash = slice(0,nn,n+1)

Which could have been written in a manner completely consistent with
other python shorthand notations and for which python "cannot
possibly" use the notation for some other purpose,

edge = 4
indexes = []
n = edge
nn = n**2
for i in range(edge):
indexes.extend([
[i*n: (i+1)*n] # rows
[i: nn: n], # cols
])

row_slices = indexes[0::2]
col_slices = indexes[1::2]
slash = [n-1: n*(n-1)+1: n-1]
backslash = [0: nn: n+1]

Dave, Dec 14, 2009

2. ### geremy condraGuest

geremy condra, Dec 14, 2009

3. ### Colin W.Guest

On 14-Dec-09 13:03 PM, Dave wrote:
> Just as sets may now be written as {3,'hi'}, I propose that slices
> should be available using [start:end] syntax. Following example comes
> from projecteuler.net problem 166. The Numeric community would also
> like this, as would the general python user. The slice notation would
> require one ":" between the brackets to differentiate it from a list,
> which is similar to the set notation requirement that disambiguates it
> from a dictionary.
>
> Several times now I've wanted python slice notation. Perhaps I'll
> write a Python Enhancement Proposal. I stored slices of vector array
>
>
> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
> indexes.extend([
> slice(i*n,(i+1)*n,1), # rows
> slice(i,nn,n), # cols
> ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = slice(n-1,n*(n-1)+1,n-1)
> backslash = slice(0,nn,n+1)
>
>
> Which could have been written in a manner completely consistent with
> other python shorthand notations and for which python "cannot
> possibly" use the notation for some other purpose,
>
>
> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
> indexes.extend([
> [i*n: (i+1)*n] # rows
> [i: nn: n], # cols
> ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = [n-1: n*(n-1)+1: n-1]
> backslash = [0: nn: n+1]

Yes, we know that PEP 3003 applies but I see no harm in discussing
possible enhancements.

The existing slice seems a little different from what you are proposing:
An object usually containing a portion of a sequence. A slice is created
using the subscript notation, [] with colons between numbers when
several are given, such as in variable_name[1:3:5].
or:
Slice objects
Slice objects are used to represent slices when extended slice syntax is
used. This is a slice using two colons, or multiple slices or ellipses
separated by commas, e.g., a[i:j:step], a[i:j, k:l], or a[..., i:j].
They are also created by the built-in slice() function.

If your scheme flies, would it be practicable to use the same syntax
as a range generator?

range(i, j, k) => i:j:k

so range(10, 2) => :10:2

i.e. we could write for i in :10:2:

or the more common:
range(10) => :10

Colin W.

Colin W., Dec 14, 2009
4. ### geremy condraGuest

> Yes, we know that PEP 3003 applies but I see no harm in discussing possible
> enhancements.

I don't think the OP knew that the moratorium was in effect. That's why I
brought it up.

Geremy Condra

geremy condra, Dec 14, 2009
5. ### Terry ReedyGuest

On 12/14/2009 1:03 PM, Dave wrote:
> Just as sets may now be written as {3,'hi'}, I propose that slices
> should be available using [start:end] syntax.

I believe this has been proposed and rejected on one of the py-dev,
py-ideas, or py-3k lists, but I would have to check to be sure.

Extended slices would also have to be allowed.

> The Numeric community would also like this,

Evidence? Are you one of the leaders thereof?

> as would the general python user.

A few might but most would find it useless since they never write
explicit slice objects and would have to learning something new to read
code like the below.

Many more people uses range objects (xrange in 2.x). A range object has
the same info as a slice object *plus* it is iterable. So it would be
MUCH more useful if that notation created a range object.

for i in [1:n]: ...

So I would oppose the slice proposal in favor of a range proposal.
However, his has also, I believe, been rejected, as an abbreviation too far.

> Several times now I've wanted python slice notation. Perhaps I'll
> write a Python Enhancement Proposal.

That could be useful, even if it gets rejected. Or perhaps this should

> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
> indexes.extend([
> slice(i*n,(i+1)*n,1), # rows
> slice(i,nn,n), # cols
> ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = slice(n-1,n*(n-1)+1,n-1)
> backslash = slice(0,nn,n+1)
>
> Which could have been written in a manner completely consistent with
> other python shorthand notations

Python avoids getting to chicken-scratchy. There was even a proposal
(rejected, see 3099) to deprecate [1,2,3], etc, in favor of list(1,2,3),
etc.

> and for which python "cannot possibly" use the notation for some

other purpose,

But it could, see above.

> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
> indexes.extend([
> [i*n: (i+1)*n] # rows
> [i: nn: n], # cols
> ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = [n-1: n*(n-1)+1: n-1]
> backslash = [0: nn: n+1]

I find this currently to be less readable.

Terry Jan Reedy

Terry Reedy, Dec 14, 2009
6. ### Terry ReedyGuest

On 12/14/2009 1:10 PM, geremy condra wrote:

The moratorium does not stop proposals for things to be added after the
moratorium ends. But it does show that Guido and the devs are reluctant
to make *any* change to the core syntax of 3.x without really good
reason. Absent that, I would not mind if the syntax remains frozen for
the rest of 3.x. A minor abbreviation that makes the language look more
like Perl will not cut it.

Terry Jan Reedy

Terry Reedy, Dec 14, 2009
7. ### Lie RyanGuest

On 12/15/2009 5:03 AM, Dave wrote:
> Just as sets may now be written as {3,'hi'}, I propose that slices
> should be available using [start:end] syntax. Following example comes
> from projecteuler.net problem 166. The Numeric community would also
> like this, as would the general python user. The slice notation would
> require one ":" between the brackets to differentiate it from a list,
> which is similar to the set notation requirement that disambiguates it
> from a dictionary.

I would prefer [a: b, ...] syntax to become an ordered dictionary
literal (if it would ever gain traction).

Lie Ryan, Dec 14, 2009
8. ### Steven D'ApranoGuest

On Mon, 14 Dec 2009 13:40:38 -0500, Colin W. wrote:

> Yes, we know that PEP 3003 applies but I see no harm in discussing
> possible enhancements.

You bored? Looking for something to do?

I've lost all enthusiasm for discussing language enhancements, regardless
of whether I'm for or against the change, knowing that there's no way it
could be added to the language, and when the Python moratorium ends the
discussion will just happen all over again.

--
Steven

Steven D'Aprano, Dec 14, 2009
9. ### Carl BanksGuest

On Dec 14, 10:03 am, Dave <> wrote:
> Just as sets may now be written as {3,'hi'}, I propose that slices
> should be available using [start:end] syntax.  Following example comes
> from projecteuler.net problem 166.  The Numeric community would also
> like this, as would the general python user.  The slice notation would
> require one ":" between the brackets to differentiate it from a list,
> which is similar to the set notation requirement that disambiguates it
> from a dictionary.
>
> Several times now I've wanted python slice notation.  Perhaps I'll
> write a Python Enhancement Proposal.  I stored slices of vector array
>
> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
>     indexes.extend([
>         slice(i*n,(i+1)*n,1),       # rows
>         slice(i,nn,n),              # cols
>         ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = slice(n-1,n*(n-1)+1,n-1)
> backslash = slice(0,nn,n+1)
>
> Which could have been written in a manner completely consistent with
> other python shorthand notations and for which python "cannot
> possibly" use the notation for some other purpose,
>
> edge = 4
> indexes = []
> n = edge
> nn = n**2
> for i in range(edge):
>     indexes.extend([
>         [i*n: (i+1)*n]                  # rows
>         [i: nn: n],                      # cols
>         ])
>
> row_slices = indexes[0::2]
> col_slices = indexes[1::2]
> slash = [n-1: n*(n-1)+1: n-1]
> backslash = [0: nn: n+1]

-1

Explicit creation of slice objects is an uncommon need and there is no
reason to support it with its own syntax.

I'd agree with Terry Reedy that range/xrange is far more commonly used
than slice objects, and if a floating slice syntax were ever added to
Python it ought to be used for range.

If you need to use a lot of slice objects you can lower your code
footprint by defining a helper class like this (adapt as needed):

class SliceCreator(object):
def __getitem__(self,loc):
if not isinstance(loc,slice):
raise TypeError
return loc
slc = SliceCreator()

slash = slc[n-1: n*(n-1)+1: n-1]

It might have been a reasonable idea for slice (and, perhaps, range)
to use slice notation rather than a function call, on the thinking
that the notational convenience outweighs the fact that you're not
actually getting an item, but it's too late for that.

Carl Banks

Carl Banks, Dec 14, 2009
10. ### NobodyGuest

On Mon, 14 Dec 2009 10:03:16 -0800, Dave wrote:

> Just as sets may now be written as {3,'hi'}, I propose that slices
> should be available using [start:end] syntax. Following example comes
> from projecteuler.net problem 166. The Numeric community would also
> like this, as would the general python user. The slice notation would
> require one ":" between the brackets to differentiate it from a list,
> which is similar to the set notation requirement that disambiguates it
> from a dictionary.
>
> Several times now I've wanted python slice notation. Perhaps I'll
> write a Python Enhancement Proposal.

Would it suffice to add the equivalent of numpy.s_ as a builtin?

> from numpy import s_
> s_[1:2:3]
slice(1, 2, 3)
> s_[1:2:3, ..., 4:5]
(slice(1, 2, 3), Ellipsis, slice(4, 5, None))

Or would it be possible to define "slice" itself so that it implements
__getitem__ and __getslice__?

Nobody, Dec 14, 2009
11. ### Anh Hai TrinhGuest

>         > from numpy import s_
>         > s_[1:2:3]
>         slice(1, 2, 3)
>         > s_[1:2:3, ..., 4:5]
>         (slice(1, 2, 3), Ellipsis, slice(4, 5, None))
>
> Or would it be possible to define "slice" itself so that it implements
> __getitem__ and __getslice__?

Indeed!

Python 2.6.4 (r264:75706, Oct 27 2009, 06:25:13)
[GCC 4.4.1] on linux2

>>> class slice(object):

.... @staticmethod
.... def __getitem__(sliceobj):
.... return sliceobj

>>> slice = slice()

>>> slice[:]

slice(None, None, None)

>>> slice[1::-1]

slice(1, None, -1)

>>> range(10).__getitem__(slice[::2])

[0, 2, 4, 6, 8]

----aht

Anh Hai Trinh, Dec 15, 2009
12. ### BearophileGuest

Steven D'Aprano:

> I've lost all enthusiasm for discussing language enhancements

That's probably the main downside of the moratorium. Humans need to
play some to keep their will to work and improve things.

Bye,
bearophile

Bearophile, Dec 15, 2009
13. ### r0gGuest

Terry Reedy wrote:
> On 12/14/2009 1:10 PM, geremy condra wrote:
>
> The moratorium does not stop proposals for things to be added after the
> moratorium ends. But it does show that Guido and the devs are reluctant
> to make *any* change to the core syntax of 3.x without really good
> reason. Absent that, I would not mind if the syntax remains frozen for
> the rest of 3.x. A minor abbreviation that makes the language look more
> like Perl will not cut it.
>
> Terry Jan Reedy
>

I agree, string slicing syntax is already a little oblique, it certainly
doesn't need complicating.

Anyway...

Simple is better than complex.
If the implementation is hard to explain, it's a bad idea.

Roger.

r0g, Dec 15, 2009
14. ### Gregory EwingGuest

Terry Reedy wrote:
> So it would be
> MUCH more useful if that notation created a range object.
>
> for i in [1:n]: ...
>
> So I would oppose the slice proposal in favor of a range proposal.

Another possibility would be to unify range and slice
objects so that they're actually the same thing. Then
the same notation could be used for both purposes.

--
Greg

Gregory Ewing, Dec 17, 2009
15. ### Colin W.Guest

On 16-Dec-09 19:23 PM, Gregory Ewing wrote:
> Terry Reedy wrote:
>> So it would be MUCH more useful if that notation created a range object.
>>
>> for i in [1:n]: ...
>>
>> So I would oppose the slice proposal in favor of a range proposal.

>
> Another possibility would be to unify range and slice
> objects so that they're actually the same thing. Then
> the same notation could be used for both purposes.
>

This would be good if the increment could also be handled.

Terry Reedy suggested:- for i in [1:n]: ...

Are the brackets really needed?

Colin W.

Colin W., Dec 17, 2009
16. ### NobodyGuest

On Mon, 14 Dec 2009 14:18:49 -0500, Terry Reedy wrote:

> Many more people uses range objects (xrange in 2.x). A range object has
> the same info as a slice object *plus* it is iterable.

This isn't quite true, as a range cannot have a stop value of None, i.e.
you can't represent [n:] or [:] etc as a range. Similarly for using
negative stop values for indices relative to the end of the sequence being
sliced.

Also, aside from the semantics of slice objects themselves, slice notation
isn't limited to a single slice object; it can also return a tuple of
slices and values, e.g.:

> numpy.s_[1::2,...,3,4:5:6]
(slice(1, None, 2), Ellipsis, 3, slice(4, 5, 6))

For a single slice, enumerating over a slice with an unspecified stop
value would be equivalent to itertools.count(). Negative stop values won't
work.

For a multi-dimensional slice, with everything specified, you would
probably want to iterate over the cartesian product (i.e. N nested loops
for an N-dimensional slice). But this won't work if anything other than
the outermost loop has an unspecified stop value, or if you use an
ellipsis within a slice.

Oh, and being able to slice a slice could be quite useful, i.e.:

[10:90:10][2::2] == [30:90:20]

cf:
> numpy.arange(100)[10:90:10][2::2]
array([30, 50, 70])
> numpy.arange(100)[30:90:20]
array([30, 50, 70])

Nobody, Dec 18, 2009
17. ### Colin W.Guest

On 17-Dec-09 20:00 PM, Nobody wrote:
> On Mon, 14 Dec 2009 14:18:49 -0500, Terry Reedy wrote:
>
>> Many more people uses range objects (xrange in 2.x). A range object has
>> the same info as a slice object *plus* it is iterable.

>
> This isn't quite true, as a range cannot have a stop value of None, i.e.
> you can't represent [n:] or [:] etc as a range. Similarly for using
> negative stop values for indices relative to the end of the sequence being
> sliced.
>
> Also, aside from the semantics of slice objects themselves, slice notation
> isn't limited to a single slice object; it can also return a tuple of
> slices and values, e.g.:
>
> > numpy.s_[1::2,...,3,4:5:6]
> (slice(1, None, 2), Ellipsis, 3, slice(4, 5, 6))
>
> For a single slice, enumerating over a slice with an unspecified stop
> value would be equivalent to itertools.count(). Negative stop values won't
> work.
>
> For a multi-dimensional slice, with everything specified, you would
> probably want to iterate over the cartesian product (i.e. N nested loops
> for an N-dimensional slice). But this won't work if anything other than
> the outermost loop has an unspecified stop value, or if you use an
> ellipsis within a slice.
>
> Oh, and being able to slice a slice could be quite useful, i.e.:
>
> [10:90:10][2::2] == [30:90:20]
>
> cf:
> > numpy.arange(100)[10:90:10][2::2]
> array([30, 50, 70])
> > numpy.arange(100)[30:90:20]
> array([30, 50, 70])
>

You don't say, but seem to imply that the slice components include None.

Section 5.3.3 of the Python doc for 2.6.4 has

The lower and upper bound expressions, if present, must evaluate to
plain integers; defaults are zero and the sys.maxint, respectively. If
either bound is negative, the sequenceâ€™s length is added to it. The
slicing now selects all items with index k such that i <= k < j where i
and j are the specified lower and upper bounds. This may be an empty
sequence. It is not an error if i or j lie outside the range of valid
indexes (such items donâ€™t exist so they arenâ€™t selected).

Colin W.

Colin W., Dec 18, 2009
18. ### Colin W.Guest

On 17-Dec-09 20:00 PM, Nobody wrote:
> On Mon, 14 Dec 2009 14:18:49 -0500, Terry Reedy wrote:
>
>> Many more people uses range objects (xrange in 2.x). A range object has
>> the same info as a slice object *plus* it is iterable.

>
> This isn't quite true, as a range cannot have a stop value of None, i.e.
> you can't represent [n:] or [:] etc as a range. Similarly for using
> negative stop values for indices relative to the end of the sequence being
> sliced.
>
> Also, aside from the semantics of slice objects themselves, slice notation
> isn't limited to a single slice object; it can also return a tuple of
> slices and values, e.g.:
>
> > numpy.s_[1::2,...,3,4:5:6]
> (slice(1, None, 2), Ellipsis, 3, slice(4, 5, 6))
>
> For a single slice, enumerating over a slice with an unspecified stop
> value would be equivalent to itertools.count(). Negative stop values won't
> work.
>
> For a multi-dimensional slice, with everything specified, you would
> probably want to iterate over the cartesian product (i.e. N nested loops
> for an N-dimensional slice). But this won't work if anything other than
> the outermost loop has an unspecified stop value, or if you use an
> ellipsis within a slice.
>
> Oh, and being able to slice a slice could be quite useful, i.e.:
>
> [10:90:10][2::2] == [30:90:20]
>
> cf:
> > numpy.arange(100)[10:90:10][2::2]
> array([30, 50, 70])
> > numpy.arange(100)[30:90:20]
> array([30, 50, 70])
>

You don't say, but seem to imply that the slice components include None.

Section 5.3.3 of the Python doc for 2.6.4 has

The lower and upper bound expressions, if present, must evaluate to
plain integers; defaults are zero and the sys.maxint, respectively. If
either bound is negative, the sequenceâ€™s length is added to it. The
slicing now selects all items with index k such that i <= k < j where i
and j are the specified lower and upper bounds. This may be an empty
sequence. It is not an error if i or j lie outside the range of valid
indexes (such items donâ€™t exist so they arenâ€™t selected).

Colin W.

Colin W., Dec 18, 2009
19. ### NobodyGuest

On Fri, 18 Dec 2009 09:49:26 -0500, Colin W. wrote:

> You don't say, but seem to imply that the slice components include None.

That's how missing components are implemented at the language level:

> class foo:
= def __getitem__(self, s):
= return s
=
> x = foo()
> x[::]
slice(None, None, None)
> x[1::2]
slice(1, None, 2)

The defaults of zero, sys.maxint and one apply to built-in types, but
nothing forces user-defined types to behave this way.

Or maybe I misunderstood your point.

Nobody, Dec 19, 2009
20. ### Colin W.Guest

On 18-Dec-09 23:16 PM, Nobody wrote:
> On Fri, 18 Dec 2009 09:49:26 -0500, Colin W. wrote:
>
>> You don't say, but seem to imply that the slice components include None.

>
> That's how missing components are implemented at the language level:
>
> > class foo:
> = def __getitem__(self, s):
> = return s
> =
> > x = foo()
> > x[::]
> slice(None, None, None)
> > x[1::2]
> slice(1, None, 2)
>
> The defaults of zero, sys.maxint and one apply to built-in types, but
> nothing forces user-defined types to behave this way.
>
> Or maybe I misunderstood your point.
>

No, it seems that the implementation is a little different from the doc.

You are right:
*** Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
(Intel)] on win32. ***
>>> a= range(10)
>>> a[2:8:2]

[2, 4, 6]
>>> a[2::2]

[2, 4, 6, 8]
>>> a[2:None:2]

[2, 4, 6, 8]
>>>

I had expected the last to be rejected, but it fits with the overall
philosophy.

Colin W

Colin W., Dec 19, 2009