negative stride list slices

[David Abrahams]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

>>> print range(10)[:-3:-1]

[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

Thanks,

 

[Reid Nichol]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

print range(10)[:-3:-1]

[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

Thanks,

http://www.python.org/doc/current/tut/node5.html#SECTION005120000000000000000

 

[Russell Blau]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

print range(10)[:-3:-1]

[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

http://www.python.org/doc/current/tut/node5.html#SECTION005120000000000000000

You probably meant to reference the next section (that bookmark relates to
strings, the next one is lists), but in either case none of the examples in
the tutorial has yet been updated to deal with slices that include a
"stride" as a third parameter.

 

[Peter Otten]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

print range(10)[:-3:-1]

[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

Well, here is my attempt to emulate the algorithm. The trick seems to be to
substitute the start/stop parameters with 0 or length-1 depending on the
sign of step.

# no warranties!
def indices(length, start, stop, step):
if step is None:
step = 1
if step < 0:
if start is None:
start = length-1
elif start < 0:
start += length

if stop is None:
stop = -1
elif stop < 0:
stop += length
else:
if start is None:
start = 0
elif start < 0:
start += length

if stop is None:
stop = length
elif stop < 0:
stop += length

if start > stop:
while start > stop:
yield start
start += step
else:
while start < stop:
yield start
start += step

assert list(indices(10, None, -3, -1)) == range(10)[:-3:-1]
assert list(indices(10, None, -3, -2)) == range(10)[:-3:-2]
assert list(indices(10, 9, -3, -2)) == range(10)[9:-3:-2]
assert list(indices(10, None, None, None)) == range(10)[::]
assert list(indices(10, None, 5, 2)) == range(10)[:5:2]

I have to admit (late but better than never) that Raymond Hettinger's new
builtin reversed() has some merits:

list(reversed(range(10)[-2:])) == range(10)[:-3:-1]

True

Peter

 

[Reid Nichol]

Can anyone explain the logic behind the behavior of list slicing with
negative strides? For example:

print range(10)[:-3:-1]
[9,8]

I found this result very surprising, and would just like to see the
rules written down somewhere.

http://www.python.org/doc/current/tut/node5.html#SECTION005120000000000000000

You probably meant to reference the next section (that bookmark relates to
strings, the next one is lists), but in either case none of the examples in
the tutorial has yet been updated to deal with slices that include a
"stride" as a third parameter.

This was the part that I was refering to:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1


Does it not all work the same in practice?

 

[Peter Hansen]

This was the part that I was refering to:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

Does it not all work the same in practice?

Can you explain how the above diagram explains David's
example then:

>>> s = 'HelpA'
>>> s[:-3:-1]

'Ap'

I don't find much connection between the diagram and
where the -3 is actually slicing either.

-Peter

 

[Shalabh Chaturvedi]

This was the part that I was refering to:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1


Does it not all work the same in practice?

I'd like to add:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

(Is this correct?)

> Can anyone explain the logic behind the behavior of list slicing with
> negative strides? For example:
>

> >>> print range(10)[:-3:-1]

> [9,8]
>
> I found this result very surprising

What were you expecting?

Thanks,
Shalabh

 

[=?ISO-8859-1?Q?Julio_O=F1a?=]

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

regards.

This was the part that I was refering to:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

Does it not all work the same in practice?

Can you explain how the above diagram explains David's
example then:

s = 'HelpA'
s[:-3:-1]

'Ap'

I don't find much connection between the diagram and
where the -3 is actually slicing either.

-Peter

 

[=?ISO-8859-1?Q?Julio_O=F1a?=]

http://www.python.org/doc/2.3.4/whatsnew/section-slices.html

There is more information on extended slicing

Regards.

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

regards.

This was the part that I was refering to:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

Does it not all work the same in practice?

Can you explain how the above diagram explains David's
example then:

s = 'HelpA'
s[:-3:-1]

'Ap'

I don't find much connection between the diagram and
where the -3 is actually slicing either.

-Peter

 

[Russell Blau]

I'd like to add:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

(Is this correct?)

The tutorial says, "The slice from i to j consists of all characters between
the edges labeled i and j, respectively." So [1:3] starts at the left edge
of the character with index 1, and ends at the left edge of the character
with index 3, returning "el". However, a slice from 3:1:-1 starts at the
*right* edge of the character with index 3, and ends at the *right* edge of
the character with index 1, returning "pl". So the above diagram is not
correct for negative steps (the "backwards" slice consists of characters
between the edges labeled j+1 and i+1, in reverse order).

Is this difference intended, or a bug? (One consequence of this difference
is that a slice of [5:0:-1] returns something different from a slice of [5:
:-1]!)

 

[Shalabh Chaturvedi]

I'd like to add:

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

(Is this correct?)

Apparently not.

>>> "HelpA"[-1:-2:-1]

'A'

I would expect 'p'. Also,

>>> "HelpA"[:-1:-1]

''

And I definitely expected 'A'. Now I find it confusing too.

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

This is only true for S[::1] and S[::-1]. In other cases, for -ve step,
the -ve indices seem to indicate a shifted position. I'm using Python 2.3.3.

 

[Peter Hansen]

(Please don't top-post. It really buggers up the quoting
and makes discussion hard.)

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

-Peter

 

[=?ISO-8859-1?Q?Julio_O=F1a?=]

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

Sorry Peter to be to lassy.

Well in the case [:-3:-1] I think this is what is happening
- revers the list
- this left with [0:2] (remember [-3:] returns 2 characters only

in the case [-1:-2:-1]
- revers the string 'Apleh'
- this left with [:1] (this is tricky but -1 is equivalent to 0 as
-2 is to 1, remember you are counting backwards from the end)

The problem is the mening of -1 when reversing the list. The slice is
closed in the lower side and open in the upper one. When reversing the
list the lower end is open and no close as with positive step.

Any comments?

 

[Tim Hochberg]

(Please don't top-post. It really buggers up the quoting
and makes discussion hard.)

Slice has three arguments, [begin:end:step]

when doing s[:-3:-1] you are asking the las to elements of the list in
reversed order.

Uh, yeah. Okay. So let's say that part was obvious. Now
please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value. Presumably one of those refers to the beginning
or end of the list, while the other one refers to something
that is referenced as "-3". Can you point to a diagram or
description in the documentation or tutorial which actually
explains this? Or can you (or anyone) explain it in your
own words? Or is everyone who thinks this makes sense just
pretending to actually understand it?

I've been using extended slicing in Numeric and now numarray for years
and I even wrote some classes that implement extended slicing, but I
still often find it confusing to use negative indices in practice.
However, despite that confusion, I'll give a shot at explaining it.

There are (at least) two issues: what to do with missing indices and
given all the indices, what does it all mean. I'll start with the first
issue, since it's simpler. Of the three indices (start, stop and step),
only start and stop are potentially confusing, since step simply
defaults to 1. Start defaults to the first element (0) if step is
greater than zero, otherwise it defaults to the last element (len(s)-1).
Stop, on the other hand, defaults to one past the edge of the sequence.
If step is positive, it's one past the far edge (len(s)), otherwise it's
one past the near edge. One would be tempted to write this as -1, but
that won't works since that means the end of the sequence, so we'll call
it (-len(s)-1). In summary the missing indices are supplied as:

start = 0 if (step > 0) else len(s)-1
stop = len(s) is (step > 0) else -len(s)-1
step = 1

On to issue number two. I've always understood extended slicing in
relation to the range. In Numeric this relation is best expressed using
Numeric.take, however for general python consumption I'll cast it in
terms of list comprehensions. Since we've already disposed of how to
supply the missing parts of the slice, this will only deal with the case
where all parts are supplied. In addition, if start or stop is less than
zero, we add len(s) so that all values are nonnegative unless stop was
-len(s)-1, in which case it ends up as -1. With that wordy preamble,
here's how I've always understood extended slicing:

s[start:stop:step] ~ [s for i in range(start,stop,step)]

Let me show a few examples:

>>> s = 'python'
>>> s[:-3:-1] 'no'
>>> s[len(s)-1:-3:-1] # Fill in the default values 'no'
>>> [s for i in range(len(s)-1, len(s)-3,-1)] # equiv to s[:-3:-1]

['n', 'o']

>>> s[4::] 'on'
>>> s[4:len(s):1] # Fill in the default values 'on'
>>> [s for i in range(4,len(s),1)] # equiv to s[4::]

['o', 'n']

>>> s[4::-1] 'ohtyp'
>>> s[4:-len(s)-1:-1] # Fill in the default values 'ohtyp'
>>> [s for i in range(4,-1,-1)] # equiv to s[4::-1]

['o', 'h', 't', 'y', 'p']


Is this confusing? Probably. However, I suspect it's a natural
consequence of Python's zero based indexing. Zero based indexing is
great in a lot of ways, but it has some unfortunate corner cases for
negative indices (another topic) and gets downright odd for negative
strides. So it goes.

I-hope-I-didn't screw-that-up-too-badly-ly yours,

-tim

 

[Peter Hansen]

please explain *which* elements are being listed in reverse
order, referring to the index value -3 and the elided
index value.

Well in the case [:-3:-1] I think this is what is happening
- revers the list
- this left with [0:2] (remember [-3:] returns 2 characters only

in the case [-1:-2:-1]
- revers the string 'Apleh'
- this left with [:1] (this is tricky but -1 is equivalent to 0 as
-2 is to 1, remember you are counting backwards from the end)

The problem is the mening of -1 when reversing the list. The slice is
closed in the lower side and open in the upper one. When reversing the
list the lower end is open and no close as with positive step.

Any comments?

Okay, so let me see how this looks with the previous diagram.

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-5 -4 -3 -2 -1

But that only works for the forward or positive step case.
With a negative/reverse step, you first have to reverse
the string:

+---+---+---+---+---+
| A | p | l | e | H |
+---+---+---+---+---+
-1 -2 -3 -4 -5 -6

And now you're in effect going from the start (i.e. the
missing value before the colon) to the -3 point.

Thus 'Ap'.

I don't find it at all intuitive, and certainly the existing
diagram needs to be updated if this is really how it works,
but at least you have an explanation that corresponds to the
values, so thank you.

It does seem very odd that the step of -1 in effect changes
the normal meaning or position of the slice indices, by
first reversing the string. I'm guessing that this is
necessary because any other interpretation leads to even
worse situations than the confusion this inspires, but
perhaps that's not true.

I would have found either 'Apl' or 'eH' more intuitive...

-Peter

 

[Christopher Koppler]

Apparently not.

That should be

+---+---+---+---+---+
| H | e | l | p | A |
+---+---+---+---+---+
0 1 2 3 4 5
-6 -5 -4 -3 -2 -1
| |
start end (defaults for +ve step)
end start (defaults for -ve step)

"HelpA"[-1:-2:-1]

'A'

I would expect 'p'. Also,

"HelpA"[:-1:-1]

''

And I definitely expected 'A'. Now I find it confusing too.

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

As others have explained, that's not easily possible because of Python's
zero-based indexing, which in general is great, but can lead to confusion
because using negative strides indices begin with -1.

This is only true for S[::1] and S[::-1]. In other cases, for -ve step,
the -ve indices seem to indicate a shifted position. I'm using Python 2.3.3.

The defaults here map to S[0:len(S):1] and S[-1:-len(S)-1:-1].

 

[Christopher T King]

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

No. S[a:b:1] includes the start index (a), but excludes the stop index
(b). Similarly, S[b:a:-1] includes the start index (b), but excludes the
stop index (a). Only if the stop indices were included (or the start
indices were excluded), thus making the indexing symmetrical, would I
expect the results to be symmetric.

 

[Michael Hudson]

Can anyone explain the logic behind the behavior of list slicing with
negative strides?

You can read PySlice_GetIndicesEx in Objects/sliceobject.c if you're
really keen...

For example:

print range(10)[:-3:-1]

[9,8]

I found this result very surprising,

What would you have expected instead?

range(10-1, 10-3, -1) == [9,8]

was my guiding principle when I wrote the code.

and would just like to see the rules written down somewhere.

It seems to be hardly any easier to write the rules than to write the
algorithm, in practice...

In practice, I only ever use negative strides as "[::-1]", the result
of which hopefully surprises noone.

Cheers,
mwh

 

[Shalabh Chaturvedi]

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

No. S[a:b:1] includes the start index (a), but excludes the stop index
(b). Similarly, S[b:a:-1] includes the start index (b), but excludes the
stop index (a). Only if the stop indices were included (or the start
indices were excluded), thus making the indexing symmetrical, would I
expect the results to be symmetric.

That's a good explanation. My initial confusion arose because I
remembered how slices work by the suggestion (and diagram) under:

http://docs.python.org/tut/node5.html#SECTION005120000000000000000

"The best way to remember how slices work is to think of the indices as
pointing /between/ characters". When negative step is in the picture,
the indices are no longer in the same inbetween places and need to be
shifted to the right. Associating indices with characters themselves
dissipates the confusion.

Thanks,
Shalabh

 

[Alex Martelli]

Does anyone else find it intuitive to expect that S[b:a:-1] be exactly
reverse of S[a:b:1]?

Not me, because I _know_ that S[x:y:anything] INCLUDES S[x] and EXCLUDES
S[y]. So, swappiing x and y can't POSSIBILY "exactly reverse' things,
*ever*, NO MATTER HOW 'anything' changes.

That's a good reason why 2.4 introduces the new built-in 'reversed': if
you want to loop on "exactly the reverse of" S[a:b], you loop on
reversed(S[a:b]) and live happily ever after.


Alex