Creating slice notation from string

[bvdp]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!

Is it possible to take an arbitrary string in the form "1:2", "1",
":-1", etc. and feed it to slice() and then apply the result to an
existing list?

For example, I have a normal python list. Let's say that x = [1,2,3,4]
and I have a string, call it "s', in the format "[2:3]". All I need to
do is to apply "s" to "x" just like python would do.

I can, of course, convert "x" to a list with split(), convert the 2
and 3 to ints, and then do something like: x[a:b] ... but I'd like
something more general. I think the answer is in slice() but I'm lost.

Thanks.

 

[Robert Kern]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!

Is it possible to take an arbitrary string in the form "1:2", "1",
":-1", etc. and feed it to slice() and then apply the result to an
existing list?

For example, I have a normal python list. Let's say that x = [1,2,3,4]
and I have a string, call it "s', in the format "[2:3]". All I need to
do is to apply "s" to "x" just like python would do.

I can, of course, convert "x" to a list with split(), convert the 2
and 3 to ints, and then do something like: x[a:b] ... but I'd like
something more general. I think the answer is in slice() but I'm lost.

For a one-liner:

x[slice(*map(int, x[1:-1].split(':')))]

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[MRAB]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!

Is it possible to take an arbitrary string in the form "1:2", "1",
":-1", etc. and feed it to slice() and then apply the result to an
existing list?

For example, I have a normal python list. Let's say that x = [1,2,3,4]
and I have a string, call it "s', in the format "[2:3]". All I need to
do is to apply "s" to "x" just like python would do.

I can, of course, convert "x" to a list with split(), convert the 2
and 3 to ints, and then do something like: x[a:b] ... but I'd like
something more general. I think the answer is in slice() but I'm lost.

>>> x = [1,2,3,4]
>>> s = "[2:3]"
>>> # Using map.
>>> x[slice(*map(int, s.strip("[]").split(":")))] [3]
>>> # Using a list comprehension.
>>> x[slice(*[int(i) for i in s.strip("[]").split(":")])]

[3]

 

[Jan Kaliszewski]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!
Is it possible to take an arbitrary string in the form "1:2", "1",
":-1", etc. and feed it to slice() and then apply the result to an
existing list?
For example, I have a normal python list. Let's say that x = [1,2,3,4]
and I have a string, call it "s', in the format "[2:3]". All I need to
do is to apply "s" to "x" just like python would do.
I can, of course, convert "x" to a list with split(), convert the 2
and 3 to ints, and then do something like: x[a:b] ... but I'd like
something more general. I think the answer is in slice() but I'm lost.

x = [1,2,3,4]
s = "[2:3]"
# Using map.
x[slice(*map(int, s.strip("[]").split(":")))] [3]
# Using a list comprehension.
x[slice(*[int(i) for i in s.strip("[]").split(":")])]

[3]

Of course, you could also do something like this:

eval('x' + s)
or
eval(str(x) + s)

-- but it's worse: less secure (e.g. if s could be user-typed) and most
probably much more time-consuming (especially the latter).

Cheers,
*j

 

[Jan Kaliszewski]

Erratum:

eval(str(x) + s)

-- but it's worse: less secure (e.g. if s could be user-typed) and most
probably much more time-consuming (especially the latter).

There should be *repr* instead of *str*.

*j

 

[Bob van der Poel]

For a one-liner:

   x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with [2:].

Ideas?

 

[Bob van der Poel]

Of course, you could also do something like this:

     eval('x' + s)
or
     eval(str(x) + s)

Yes, I have user inputed 's'. So, if I can't get the generalized list
version from Robert working I'll have to use this. Speed is not a big
deal in this. As to malicious input, I could pretty easily check to
see that all the values are integers.

I tried something like this earlier but could not get it to work.
Didn't think of converting my x[] to a string. Makes sense.

Thanks.

 

[Rhodri James]

Yes, I have user inputed 's'. So, if I can't get the generalized list
version from Robert working I'll have to use this. Speed is not a big
deal in this. As to malicious input, I could pretty easily check to
see that all the values are integers.

If you've done that check, you've parsed the input so you might as well
use the values you've derived rather than waste time and add risk by
using eval().

 

[Ethan Furman]

For a one-liner:

x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with [2:].

Ideas?

try:
start, end = s[1:-1].split(':')
except ValueError:
start = int(s[1:-1] # only one value specified
end = start+1
start = int(start) if start else 0
end = int(end) if end else len(x)
x[start:end]

~Ethan~

 

[Robert Kern]

For a one-liner:

x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
File "<stdin>", line 1, in<module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with [2:].

Ideas?

Expanding out to a couple of lines now:

slice_args = []
for i in s.strip("[]").split(':'):
if i.strip() == '':
slice_args.append(None)
else:
slice_args.append(int(i))

y = x[slice(*slice_args)]


--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Bob van der Poel]

If you've done that check, you've parsed the input so you might as well
use the values you've derived rather than waste time and add risk by
using eval().

Not sure exactly what you mean here? Checking the values is (fairly)
trivial. Probably split on ":" and check the resulting list, etc.

But, translating 1, 2 or 3 ints into a valid splice isn't quit that
easy? I could figure each value, and convert them to either int or
None (key is the None! From my previous try '' doesn't work!)

But, I still need three possible lines:

if len(i) == 1:
x=x[i(0)]
else if len(i) == 2:
x=x[i(0):i(1)]
....


But, I'm wondering if just doing a split() on the index list and
seeing if they are all ints or None isn't safe?

 

[Jan Kaliszewski]

For a one-liner:

   x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with [2:].

Ideas?

x = [1,4,3,5,4,6,5,7]
s = '[3:6]'

x[slice(*((int(i) if i else None)
for i in s.strip("[]").split(":")))]

Cheers,
*j

 

[Bob van der Poel]

For a one-liner:
   x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with  [2:].

Ideas?

     x = [1,4,3,5,4,6,5,7]
     s = '[3:6]'

     x[slice(*((int(i) if i else None)
               for i in s.strip("[]").split(":")))]

Thanks. I think this will work fine. If I paste the above in a python
shell it's perfect. And if I paste it into my code it errors out ...
so I have to look a bit more. Always the problem with one-liners if
figuring out where the error is. I think it's just a naming thing ...
I'm sure I'll get it soon. I'll shout if I find more problems.

Best,

 

[Steven D'Aprano]

But, translating 1, 2 or 3 ints into a valid splice isn't quit that
easy? I could figure each value, and convert them to either int or None
(key is the None! From my previous try '' doesn't work!)

But, I still need three possible lines:

if len(i) == 1:
x=x[i(0)]
else if len(i) == 2:
x=x[i(0):i(1)]
....

items = [int(n) for n in slice_string.split(":")]
slice(*items)

 

[Bob van der Poel]

But, translating 1, 2 or 3 ints into a valid splice isn't quit that
easy? I could figure each value, and convert them to either int or None
(key is the None! From my previous try '' doesn't work!)

But, I still need three possible lines:

 if len(i) == 1:
    x=x[i(0)]
  else if len(i) == 2:
    x=x[i(0):i(1)]
   ....

items = [int(n) for n in slice_string.split(":")]
slice(*items)

Actually, nither this or Jan's latest is working properly. I don't
know if it's the slice() function or what (I'm using python 2.5). But:

x = [1,2,3,4,5]
slice_string="2"
items = [int(n) if n else None for n in slice_string.split(":")]
[slice(*items)]
[1, 2]

not the expected:

[3]

Things like -1 don't work either.

I'm really not sure what's going on, but I suspect it's the way that
slice() is getting filled when the slice string isn't a nice one with
each ":" present?

 

[Bob van der Poel]

Bob van der Poel wrote:

For a one-liner:
  x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with  [2:].

Ideas?

try:
     start, end = s[1:-1].split(':')
except ValueError:
     start = int(s[1:-1] # only one value specified
     end = start+1
start = int(start) if start else 0
end = int(end) if end else len(x)
x[start:end]

~Ethan~

Yes ... I see. I'm thinking that eval() is looking very nice. If we do
it the above way we also have to check for empties using things like
[1::2] and I'm really getting confused about the possibilities

 

[Ethan Furman]

Bob van der Poel wrote:

For a one-liner:

  x[slice(*map(int, x[1:-1].split(':')))]

Thanks.

Almost works

For s="[2]" and s="[1:2]" it's fine. But, if I have

s = "[:2]" then I get:

x[slice(*[int(i) for i in s.strip("[]").split(":")])]

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: invalid literal for int() with base 10: ''

Similar problem with  [2:].

Ideas?

try:
     start, end = s[1:-1].split(':')
except ValueError:
     start = int(s[1:-1] # only one value specified
     end = start+1
start = int(start) if start else 0
end = int(end) if end else len(x)
x[start:end]

~Ethan~

Yes ... I see. I'm thinking that eval() is looking very nice. If we do
it the above way we also have to check for empties using things like
[1::2] and I'm really getting confused about the possibilities

How about:

[untested]
s = s[1:-1] # strip the brackets
if s.count(':') == 0:
return x[int(s)]
while s.count(':') < 2:
s += ':'
start, stop, step = s.split(':')
start = int(start) if start else 0
end = int(stop) if stop else len(x)
step = int(step) if step else 1
return x[start:stop:step)

~Ethan~

 

[Steven D'Aprano]

Actually, nither this or Jan's latest is working properly. I don't know
if it's the slice() function or what (I'm using python 2.5). But:

x = [1,2,3,4,5]
slice_string="2"
items = [int(n) if n else None for n in slice_string.split(":")]
[slice(*items)]
[1, 2]

It's not clear what is input and what is output. I'm *guessing* that the
first four lines are input and the fifth is output.

By the way, nice catch for the "else None". But why are you wrapping the
call to slice() in a list in the fourth line?


I can't replicate your results. I get the expected results:

slice_string="2"
items = [int(n) if n else None for n in slice_string.split(":")]
[slice(*items)]

[slice(None, 2, None)]

exactly the same as:
slice(None, 2, None)

Testing this, I get the expected result:

x = [1,2,3,4,5]
x[slice(*items)]

[1, 2]

which is exactly the same if you do this:

x[:2:]

[1, 2]

not the expected:

[3]

Why would you expect that? You can't get that result from a slice based

x[2::] [3, 4, 5]
x[:2:] [1, 2]
x[::2]

[1, 3, 5]

There is no slice containing *only* 2 which will give you the result you
are asking for. You would need to do this:

x[2:3]

[3]


Perhaps what you are thinking of is *indexing*:

x[2]

3

but notice that the argument to list.__getitem__ is an int, not a slice,
and the result is the item itself, not a list.

To get the behaviour you want, you need something more complicated:

def strToSlice(s):
if ':' in s:
items = [int(n) if n else None for n in s.split(':')]
else:
if s:
n = int(s)
items = [n, n+1]
else:
items = [None, None, None]
return slice(*items)

Things like -1 don't work either.

They do for me:

slice_string="2:-2"
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
items = [int(n) if n else None for n in slice_string.split(":")]
x[ slice(*items) ] [3, 4, 5, 6, 7, 8]
x[2:-2]

[3, 4, 5, 6, 7, 8]

I'm really not sure what's going on, but I suspect it's the way that
slice() is getting filled when the slice string isn't a nice one with
each ":" present?

I think you're confused between __getitem__ with a slice argument and
__getitem__ with an int argument.

 

[Paul McGuire]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!

Well, this is a nice puzzler, better than a sudoku. Maybe a quick
parser with pyparsing will give you some guidance on how to do this
without a parser library:

from pyparsing import *

# relevant punctuation, suppress after parsing
LBR,RBR,COLON = map(Suppress,"[]:")

# expression to parse numerics and convert to int's
integer = Regex("-?\d+").setParseAction(lambda t: int(t[0]))

# first try, almost good enough, but wrongly parses "[2]" -> [2::]
sliceExpr = ( LBR + Optional(integer,default=None) +
Optional(COLON + Optional(integer,default=None),
default=None) +
Optional(COLON + Optional(integer,default=None),
default=None) +
RBR )

# better, this version special-cases "[n]" -> [n:n+1]
# otherwise, just create slice from parsed int's
singleInteger = integer + ~FollowedBy(COLON)
singleInteger.setParseAction(lambda t : [t[0],t[0]+1])

sliceExpr = ( LBR +
(singleInteger |
Optional(integer,default=None) +
Optional(COLON + Optional(integer,default=None),
default=None) +
Optional(COLON + Optional(integer,default=None),
default=None)
) +
RBR )

# attach parse action to convert parsed int's to a slice
sliceExpr.setParseAction(lambda t: slice(*(t.asList())))


tests = """\
[2]
[2:3]
[2:]
[2::2]
[-1:-1:-1]
[:-1]
[::-1]
[:]""".splitlines()

testlist = range(10)
for t in tests:
parsedSlice = sliceExpr.parseString(t)[0]
print t, parsedSlice, testlist[parsedSlice]


Prints:

[2] slice(2, 3, None) [2]
[2:3] slice(2, 3, None) [2]
[2:] slice(2, None, None) [2, 3, 4, 5, 6, 7, 8, 9]
[2::2] slice(2, None, 2) [2, 4, 6, 8]
[-1:-1:-1] slice(-1, -1, -1) []
[:-1] slice(None, -1, None) [0, 1, 2, 3, 4, 5, 6, 7, 8]
[::-1] slice(None, None, -1) [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
[:] slice(None, None, None) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]


Yes, it is necessary to handle the special case of a "slice" that is
really just a single index. If your list of parsed integers has only
a single value n, then the slice constructor creates a slice
(None,n,None). What you really want, if you want everything to create
a slice, is to get slice(n,n+1,None). That is what the singleInteger
special case does in the pyparsing parser.

-- Paul

 

[Falcolas]

I'm trying to NOT create a parser to do this .... and I'm sure that
it's easy if I could only see the light!

Is it possible to take an arbitrary string in the form "1:2", "1",
":-1", etc. and feed it to slice() and then apply the result to an
existing list?

For example, I have a normal python list. Let's say that x = [1,2,3,4]
and I have a string, call it "s', in the format "[2:3]". All I need to
do is to apply "s" to "x" just like python would do.

I can, of course, convert "x" to a list with split(), convert the 2
and 3 to ints, and then do something like: x[a:b] ... but I'd like
something more general. I think the answer is in slice() but I'm lost.

Thanks.

You might also consider looking at the operator module, which provides
to potentially useful methods - itemgetter and getslice. Neither will
directly use the string containing the bare slice notation, however
they do provide another alternative to directly using eval.

~G