slice notation as values?

[Antoon Pardon]

Now slices are objects in python, I was wondering if slice
notation will be usable outside subscribtion in the future.

Will it ever be possible to write things like:

a = 4:9
for key, value in tree.items('alfa.': 'beta.'):

 

[Steve Holden]

Now slices are objects in python, I was wondering if slice
notation will be usable outside subscribtion in the future.

Will it ever be possible to write things like:

a = 4:9
for key, value in tree.items('alfa.': 'beta.'):

Do you mean

for key, value in tree.items()['alfa.': 'beta.']:

What would this mean?

regards
Steve

 

[Antoon Pardon]

Now slices are objects in python, I was wondering if slice
notation will be usable outside subscribtion in the future.

Will it ever be possible to write things like:

a = 4:9
for key, value in tree.items('alfa.': 'beta.'):

Do you mean

for key, value in tree.items()['alfa.': 'beta.']:

No, the slice is meant to be a parameter to the method.
It would be a more convenient way to write:

for key, value in tree.items(slice('alfa.', 'beta.'))

 

[Duncan Booth]

Will it ever be possible to write things like:

a = 4:9
for key, value in tree.items('alfa.': 'beta.'):

The first of these works fine, except you need to use the correct syntax:

a = slice(4,9)
range(10)[a] [4, 5, 6, 7, 8]

The second also works fine, provide tree is a type which supports it and
you rewrite the call as tree.items(slice('alfa','beta.')) or perhaps
tree['alfa':'beta.'].items(). To support slicing directly on a dictionary
you could do:
def __getitem__(self, item):
if isinstance(item, slice):
return self.__class__((k,v) for (k,v) in self.iteritems()
if item.start <= k < item.stop)
return dict.__getitem__(self, item)

d = sliceable({'alpha': 1, 'aaa': 2, 'beta': 3, 'bee': 4 })
d['alpha':'beta'] {'alpha': 1, 'bee': 4}
d['alpha':'beta.'] {'alpha': 1, 'beta': 3, 'bee': 4}
for key, value in d['alpha':'beta.'].items():

print key, value


alpha 1
beta 3
bee 4

It seems unlikely that this will make it into the builtin dict type, but
you never know.

 

[Antoon Pardon]

The first of these works fine, except you need to use the correct syntax:

Sure, I know that. But why a different syntax?

If we have lst = range(10), we can write

lst[slice(3,7)]

instead of

lst[3:7]

Now my impression is that should we only have the upper notation, slices
would be less usefull, because it would make using them more cumbersome.

I think that having this easy notation for slices available in more
general circumstances, would make the use of them in other situations
more easy too.

a = slice(4,9)
range(10)[a] [4, 5, 6, 7, 8]

The second also works fine, provide tree is a type which supports it and
you rewrite the call as tree.items(slice('alfa','beta.')) or perhaps
tree['alfa':'beta.'].items(). To support slicing directly on a dictionary
you could do: def __getitem__(self, item):
if isinstance(item, slice):
return self.__class__((k,v) for (k,v) in self.iteritems()
if item.start <= k < item.stop)
return dict.__getitem__(self, item)

d = sliceable({'alpha': 1, 'aaa': 2, 'beta': 3, 'bee': 4 })
d['alpha':'beta'] {'alpha': 1, 'bee': 4}
d['alpha':'beta.'] {'alpha': 1, 'beta': 3, 'bee': 4}
for key, value in d['alpha':'beta.'].items():

print key, value

alpha 1
beta 3
bee 4

It seems unlikely that this will make it into the builtin dict type, but
you never know.

It doesn't need to be in the buildin dict type, I just would think it
could make the interface to my own treedict type more intuitive.

In my treedict the keys are accessible in order. If you have dictionary
with strings as keys doing:

for key in tree:

will give you the keys in alfabetical order. Doing

for key in tree['a':'b']:

will give you all the keys that start with an 'a'.

Now there are a number of methods with similar results.
keys, values, items and there iter variants iterkeys,
itervalues and iteritems. These methods are 'sliceable'
too and I think the possibilty of giving such a method
a slice as parameter, with the same notation as in
a subscript, would be the clearest way for the user
to provide slice information.

If the user can write:

for key in tree['a':'b']:

Why shouldn't he be able to write:

for key, value in tree.iteritems('a':'b'):

 

[Duncan Booth]

Antoon Pardon asked:

If we have lst = range(10), we can write

lst[slice(3,7)]

instead of

lst[3:7]

Now my impression is that should we only have the upper notation, slices
would be less usefull, because it would make using them more cumbersome.

Quite right, but the syntax for the slice only applies inside the square
brackets and there would be big problems making it work outside a
subscript. If you allowed a:b you get syntactic ambiguities: e.g. is this a
slice or a complete if statement:

if a:b

If you allow a slice on its own but require the square brackets still to be
there then you don't (I think) get any syntax ambiguities, but the result
looks like it should be some weird list comprehension so its just too
confusing:

myslice = [a:b]

I think the case for freestanding slices is there, but they aren't common
enough to justify special syntax.

If the user can write:

for key in tree['a':'b']:

Why shouldn't he be able to write:

for key, value in tree.iteritems('a':'b'):

....
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
....

If the user can write

for key in tree['a':'b']:

then he can write:

for key in tree['a':'b'].iteritems():

so why add a second way to do that?

 

[Antoon Pardon]

Antoon Pardon asked:

If we have lst = range(10), we can write

lst[slice(3,7)]

instead of

lst[3:7]

Now my impression is that should we only have the upper notation, slices
would be less usefull, because it would make using them more cumbersome.

Quite right, but the syntax for the slice only applies inside the square
brackets and there would be big problems making it work outside a
subscript. If you allowed a:b you get syntactic ambiguities: e.g. is this a
slice or a complete if statement:

if a:b

If you allow a slice on its own but require the square brackets still to be
there then you don't (I think) get any syntax ambiguities, but the result
looks like it should be some weird list comprehension so its just too
confusing:

myslice = [a:b]

I don't see the problem. The syntax for tuples can create syntactic
ambiguities too. Take the following statement:

f(a,b)

Does the call have two parameters or one that is a tuple? In practice
one uses parenthesis to disambigue the stament. So in the above case
it would be a complete if statement. And myslice = a:b wouldn't
cause a problem.

I think the case for freestanding slices is there, but they aren't common
enough to justify special syntax.

If the user can write:

for key in tree['a':'b']:

Why shouldn't he be able to write:

for key, value in tree.iteritems('a':'b'):

import this

...
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
...

If the user can write

for key in tree['a':'b']:

then he can write:

for key in tree['a':'b'].iteritems():

No he can't. tree['a':'b'] would provide a list
of keys that all start with an 'a'. Such a list
doesn't have an iteritems method. It wouldn't even
contain the information to construct items.

 

[Duncan Booth]

If the user can write

for key in tree['a':'b']:

then he can write:

for key in tree['a':'b'].iteritems():

No he can't. tree['a':'b'] would provide a list
of keys that all start with an 'a'. Such a list
doesn't have an iteritems method. It wouldn't even
contain the information to construct items.

Why would it produce a list?

Slicing a *list* produces a list, slicing a tuple produces a tuple, slicing
a string produces a string. I would expect slicing any other type would
also return you a new object of the same type. Thats what the code sample I
posted earlier does.

 

[Antoon Pardon]

If the user can write

for key in tree['a':'b']:

then he can write:

for key in tree['a':'b'].iteritems():

No he can't. tree['a':'b'] would provide a list
of keys that all start with an 'a'. Such a list
doesn't have an iteritems method. It wouldn't even
contain the information to construct items.

Why would it produce a list?

Correction, it produces an iterator.

Slicing a *list* produces a list, slicing a tuple produces a tuple, slicing
a string produces a string. I would expect slicing any other type would
also return you a new object of the same type.

But iterating over a list, tuple or string, give you the elements
of a list, tuple or string one by one. Iterating over a dict
doesn't give you the elements one by one, but only the keys.

In general I use slices over a tree because I only want to iterate
over a specific subdomain of the keys. I'm not iterested in make
a tree over the subdomain. Making such a subtree would be an
enormous waste of resources.

So even if we agree that tree['a':'b'] should create a subtree.
Then I still would want a way to iterate over a subdomain of
the keys in the tree without creating a subtree. Yes this
would create more than one way to do the same thing. But
this is already so with builtin dicts where a number of methods
give you different ways to do things, apparantly because
according to circumstances one is more efficient than the other.

So lets agree that tree['a':'b'] would produce a subtree. Then
I still would prefer the possibility to do something like:

for key in tree.iterkeys('a':'b')

Instead of having to write

for key in tree['a':'b'].iterkeys()

Sure I can now do it like this:

for key in tree.iterkeys('a','b')

But the way default arguments work, prevents you from having
this work in an analague way as a slice.

With slice notation you could have the following two cases:

for key in tree.iterkeys('a'

for key in tree.iterkeys'b')

But you can't do

for key in tree.iterkeys('a',)

or more regrettably, you can't do:

for key in tree.iterkeys(,'b')


I also think that other functions could benefit. For instance suppose
you want to iterate over every second element in a list. Sure you
can use an extended slice or use some kind of while. But why not extend
enumerate to include an optional slice parameter, so you could do it as
follows:

for el in enumerate(lst,::2)

If you have an iterable, you sometime want to iterate over only a part
of it. The only way now to do so seems to be to either use a while
loop or create a second iterable over your limited domain and iterate
over the subiterable. I think it would be a an improvement if an
iterator could be constructed to just go over the subdomain within
your iterable. I also think that the most natural way to define
that subdomain would be by using slice notation.

 

[Steven Bethard]

So lets agree that tree['a':'b'] would produce a subtree. Then
I still would prefer the possibility to do something like:

for key in tree.iterkeys('a':'b')

Instead of having to write

for key in tree['a':'b'].iterkeys()

Sure I can now do it like this:

for key in tree.iterkeys('a','b')

But the way default arguments work, prevents you from having
this work in an analague way as a slice.

How so? Can't you just pass the *args to the slice contstructor? E.g.::

def iterkeys(self, *args):
keyslice = slice(*args)
...

Then you can use the slice object just as you would have otherwise.

STeVe

 

[Antoon Pardon]

So lets agree that tree['a':'b'] would produce a subtree. Then
I still would prefer the possibility to do something like:

for key in tree.iterkeys('a':'b')

Instead of having to write

for key in tree['a':'b'].iterkeys()

Sure I can now do it like this:

for key in tree.iterkeys('a','b')

But the way default arguments work, prevents you from having
this work in an analague way as a slice.

How so? Can't you just pass the *args to the slice contstructor? E.g.::

def iterkeys(self, *args):
keyslice = slice(*args)
...

Then you can use the slice object just as you would have otherwise.

This doesn't work for a number of reasons,

1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: slice expected at least 1 arguments, got 0


2) It doens't give a clear way to indicate the following
kind of slice: tree.iterkeys('a'. Because of the
follwing:
slice(None, 'a', None)

which would be equivallent to tree.iterkeys'a')

 

[Duncan Booth]

In general I use slices over a tree because I only want to iterate
over a specific subdomain of the keys. I'm not iterested in make
a tree over the subdomain. Making such a subtree would be an
enormous waste of resources.

Probably not unless you have really large data structures. If you have
something like a dbhash which would be inefficient to copy then both the
original view of the data structure and the slices could share the same
data. Creating a slice doesn't *have* to copy anything just so long as the
semantics are clear.

With slice notation you could have the following two cases:

for key in tree.iterkeys('a'

for key in tree.iterkeys'b')

x['a':] is short for x['a':None]
x[:'b'] is short for x[None:'b']

But you can't do

for key in tree.iterkeys('a',)

or more regrettably, you can't do:

for key in tree.iterkeys(,'b')

If your datatype defines iterkeys to accept start and end arguments then
you can do:

for key in tree.iterkeys('a',None)
for key in tree.iterkeys(None,'b')

which is directly equivalent to the slices, or you can do:

for key in tree.iterkeys(start='a')
for key in tree.iterkeys(stop='b')

which is more explicit.

I also think that other functions could benefit. For instance suppose
you want to iterate over every second element in a list. Sure you
can use an extended slice or use some kind of while. But why not
extend enumerate to include an optional slice parameter, so you could
do it as follows:

for el in enumerate(lst,::2)

'Why not'? Because it makes for a more complicated interface for something
you can already do quite easily.

 

[Antoon Pardon]

In general I use slices over a tree because I only want to iterate
over a specific subdomain of the keys. I'm not iterested in make
a tree over the subdomain. Making such a subtree would be an
enormous waste of resources.

Probably not unless you have really large data structures. If you have
something like a dbhash which would be inefficient to copy then both the
original view of the data structure and the slices could share the same
data. Creating a slice doesn't *have* to copy anything just so long as the
semantics are clear.

With slice notation you could have the following two cases:

for key in tree.iterkeys('a'

for key in tree.iterkeys'b')

x['a':] is short for x['a':None]
x[:'b'] is short for x[None:'b']

That is beside the point. The user doesn't have to know that
in order to use slices. In point of fact I think that if
tomorrow they changed the default to something different
not a single program would break.

If your datatype defines iterkeys to accept start and end arguments then
you can do:

for key in tree.iterkeys('a',None)
for key in tree.iterkeys(None,'b')

which is directly equivalent to the slices, or you can do:

for key in tree.iterkeys(start='a')
for key in tree.iterkeys(stop='b')

which is more explicit.

Yes we could do all that. The question remains why we should burden
the user with all this extra information he has to know now in order
to use this method, while there is a clear notation he can use
without all this.

The user doesn't has to type:

lst[5:None] or lst[None:7],

Neither has he to type something like

lst[start=5] or lst[stop=7]


There are circumstances when the user needs to provide slice
information to a function. Why not allow him to use the
same notation as he can use in subscribtion. What reason
is there to limit a specific notation for a value to
specific circumstances. To me this looks as arbitrary
as would bracket notation not be allowed as an argument
but that you would be obligated to use list((a,b,...)) in
calls instead of [a,b,...]

It wouldn't prevent you to do anything from what you can
do now with python, it would only make a number of things
unnecesary cumbersome.

So yes, my proposal will not allow you to do anything you
can;t already do now. It would just allow you to do a number
of things in a less cumbersome way.

'Why not'? Because it makes for a more complicated interface for something
you can already do quite easily.

Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]),
(4,lst[4]) ...

I haven't found a way to do this easily. Except for something like:

start = 0:
while start < len(lst):
yield start, lst[start]
start += 2

But if you accept this, then there was no need for enumerate in the
first place. So eager to learn something new, how do you do this
quite easily?

 

[Devan L]

'Why not'? Because it makes for a more complicated interface for something
you can already do quite easily.

Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]),
(4,lst[4]) ...

I haven't found a way to do this easily. Except for something like:

start = 0:
while start < len(lst):
yield start, lst[start]
start += 2

But if you accept this, then there was no need for enumerate in the
first place. So eager to learn something new, how do you do this
quite easily?

lst = ['ham','eggs','bacon','spam','foo','bar','baz']
list(enumerate(lst))[::2]

[(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')]

No changes to the language necessary.

 

[Brian Beck]

Will it ever be possible to write things like:

a = 4:9

I made a silly recipe to do something like this a while ago, not that
I'd recommend using it. But I also think it wouldn't be too far-fetched
to allow slice creation using a syntax like the above...

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415500

 

[Steven Bethard]

So lets agree that tree['a':'b'] would produce a subtree. Then
I still would prefer the possibility to do something like:

for key in tree.iterkeys('a':'b')

Instead of having to write

for key in tree['a':'b'].iterkeys()

Sure I can now do it like this:

for key in tree.iterkeys('a','b')

But the way default arguments work, prevents you from having
this work in an analague way as a slice.

How so? Can't you just pass the *args to the slice contstructor? E.g.::

def iterkeys(self, *args):
keyslice = slice(*args)
...

Then you can use the slice object just as you would have otherwise.

This doesn't work for a number of reasons,

1)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: slice expected at least 1 arguments, got 0

I wasn't sure whether or not the slice argument was optional.
Apparently it's intended to be, so you have to make one special case:

def iterkeys(self, *args):
keyslice = args and slice(*args) or slice(None, None, None)

2) It doens't give a clear way to indicate the following
kind of slice: tree.iterkeys('a'. Because of the
follwing:

slice(None, 'a', None)

which would be equivallent to tree.iterkeys'a')

Well, it certainly gives a way to indicate it:

tree.iterkeys(None, 'a')

Whether or not it's a "clear" way is too subjective of a topic for me to
get into. That's best left to Guido[1]. My point is that it *does*
work, and covers (or can be slightly altered to cover) all the
functionality you want. That doesn't mean you have to like the API for
it, of course.

STeVe

[1] By which I mean that you should submit a PEP on the idea, and let
Guido decide which way is prettier. Just be sure to give all the
equivalent examples - i.e. calling the slice constructor with the
appropriate arguments.

 

[Bengt Richter]

I also think that other functions could benefit. For instance suppose
you want to iterate over every second element in a list. Sure you
can use an extended slice or use some kind of while. But why not
extend enumerate to include an optional slice parameter, so you could
do it as follows:

for el in enumerate(lst,::2)

'Why not'? Because it makes for a more complicated interface for something
you can already do quite easily.

Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]),
(4,lst[4]) ...

I haven't found a way to do this easily. Except for something like:

start = 0:
while start < len(lst):
yield start, lst[start]
start += 2

But if you accept this, then there was no need for enumerate in the
first place. So eager to learn something new, how do you do this
quite easily?

lst = ['ham','eggs','bacon','spam','foo','bar','baz']
list(enumerate(lst))[::2]

[(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')]

No changes to the language necessary.

Or, without creating the full list intermediately,

>>> lst = ['ham','eggs','bacon','spam','foo','bar','baz']
>>> import itertools
>>> list(itertools.islice(enumerate(lst), 0, None, 2))

[(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')]

Regards,
Bengt Richter

 

[Duncan Booth]

I made a silly recipe to do something like this a while ago, not that
I'd recommend using it. But I also think it wouldn't be too far-fetched
to allow slice creation using a syntax like the above...

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/415500

Another possibility would be to make the slice type itself sliceable, then
you could write things like:

a = slice[4:9]
a

slice(4, 9, None)

Sample implementation:
def __getitem__(cls, item):
return item
def __init__(self, *args, **kw):
return super(MetaSlice,self).__init__(self, *args, **kw)

__metaclass__=MetaSlice

Slice[2:3] slice(2, 3, None)
Slice[:3] slice(None, 3, None)
Slice[:3:-1]

slice(None, 3, -1)

 

[Antoon Pardon]

I also think that other functions could benefit. For instance suppose
you want to iterate over every second element in a list. Sure you
can use an extended slice or use some kind of while. But why not
extend enumerate to include an optional slice parameter, so you could
do it as follows:

for el in enumerate(lst,::2)

'Why not'? Because it makes for a more complicated interface for something
you can already do quite easily.

Do you think so? This IMO should provide (0,lst[0]), (2,lst[2]),
(4,lst[4]) ...

I haven't found a way to do this easily. Except for something like:

start = 0:
while start < len(lst):
yield start, lst[start]
start += 2

But if you accept this, then there was no need for enumerate in the
first place. So eager to learn something new, how do you do this
quite easily?

lst = ['ham','eggs','bacon','spam','foo','bar','baz']
list(enumerate(lst))[::2]

[(0, 'ham'), (2, 'bacon'), (4, 'foo'), (6, 'baz')]

It is not about what is needed, but about convenience.

Now let me see, in order to just iterate over the even elements
of a list with the index of the element, you turned an iterator
into a list, which you use to create an other list which you
will finaly iterate over.

If this is the proposed answer, I wonder why iterators were introduced
in the first place. I thought iterator were went to avoid the need
to construct and copy list when all you want is iterate and when
I ask how to get a specific iterator you come with a construct that
makes rather heavily use of list constructions.

 

[Antoon Pardon]

I made a silly recipe to do something like this a while ago, not that
I'd recommend using it. But I also think it wouldn't be too far-fetched
to allow slice creation using a syntax like the above...

The point is that the syntax "4:9" is already used for slice creation.

The python grammer is essentally saying that something like 4:9 is a
literal, just like strings and numbers, but that this specific literal
can only be used in a subscription.

Look at the following:
.... lst[[2,3,5]]
.... lst[8:13:21]
.... 2 0 LOAD_GLOBAL 0 (lst)
3 LOAD_CONST 1 (2)
6 LOAD_CONST 2 (3)
9 LOAD_CONST 3 (5)
12 BUILD_LIST 3
15 BINARY_SUBSCR
16 POP_TOP

3 17 LOAD_GLOBAL 0 (lst)
20 LOAD_CONST 4 (8)
23 LOAD_CONST 5 (13)
26 LOAD_CONST 6 (21)
29 BUILD_SLICE 3
32 BINARY_SUBSCR
33 POP_TOP
34 LOAD_CONST 0 (None)
37 RETURN_VALUE

So you see that the slice is treated in an similar way
as the list. There is no reason why this shouldn't work
in case we want a slice in an assignment or a function
call.