"index" method only for mutable sequences??

[Antoon Pardon]

There is a cost to every new language feature: it has to be implemented,
documented, maintained, and above all learned by the users. Good design
involves, in part, not adding to these burdens except where there is a
benefit at least equal to the cost.

So what is the easiest to learn: "All sequences have an index method" or
"Such and so sequences have an index method and others don't"

Which of the above is the easiest to document?

Now with implementation and maintaining. If you would start with a class
of sequence which classes like tuple and list would inherit from, then
there also would be a single function to be implemented and maintained.
It would just be usable in more types.

Why not try them? Pose a few useful cases where the existence of
tuple.index() would actually give a benefit and see what response you get.

You would have to meet a few obvious conditions: the use case would need to
be one where a tuple was preferable to a list (the obvious one being where
you need to hash it), the tuple would have to be sufficiently long or the
calls to 'index' sufficiently frequent to make casting to a list so you can
call 'list(obj).index(...)' unattractive, and you actually need a reason to
be using 'index' rather than just the 'in' operator.

If someone states: "Show me your use case for using tuple.index and I
will show you how to avoid it." or words to that effect I think there
is little use trying.

Besides I have seen how such things evolve here before. Every use case
that will be presented, will be rewritten in a way that doesn't need
tuple.index and these rewritals will be used as an argument for not
having tuple.index. If you need to hash it, you can simply subclass
list and provide a hash with your class or you can simply use tuples
and write an index function yourself.

The same happened with the ternary operator. Every use case someone
could come up with was rejected by rewriting the code without using
a ternary operator. AFAICS the only reason the ternary operator
finaly got introduced was because a python developer was bitten by an
illusive bug, introduced by one of the idioms that was often used as a
way to simulate a ternary operator.

 

[Paul Boddie]

So what is the easiest to learn: "All sequences have an index method" or
"Such and so sequences have an index method and others don't"

I think this observation leads to insights both at the end-user level
and at the implementer level. Tuples and lists are sequences; the
index method can be defined generically for all sequences; adding such
a method to the tuple type can be done with barely any changes to the
implementation taken from the list type. This leads to the observation
that a generic index method (defined as a function in the
implementation) could be made to work with both lists and tuples, and
that various other methods might also be defined similarly, although
things like append, extend and other mutating methods wouldn't be
appropriate for a tuple.

Which of the above is the easiest to document?

Now with implementation and maintaining. If you would start with a class
of sequence which classes like tuple and list would inherit from, then
there also would be a single function to be implemented and maintained.
It would just be usable in more types.

There isn't a "big win" in this case: the intersection of useful
methods between mutable and immutable sequence types is rather small.
Nevertheless, providing a slightly deeper abstract class hierarchy
might be appropriate, and this is being considered for Python 3000.

[...]

The same happened with the ternary operator. Every use case someone
could come up with was rejected by rewriting the code without using
a ternary operator. AFAICS the only reason the ternary operator
finaly got introduced was because a python developer was bitten by an
illusive bug, introduced by one of the idioms that was often used as a
way to simulate a ternary operator.

The ternary operator, whilst providing new and occasionally useful
functionality, is rather "badly phrased" in my opinion: when used,
it's a bit like reading one natural language and suddenly having the
grammar of another in use for the rest of the sentence.

Paul

 

[Antoon Pardon]

There isn't a "big win" in this case: the intersection of useful
methods between mutable and immutable sequence types is rather small.
Nevertheless, providing a slightly deeper abstract class hierarchy
might be appropriate, and this is being considered for Python 3000.

Well I wasn't trying to show this aspect as a big win. Just illustrating
it doesn't have to be a big cost in this case.

The ternary operator, whilst providing new and occasionally useful
functionality, is rather "badly phrased" in my opinion: when used,
it's a bit like reading one natural language and suddenly having the
grammar of another in use for the rest of the sentence.

I agree. The structure of the if-expression doesn't resemble the
structure of the if-statement. I think it was a bad choice to
have the two so dissimilar.

 

[Carsten Haese]

If you have a use case for tuple.index, please show it to me, and I'll
show you what you should be using instead of a tuple.

No wonder no convincing use cases for tuples have shown up. You just
defined use cases of tuples as unconvincing.[/QUOTE]

It's not a definition, it's an observation. I'm simply saying that all use
cases I've seen for tuple.index could be written in a clearer fashion by using
something else. Please prove me wrong by supplying a use case of tuple.index
that can not be improved by rewriting.

Note that I have proved elsewhere on this thread that any real use case for
tuple.index will involve an operation to explicitly use an index different
from the one obtained by the call to tuple.index(). I'd like to hear your
opinion on this.

My personal opinion is that lack of use cases are used too much as an
excuse. Limiting the development of your language to use cases makes
your language less consistent and so makes your language less easy to
learn.

That is your opinion, and you are entitled to your opinion. My opinion is that
adding features that don't have any uses just weigh the language down
unnecessarily, and they distract programmers from finding the best solution to
their problem.

Of course it also limits the language to what the developers can
think off.

Initially, yes, but if enough use cases exist for a feature that the
developers didn't think of, it will be implemented.

-Carsten

 

[Carsten Haese]

On 10 Apr 2007 09:48:41 GMT, Antoon Pardon wrote

If someone states: "Show me your use case for using tuple.index and I
will show you how to avoid it." or words to that effect I think there
is little use trying.

Or maybe you just can't think of any good use cases, and that's annoying you
because it proves my point. Please, prove me wrong by showing use cases for
tuple.index that can't be rewritten.

-Carsten

 

[Antoon Pardon]

No wonder no convincing use cases for tuples have shown up. You just
defined use cases of tuples as unconvincing.

It's not a definition, it's an observation. I'm simply saying that all use
cases I've seen for tuple.index could be written in a clearer fashion by using
something else. Please prove me wrong by supplying a use case of tuple.index
that can not be improved by rewriting.[/QUOTE]

No it is a defintion because it states this can be done for every
possible case, even cases you have no idea about.

Note that I have proved elsewhere on this thread that any real use case for
tuple.index will involve an operation to explicitly use an index different
from the one obtained by the call to tuple.index(). I'd like to hear your
opinion on this.

And what relevance would such a proof have?

That is your opinion, and you are entitled to your opinion. My opinion is that
adding features that don't have any uses just weigh the language down
unnecessarily, and they distract programmers from finding the best solution to
their problem.

Adding the index method to tuples is not adding a feature. It is
removing a limitation. Writing an index function in python that
works with whatever sequence is dead easy. So if the python
core implementation only works with a specific sequence that
is putting on unnecessary limitations.

Initially, yes, but if enough use cases exist for a feature that the
developers didn't think of, it will be implemented.

Which means your language will not be that usefull for a number of
things until you have frustrated enough people.

 

[Duncan Booth]

So what is the easiest to learn: "All sequences have an index method"
or "Such and so sequences have an index method and others don't"

Which of the above is the easiest to document?

The second would be if it were true. However it would mean you would have
to add an index method to *all* sequences.

FWIW, The current documentation says that 's.index' is a method defined on
all *mutable* sequence types: almost as simple as your second wording but
unfortunately misleading since strings also have index.

If someone states: "Show me your use case for using tuple.index and I
will show you how to avoid it." or words to that effect I think there
is little use trying.

I genuinely cannot think of a use case. I didn't ask you to suggest one so
I could show you how to avoid it, I asked you to suggest one so that we
could take the discussion from the purely hypothetical to a more concrete
discussion of whether it would be a worthwhile enhancement.

Fair enough though. I'll assume you cannot think of a suitable use case
either.

 

[Carsten Haese]

No it is a defintion because it states this can be done for every
possible case, even cases you have no idea about.

That would be a theorem or conjecture, not a definition. You can call it
definition or banana or screwdriver if you like, but that's not going to
disprove it.

And what relevance would such a proof have?

It proves at that every conceivable use case for tuple.index, including
the ones I am not thinking of, will never directly use the index
obtained from calling tuple.index(). Such a use case will be poorly
self-documented, and a clearer implementation can be found by avoiding
tuple.index().

I'll illustrate this on the one concrete use case that was suggested on
this thread: In a game where the sequence of players is given by the
tuple p, find the opponents of the current player.

One hypothetical solution is to find the index if the current player and
then slice and reassemble the tuple:

i = p.index(current_player)
opponents = p[:i-1] + p[i+1:]

An alternative is this:

opponents = tuple(x for x in p if x is not current_player)

You may disagree, but in my opinion, the alternative is better because
it is a more natural translation of the concept that the opponents of
the current player are all players that are not the current player.

Adding the index method to tuples is not adding a feature. It is
removing a limitation.

The non-existence of tuple.index is only a limitation if there is a need
for the method to exist. Please prove that this need exists.

-Carsten

 

[Antoon Pardon]

On 10 Apr 2007 09:48:41 GMT, Antoon Pardon wrote

Or maybe you just can't think of any good use cases, and that's annoying you
because it proves my point. Please, prove me wrong by showing use cases for
tuple.index that can't be rewritten.

No you just have proven my point. I predicted that whatever use case
would be given, people would stand ready to rewrite is and use those
rewritals as argument againt the use case. Here you are ready to do
just that.

Since you can just write an index function that works with any sequence
or you could simply write something like list(tup).index('...'), any
code that would use tupple.index can be rewritten to do without.

But that is not such a strong argument. Should the case be reversed
and tuples have an index method and lists not, you would be able
to rewrite any code that would use list.index into code that
wouldn't. But if you are so eager to rewrite, how about the following:

I am using the struct module to get binary data from a file.
Sometimes I want to skip until I find a particular binary
number. Somewhat simplified it looks like this:


class Itemfile:
def __init__(self, fn):
self.fl = open(fn)
self.ix = 80

def nextitem(self):
if self.ix == 80:
self.buf = struct.unpack("80i", self.fl.read(320))
self.ix = 0
result = self.buf[self.ix]
self.ix += 1
return result

def skipuntil(self, val):
done = False
while not done:
try:
self.ix = self.buf.index(val, self.ix)
done = True
except ValueError:
self.ix = 0
self.buf = struct.unpack("80i", self.fl.read(320))


Now I'm sure you can rewrite this without the need of tuple.index.
It just seems odd that I have to go through extra hoops here to
get the effect of tuple.index because struct.unpack returns its result
in a tuple and a tuple doesn't provide index.

 

[Antoon Pardon]

The second would be if it were true. However it would mean you would have
to add an index method to *all* sequences.

FWIW, The current documentation says that 's.index' is a method defined on
all *mutable* sequence types: almost as simple as your second wording but
unfortunately misleading since strings also have index.

Which illustrate that the mutable unmutable distinction is not very
usefull to decide whether a sequence could use "index" or not.

I genuinely cannot think of a use case. I didn't ask you to suggest one so
I could show you how to avoid it, I asked you to suggest one so that we
could take the discussion from the purely hypothetical to a more concrete
discussion of whether it would be a worthwhile enhancement.

I have given a use case in an other article in this thread. Feel free to
comment.

 

[Antoon Pardon]

The non-existence of tuple.index is only a limitation if there is a need
for the method to exist. Please prove that this need exists.

It doesn't because "need" is such a strong word, that just the fact that
you can write your own index function means the method isn't needed.

 

[Steve Holden]

I think this observation leads to insights both at the end-user level
and at the implementer level. Tuples and lists are sequences; the
index method can be defined generically for all sequences; adding such
a method to the tuple type can be done with barely any changes to the
implementation taken from the list type. This leads to the observation
that a generic index method (defined as a function in the
implementation) could be made to work with both lists and tuples, and
that various other methods might also be defined similarly, although
things like append, extend and other mutating methods wouldn't be
appropriate for a tuple.

A generic definition of index would be tricky, though, if you wanted to
include strings as sequences. In that case you aren't testing for the
presence of a single element but a sub-sequence - I think this change
was introduced in 2.4 to allow checks like

"men" in "three good men"

to succeed where formerly only single characters could be checked for.
One might perversely allow extension to lists and tuples to allow

[3, 4] in [1, 2, 3, 4, 5, 6]

to succeed, but that's forcing the use case beyond normal limits. The
point I am trying to make is that circumstances alter cases, and that we
can't always rely on our intuition to determine how specific methods
work, let alone whether they are available.

I hear the screams of "just add the index() method to tuples and have
done with it" and, to an extent, can sympathize with them. But that way
lies creeping featurism and the next thing you know we'll have a ternary
operator in the language - oh wait, we do now!

Which of the above is the easiest to document?

Now with implementation and maintaining. If you would start with a class
of sequence which classes like tuple and list would inherit from, then
there also would be a single function to be implemented and maintained.
It would just be usable in more types.

There isn't a "big win" in this case: the intersection of useful
methods between mutable and immutable sequence types is rather small.
Nevertheless, providing a slightly deeper abstract class hierarchy
might be appropriate, and this is being considered for Python 3000.

[...]

The same happened with the ternary operator. Every use case someone
could come up with was rejected by rewriting the code without using
a ternary operator. AFAICS the only reason the ternary operator
finaly got introduced was because a python developer was bitten by an
illusive bug, introduced by one of the idioms that was often used as a
way to simulate a ternary operator.

The ternary operator, whilst providing new and occasionally useful
functionality, is rather "badly phrased" in my opinion: when used,
it's a bit like reading one natural language and suddenly having the
grammar of another in use for the rest of the sentence.

Indeed the syntax is deliberately "crippled" - Guido's reasoning being,
I believe, that if it were too easy and natural to use then people would
use it inappropriately and too frequently.

regards
Steve

 

[Carsten Haese]

But if you are so eager to rewrite, how about the following:

I am using the struct module to get binary data from a file.
Sometimes I want to skip until I find a particular binary
number. Somewhat simplified it looks like this:


class Itemfile:
def __init__(self, fn):
self.fl = open(fn)
self.ix = 80

def nextitem(self):
if self.ix == 80:
self.buf = struct.unpack("80i", self.fl.read(320))
self.ix = 0
result = self.buf[self.ix]
self.ix += 1
return result

def skipuntil(self, val):
done = False
while not done:
try:
self.ix = self.buf.index(val, self.ix)
done = True
except ValueError:
self.ix = 0
self.buf = struct.unpack("80i", self.fl.read(320))


Now I'm sure you can rewrite this without the need of tuple.index.
It just seems odd that I have to go through extra hoops here to
get the effect of tuple.index because struct.unpack returns its result
in a tuple and a tuple doesn't provide index.

Your data is an array. Choosing a data structure that doesn't fit your
data is always going to cause pain. Instead of using struct.unpack, you
should use array.array, and voila!, you get an index method.

-Carsten

 

[Carsten Haese]

I hear the screams of "just add the index() method to tuples and have
done with it" and, to an extent, can sympathize with them. But that way
lies creeping featurism and the next thing you know we'll have a ternary
operator in the language - oh wait, we do now!

It would indeed be much easier to just give up. However, the resistance
to tuple.index is more than a generic resistance to feature creep. As I
have demonstrated elsewhere on this thread, any use case for tuple.index
will be inherently obfuscated. Code clarity is a major design goal of
Python, and adding tuple.index would be contrary to this goal.

I'm just a user with no influence on the development of Python itself,
but in my humble opinion, the non-existence of tuple.index is more
pythonic than its existence would be.

-Carsten

 

[Paul Boddie]

The point I am trying to make is that circumstances alter cases, and that we
can't always rely on our intuition to determine how specific methods
work, let alone whether they are available.

But it's telling that by adopting precisely the implementation that we
currently have for lists, we can have a tuple method which does what
most people would reasonably expect. "Why are we suddenly getting
single characters instead of whole strings?" people have presumably
exclaimed often enough, illustrating that the sequence nature of
strings is a controversial topic. Lists and tuples, however, don't
have such controversial baggage.

I hear the screams of "just add the index() method to tuples and have
done with it" and, to an extent, can sympathize with them. But that way
lies creeping featurism and the next thing you know we'll have a ternary
operator in the language - oh wait, we do now!

Yes, but the cost of adding index to tuples is minimal, and the mental
cost to programmers is arguably negative. Meanwhile, we now have to
put up with the syntactic bodge that is the ternary operator until the
time comes when it gets deprecated as something that didn't work out
or wasn't really necessary (in Python 4000, perhaps), meaning that we
now have to read third-party code more carefully, the people writing
editors and tools have to change their lexers/parsers again, and so
on.

Paul

 

[Steve Holden]

It would indeed be much easier to just give up. However, the resistance
to tuple.index is more than a generic resistance to feature creep. As I
have demonstrated elsewhere on this thread, any use case for tuple.index
will be inherently obfuscated. Code clarity is a major design goal of
Python, and adding tuple.index would be contrary to this goal.

I'm just a user with no influence on the development of Python itself,
but in my humble opinion, the non-existence of tuple.index is more
pythonic than its existence would be.

I quite agree. I was not advocating it as a serious course of action,
more admiring its noise-reduction potential. I'm no great fan of the if
.... else expression either, come to that.

regards
Steve

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

i = p.index(current_player)
opponents = p[:i-1] + p[i+1:]

An alternative is this:

opponents = tuple(x for x in p if x is not current_player)

You may disagree, but in my opinion, the alternative is better because
it is a more natural translation of the concept that the opponents of
the current player are all players that are not the current player.

Your alternative is wrong because it wont raise ValueError if
current_player is not present in the tuple. Please revise your
"solution."

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

I think this observation leads to insights both at the end-user level
and at the implementer level. Tuples and lists are sequences; the
index method can be defined generically for all sequences; adding such
a method to the tuple type can be done with barely any changes to the
implementation taken from the list type. This leads to the observation
that a generic index method (defined as a function in the
implementation) could be made to work with both lists and tuples, and
that various other methods might also be defined similarly, although
things like append, extend and other mutating methods wouldn't be
appropriate for a tuple.

A generic definition of index would be tricky, though, if you wanted to
include strings as sequences. In that case you aren't testing for the
presence of a single element but a sub-sequence - I think this change
was introduced in 2.4 to allow checks like

"men" in "three good men"

to succeed where formerly only single characters could be checked for.
One might perversely allow extension to lists and tuples to allow

[3, 4] in [1, 2, 3, 4, 5, 6]

to succeed, but that's forcing the use case beyond normal limits. The
point I am trying to make is that circumstances alter cases, and that we
can't always rely on our intuition to determine how specific methods
work, let alone whether they are available.

I'd love to have that! There are at least one million use cases for
finding a sequence in a sequence and implementing it yourself is
non-trivial. Plus then both list and tuple's index methods would work
*exactly* like string's. It would be easier to document and more
useful. A big win.

Indeed the syntax is deliberately "crippled" - Guido's reasoning being,
I believe, that if it were too easy and natural to use then people would
use it inappropriately and too frequently.

There are no appropriate use cases for that feature. Maybe not for
tuple.index either, but increasing consistency and being able to say
"ALL sequences have an index method that works like THIS" would be a
big win. Unfortunately, it really is the string type that screws up
the symmetry.

 

[Steve Holden]

But it's telling that by adopting precisely the implementation that we
currently have for lists, we can have a tuple method which does what
most people would reasonably expect. "Why are we suddenly getting
single characters instead of whole strings?" people have presumably
exclaimed often enough, illustrating that the sequence nature of
strings is a controversial topic. Lists and tuples, however, don't
have such controversial baggage.

You can call something non-controversial when it generates a thread like
this? It's really a storm in a teacup. The acid test would be to
generate a patch that added the method and then see if you could get a
committer to commit it. All else (including my own contributions) is
mere hot air.

Yes, but the cost of adding index to tuples is minimal, and the mental
cost to programmers is arguably negative. Meanwhile, we now have to
put up with the syntactic bodge that is the ternary operator until the
time comes when it gets deprecated as something that didn't work out
or wasn't really necessary (in Python 4000, perhaps), meaning that we
now have to read third-party code more carefully, the people writing
editors and tools have to change their lexers/parsers again, and so
on.

What can I say? Every language has warts. Some people were as anxious to
see if ... else (which I regard as a wart) put in as others are to see
tuple.index().

regards
Steve

 

[Carsten Haese]

i = p.index(current_player)
opponents = p[:i-1] + p[i+1:]

An alternative is this:

opponents = tuple(x for x in p if x is not current_player)

You may disagree, but in my opinion, the alternative is better because
it is a more natural translation of the concept that the opponents of
the current player are all players that are not the current player.

Your alternative is wrong because it wont raise ValueError if
current_player is not present in the tuple. Please revise your
"solution."

You have a point. Here is my revised solution:

assert current_player in p
opponents = tuple(x for x in p if x is not current_player)

The added advantage is that AssertionError is better than IndexError for
conveying that a severe program bug has occurred.

-Carsten