"index" method only for mutable sequences??

[Carsten Haese]

i = p.index(current_player)
opponents = p[:i-1] + p[i+1:]

An alternative is this:

opponents = tuple(x for x in p if x is not current_player)

You may disagree, but in my opinion, the alternative is better because
it is a more natural translation of the concept that the opponents of
the current player are all players that are not the current player.

Your alternative is wrong because it wont raise ValueError if
current_player is not present in the tuple. Please revise your
"solution."

You have a point. Here is my revised solution:

assert current_player in p
opponents = tuple(x for x in p if x is not current_player)

The added advantage is that AssertionError is better than IndexError for
conveying that a severe program bug has occurred.

_.replace("IndexError", "ValueError"), of course.

-Carsten

 

[Paul Boddie]

You can call something non-controversial when it generates a thread like
this? It's really a storm in a teacup. The acid test would be to
generate a patch that added the method and then see if you could get a
committer to commit it. All else (including my own contributions) is
mere hot air.

The patch is already submitted:

http://sourceforge.net/tracker/index.php?func=detail&aid=1696444&group_id=5470&atid=305470

I won't miss tuple.index very often if it never gets in, but it's
always enlightening/entertaining to see the rationales given for the
rejection of this and other features, in contrast to things like "y if
x else z" which just seem to mysteriously acquire momentum and then
power their way in regardless.

Paul

 

[Paul Boddie]

You have a point. Here is my revised solution:

assert current_player in p
opponents = tuple(x for x in p if x is not current_player)

The added advantage is that AssertionError is better than IndexError for
conveying that a severe program bug has occurred.

Unless you're running python with the -O flag. So, instead of the
"unpythonic"...

i = p.index(current_player)
opponents = p[:i] + p[i+1:]

....we now have...

if current_player not in p:
raise ValueError, current_player
opponents = tuple(x for x in p if x is not current_player)

Sure, p would probably be a list for a lot of games, and as I've noted
previously, since you have to specify all of the elements at once to
initialise the tuple, you should know where the player is. But this
only applies to situations where you have control over the code
needing to know the index *and* the code making the tuple in the first
place.

Paul

 

[Antoon Pardon]

But if you are so eager to rewrite, how about the following:

I am using the struct module to get binary data from a file.
Sometimes I want to skip until I find a particular binary
number. Somewhat simplified it looks like this:


class Itemfile:
def __init__(self, fn):
self.fl = open(fn)
self.ix = 80

def nextitem(self):
if self.ix == 80:
self.buf = struct.unpack("80i", self.fl.read(320))
self.ix = 0
result = self.buf[self.ix]
self.ix += 1
return result

def skipuntil(self, val):
done = False
while not done:
try:
self.ix = self.buf.index(val, self.ix)
done = True
except ValueError:
self.ix = 0
self.buf = struct.unpack("80i", self.fl.read(320))


Now I'm sure you can rewrite this without the need of tuple.index.
It just seems odd that I have to go through extra hoops here to
get the effect of tuple.index because struct.unpack returns its result
in a tuple and a tuple doesn't provide index.

Your data is an array. Choosing a data structure that doesn't fit your
data is always going to cause pain. Instead of using struct.unpack, you
should use array.array, and voila!, you get an index method.

No it is not. This is exactly what I thought was going to happen. One
simplifies a problem so that the code is not too big to discuss here
and people will use characteristics of the simplified code to suggest
how one should have solved the problem differently.

I'm not interrested in going through such a merry around again.

As I said, writing an index function that works with any kind
of sequence is easy enough and once written can be used as
often as one whishes. So although I prefer more consistency
from a language that python currently provides the language
has enough going for it to stick with it dispite these kind
of warts. So having that function is a practical enough solution
for me. And curiously, having that function makes using
tuples no longer painfull in situations where other would
argue that tuples don't fit my data. If tuples don't fit
my data, I sure find it strange that one little function
causes you to no longer experience it as such.

 

[bearophileHUGS]

BJörn Lindqvist:

One might perversely allow extension to lists and tuples to allow
[3, 4] in [1, 2, 3, 4, 5, 6]
to succeed, but that's forcing the use case beyond normal limits. The
point I am trying to make is that circumstances alter cases, and that we
can't always rely on our intuition to determine how specific methods
work, let alone whether they are available.

I'd love to have that! There are at least one million use cases for
finding a sequence in a sequence and implementing it yourself is
non-trivial. Plus then both list and tuple's index methods would work
*exactly* like string's. It would be easier to document and more
useful. A big win.

Sublist search (and generally adding a bit of pattern matching
features to Python) looks far from being perverse, it may even become
pythonic ;-)

Bye,
bearophile

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

You have a point. Here is my revised solution:

assert current_player in p
opponents = tuple(x for x in p if x is not current_player)

The added advantage is that AssertionError is better than IndexError for
conveying that a severe program bug has occurred.

Assertions are not checked when you run scripts with -O. Furthermore,
changing the exception type and the message it produces, is quite a
big deviation from list.index.

 

[Carsten Haese]

Assertions are not checked when you run scripts with -O.

Right. Which is why you use assert to guard against situations that
should never happen, and you determine in unit tests that they, in fact,
don't happen. Realistically, in a game loop such as

while not game_has_ended:
for current_player in p:
player_does_something(current_player)

it's obvious that the assertion "current_player in p" will never fail.

Furthermore,
changing the exception type and the message it produces, is quite a
big deviation from list.index.

What's your point? I wasn't talking about coding a drop-in replacement
for list.index. I was suggesting one possible solution to a problem that
may or may not be solved by using tuple.index.

-Carsten

 

[Chris Mellon]

Right. Which is why you use assert to guard against situations that
should never happen, and you determine in unit tests that they, in fact,
don't happen. Realistically, in a game loop such as

while not game_has_ended:
for current_player in p:
player_does_something(current_player)

I'm curious why someone would even consider using a tuple in this case
regardless. I think that much of the desire for tuple.index is because
people use a tuple where they could have a list, but either a) some
vestige of B&D language programming comes out and they want to make
sure a caller can't mutate it or b) they decide it's not going to
change and use the "immutable list" version of the tuple.

The first reason is clearly perverse, and can be discounted.

The second means this is not so much about tuple.index as it is about
appropriate data types. It's not going to be resolved by use cases,
because clearly the only use of tuple.index is when you're using it as
an immutable list, as in this case. Any use case where you'd want to
search a tuple can be rewritten (trivially) as a list.

So the solution for the dissenters is to justify the need for a frozen
list, not to justify index on tuples.

The only use case which even approaches reasonableness in this thread
is the binary parsing example (where the position of a value in an
unpacked binary blob might be something you need to know). This is a
rare enough use case and is easy enough to work around (convert it to
a list, write a helper function) that I don't think it's worth any
language change overhead at all.

 

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

while not game_has_ended:

I'm curious why someone would even consider using a tuple in this case
regardless. I think that much of the desire for tuple.index is because
people use a tuple where they could have a list, but either a) some
vestige of B&D language programming comes out and they want to make

Maybe someone had to much alcohol when they were coding? Maybe they
don't know better? Maybe they thought that an index method on a
sequence made sense? Who are you to spoil their fun? Could it be that
YOU are the B&D person?

 

[Terry Reedy]

One might perversely allow extension to lists and tuples to allow

[3, 4] in [1, 2, 3, 4, 5, 6]

to succeed, but that's forcing the use case beyond normal limits.

I'd love to have that! There are at least one million use cases for
finding a sequence in a sequence and implementing it yourself is
non-trivial. Plus then both list and tuple's index methods would work
*exactly* like string's. It would be easier to document and more
useful. A big win.

=======================
It would be ambiguous: [3,4] in [[1,2], [3,4], [5,6]] is True now.

Strings are special in that s can only be a (sub)string of length 1.
'b' in 'abc' is True. This makes looking for longer substrings easy.

However, [2] in [1,2,3] is False. IE, list is not normally a list. So
looking for sublists is different from looking for items.

Terry Jan Reedy

 

[Terry Reedy]

| always enlightening/entertaining to see the rationales given for the
| rejection of this and other features, in contrast to things like "y if
| x else z" which just seem to mysteriously acquire momentum and then
| power their way in regardless.

No mystery. It was Guido's initial proposal and he never changed his mind.

 

[Paul Rubin]

You have a point. Here is my revised solution:

assert current_player in p
opponents = tuple(x for x in p if x is not current_player)

Still wrong on two counts. First, assert is a no-op if optimization
is turned on. Second, your version returns a different result from
the original if current_player occurs in p more than once.

 

[Carsten Haese]

Still wrong on two counts. First, assert is a no-op if optimization
is turned on.

Right. I already responded to this when Bjorn made the same objection.
Please try to keep up.

Second, your version returns a different result from
the original if current_player occurs in p more than once.

First of all, in the use case we're talking about, current_player
shouldn't occur in p more than once. And if it did, is it really
reasonable to demand that that player be their own opponent? I don't
think so. So yes, the result is different because it's correct, and the
tuple.index-based solution is actually incorrect in this case.

-Carsten

 

[Hendrik van Rooyen]

I'm just a user with no influence on the development of Python itself,
but in my humble opinion, the non-existence of tuple.index is more
pythonic than its existence would be.

I really cannot follow the logic behind this statement.
I can write:

L = [a,b,c,d,e,f]
T= (a,b,c,d,e,f)

The difference between the two things is that I can add to
and change L, but not T.

Now it seems to me that whatever argument is used to
justify the existence of:

n = L.index(d)

can be used to justify the existence of:

n = T.index(d)

and vice versa.

Cut down to these basics, it seems to me that the arguments
against the latter construct are simply knee jerk reactions
to preserve the status quo.

If an index method for tuples is such a very bad thing,
then the same arguments can be used to justify the
removal of the list index method from the language.

I happen to agree with Antoon - there is a LOT of merit
in consistency, as it makes things easy to learn and remember.

And I would heretically go even further, and argue that it
should be possible to write stuff like:

T = T.append(x)

And get back a new tuple bound to the old name...

- Hendrik

 

[Paul Boddie]

This is a rare enough use case and is easy enough to work around (convert it to
a list, write a helper function) that I don't think it's worth any
language change overhead at all.

It isn't a language change: it's a change to the interface of a data
type, and not a particularly big one, either.

Paul

 

[Steven D'Aprano]

I can write:

L = [a,b,c,d,e,f]
T= (a,b,c,d,e,f)

The difference between the two things is that I can add to
and change L, but not T.

No, that's *one* difference between the two things. There are other
differences, e.g. the obvious is that they are different types (and
therefore different string representations), but also some not so obvious:

-378539185

hash([1,2,3])

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: list objects are unhashable

Now it seems to me that whatever argument is used to justify the
existence of:

n = L.index(d)

can be used to justify the existence of:

n = T.index(d)

and vice versa.

That depends on what you the purpose of lists and tuples are. Yes, they
are both sequences, but they have different uses. Ball-peen hammers and
tack hammers are both hammers, but they have different purposes and
therefore different functionality. Likewise for tuples and lists:

Lists are designed for sequences of homogeneous items, e.g.:

L = [1, 2, 4, 8, 16, 32]

while tuples are designed to be more like structs or records, with
heterogeneous items, e.g.:

T = ("Fred", 32, 12.789, {}, None, '\t')

So according to these intended usages, it makes sense to ask for
L.index(32) because in an application you have no way of telling which
item (if any) 32 would be in. But T.index(32) is seen as meaningless,
because you know that either 32 is in the second slot or it isn't --
you're never going to find yourself in a situation knowing that 32 is
*somewhere* in the tuple without knowing where the one place it *could* be.

(There is one other option: you care that 32 is somewhere in the tuple,
but you don't care where. That's when you use the "in" operator.)

Anyway, that was the original design. When you see tuple, think struct. If
you have a struct, it doesn't make a whole lot of sense to ask "which
field contains 32?", and so according to this intended usage, giving
tuples index and count methods would be a Bad Idea: it just makes extra
work for the Python Dev team, for no benefit.

Personally, I think that tuples do double-duty as *both* immutable lists
and structs/records. So even though index and count methods don't make
sense for a struct, it does make sense for an immutable list, and I for
one would not object to seeing tuples grow those two methods.

[snip]

And I would heretically go even further, and argue that it should be
possible to write stuff like:

T = T.append(x)

And get back a new tuple bound to the old name...

That's a design decision, and one that's not likely to be accepted because
it's so easy to do:


T = T + (x,) # like T.append()
T = T + (x, y, z) # like T.extend()
T = tuple(sorted(T)) # like T.sort()
T = T[:4] + T[4:] # like del T[4]
etc.

It makes sense for lists to have these methods because lists can make the
changes in place, but tuples can't.

 

[Antoon Pardon]

I can write:

L = [a,b,c,d,e,f]
T= (a,b,c,d,e,f)

The difference between the two things is that I can add to
and change L, but not T.

No, that's *one* difference between the two things. There are other
differences, e.g. the obvious is that they are different types (and
therefore different string representations), but also some not so obvious:

hash((1,2,3))

-378539185

hash([1,2,3])

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: list objects are unhashable

Now it seems to me that whatever argument is used to justify the
existence of:

n = L.index(d)

can be used to justify the existence of:

n = T.index(d)

and vice versa.

That depends on what you the purpose of lists and tuples are. Yes, they
are both sequences, but they have different uses. Ball-peen hammers and
tack hammers are both hammers, but they have different purposes and
therefore different functionality. Likewise for tuples and lists:

Lists are designed for sequences of homogeneous items, e.g.:

L = [1, 2, 4, 8, 16, 32]
while tuples are designed to be more like structs or records, with
heterogeneous items, e.g.:

T = ("Fred", 32, 12.789, {}, None, '\t')

I think you are confused. Last time I heard this homogeneous items stuf,
it had nothing to do with the types being the same. They were homogeneous
because they somehow belonged together and heterogeneous because they
just happened to live together. Similarity of type played no part in
calling the data homogeneous or heterogeneous.

 

[Marc 'BlackJack' Rintsch]

Lists are designed for sequences of homogeneous items, e.g.:

L = [1, 2, 4, 8, 16, 32]
while tuples are designed to be more like structs or records, with
heterogeneous items, e.g.:

T = ("Fred", 32, 12.789, {}, None, '\t')

I think you are confused. Last time I heard this homogeneous items stuf,
it had nothing to do with the types being the same. They were homogeneous
because they somehow belonged together and heterogeneous because they
just happened to live together. Similarity of type played no part in
calling the data homogeneous or heterogeneous.

Then you are confused. The typical use case for tuples are database
records. The columns in the table can have completely different types but
the values in a row, represented as a Python tuple, of course belong
together.

The homogeneous objects in lists must not be of the same type but share
some behavior so it makes sense to apply some operation on all the
elements. For example get the length of each item or sum them all up.

Ciao,
Marc 'BlackJack' Rintsch

 

[Antoon Pardon]

Lists are designed for sequences of homogeneous items, e.g.:

L = [1, 2, 4, 8, 16, 32]
while tuples are designed to be more like structs or records, with
heterogeneous items, e.g.:

T = ("Fred", 32, 12.789, {}, None, '\t')

I think you are confused. Last time I heard this homogeneous items stuf,
it had nothing to do with the types being the same. They were homogeneous
because they somehow belonged together and heterogeneous because they
just happened to live together. Similarity of type played no part in
calling the data homogeneous or heterogeneous.

Then you are confused. The typical use case for tuples are database
records. The columns in the table can have completely different types but
the values in a row, represented as a Python tuple, of course belong
together.

Don't blame me. I don't agree with the view. But that was sort of the
explanation that was given here last time I remember this topic came
up in defending why tuples and lists differ in a number of ways that
are less obvious.

They wrote about lists containing homogeneous items and tuples
containing hetergenous items but stressed rather strongly that
this shouldn't be understood in terms of type similarities.

The homogeneous objects in lists must not be of the same type but share
some behavior so it makes sense to apply some operation on all the
elements. For example get the length of each item or sum them all up.

No they don't. The counter example is using a list as a stack when
evaluating expressions. You can use one stack to store the still
to be treated numbers and operands and those two don't need
common behaviour.

 

[Chris Mellon]

-----Original Message-----
From: [email protected] [mailtoython-
[email protected]] On Behalf Of Steven D'Aprano
Sent: Wednesday, April 11, 2007 7:49 AM
To: (e-mail address removed)
Subject: Re: tuples, index method, Python's design

(There is one other option: you care that 32 is somewhere in the tuple,
but you don't care where. That's when you use the "in" operator.)

Anyway, that was the original design. When you see tuple, think struct. If
you have a struct, it doesn't make a whole lot of sense to ask "which
field contains 32?", and so according to this intended usage, giving
tuples index and count methods would be a Bad Idea: it just makes extra
work for the Python Dev team, for no benefit.

Personally, I think that tuples do double-duty as *both* immutable lists
and structs/records. So even though index and count methods don't make
sense for a struct, it does make sense for an immutable list, and I for
one would not object to seeing tuples grow those two methods.

From another function, you receive a tuple of data that it extracted

from a stream. Within that tuple is a marker that indicates where the
head of the incoming stream's data structure is. You need to find the
marker and scan from that location on to sync your local data structure
to the incoming stream's data.

Should the external function provide the stream data in a list rather
than a tuple? Probably, but someone else wrote the function so that's
out of your control. Can you cast the tuple to a list? Sure, but for a
large tuple that's potentially a large speed and memory hit.

That probably the biggest general use case for tuple.index(). A
third-party module returns a tuple in which you need to find a piece of
data.

So, when you have a) a third party module that you cannot change and
b) it shouldn't return a tuple but it does anyway and c) it's a big
enough tuple that is large enough that conversion to a list is
prohibitive, that's a "general" use case for tuple.index?

Has this supposedly general and common use case actually happened?