Consistency in Python

[Hendrik van Rooyen]

Hi,

for S where S is a Standard Python type:
The slice notation S[n] returns either:
The n'th element of S, or
The value of the dictionary entry whose key is n.

This is beautiful because as a programmer you don't have to worry what S is...
(and as an aside - This consistency has already saved my butt when I thought I
was working with a string that turned out to be a tuple instead - and still
worked perfectly as expected...)

Now consider what you have to do to add an element to S...
(where "add" is used in its meaning of increasing the size of a set, and not 1 +
1 = 2)

There seems to be no common methods such as-
"prepend" - for adding something to the beginning
"append" - for adding something to the end
"insert[j]" - for adding something somewhere in the middle

Or have I missed something ?

Is there a reason for this lack - or is it simply that the language has "just
growed" ?

Are there any plans afoot to do this kind of tidying up in the places where it
makes sense?

BTW - I understand that some things are immutable - but that is an
implementation detail, not a language issue. - the fact that you get the name S
bound to a new object is irrelevant to a discussion about how you tell the
interpreter to do something...

Any other thoughts - such as making a kind of opposite to del instead ?

- Hendrik

 

[Diez B. Roggisch]

Hi,

for S where S is a Standard Python type:
The slice notation S[n] returns either:
The n'th element of S, or
The value of the dictionary entry whose key is n.

This is beautiful because as a programmer you don't have to worry what S is...
(and as an aside - This consistency has already saved my butt when I thought I
was working with a string that turned out to be a tuple instead - and still
worked perfectly as expected...)

Now consider what you have to do to add an element to S...
(where "add" is used in its meaning of increasing the size of a set, and not 1 +
1 = 2)

There seems to be no common methods such as-
"prepend" - for adding something to the beginning
"append" - for adding something to the end
"insert[j]" - for adding something somewhere in the middle

Or have I missed something ?

Yes, the nature of collections. dictionaries have no notion of
"somewhere in the middle". Most of the time they are unordered. If they
are ordered, they can be ordered by insertion time, key or value value.
And they always need key, value

So - all these three methods only make sense on sequences which imply a
key (the index), and are mutable of course - which is why they are
available on lists only.

Diez

 

[Paul Boddie]

There seems to be no common methods such as-
"prepend" - for adding something to the beginning
"append" - for adding something to the end
"insert[j]" - for adding something somewhere in the middle

Or have I missed something ?
[...]

BTW - I understand that some things are immutable - but that is an
implementation detail, not a language issue. - the fact that you get the name S
bound to a new object is irrelevant to a discussion about how you tell the
interpreter to do something...

You haven't missed one of the principal reasons exactly, but you
underestimate its importance. There aren't such methods as append or
insert on strings or tuples precisely because those objects are
immutable, whilst such methods on lists and other mutable objects
change the contents of such objects. Moreover, append and insert return
no result because the change occurs within an existing object - if you
were to return a reference to the changed object, it would be the same
reference as the one you already had.

# Append on lists:
l.append(something) # returns nothing (you'll get None)

Now, you could argue that insert and append should always return a
reference to some object, and that for lists (and other mutable
objects) it should return the same reference (to the changed object),
whereas for strings (and other immutable objects) it should return a
different reference (to an object which has the changed contents of the
original object).

# Fictional append on strings:
s2 = s.append(sometext) # s would be the same, s2 different
# Fictional append variant on lists:
l2 = l.append(something) # l and l2 would be the same

However, there's a sort of unwritten guarantee - although it could be
in the documentation - that the append and insert methods specifically
mutate objects and that they therefore have no place in immutable
objects. Certainly, the behaviour illustrated above could be surprising
in some kinds of programs because the append method on strings would be
more like a factory or a copy constructor rather than a "mutator".

Now, you could argue that the name involved could be hacked in some
magical way, thus preserving the illusions that strings are mutable,
but what happens when a name is not involved?

s.append(sometext) # s gets replaced
(text + moretext).append(sometext) # what gets replaced?

Mutability is more than a mere implementation detail: in imperative
languages, it's probably one of the most underrated mechanisms
influencing the correct behaviour of programs.

Paul

 

[Paul McGuire]

Moreover, append and insert return
no result because the change occurs within an existing object - if you
were to return a reference to the changed object, it would be the same
reference as the one you already had.

There's nothing wrong with returning self from a mutator. This was a common
idiom in Smalltalk (the syntax for this was "^self", which was probably the
most common statement in any Smalltalk program), and permitted the chaining
of property mutators into a single line, each invoking a mutator and
returning self, which was then used to invoke the next mutator, etc. For
instance, you could have a class like this:

class Box(object):
def __init__(self):
self.ln = 0
self.wd = 0
self.ht = 0
def length(self,ln):
self.ln = ln
return self
def width(self,w):
self.wd = w
return self
def height(self,h):
self.ht = h
return self
volume = property(lambda self:self.ln*self.wd*self.ht)

bx = Box().length(100).width(50).height(20)
print bx.volume


I guess this is sort of a trivial example, and in its triviality, even
borders on un-Pythonic. But it is useful when you want to support a large
set of object attributes, without creating an init method with many, many
args. Even in this case, an init method that takes length, width and height
args must always get them specified in the correct order, or use named args.
But with mutators that return self, a client could write any of these:

bx = Box().length(100).width(50).height(20)
bx = Box().width(50).height(20).length(100)
bx = Box().width(50).length(100).height(20)
....etc...

and the results are the same.

-- Paul

 

[Paul Boddie]

There's nothing wrong with returning self from a mutator. This was a common
idiom in Smalltalk (the syntax for this was "^self", which was probably the
most common statement in any Smalltalk program), and permitted the chaining
of property mutators into a single line, each invoking a mutator and
returning self, which was then used to invoke the next mutator, etc.

Various Java APIs encourage this kind of behaviour, too, or at least
encouraged it in earlier times. However, as I wrote, there is some kind
of understanding that undoubtedly results from a policy decision in the
design of Python's built-in types that saves the programmer from having
to question the nature of a return value in such a way.

[...]

But with mutators that return self, a client could write any of these:

bx = Box().length(100).width(50).height(20)
bx = Box().width(50).height(20).length(100)
bx = Box().width(50).length(100).height(20)
...etc...

and the results are the same.

This is very convenient, and I've often thought about doing such things
in my own APIs, but there can sometimes be subtle problems introduced
into programs when you decide to change the nature of such a mutator,
or if you have another type/class whose mutators are of a different
nature, such that the width method produces a new Box object (which is
quite similar to what the questioner seemed to have in mind) instead of
mutating (or failing to mutate) the existing object.

Indeed, it may be important to know whether you're creating new objects
or not, and without metadata the most convenient way to communicate
this is quite probably to define a rigid interface (after all, why
should width or append return an object?) which cannot be implemented
on immutable things.

Paul

 

[Hendrik van Rooyen]

| Hendrik van Rooyen wrote:
| >
| > There seems to be no common methods such as-
| > "prepend" - for adding something to the beginning
| > "append" - for adding something to the end
| > "insert[j]" - for adding something somewhere in the middle
| >
| > Or have I missed something ?
|
| [...]
|
| > BTW - I understand that some things are immutable - but that is an
| > implementation detail, not a language issue. - the fact that you get the
name S
| > bound to a new object is irrelevant to a discussion about how you tell the
| > interpreter to do something...
|
| You haven't missed one of the principal reasons exactly, but you
| underestimate its importance. There aren't such methods as append or
| insert on strings or tuples precisely because those objects are
| immutable, whilst such methods on lists and other mutable objects
| change the contents of such objects. Moreover, append and insert return
| no result because the change occurs within an existing object - if you
| were to return a reference to the changed object, it would be the same
| reference as the one you already had.
|
| # Append on lists:
| l.append(something) # returns nothing (you'll get None)
|
| Now, you could argue that insert and append should always return a
| reference to some object, and that for lists (and other mutable
| objects) it should return the same reference (to the changed object),
| whereas for strings (and other immutable objects) it should return a
| different reference (to an object which has the changed contents of the
| original object).
|
| # Fictional append on strings:
| s2 = s.append(sometext) # s would be the same, s2 different
| # Fictional append variant on lists:
| l2 = l.append(something) # l and l2 would be the same

Lovely - this is exactly what I am thinking about...

|
| However, there's a sort of unwritten guarantee - although it could be
| in the documentation - that the append and insert methods specifically
| mutate objects and that they therefore have no place in immutable
| objects. Certainly, the behaviour illustrated above could be surprising
| in some kinds of programs because the append method on strings would be
| more like a factory or a copy constructor rather than a "mutator".
|
| Now, you could argue that the name involved could be hacked in some
| magical way, thus preserving the illusions that strings are mutable,
| but what happens when a name is not involved?
|
| s.append(sometext) # s gets replaced
| (text + moretext).append(sometext) # what gets replaced?

nothing really - this thing standing on its own is kind of "outside" the
language - If I do not have a name for you, I can't order you about - and the
same is true here - the result is just a string that is used wherever its
needed, as a literal would be : "this is a string" + " Here is some more"

("is" being used in the sense of "should be" - we are talking hypothetics here
and not how stuff actually "is")

|
| Mutability is more than a mere implementation detail: in imperative
| languages, it's probably one of the most underrated mechanisms
| influencing the correct behaviour of programs.
|
| Paul

This could be true - its just that, as an assembler type person - the notion
that some parts of memory is "cast in concrete" is kind of strange to me...

I understand that when you do something like giving a dict a key and tie a value
to the key - its dangerous to change the key - it would be a very smart
implementation indeed that can track the changes to such a key - but I can see
no value in solving the problem by making the key "Immutable" - it is a natural
consequence (at least in my mind) that when you store something in one way, you
have to use exactly the same way to retrieve it - but I don't think it should be
a language rule to try to protect a programmer from his own stupidity in such
cases -

so if I do something like:

PaulsName = "Paul Buddie"

and I use that as a key in my telephone directory,
and then I see the error, and I do

PaulsName = "Paul Boddie"

and I try to get at the information I have stored earlier, using the same
symbolic name - it aint gonna work...

And I accept this - its not what I am on about - I still think it would be nicer
if there were these commands to do the prepend insert append job, instead of
doing it laboriously using slicing.

(Aside: note that in the above the "value" tied to the symbolic name "PaulsName"
has actually changed - so what is immutable about it from this perspective? if
I can do this - why bother with immutability? )

Think for instance of the hoops you have to jump through to calculate the
checksum on an Intel hex, or Motorola s file, and to make a string that you can
write to a file or send to a serial port...

Thanks for the reasoned reply, btw - I appreciate it!

- Hendrik

 

[Hendrik van Rooyen]

| Hendrik van Rooyen schrieb:
| > Hi,
| >
| > for S where S is a Standard Python type:
| > The slice notation S[n] returns either:
| > The n'th element of S, or
| > The value of the dictionary entry whose key is n.
| >
| > This is beautiful because as a programmer you don't have to worry what S
is...
| > (and as an aside - This consistency has already saved my butt when I thought
I
| > was working with a string that turned out to be a tuple instead - and still
| > worked perfectly as expected...)
| >
| > Now consider what you have to do to add an element to S...
| > (where "add" is used in its meaning of increasing the size of a set, and not
1 +
| > 1 = 2)
| >
| > There seems to be no common methods such as-
| > "prepend" - for adding something to the beginning
| > "append" - for adding something to the end
| > "insert[j]" - for adding something somewhere in the middle
| >
| > Or have I missed something ?
|
| Yes, the nature of collections. dictionaries have no notion of
| "somewhere in the middle". Most of the time they are unordered. If they
| are ordered, they can be ordered by insertion time, key or value value.
| And they always need key, value
|
| So - all these three methods only make sense on sequences which imply a
| key (the index), and are mutable of course - which is why they are
| available on lists only.
|
| Diez

I understand that dicts are actually a bit special - Its very simple to add
something to a dict - you just do it - but the in the other cases what I have in
mind is more in line with some of what Paul Boddie wrote...

- Hendrik