Thoughts about Python

[Greg Ewing (using news.cis.dfn.de)]

I know that tuples can be used as keys... but how many times do
you need tuples as dictionary keys (it would be simple to turn a list
into an immutable string if really needed ("::".join(["a","b"]).

Having used other languages where dictionary keys have to
be strings, I very much like the fact that I *don't* have to
do this in Python. Uniquely deriving a string from a non-flat
data structure is fraught with difficulties (e.g. in your example,
what if one of the list elements contains "::"? Or what if you
wanted to stringify [42, "elephant", myFunkyClassInstance]?) You
end up having to carefully design a stringifying scheme for each
particular case. It's a big can of hassles that I can do without.

And using a tuple as a dict key is a more common thing to do than
you might think. It synergises with another feature of the Python
syntax, that commas (and not parentheses) are what generate tuples,
to give a very nice way of using a dict as a multi-dimensional table:

my_sparse_matrix = {}
my_sparse_matrix[17, 42] = 88

This works because '17, 42' is an expression yielding a
tuple, and tuples can be used as dict keys.

I hope you can see now that expecting Python programmers to
"simply" turn their lists into strings for use as dict
keys would *not* go down well in the Python community.

 

[Greg Ewing (using news.cis.dfn.de)]

Well, yeah, this is true and isn't. It is true that there is no rule
how to name a class. To me a list / dict / string is a (basic) class
(you take about types - the manuals does also) but why introduce new
words, when the class-idiom is enough to explain everything.

Types and classes used to be very different things, but somewhere
around Python 2.1 or 2.2 a project was begun to unify the two
concepts, and nowadays the terms "type" and "class" are very
nearly synonymous. (There still exist what are now called "old-style
classes", but these are expected to disappear eventually, at which
time we will be able to stop talking about types at all and just
speak of classes.)

Before the type/class unification, builtins such as list and str
were functions. After the unification, it made sense to re-define
them as types/classes for compatibility, and for consistency the
new ones such as dict were named in the same style.

You're right that naming conventions (or lack thereof) in Python
leave something to be desired, but we're stuck with many of them
for historical reasons. Personally, I *like* that the most
fundamental built-in functions and classes have the shortest
and simplest names.

Your suggested convention (classes start with uppercase, functions
with lowercase) has its uses, and I tend to follow it in most of
the code that I write, but it has its drawbacks as well. A
counterargument often put forward is that users of a class usually
don't care whether something is really a class or whether it's
a factory function, and often you would like to be free to change
the implementation from one to the other without affecting client
code. With a naming convention that distinguishes classes from
functions, you can't do that.

 

[Paul Prescod]

....
You're right that naming conventions (or lack thereof) in Python
leave something to be desired, but we're stuck with many of them
for historical reasons.

Not really. If "open" can be renamed "file" it could also have been
renamed File. To be honest I think that the real problem is that Guido
is allergic to caps. I don't know how None snuck in.

Your suggested convention (classes start with uppercase, functions
with lowercase) has its uses, and I tend to follow it in most of
the code that I write, but it has its drawbacks as well. A
counterargument often put forward is that users of a class usually
don't care whether something is really a class or whether it's
a factory function, and often you would like to be free to change
the implementation from one to the other without affecting client
code. With a naming convention that distinguishes classes from
functions, you can't do that.

Changing from a function to a class is _always_ easy.

def open(filename, flags):
return File(filename, flags)

The trickier issue is having the class itself export the interface the
client code is used to.

Changing from a class to a function is never really very safe:

class myobject(File):
...

if isinstance(myobject, File):
....

How can you change File to be just a function safely? But anyhow, why
would you want to?

Paul Prescod

 

[Greg Ewing (using news.cis.dfn.de)]

Changing from a class to a function is never really very safe:

class myobject(File):
...

if isinstance(myobject, File):
....

How can you change File to be just a function safely?

You can't, obviously. This demonstrates that there *is*
an important distinction to be made between classes and
functions -- you can subclass one and not the other!

Arguments like this (and others, such as the capability-based
security model problems) are making me re-consider whether
having classes also be factory functions is really such
a good idea. There are two distinct ways of using classes
in Python:

1. As a base class
2. As a factory function

and these have different degrees of coupling to the
implementation. Use (1) requires it to really be a
class, whereas (2) only requires it to be a callable
object.

For use (1), there is an advantage in having classes
named differently from functions -- if it's named like
a class, you know it's a class and can therefore
subclass it.

But for use (2), it's a disadvantage, since if something
starts out as a factory function and later changes into
a (published) class, all code using it has to change.

The only way I can see of reconciling these is to adopt
the convention that whenever you publish a class Foo,
you also publish a factory function foo() that instantiates
Foo. Users of it are expected to use Foo when subclassing
and foo() when instantiating.

Now, if you change the implementation so that Foo is
no longer a class, code which subclasses Foo will break --
there's no avoiding that. But code which only instantiates,
using foo(), will keep working.

 

[Rainer Deyke]

"""
obj.freeze -> obj

Prevents further modifications to obj. A TypeError will be raised if
modification is attempted. There is no way to unfreeze a frozen
object. See also Object#frozen? .
"""

An interesting idea. This would also allow other mutable types such as
dictionaries to be used as dictionary keys. I'm not sure if I like it, but
I am sure that I am not happy with the current list/tuple situation.

An alternate approach would be for the dictionary to lock the key objects
when they are inserted and unlock them when they are removed. Another would
be for dictionaries to keep a private copy of their keys.

 

[Joe Mason]

Having used other languages where dictionary keys have to
be strings, I very much like the fact that I *don't* have to
do this in Python. Uniquely deriving a string from a non-flat
data structure is fraught with difficulties (e.g. in your example,
what if one of the list elements contains "::"? Or what if you
wanted to stringify [42, "elephant", myFunkyClassInstance]?) You
end up having to carefully design a stringifying scheme for each
particular case. It's a big can of hassles that I can do without.

TCL-style strings provice a good universal escaping mechanism for this,
as long as you're consistent.

Joe

 

[Marco Aschwanden]

I can see that tuples have their utility as a dict key. And from other
postings as well, I conclude: Tuples as dictionary keys are common
practice and are therefore here to stay.

Thanks,
Cheers,
Marco

 

[Marco Aschwanden]

Forget the speed and memory difference. The main argument for tuples as a
separate type are to use as dictionary keys. How do you propose to handle
dictionary keys without tuples?

Maybe I don't get the point here: Why do dictionaries need tuples to
work? I know that tuples can be used as keys... but how many times do
you need tuples as dictionary keys (it would be simple to turn a list
into an immutable string if really needed ("::".join(["a","b"]).

Simple, yes. Practicable, no. Wrong, certainly. For instance, I have a
lot of use cases where I use tuples of numbers -- (3, 4, 20.0101), eg.
It'd be a *hack* to convert that into a string, and the representation
would not be unique. This ain't Perl.

'(3, 4, 20.010100000000001)' # 8o)
What is not unique about this string?
Okay, I can think of cases where this approach is problematic -
foremost when sorting is involved.
It is also a bit complicated to retrieve the values from the string
(it has to be "split"ted or "eval"ed back to a tuple or list).
If this pattern would be seldom used... but it seems, that everybody
except me uses this dicts as keys.

And yes, luckily this is not Perl.

Cheers,
Marco

 

[Marco Aschwanden]

*** Problem: clumsy properties for classes

The problem is currently under discussion. Your fix is based on thinking
'self' to be a keyword, which it is not, instead of a usage convention for
English language programmers.

Thanks! Now I know where my thinking went wrong... self is "another"
soft Python contract. This was now a real enlightment! I intermingled

Quiz: how would you rewrite map(len, ['abc', (1,2,3,4), [5,6]]) if len()
were a method?

The quiz is already (and very elegantly and better readable) solved
with a comprehension list. The OO-technique used is called
polymorphism... 8o).

Thanks for the hint on <self> is not "system var".
Cheers,
Marco

 

[Marco Aschwanden]

*** Problem: tuples are not necessary

Frankly, there is very little in any programming language that is
necessary. Back when I brushed up against computability theory,
I found that all programs can be written in a language with one operation:
a combination of subtract and conditional branch. That's hardly
practical, though. The question is whether a feature serves a useful
purpose.

I once read an interesting article that a programming language can be
reduced to 4 instructions: (1) input (2) output (3) if-then (4) goto.
All other commands / functions are built upon those 4 "basic ideas".

I would say: Python struggles for a maximum functionalty by preserving
optimal "simplicity". And it does a good job at that! That's why I
like it so much.

I am not a reductionist. And I am not asking to create a reduced
Python. I had just the feeling that to learn about all specialities of
tuples is not worth the hassle... obviously I was wrong and you bring
some more aspects in favour tuples further down below:

Tuples serve two distinct purposes. One is as a cheap version
of a C struct: that is, a data structure in which each element
serves a conceptually different purpose.

The other is as a structure that is immutable so it can be used
as a key in a dict.

One thing to understand about this is that immutability is key:
dicts depend on having a hash key that is somehow derived
from the content of the object, and if the content of a key
changes that key will probably simply be unlocatable: it will be
filed in the dict under the old rather than the new hash code.

Cheers,
Marco

 

[Marco Aschwanden]

I agree with you in all but that tuples are not

necessary. Lists are the ones that are actually a
hack for speed!

Did you do some time profiling? I would be interested in the results.

3_ I prefer reading from left to right instead from right
to left (it makes the parenthesis less confusing too).
Compare to:

count=len(reversed(sorted(appended(lst,3)))).

Maybe I should learn Hebrew
Or we could always change it to

$ lst|append -3|sort|reverse|len

(in the end all are just filter patterns )

This is an excellent example!
count = lst.append(3).sort().reverse().len()
is extremly readable but chaining of method calls is not supported in
Python. And I can live with it... Python has a more "expressive" path:

lst = []
lst.append(3)
lst.sort()
lst.reverse()
count = len(lst)

The last line looks a bit odd after all the dot-notations to see a
function-call in the last line.

4_ If the python interpreter was smart enough sort()
[...]
Inmutable types are usually less "magical" than their
mutable counterpart.

For instance:

lst1=[1,2]
lst2=lst1
lst2.append(3)
lst1 [1, 2, 3]
lst2 [1, 2, 3]
tup1=(1,2)
tup2=tup1
tup2=tup2+(3,)
tup1 (1, 2)
tup2 (1, 2, 3)

Do you think a newbie would expect that a modification
on one variable would have a effect on another?

Good point.

Thanx,
Marco

 

[Mike C. Fletcher]

Marco Aschwanden wrote:
....

'(3, 4, 20.010100000000001)' # 8o)
What is not unique about this string?
Okay, I can think of cases where this approach is problematic -
foremost when sorting is involved.

Actually, this is the classic problem of data-escaping. It's *possible*
to guarantee unique values when encoding as a string, but it's a *lot*
harder than just doing str( data ). You need to make sure that
different things are going to show up with different values:

d = {}
myWorkingList = (3,4,20.010100000000001)
somePieceOfDataFromSomewhere = '(3, 4, 20.010100000000001)'
d[ myWorkingList ] = 32
d[ somePieceOfDataFromSomewhere ] = 34

That is, you'd need to encode into your string information regarding the
*type* and internal structure of the object to avoid collisions where
something just happens to look the same as another object, so something
like:

'(I3\nI4\nF20.010100000000001\nt.' # for the tuple and
"S'(3,4,20.010100000000001)'\np1\n." # for the string

Thing is, creating those static string representations is a major PITA
(in case you're wondering, the above are pickles of the values), and
would waste a lot of memory and processing cycles (to build the safe
string representation, (a task which is somewhat involved)) compared to
just using a pointer to the object and the object's built-in __hash__
and comparison methods.

It is also a bit complicated to retrieve the values from the string
(it has to be "split"ted or "eval"ed back to a tuple or list).
If this pattern would be seldom used... but it seems, that everybody
except me uses this dicts as keys.

Round-trip-eval-capable objects are considered good style, but are not
by any means universal even among the built-in objects:
<object object at 0x00C7D3B8>

pickle/cPickle is closer to the idea of a robust mechanism for coercion
to/from string values, but even it has its limits, particularly with
respect to older built-in types that assumed they'd never be pickled.

wrt sorting, it's not really a huge issue, dictionaries are not ordered,
so to produce a sorted set you'd need to produce the keys as a
list/sequence of reconstituted objects anyway.

Have fun, and enjoy yourself,
Mike

_______________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/

 

[Michael Hudson]

*** Problem: tuples are not necessary

Says you! Here's a hint: hash([]).

[...]

I suppose this is the argument: list are not immutable that is why
they cannot be used a dict key... I wonder how many people use a tuple
as dict key.

I do, on occasion.

This seems to be a common practice to justify its own "type". I
admit, I can think of simple workarounds when there wouldn't be
tuples (eg. Instead of transforming a list into tuple why not
transforming it into a string).

Oh yeah, transforming it into a string is really clean.

un-Pythonic... this bytes ... hmm... why? Reading your comments it
does not justify such a hard judgment - or does it?

I was trying to be polite...

I've been reading this list for about six (eek!) years now. You kind
of get used to posts like yours, and also to their posters realizing
that there is perhaps more sense behind Python's existing form than
they first imagine.

Cheers,
mwh

 

[b-blochl]

...............
I once read an interesting article that a programming language can be
reduced to 4 instructions: (1) input (2) output (3) if-then (4) goto.
All other commands / functions are built upon those 4 "basic ideas".
.....................

I am absolutely horrified to find goto as a necessary instruction of
programming! Real (modern) programming languages dont have a goto
statement, since extensive use is considered bad programming practice
in every language. If there is a goto statement - ignore it. "jump" or
"goto" hides the difference between selection and repetition und
unclarifies the program structure.

Bohm C. and G. Jacopini "Flow Diagrams, Turing Machines, and Languages
with Only Two Formation Rules." Communications of the ACM, Vol 9, No.5,
May 1966, pp336-371.
In this article they have shown, that any progrm can be written without
any goto statement. Structured programming is common goal since about
1970. (See O. J. Dahl and C. A. R Hoare, Notes on Structured
Programming, 1972)

Bernhard

 

[Richard Brodie]

I am absolutely horrified to find goto as a necessary instruction of
programming!

..Bohm C. and G. Jacopini "Flow Diagrams, Turing Machines, and Languages
with Only Two Formation Rules."

You'll be sticking to the infinitely long tape then.

 

[Terry Reedy]

(e-mail address removed) (David M. Cooke) wrote in message
'(3, 4, 20.010100000000001)' # 8o)
What is not unique about this string?
Okay, I can think of cases where this approach is problematic -
foremost when sorting is involved.

Mike pointed out the problem of making sure that different things get
different strings. For dict keys, there is also the problem of making sure
that 'different' things that hash equal and which have the same value as a
key get the same string, and you cannot do that without inventing a new
*non-invertible* stringify function. In particular, numbers are hashed by
value, with 0 == 0L == 0.0 => hash(0) == hash(0L) == hash(0.0), whereas
these would give three different strings with either str() or repr().
Example:

d={1:'one'}
d[1.0]

'one'

So, you make newstr((1,2)) == newstr((1.0,2L)). Which means you must
eliminate the number type information and make newstr *not* invertible! Or
you could change the dict key rules;=).

Terry J. Reedy

 

[Rainer Deyke]

Actually, this is the classic problem of data-escaping. It's
*possible* to guarantee unique values when encoding as a string, but
it's a *lot* harder than just doing str( data ).

In Python it's one extra line of code.

from pickle import dumps
dumps(data)

 

[Mike C. Fletcher]

Mike C. Fletcher wrote:



In Python it's one extra line of code.

from pickle import dumps
dumps(data)

Sure, even used that to create the examples , but it instantiates and
invokes a huge class (~500 lines of Python code) built using multiple
other dictionaries, each of which would need to be serviced themselves
without getting into weird recursion problems. Pickling machinery is
*really heavy* (i.e. a lot harder for the computer to do, and a lot more
hairy to program) than a few structure lookups to find functions
computing hashes and/or equalities and the inclusion of a static
sequence type.

Peace and lava lamps,
Mike

_______________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/

 

[Paul Prescod]

...

Now, if you change the implementation so that Foo is
no longer a class, code which subclasses Foo will break --
there's no avoiding that. But code which only instantiates,
using foo(), will keep working.

Okay, but is this ever a serious problem? Is changing classes into
functions and vice versa actually a common enough problem to worry about?

Paul Prescod

 

[Bob Ippolito]

Did you do some time profiling? I would be interested in the results.



This is an excellent example!
count = lst.append(3).sort().reverse().len()
is extremly readable but chaining of method calls is not supported in
Python. And I can live with it... Python has a more "expressive" path:

You can 'chain method calls' all you want.. what you're missing here is
a python idiom: methods that mutate an object shall not return the
same object.

If the above code were to actually work, then "lst" would contain an
extra element and be stored in reverse sorted order, while "count"
would be the number you were trying to extract. The intent probably
was not to modify "lst", so this particular idiom just saved you the
time of debugging your broken code by encouraging you to do it the
right way the first time.

-bob