Language design

[Oscar Benjamin]

More likely, JP Morgan's mail system added that footer to the message
on the way out the virtual door. My recommendation would be to not
post using your company email address. Get a free email address.

It wouldn't surprise me if JPMorgan would block free email addresses on site.


Oscar

 

[Nobody]

and most routines that handle file names accept either text strings or
bytes strings:

I was going to say "that just leaves environ and argv". But I see that
os.environb was added in 3.2. Which just leaves argv.

 

[Markus Rother]

() == [] False

But:

bool(().__eq__([]))

True

This is not a trap, this is simply the wrong way to do it. The magic
methods (aka dunder methods) are there for Python to call, not you
(except under special circumstances, such as when writing your own
dunder methods).

While trying to do it, I learned that its not the right way to do it.
However, I was not satisfied with the fact, that there is no built in
pure function for operations on primitives. Such that
.... def do_stuff(x,y):
.... return fn(x,y)
.... return do_stuff

I understand that python is not a functional language, but it
frustrates me at times.

Markus

 

[Ethan Furman]

() == []
False

But:

bool(().__eq__([]))
True

This is not a trap, this is simply the wrong way to do it. The magic
methods (aka dunder methods) are there for Python to call, not you
(except under special circumstances, such as when writing your own
dunder methods).

While trying to do it, I learned that its not the right way to do it.
However, I was not satisfied with the fact, that there is no built in
pure function for operations on primitives. Such that
... def do_stuff(x,y):
... return fn(x,y)
... return do_stuff

I understand that python is not a functional language, but it
frustrates me at times.

--> import operator
--> operator.__all__ # public api methods are in __all__
['abs', 'add', 'and_', 'attrgetter', 'concat', 'contains', 'countOf', 'delitem', 'eq', 'floordiv', 'ge', 'getitem',
'gt', 'iadd', 'iand', 'iconcat', 'ifloordiv', 'ilshift', 'imod', 'imul', 'index', 'indexOf', 'inv', 'invert', 'ior',
'ipow', 'irshift', 'is_', 'is_not', 'isub', 'itemgetter', 'itruediv', 'ixor', 'le', 'length_hint', 'lshift', 'lt',
'methodcaller', 'mod', 'mul', 'ne', 'neg', 'not_', 'or_', 'pos', 'pow', 'rshift', 'setitem', 'sub', 'truediv', 'truth',
'xor']

Imports are a Good Thing; not everything has to be built in.

 

[Neil Cerutti]

() == []
False

But:

bool(().__eq__([]))
True

This is not a trap, this is simply the wrong way to do it. The magic
methods (aka dunder methods) are there for Python to call, not you
(except under special circumstances, such as when writing your own
dunder methods).

While trying to do it, I learned that its not the right way to do it.
However, I was not satisfied with the fact, that there is no built in
pure function for operations on primitives. Such that
... def do_stuff(x,y):
... return fn(x,y)
... return do_stuff

I understand that python is not a functional language, but it
frustrates me at times.

True

 

[Markus Rother]

3. The default return value of methods is None instead of self.
If it was self, it would be possible to chain method calls (which
is called a cascade in smalltalk).

lst = []
lst.append(1).append(2).append(3) ## FAIL

Traceback (most recent call last):
...
AttributeError: 'NoneType' object has no attribute 'append'

That's a policy decision: a method (or function) will *EITHER* return
a value, *OR* mutate its primary argument (in the case of a method,
that's self).

You are stating: "All getters must be free of side effects".
That is not the case. Furthermore, the demand for getters with hidden
side effects is the reasoning behind properties.

The policy could as well be: a method (not a function) will either
return a value, or return self, whether or not the object was mutated.

Why should that be split into two statements? Or alternatively, why
should an extra copy of the list be created (if you use Python's
sorted() here)? But for the new programmer, this is a convenient
safety-net, and if list.sort() worked the other way, it'd be just as
much a gotcha ("I ask for a sorted list, and it also changed the
original?!??").

I understand the point you are making in the end, in the interest of
having an easy to start with language.

Markus

 

[Markus Rother]

What design mistakes, traps or gotchas do you think Python has? Gotchas
are not necessarily a bad thing, there may be good reasons for it, but
they're surprising.

I have one more:

Dictionaries should iterate over their items instead of their keys.

Looking forward to contrary opinions.

Greets,
Markus

 

[Ned Batchelder]

I have one more:

Dictionaries should iterate over their items instead of their keys.

I understand the natural desire for this, and I sometimes forget and
have to add a ".items()" to my loops. But if you consider how "in"
should work with dicts, it's only reasonable that "k in d" examine if a
key is in a dict, not if an item is in a dict. And once you have "in"
working like that, it's reasonable for iteration to work on keys.

Dicts act like collections of keys, each with an associated value.

--Ned.

 

[Terry Reedy]

Dictionaries should iterate over their items instead of their keys.

Dictionaries *can* iterate by keys, values, or items. You would prefer
that the default iteration be by items rather than keys.

Looking forward to contrary opinions.

When the issue was discussed and decided, a majority of developers
preferred keys. A large factor was experience and expectation that
iterating by keys is more common than than iterating by items, coupled
with the idea that the most common usage should be the default.

 

[Neil Cerutti]

I have one more:

Dictionaries should iterate over their items instead of their keys.

Looking forward to contrary opinions.

Consider the 'in' operator.

 

[Grant Edwards]

No, your message was not confidential. You sent it to a public
mailing list, presumably by choice. So please omit such false and
pointless legal rubbish from your messages here.

OTOH, perhaps it was confidential. In which case, J. P. Morgan might
want to know that ramit.prasad is sending confidential information to
a pulic mailing list.

 

[Chris Angelico]

You are stating: "All getters must be free of side effects".
That is not the case. Furthermore, the demand for getters with hidden
side effects is the reasoning behind properties.

This isn't a language feature here, just a stdlib policy. It's more
akin to Ruby's habit of adorning the mutating methods with an
exclamation mark - it's a way of stopping you from accidentally doing
what you didn't mean to do. There's a sharp distinction between
list.sort(), which mutates in place and returns None, and sorted(),
which doesn't touch its argument and returns a new list.

ChrisA

 

[Mark Janssen]

Really? Are you saying you (and the community at-large) always derive

Not directly, that would be silly.

Silly? "Explicit is better than implicit"... right?

Depends on whether I'm standing on my head or not.

Or more importantly, it depends on whether I visualise my hierarchy going
top->down or bottom->up. Both are relevant, and both end up with the
*exact same hierarchy* with only the direction reversed.

Ha, "only the direction reversed". That little directionality that
you're passing by so blithely is the difference between whether you're
talking about galaxies or atoms. Please.

The simplicity of Python has seduced you into making an "equivocation"
of sorts. It's subtle and no one in the field has noticed it. It
crept in slowly and imperceptively.

No you don't. You get a bunch of ill-defined methods that don't make
sense on dicts.

They are not necessarily ill-defined. Keep in mind Python already
chose (way back in 1.x) to arbitrary overwrite the values in a key
collision. So this problem isn't new. You've simply adapted to this
limitation without knowing what you were missing.

Right. The dict literal should be {:} -- the one obvious way to do it.

I don't agree it is obvious. It is as obvious as (,) being the empty tuple
or [,] being the empty list.

You're just being argumentative. If there are sets as built-ins, then
{:} is the obvious dict literal, because {} is the obvious one for
set. You don't need [,] to be the list literal because there is no
simpler list-type.

What does this have to do with Python 3? It works fine in Python 2.

I mean, you're suggestions are coming from a "believer", not someone
wanting to understand the limitations of python or whether v3 has
succeeded at achieving its potential.

I don't even understand what you are talking about here. "[reference]
variables"? What does that mean?

It's a just a tricky point, that I will wait to comment on.

I'm looking forward to an explanation, as I'm intrigued.

Well, wer'e here at junior-high. It will take some time....

 

[Steven D'Aprano]

Silly? "Explicit is better than implicit"... right?

If I'm inheriting from str, I inherit from str explicitly:

class MyStr(str): ...

and then str in turn inherits from object explicitly. I certainly do not
inherit from object and then re-implement all the string methods from
scratch:

class MyStr(object):
def __new__(cls, value): ...
def upper(self): ...
def lower(self): ...
# and so on...

That would be ridiculous, and goes against the very idea of inheritance.
But nor do I feel the need to explicitly list the entire superclass
hierarchy:

class MyStr(str, object):
...


which would be silly. Only somebody who doesn't understand how
inheritance works in Python would do that. There's simply no need for it,
and in fact it would be actively harmful for larger hierarchies.

Ha, "only the direction reversed". That little directionality that
you're passing by so blithely is the difference between whether you're
talking about galaxies or atoms.

It makes no difference whether I write:

atoms -> stars -> galaxies

or

galaxies <- stars <- atoms

nor does it make any difference if I write the chain starting at the top
and pointing down, or at the bottom and pointing up.

Your objection implies that writing family trees with the most distant
ancestor (the root of the tree) at the top of the page somehow confuses
people into thinking that perhaps they are the progenitor of people who
lived generations earlier. That's absurd -- people simply are not as
stupid as you think.

The simplicity of Python has seduced you into making an "equivocation"
of sorts. It's subtle and no one in the field has noticed it. It crept
in slowly and imperceptively.

Ah, and now we come to the heart of the matter -- people have been
drawing tree-structures with the root at the top of the page for
centuries, and Mark Janssen is the first person to have realised that
they've got it all backwards.

They are not necessarily ill-defined. Keep in mind Python already chose
(way back in 1.x) to arbitrary overwrite the values in a key collision.
So this problem isn't new. You've simply adapted to this limitation
without knowing what you were missing.

No, Python didn't "arbitrarily" choose this behaviour. It is standard,
normal behaviour for a key-value mapping, and it is the standard
behaviour because it is the only behaviour that makes sense for a general
purpose mapping.

Python did not invent dicts (although it may have invented the choice of
name "dict").

If you think of inheritance in the Liskov Substitution sense, then you
might *consider* building dicts on top of sets. But it doesn't really
work, because there is no way to sensibly keep set-behaviour for dicts.

For example, take intersection of two sets s and t. It is a basic
principle of set intersection that s&t == t&s.

But now, take two dicts with the same keys but different values, d and e.
What values should be used when calculating d&e compared to e&d? Since
they are different values, we can either:

* prefer the values from the left argument over that from the right;
* prefer the values from the right argument over that from the left;
* refuse to choose and raise an exception;
* consider the intersection empty

The first three choices will break the Liskov Substitution Principle,
since now dicts *cannot* be substituted for sets. The fourth also breaks
Liskov, but for a different reason:

# sets
(key in s) and (key in t) implies (key in s&t);

but

# dicts
(key in d) and (key in e) *does not* imply (key in d&e)


So whatever you do, you are screwed Liskov-wise. You cannot derive dicts
from sets and still keep the Liskov Substitution Principle.

Of course, LSP is not the only way to design your inheritance
hierarchies. An alternative is to design them in terms of delegating
implementation to the superclass. As Raymond Hettinger puts it, your
classes tells it's parent to do some of the work. But in this case,
you're still screwed: you can derive sets from dicts, and in fact the
first implementation of sets in Python did exactly that, but you cannot
derive dicts from sets.

So either way, whether you are an OOP purist who designs your classes
with type-theoretic purity and the Liskov Substitution Principle in mind,
or a pragmatist who designs your classes with delegation of
implementation in mind, you can't sensibly derive dicts from sets.

3) It used the set literal for dict, so that there's no obvious way
to do it. This didn't get changed in Py3k.

No, it uses the dict literal for dicts.

Right. The dict literal should be {:} -- the one obvious way to do
it.

I don't agree it is obvious. It is as obvious as (,) being the empty
tuple or [,] being the empty list.

You're just being argumentative. If there are sets as built-ins, then
{:} is the obvious dict literal, because {} is the obvious one for set.
You don't need [,] to be the list literal because there is no simpler
list-type.

The point is that the obvious way to write an empty collection is using a
pair of delimiters, not to shove an arbitrary separator separating
nothing at all in there:

[] is an empty list, not [,]
() is an empty tuple, not (,)
{} is an empty (dict|set), not {,} or {:}


We can't have {} be both an empty set and an empty dict, so one of the
two has to miss out. Neither is more obvious than the other. dicts are
more important data structures, and they have historical precedence, so
they win. If Python was being re-invented from scratch now, or if sets
had been around just as long as dicts, people might have chosen to given
sets higher priority than dicts, but frankly I doubt it. It's much more
common to want an empty dict than an empty set, at least in my experience.

I mean, you're suggestions are coming from a "believer", not someone
wanting to understand the limitations of python or whether v3 has
succeeded at achieving its potential.

"not someone wanting to understand the limitations of python..." -- are
you aware that I started this thread?

 

[Antoon Pardon]

Op 10-09-13 12:20, Chris Angelico schreef:

Significant indentation. It gets someone every day, it seems.

Not only that. There are a lot of python code snippets on the net
that for whatever reason lost their indentation. There is no
algorithm that can restore the lost structure.

 

[Chris Angelico]

For example, take intersection of two sets s and t. It is a basic
principle of set intersection that s&t == t&s.

Note that, while this is true, the two are not actually identical:
True

I'd actually posit that Python has this particular one backward (I'd
be more inclined to keep the left operand's value), but it's
completely insignificant to most usage. But in any case, there's
already the possibility that a set union can be forced to make a
choice between two equal objects, so we're already a bit beyond the
purity of mathematics. Python could have implemented dicts much more
like "sets with values", with set semantics maintained throughout, but
it'd require some oddities:
False

"Sets with values" semantics would demand that these both be True,
which is grossly unintuitive.

So while it may be true in pure mathematics that a set is-a dict (or a
dict is-a set), it's bound to create at least as many gotchas as it
solves.

ChrisA

 

[Steven D'Aprano]

Op 10-09-13 12:20, Chris Angelico schreef:
Not only that. There are a lot of python code snippets on the net that
for whatever reason lost their indentation. There is no algorithm that
can restore the lost structure.

Is there an algorithm that will restore the lost structure if you delete
all the braces from C source code?

Perhaps if web sites and mail clients routinely deleted braces, we'd see
the broken-by-design software being fixed instead of blaming the language.

 

[Chris Angelico]

Is there an algorithm that will restore the lost structure if you delete
all the braces from C source code?

Perhaps if web sites and mail clients routinely deleted braces, we'd see
the broken-by-design software being fixed instead of blaming the language.

While I don't deny your statement, I'd like to point out that English
usually isn't overly concerned with formatting. You can take this
paragraph of text, unwrap it, and then reflow it to any width you
like, without materially changing my points. C follows a rule of
English which Python breaks, ergo software designed to cope only with
English can better cope with C code than with Python code. Removing
all braces would be like removing all punctuation - very like, in fact
- a very real change to the content, and destruction of important
information. Python is extremely unusual in making indentation
important information, thus running afoul of systems that aren't meant
for any code.

But if you look at the quoted text above, I specifically retained your
declaration that "Gotchas are not necessarily a bad thing" when citing
significant indentation. I'm not here to argue that Python made the
wrong choice; I'm only arguing that it frequently confuses people.

ChrisA

 

[Antoon Pardon]

Op 13-09-13 12:13, Steven D'Aprano schreef:

Is there an algorithm that will restore the lost structure if you delete
all the braces from C source code?

Yes, almost. Just look at the indentation of the program and you will
probably be able to restore the braces in 99% of the programs.

Perhaps if web sites and mail clients routinely deleted braces, we'd see
the broken-by-design software being fixed instead of blaming the language.

The world is not perfect. If products in your design are hard to repair
after some kind of hiccup, then I think the design can be blamed for
that. Good design is more than being ok when nothing goes wrong. Good
design is also about being recoverable when things do go wrong.

 

[Terry Reedy]

I believe tabs are worse than spaces with respect to getting lost.

While I don't deny your statement, I'd like to point out that English
usually isn't overly concerned with formatting.

Poetry, including that in English, often *is* concerned with formatting.
Code is more like poetry than prose.

You can take this
paragraph of text, unwrap it, and then reflow it to any width you
like, without materially changing my points.

But you cannot do that with poetry! Or mathematical formulas. Or tables.
Or text with headers and paragraphs and indented quotations. Etc. What
percentage of published books on your bookshelf have NO significant
indentation? As far as I know for mine, it is 0.

C follows a rule of English

which you just made up, and which is drastically wrong,

which Python breaks,
ergo software designed to cope only with English

impoverished plain unformatted prose

can better cope with C code than with Python code.

Software that removes formatting info is broken for English as well as
Python.

Python is extremely unusual in making indentation
important information

You have it backwards. Significant indentation is *normal* in English. C
in unusual is being able to write a whole text on a single line.

When I was a child, paragraphs were marked by tab indents. The change to
new-fangled double spacing with no indent seems to have come along with
computer text processing. Perhaps this is because software is more prone
to dropping tabs that return characters.