Python's biggest compromises

[Anthony_Barker]

I have been reading a book about the evolution of the Basic
programming language. The author states that Basic - particularly
Microsoft's version is full of compromises which crept in along the
language's 30+ year evolution.

What to you think python largest compromises are?

The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled. These three put
python under fire and cause some large projects to move off python or
relegate it to prototyping.

Whitespace is an esthetic preference that make Andrew Hunt and David
Thomas (of Pragmatic Programmer fame) prefer Ruby. Personally, I love
it - but I can see why some people might not like it (30 years of
braces).

Dynamic typing causes the most fuss. I like Guido's answer to the
question -

"Doesn't dynamic typing cause more errors to creep into the code

because you catch them later than compile time?".

"No, we use Unit Testing in Zope".

That said, obvious Basic compromised by using things such as "Option
Explicit", thereby allowing both dynamic and more static style
variables. Yahoo groups moved from python to C due to dynamic typing.

Non-compiled - obviously there are times when performance matters more
than other things. Google I believe uses python to prototype (or used)
and then turns to c++ for heavy lifting.

What about immutable strings? I'm not sure I understand Guido's
preference for them.

Anthony
http://xminc.com/anthony

 

[Christopher Koppler]

I have been reading a book about the evolution of the Basic
programming language. The author states that Basic - particularly
Microsoft's version is full of compromises which crept in along the
language's 30+ year evolution.

What to you think python largest compromises are?

[snip whitespace, dynamic typing, interpreted]

I don't see those as compromises, but mostly as assets.

Significant whitespace (you probably mean significant indentation -
whitespace isn't more or less significant in Python than in other
modern languages) I have only experienced as a boost in readability,
clarity and, most of all, consistence; and there's no possibility of
'brace style wars'.

Dynamic typing vs. static typing has already long ago reached the
status of holy war, so I'll decline to comment.

That python is not (yet) compiled, is mostly a non-issue (and if PyPy
is a success, it won't even be that). If it was just about
performance, then coding the really performance-intensive parts in C
should suffice, apart from kernel hacking and similar . In my
experience, the decision to convert a (successfully functioning)
project from 'a scripting language' to C/C++/Java has always been a
political one, and not really based on technical considerations.

That said, the only large compromise in Python language design I can
detect, is the decision to be quite strictly backwards-compatible
between versions, which is definitely not a bad thing, as long as the
language doesn't go baroque because of it. And Python 3.0 will
hopefully throw out any accumulated cruft.


--Christopher

 

[Edward K. Ream]

What to you think python largest compromises are?

There aren't any.

You want to base significant projects on the highest level, most dynamic
tools available (Python), then use Python as a wrapper to hide static
inflexibilities and inferiorities when descending to lower levels for
whatever (usually spurious) reasons. For example, there is a huge
difference between using wxWindows and wxPython, but the performance
difference between wxWindows and wxPython is insignificant.

Edward

 

[Bruno Desthuilliers]

Anthony_Barker wrote:
(snip)

What to you think python largest compromises are?

The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled.

IMHO these are not compromises, but features.

Bruno

 

[Robin Becker]

......
High performance isn't Python's target. If PyPy ever gets their act
off the ground, then we will have a shot at a good quality JIT
interpreter. Then watch it fly.

..... doesn't psyco already attempt to do JIT? It certainly doesn't
speed things up that much. If all the variables are known to be of a
specific type then you could expect C like speeds from a good JIT, but
without global analysis it's hard to see how we infer/guarantee python
types.

 

[John J. Lee]

That said, I've come to the conclusion that the editor should take
care of these things for you. If you prefer a brace free notation,
you should be able to tell your editor to present the program to you
that way. If you prefer braces, then it should be able do that for
you as well. That kind of stylistic thing doesn't belong in the
language.

100% agreed: once-and-only-once dictates this. Manually maintaining
both braces and indentation (as in C, with some editors) is Just Plain
Bad for this reason.

In fact, if I didn't have to deal with the braces, I think I'd come
around to the view that the text should have them. Explicit is
better than implicit,

At least in the absence of proper research results, I think this part
of it *is* religious. To some people, it's obvious that whitespace is
better on the eyes. To others, it's obvious that whitespace + braces
is better on the eyes. One group may well be wrong <0.5 wink>.

This argument (that braces are another visual cue which make code
easier to read -- provided that they're automatically maintained in
sync with the indentation), is the only argument against
pure-whitespace that has ever made any sense to me. Irrespective of
whether you pick whitespace or braces as the thing the compiler takes
notice of, though, the optimal solution probably still involves an
editor that supports showing/hiding braces, so the syntax is
(theoretically!) a non-issue. In practice, editors don't seem to
currently support showing and hiding braces automatically. At least
editors do get you out of most mistakes caused by brace-y languages.

Of course, whitespace *is* still superior because the storage format
doesn't duplicate state -- so people with poor editors can't produce
ambiguous code (remember code can be ambiguous to *people* even if
it isn't to a compiler). OTOH, *Python's* scheme is inferior to a
pure-space-character indentation scheme because off the tab-vs.-space
issue :-(

and there are a few things that the
automatic indentation makes rather difficult in the design area.

[...]

What are those things??


John

 

[Mark VandeWettering]

I have been reading a book about the evolution of the Basic
programming language. The author states that Basic - particularly
Microsoft's version is full of compromises which crept in along the
language's 30+ year evolution.

What to you think python largest compromises are?

The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled. These three put
python under fire and cause some large projects to move off python or
relegate it to prototyping.

I don't view any of these as "compromises". That word suggests that
something was conceded, or that an intermediate position between two
extremes was chosen to appease. I don't think that either sense really
applies to these features.

The three items that you listed are merely design choices. While arguments
over them are continuous, two of the design choices (interpreter, dynamic
typing) are consistent with Python's intended use as a language which
excels at rapid prototyping. The third (white space) is merely a stylistic
choice which is designed to encourage readable programs.

"Compromises" in language design occur usually when a committee tries to
standardize a language, and each has differing views about how the language
should be used. While this occurs somewhat in Python, other languages
have suffered more mightily from this particular disorder.

Mark

 

[Aaron Leung]

It seems to me that a big compromise/feature is that all kinds of
namespaces are usually represented by dictionaries, and that Python
exposes this fact to the programmer. This would seem to limit the
possible optimizations that can easily be performed by a compiler.

BTW, I have only read about Python out of interest, and haven't
actually used it for anything, so I hope my remark isn't ignorant.

Best regards,
Aaron

 

[Skip Montanaro]

Anthony> What to you think python largest compromises are?

Anthony> The three that come to my mind are significant whitespace,
Anthony> dynamic typing, and that it is interpreted - not compiled.
Anthony> These three put python under fire and cause some large projects
Anthony> to move off python or relegate it to prototyping.

Your message is sure to get the pot boiling. I don't think of any of the
above as compromises. They were all design decisions. Considering the
whitespace issue, calling it a compromise suggests that Guido had to cave in
to some outside forces. He couldn't decide between BEGIN/END or {/} as
block delimiters, so he chose significant whitespace. It doesn't make
sense.

Anthony> What about immutable strings? I'm not sure I understand Guido's
Anthony> preference for them.

Performance is one reason. The Python interpreter creates a huge number of
strings at runtime. Knowing exactly how long the string is going to be and
that it will not grow means that a single malloc can be used to allocate the
object header and the storage for the data. If strings were mutable, the
structure of the string object storage would probably be much different and
you'd need at minimum two mallocs per string, one for the object header and
one for the data itself.

Skip

 

[Ian Bicking]

The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled. These three put
python under fire and cause some large projects to move off python or
relegate it to prototyping.

I think these are valid as "compromises", not to be confused with
flaws. These are all explicit choices, and ones for which the original
justifications remain valid. A lot of stuff in Basic is simply flaws,
or based on justifications that no longer apply to today's programming
world.


Anyway, I might add mine: the nature of modules as executed code is a
compromise. That is, a module isn't a declaration, it's a program to be
executed in its own namespace. When you import a module, you are
executing the module then looking at the namespace.

There are some advantages to this, particularly in the transparency of
the implementation -- things don't always work the way you might want
(e.g., circular imports), but it's usually not that hard to understand
why (and often the way you want things to work has nasty gotchas that
you wouldn't have thought of). It also opens up a lot of possibilities
for dynamicism in class and function declaration, like doing this in the
top level:

if something:
def func(x): ...
else:
def func(x): ...

But it's a compromise, because it makes things more difficult as well.
It's a *big* problem in any long-running process, where you may want to
modify code underlying the system without rebuilding the entire state.
Classes aren't declared, they are simply constructed, so by reloading a
module all the persistent instances still exist and refer to the defunct
class. You can modify classes at runtime, but this is different from
simply rerunning the class definition. (A clever metaclass *could* make
those equivalent, though... hmmm...)

A related problem is that Python objects generally can accept any
instance variable names, and names are not declared anywhere. Again,
this makes it difficult to deal with long-lived objects. If you change
the class so that all instances get a new attribute, old objects won't
be updated. I'm thinking about both of these things in terms of
Smalltalk, where they make tools possible that really add to the ease of
developing in its environment.

Not that I don't like the fun tricks Python lets you do. Prototype-like
programming (as in Self) is very accessible in Python, and classes are
only a suggestion not a dominant concept. So, it's a compromise.

There are lots and lots of compromises in Python -- every aspect has
pluses and minuses to it. Personally I like whitespace sensitivity well
enough, but in the larger sense I think it probably was the wrong choice
-- but that's based on how I weigh various benefits and problems, and
other people will validly weigh them differently.

Ian

 

[Terry Reedy]

What to you think python largest compromises are?

A compromise is an in-between position or decision. Example: wife
wants to go to a horse show, husband to an auto race, so they
compromise and go to a horse race.

The three that come to my mind are significant whitespace, dynamic
typing, and that it is interpreted - not compiled.

The first two are end-point positions, not in-between compromises.
The third is a matter of definition and implementation. CPython
compiles to version-dependent but otherwise portable PyCode. PyRex,
Weave, and Psyco all compile to C or machine code.

These three put python under fire

Anything can bo put under fire by anyone who wants to shoot.

and cause some large projects to move off python or

This sort of statement remains an opinion or impression until backed
by evidence.

relegate it to prototyping.

This is one of its intended uses.

Whitespace is an esthetic preference that make Andrew Hunt and David
Thomas (of Pragmatic Programmer fame) prefer Ruby.

Evidence? And I mean evidence that whitespace is *the* reason and not
just a convenient summary of an overall esthetic preference.

In any case, so what? Different strokes for different folks. Do they
also use indentation for human readers? If so, they have assigned
themselves the task of keeping brackets and indents in sync so that
human and machine 'see' the same structure.

I see two good uses for brackets:
1. machine-generated code never intended for human eyes
2. redundancy for transmission error detection by a processor that
compares brackets and indents and raises a flag on mismatches.

A compromise in the area of structure indication would be accepting
either brackets or indents or both.

Yahoo groups moved from python to C due to dynamic typing.

Evidence? Evidence as to what exactly happened (it is not common
knowledge that I know of) and that any such change was a reasoned
technical decision and not politics.

If memory serves me right, Guido has tried a couple of compromises to
slightly limit dynamicity that he could not see much use for and has
backed off at least partly when current users presented use cases that
the change would break.

Non-compiled - obviously there are times when performance matters more
than other things. Google I believe uses python to prototype (or used)
and then turns to c++ for heavy lifting.

This is a simple matter of economic tradeoff. A week of programmer
time costs roughly the same as, say, a year of pc time. A roaring
success like Google has hundreds (thousands?) of servers around the
world running the same relatively stable code. Faster code means
machines not bought, installed, and maintained. But I imagine that
their main production code is pretty far out on the frequency-of-use
curve.

Terry J. Reedy

 

[John Roth]

.... doesn't psyco already attempt to do JIT? It certainly doesn't
speed things up that much. If all the variables are known to be of a
specific type then you could expect C like speeds from a good JIT, but
without global analysis it's hard to see how we infer/guarantee python
types.

Well, that's certainly a problem, but Bicycle Repair Man seems to
do a pretty good job of type inference, at least for refactoring.

One of the things to consider here is that a decent JIT interpreter
would automatically change the playing field for what is good
practice and what isn't, at least if you expect performance.

John Roth

 

[Ian Bicking]

Well, that's certainly a problem, but Bicycle Repair Man seems to
do a pretty good job of type inference, at least for refactoring.

And Java's JIT is based on (at least originally) work done on Self,
which had to do type inference. And actually in many circumstances Java
requires type inference, because you can substitute in an instance of a
subclass.

Anyway, JIT is all about runtime analysis -- if you could infer types
completely before running the program, you would just put in the
optimizations statically (i.e., compiling optimizations). JIT does
those optimizations at runtime by definition.

And Bicycle Repair Man is inspired by the Refactoring Browser, an IDE
tool based on another dynamic language (Smalltalk), not on a tool from a
static language (like Java).

Ian

 

[=?ISO-8859-1?Q?Hannu_Kankaanp=E4=E4?=]

What to you think python largest compromises are?

I think reference counting is. Added to this the fact that
garbage collection is also possible (as in Jython). So we get
the worst of both worlds.

1. Idioms like this won't work portably:

def foo():
f = file('x')
return f.read()

Even though f.__del__ closes the file, in garbage
collected environment the f object might not get deleted
in a good while, and would cause problems.

2. And then, we still need to use weakref to ensure
that our crossreferenced stuff works both with and without GC.

Worst of both indeed. Maybe the decision to choose reference
counting was driven by speed considerations. That might've
been reasonable back in early 90's, but GC techniques have
evolved from those days and so GC would be a superior
technique now.

Well, since no one else pointed this out yet, maybe there's
some flaw in my reasoning.

 

[Andrew Dalke]

Dennis Lee Bieber:

Whereas BASIC started life as an interpreted language wherein every
statement had a line number, conditionals (IF) did jumps to line
numbers, and all variables were global (even across subroutine calls).

Not interpreted.
See http://wombat.doc.ic.ac.uk/foldoc/foldoc.cgi?Dartmouth+BASIC
] Dartmouth BASIC
] <language> The original BASIC language [...] Unlike most later
] BASIC dialects, Dartmouth BASIC was compiled

or a more detailed description at http://www.kbasic.org/1/history.php3
which says the first interpreted BASIC was the Micro-Soft one for
the Altair.

In more seriousness, compiled v. interpreted is an implementation
detail which has very little impact on the language. The other examples
you gave are more relevant.

One thing to consider is - what's the *compromise* in the different
versions of BASIC? That is, how was the support for backwards-
compatible operations in BASIC (which you list as a compromise)
any different than Python's backwards compatibility support?

Andrew
(e-mail address removed)

 

[Gerald Klix]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Anthony_Barker wrote:
| I have been reading a book about the evolution of the Basic
| programming language. The author states that Basic - particularly
| Microsoft's version is full of compromises which crept in along the
| language's 30+ year evolution.
|
| What to you think python largest compromises are?
|
IHMO it is the lambda expression.
These "functions" are not real functions. You can not use
statements in them.

What Python realy needs here is some means to make an expression
from a list of statements (called suite in the syntax defintion).
That given PEP 308 (If-then-else expression) or PEP 318 (Function/Method
Decorator Syntax) are mostly pointless.

Let me give an example:

Suppose there some special braces like and a exit operator like a
unary ^ one can write a conditional expession like that:

~ if f():
~ ^trueValue
~ else:
~ ^falseValue

classmethods can be declared as follows:
cm =
~ def im( arg0, arg1 ):
~ return answer
~ ^classmethod( im )

or
cm = classmethod(
~ def im( arg0, arg1 ):
~ return answer
~ ^im
~ )

obvously this demands for some means to write anonymous functions like

def ( arg0, arg1 ):
~ return arg0 - arg1

semanticly this should transform to


~ def newName( arg0, arg1 ):
~ return arg0 - arg1
~ ^newName

giving

cm = def ( arg0, arg1 ):
~ return answer

Ok, I admit that this is difficult to be integrated in
the existing syntax. Perhaps we can not drop the braces
around such expression.

Is this worth writing a PEP?

|
|>"No, we use Unit Testing in Zope".
I am still missing a simple testing framework. doctest
is a good idea, but conflicts with syntax hilighting in most
editors.

|
|
| That said, obvious Basic compromised by using things such as "Option
| Explicit", thereby allowing both dynamic and more static style
| variables. Yahoo groups moved from python to C due to dynamic typing.
This is not a problem. They key to speed is using extension written
in C for performance. Normaly you will find one that sloves your problem.

|
| Non-compiled - obviously there are times when performance matters more
| than other things. Google I believe uses python to prototype (or used)
| and then turns to c++ for heavy lifting.
|
| What about immutable strings? I'm not sure I understand Guido's
| preference for them.
In fact the array module provides mutable strings.

|
| Anthony
| http://xminc.com/anthony

HTH,
Gerald
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Debian - http://enigmail.mozdev.org

iD8DBQE/KjWgEDg9cqFA1jQRAovzAJkBygPzdfHsoVXu9H2QHnxTD9sMmACdHvY5
dB1j4kXTzejml7fG0oSwhUg=
=WGR+
-----END PGP SIGNATURE-----

 

[Robin Becker]

And Java's JIT is based on (at least originally) work done on Self,
which had to do type inference. And actually in many circumstances Java
requires type inference, because you can substitute in an instance of a
subclass.

Anyway, JIT is all about runtime analysis -- if you could infer types
completely before running the program, you would just put in the
optimizations statically (i.e., compiling optimizations). JIT does
those optimizations at runtime by definition.

but Java does at least require specifying every type and that must at
least cut down on the amount of work required.

And Bicycle Repair Man is inspired by the Refactoring Browser, an IDE
tool based on another dynamic language (Smalltalk), not on a tool from a
static language (like Java).

Ian

I don't have any data here, but I believe Python is just a little too
weakly typed for compiling to float*float type assembler efficiently.

 

[John Roth]

but Java does at least require specifying every type and that must at
least cut down on the amount of work required.


I don't have any data here, but I believe Python is just a little too
weakly typed for compiling to float*float type assembler efficiently.

The trick with JITs is that they don't depend on absolute type
consistency. They depend on the observation that 99.44% of your
code is type consistent, and that consistency will turn up at run time. So
the code they generate depends on that discovered consistency, and
checks in front of each section to discover if the types are what the
code expects.

If it is, they execute it, if it isn't, they abandon it and go back to
the intepreter to discover what happened.

John Roth

 

[Michael Hudson]

Worst of both indeed. Maybe the decision to choose reference
counting was driven by speed considerations.

Ease of implementation, portability and playing nicely with C
extensions are more likely candidates, IMO.

That might've been reasonable back in early 90's, but GC techniques
have evolved from those days and so GC would be a superior technique
now.

<button nature="hot">
Reference counting *is* a form of garbage collection.
</button>

Saying "Ref. counting sucks, let's use GC instead" is a statement near
as dammit to meaningless.

Given the desires above, I really cannot think of a clearly better GC
strategy for Python that the one currently employed. AFAICS, the
current scheme's biggest drawback is its memory overhead, followed by
the cache-trashing tendencies of decrefs.

What would you use instead?

Cheers,
mwh

 

[John Roth]

(e-mail address removed) (Anthony_Barker) wrote in message

I think reference counting is. Added to this the fact that
garbage collection is also possible (as in Jython). So we get
the worst of both worlds.

1. Idioms like this won't work portably:

def foo():
f = file('x')
return f.read()

Even though f.__del__ closes the file, in garbage
collected environment the f object might not get deleted
in a good while, and would cause problems.

2. And then, we still need to use weakref to ensure
that our crossreferenced stuff works both with and without GC.

Worst of both indeed. Maybe the decision to choose reference
counting was driven by speed considerations. That might've
been reasonable back in early 90's, but GC techniques have
evolved from those days and so GC would be a superior
technique now.

Well, since no one else pointed this out yet, maybe there's
some flaw in my reasoning.

There's a flaw in your reasoning. The various techniques that
descend from mark and sweep (which is what you're
calling garbage collection) depend on being able to
identify all of the objects pointed to. For objects that are
owned by Python, that's a lengthy (that is, inefficient)
process, and it's not possible in general for objects that
are created by extensions.

Reference counting only depends on having the
object itself, and control of the creation and removal
of references. The latter is a frequent source of bugs
and memory leaks in extensions.

It's easy to say that various languages would be improved
by adding "real" garbage collection, but those techniques
impose significant design constraints on the implementation
model.

John Roth