Python evolution: Unease

[Iwan van der Kleyn]

Please ignore if you are allergic to ramblings

Despite a puritan streak I've always tried to refrain from language wars
or syntax bickering; call it enforced pragmatism. That's the main reason
why I've liked Python: it's elegant and simple and still dynamic and
flexible. You could do worse for a clean and pragmatic language.

I do know my Smaltalk from my Common Lisp and my Ruby from my C#, so I
think I'm quite capable of escaping the "Blub paradox"
http://c2.com/cgi/wiki?BlubParadox. I do miss some slick features in
Python. But the nice thing about Python is that in those cases I can use
its dynamism to implement it myself (and usually somebody else has done
it for me, of course).

In the end I'm not a language guru nor a framework wizard, but a mere
mortal who designs and writes programs for end-users. For that task I
need: a better standard ide, an integrated db interface with a proper
set of db drivers (!!), a better debugger, a standard widget/windows
toolkit, something akin to a standard for web programming, better
documentation, a standard lib which is better organized, a formalized
set of protocols and patterns for program construction. And an
interpreter which is fast enough to avoid using C or Pyrex in most
obvious cases.

Many will say that Van Rossum's brainstorms/proposals as depicted in

http://www.artima.com/weblogs/viewpost.jsp?thread=86641

will help in the above mentioned. And I'm certainly not against Optional
ype checking.

But I see little to no efforts from the core python team to address my
needs as listed above. They seem mainly to focus on the core attributes
and syntax of the language. Very little or no efforts are taken to
improve the infrastructure around the language.

And then I read the following sentence by Van Rossum:

"In order to make type inferencing a little more useful, I'd like to
restrict certain forms of extreme dynamic behavior in Python"

In the end, it's mindset which counts. And I think that mindset is going
to be determine the way foreward for Python: more features, increased
complexity, less dynamism. Lots of syntax crud, without addressing the
need to improve the infrastructure around the language.

In short: I symphatize Patrick Logan's feeling:

http://patricklogan.blogspot.com/2005/01/road-to-ruin.html








Regards,

Iwan van der Kleyn

 

[flamesrock]

Well, I'm not a seasoned programmer like you but I have to say Python
is the singlebest language I've worked with to date. In a matter of
weeks I learned to do things that took me months in other languages and
even found the process enjoyable.

Maybe you are right. If so, couldn't Python be forked into something
like you describe, while still remaining compatible at the core? (if
anyones willing)

Python++ anyone?

 

[Paul Rubin]

Maybe you are right. If so, couldn't Python be forked into something
like you describe, while still remaining compatible at the core? (if
anyones willing)

It's not an issue with the Python core (language); I read that post as
mostly bemoaning the poor state of the runtime library. I feel the
same concerns, however, fixing it is a lot of work.

 

[michele.simionato]

Maybe a PSF grant would help? I guess this has been considered ...
Michele Simionato

 

[Alex Martelli]

to be determine the way foreward for Python: more features, increased
complexity, less dynamism. Lots of syntax crud, without addressing the

As a student of human nature, I'm _really_ curious as to how one could
possibly read the key document:
http://www.python.org/peps/pep-3000.html
and think in consequence of "more features, increased complexity".

Also, you keep talking about "the core python team" on the basis, it
would appear, of reading one document by Guido. Have you bothered doing
a MINIMUM of homework, such as, looking at
http://www.amk.ca/diary/archives/cat_python.html
and specifically AMK's entry for September 30? I'm trying to understand
whether you completely missed doing the most elementary amount of
background searching before venting on the group, or if you did find and
read the obvious documents and somehow STILL manage to completely ignore
their contents or read them as saying exactly the opposite of what they
_do_ say...


Alex

 

[Iwan van der Kleyn]

Also, you keep talking about "the core python team" on the basis, it
would appear, of reading one document by Guido. Have you bothered doing
a MINIMUM of homework, such as, looking at
http://www.amk.ca/diary/archives/cat_python.html

Well, you being a member of that core team (meaning nog an
organisational unit, but the group of people doing the really hard job,
getting Python to work. An excellent job at that I can repect you
if not branding me a lamer at least admonishing me for not coming up
with a thorough factual statement. But like I stated: "ramblings", remember.

I'm not completely unknown with the workings of our species myself,
though. Especially when discourse and policy is dictated by a select
group of people (meaning: the one who actually create python, no
criticism there) with final abritary powers for one indidual (again, no
criticism), mindset *is* just as important as stated fact. Mindset will
dictate future discourse and action.

And I do sense (reading planet python/this newsgroup) a mindset or at
least a tendency by the people who really matter in these discussion to
keep on adding features to the syntax; to add "structure" to Python. My
personal preference would be to leave the language alone for a while and
to improve its infrastructure.

Regards,
Iwan

 

[Ville Vainio]

Iwan> And then I read the following sentence by Van Rossum:

Iwan> "In order to make type inferencing a little more useful, I'd
Iwan> like to restrict certain forms of extreme dynamic behavior
Iwan> in Python"

Iwan> In the end, it's mindset which counts. And I think that
Iwan> mindset is going to be determine the way foreward for
Iwan> Python: more features, increased complexity, less
Iwan> dynamism. Lots of syntax crud, without addressing the need
Iwan> to improve the infrastructure around the language.

What form of extreme dynamic behaviour have you been using lately? Do
you really think it's more worthwhile than the benefits provided by
type inference, least of which isn't the ability by IDEs to provide
you accurate code completion.

Also, Python is not a monolithic entity. Guido certainly isn't going
to write a better IDE for Python, so the time used on language
features isn't removed from improving the infrastructure around the
language.

 

[Paul Rubin]

Also, Python is not a monolithic entity. Guido certainly isn't going
to write a better IDE for Python, so the time used on language
features isn't removed from improving the infrastructure around the
language.

There aren't THAT many people working on Python. Any time spent on
feature X does tend to divert resources from feature Y.

I think there should be a moratorium on nontrivial language changes
(as opposed to library improvements) until PyPy is fully deployed.
Too much of Python as we know it today is shaped by the weirdness of
CPython. We ought to be able to get away from that.

 

[Ville Vainio]

Paul> There aren't THAT many people working on Python. Any time
Paul> spent on feature X does tend to divert resources from
Paul> feature Y.

But the people working on wxPython, pygtk, pyqt, pydev, whatever, are
largely not the same guys that commit stuff to CPython CVS.

Paul> fully deployed. Too much of Python as we know it today is
Paul> shaped by the weirdness of CPython. We ought to be able to
Paul> get away from that.

Type declarations are a feature that might benefit IronPython and
Jython more than they would CPython.

 

[Paul Rubin]

But the people working on wxPython, pygtk, pyqt, pydev, whatever, are
largely not the same guys that commit stuff to CPython CVS.

Right, but for that reason, they don't count as being working on
Python.

Type declarations are a feature that might benefit IronPython and
Jython more than they would CPython.

CPython seems to be what drives quite a few language design decisions.

 

[Istvan Albert]

And I do sense (reading planet python/this newsgroup) a mindset or at
least a tendency by the people who really matter in these discussion to
keep on adding features to the syntax; to add "structure" to Python. My
personal preference would be to leave the language alone for a while and
to improve its infrastructure.

In all honesty this:

http://www.artima.com/weblogs/viewpost.jsp?thread=86641

scares me too. Reminds me of Larry Wall's writings on Perl 6
that make me tune out fairly quickly. I don't have the kind of
problems that the these features will solve so I can't relate
to them at all.

But others might do. Especially when using python in an environment
where enforcing a strict contract is important. But if python
were to become overly complicated I'll find something else.
Three years ago I have not not used python at all, now I'm
using it for everything.

Languages should evolve with time, adapt to the needs
of its users. Sometimes that means that in some areas
it might feel worse. But it could also mean that the
problem is with us, so it would be unfair to spend effort
towards holding back this evolution just because
we don't need it.

Istvan.

PS. why can't decorators solve this optional type checking
problem? I clearly remember this as being one of the
selling points for having decorators in the first place...

 

[Aahz]

Maybe a PSF grant would help? I guess this has been considered ...

The first three PSF grants were all in some way not directly related to
changing the core language. One was for a library, one for improving
Jython, and one for improving docs. Giving the PSF more money increases
the chances for additional work.

 

[michele.simionato]

Aahz:

The first three PSF grants were all in some way not directly related to
changing the core language. One was for a library, one for improving
Jython, and one for improving docs. Giving the PSF more money increases
the chances for additional work.

Here is the link you forgot to post ;-)

http://www.python.org/psf/grants/

The one about the docs seems more about teaching scientists how to use
Python.

Michele Simionato

 

[Michael Hobbs]

What form of extreme dynamic behaviour have you been using lately?

One real-world example: in my new coverage analysis tool (to be
released any month now), I need to trace threads without changing any
code. To do so, I redefine the thread.start_new_thread() function from
outside the thread module, like so:


orig_start_new_thread = thread.start_new_thread

def traced_start_new_thread(func, args, kwargs={}):
return orig_start_new_thread(traced_func_call, (func, args, kwargs))

def traced_func_call(func, args, kwargs):
sys.settrace(my_trace_func)
func(*args, **kwargs)

thread.start_new_thread = traced_start_new_thread


Granted, doing something like this on a regular basis is really bad
coding style, but the ability to do something like this when I really
need it is what I love about Python.

 

[Doug Holton]

But if python
were to become overly complicated I'll find something else.
Three years ago I have not not used python at all, now I'm
using it for everything.

You're in luck, python 2.4 won't be significantly changing anytime soon.

PS. why can't decorators solve this optional type checking
problem? I clearly remember this as being one of the
selling points for having decorators in the first place...

Because they are quite obviously an ugly and overly complicated
solution. Even Guido understood this:
http://mail.python.org/pipermail/python-dev/2004-September/048518.html

"A warning: some people have shown examples of extreme uses of
decorators. I've seen decorators proposed for argument and return type
annotations, and even one that used a decorator to create an object
that did a regular expression substitution. Those uses are cute, but I
recommend being conservative when deciding between using a decorator
or some other approach, especially in code that will see a large
audience (like 3rd party library packages). Using decorators for type
annotations in particular looks tedious, and this particular
application is so important that I expect Python 3000 will have
optional type declarations integrated into the argument list."

 

[Hans Nowak]

As a student of human nature, I'm _really_ curious as to how one could
possibly read the key document:
http://www.python.org/peps/pep-3000.html
and think in consequence of "more features, increased complexity".

Also, you keep talking about "the core python team" on the basis, it
would appear, of reading one document by Guido. Have you bothered doing
a MINIMUM of homework, such as, looking at
http://www.amk.ca/diary/archives/cat_python.html
and specifically AMK's entry for September 30? I'm trying to understand
whether you completely missed doing the most elementary amount of
background searching before venting on the group, or if you did find and
read the obvious documents and somehow STILL manage to completely ignore
their contents or read them as saying exactly the opposite of what they
_do_ say...

Optimistic documents about a cleaner and smaller language (and an
improved stdlib) are all well and good, but if you look what has
actually been happening to Python over the last few years, then the OP's
worries don't seem so far-fetched. "More features, increased
complexity, less dynamism" pretty much sums it up.

Guido's posts about optional static typing seem to suggest that this
development will continue in the same vein. (He may just be putting his
thoughts on paper, but it's the BDFL, so what is one supposed to think?)

I for one will NOT welcome our new static typing overlords. ;-)

 

[Alex Martelli]

And I do sense (reading planet python/this newsgroup) a mindset or at
least a tendency by the people who really matter in these discussion to
keep on adding features to the syntax; to add "structure" to Python. My
personal preference would be to leave the language alone for a while and
to improve its infrastructure.

I happen to agree with this preference as stated here -- the time to
change the language is at 3.0 release time (say a few years from now).

It _is_, of course, perfectly sensible for Guido to start musing out
loud on the only major addition he really wants to make at that distant
time a few years from now, namely optional static typing -- it will be a
biggie when it comes, so it should not be sprung on us all by surprise;
the more (and the earlier) feedback comes to help him decide the exact
details of that addition, the merrier.

Meanwhile, over on python-dev, the discussion is on completing the
AST-branch (so the standard library may finally include a full
python-coded Python compiler when 2.5 is released, likely in 2006),
dealing (in distutils, mostly) with the fact that the Mac may use some
case-sensitive filesystems but its default one is case-preserving but
insensitive, issues with memory allocation strategies triggered by
Darwin's weird realloc behavior (a realloc there never shrinks an
allocation, eek), the best maintenance strategy for old FAQs, what
functionality is to be added to the zipfile module (e.g., removing files
from a zip archive), and the fix of a misspelling in the docs.

I kid you not: these are the threads going on today in the python-dev
list, where most of [the discussion about] Python's development really
takes place. Not sure who you believe "really matter in these
discussion" (sic), but the discussants today included Guido van Rossum,
Tim Peters, Martin v. Loewis, and many others. How one can possibly
infer from such raw data "a mindset or at least a tendency ... to keep
adding features to the syntax" is really hard to explain, at least for
me. Almost invariably, the great majority of proposals to change the
language, most particularly the syntax, come on this list, fortunately,
since that leaves python-dev essentially free to keep focusing on the
huge nitty-gritty daily work of enhancing the implementation (and other
aspects of the infrastruture, such as the spelling of the docs...);
almost invariably, moreover, the great majority of such proposals come
from clueless or semi-clueless newbies or semi-newbies.

You appear to have taken the fact that Guido appears to want lot of
early discussion about his ideas on optional static typing, presumably
to help those ideas develop, and drawn from that tiny speck of 'fact'
the most absurdly extrapolated conclusions, totally and utterly ignoring
the overwhelming countervailing mass of facts and evidence that point
exactly in the opposite direction.

Of course, what YOU think should happen regarding the infrastructure,
and what the people who donate their time to actually make things happen
want, may be totally at odds. For example, you appear to keenly want
one standard IDE to be blessed and receive all attention, one standard
web framework ditto, and so on, and so forth; most core developers, on
the other hand, appear to be very modestly inclined to spend their time
and energy in that direction, so that only a rather minor fraction of
the python-dev effort is channeled that way. Innovative efforts are
best kept OUT of the core Python standard, many of us believe, as long
as their innovative fervor continues: once something does get into the
standard Python library &c, it won't and shouldn't develop very fast,
nor at all freely, due to the tiranny of backwards compatibility, cross
platform support, and the huge increase in maintenance/support efforts
that comes when something suddenly becomes much more widespread and
widely used than it previously used to be. Enhancements in library
organization and functionality, fixing typos in the docs, or deciding
whether and how to work around some platform's quirks, are definitely
less glamorous than inventing new grand unification projects, but
nevertheless this kinds of things IS by far the vastest part of the work
in a large and mature open-source project.

Wonderfully innovative projects, *OUT* of the core language/library, are
very abundant. Come to PyCon in March and you'll probably hear several
eye-opening, possibly mind-boggling talks about such things as the type
inferencing and optimizations of the pypy project, or Enthought's
breathtaking Traits and Envisage technologies -- all "infrastructure",
btw, no language changes involved. It's perfectly fit and proper that
such work takes place exactly where it's taking place now. If and when
it matures and stabilizes, to the point of not needing any more
innovation but rather consolidation and spreading, _then_ some part of
it may be merged into the standard core. Many people contributing to
such innovative projects are also Python core committers, of course.


Alex

 

[Istvan Albert]

application is so important that I expect Python 3000 will have
optional type declarations integrated into the argument list."

I think that *optional* part of the "optional type declaration"
is a myth.

It may be optional in the sense that the language will
accept missing declarations but as soon as the feature
is available it will become "mandatory" to use it
(peer pressure, workplace practices).

Istvan.

 

[Aahz]

Aahz:

Here is the link you forgot to post ;-)

http://www.python.org/psf/grants/

Didn't forget, was lazy and short of time.

The one about the docs seems more about teaching scientists how to use
Python.

True, but that's still an expansion of Python's usability as opposed to
changing the core language.

 

[Bulba!]

Also, you keep talking about "the core python team" on the basis, it
would appear, of reading one document by Guido. Have you bothered doing
a MINIMUM of homework, such as, looking at
http://www.amk.ca/diary/archives/cat_python.html
and specifically AMK's entry for September 30?

This complaint about "syntax cruft" is really weird, as while
browsing through the old and new docs I got the impression that
Python hasn't really changed so much as just added new, and
mostly well-chosen features. What's not to like in sets for
instance?!

However, I do have to concede that Python as environment
still has a way to go - not the language features are missing, but
better _standard_ IDE and debugger. E.g. Pythonwin debugger
sometimes simply dies on me, I have no idea why.

This is not to criticize the great work that you
guys are doing and the results of which we get FOR
FREE (something in principle I don't believe), but merely
to indicate that Python has grown and so did the
expectations. People tend to get spoilt: they expect to
find in a free product the gazillion of VS-like features.
As we say here, "appetite tends to grow as you eat".

E.g. right now I would kill for a standard, built-in matrix
type that would be as flexible as lists and dictionaries are
in Python, so that I could slurp the whole CSV file or some
other table in one instruction into a matrix that could
accomodate data types likes strings and floats (just like
dictionaries do - just, say, declare, "column 1 of matrix
contains strings, and please convert the values in column 2
into floats"), and then easily do some fancy matrix
transformations.

Clean and expressive syntax plus flexibility of lists and
dictionaries are the main reasons I got into Python.

Where else could I do smth as neat as writing a
CSV header with:

DictWriterInstance.writerow(dict(zip(titles,titles)))

Note I didn't have to do any operations myself:
all that was necessary for me as a programmer was to
figure out how to connect the Lego pieces together.

We need more of this sort of expressive power in a
standard library and built-in types. This is the subconscious
power of attraction in Python I think. I'm not a language
designer. I don't know how to get there. I just would love
to see more of that around.

To summarize, it's not the language that is would be
nice to develop further. Those are:

- more of the rich, flexible data types

- more extensive standard debugger

- possibly standard IDE with more features
that are typically found in commercial products

<now the evil me sits back, relaxes and watches
how properly motivated Alex and others get to work>

No, really, guys, great thanks for all that work.