Going the PL/1 way

[A.M. Kuchling]

So do you say "them and us", Python developers vs. users?
I've always had the "we all"-feeling. Maybe that's over.

That's always been the case, between people who contribute (in time, money,
skills) and people who don't, for whatever reason. The saving grace is that
it's very easy to become a developer; just write some code, help with
development, and all of a sudden you're a developer.

--amk

 

[Istvan Albert]

My basic point is that the features added to projects are those that people
are willing to actually work on. In the case of decorators, some people
wanted decorators, for the sake of PyObjC or ctypes or whatever, and Mark
Russell actually sat down and implemented it. (Because he uses PyObjC?

I think one of the latent messages of this thread was that adding
new "gratuitous" features to the core language is a bad thing.

Just because someone sits down and codes it does not mean it
should be added to the language. Features are forever, you
cannot just can't eliminate them in the next version.

Why putter around with banalities like sorted() and reversed()
that only add to the cognitive overhead, break the consistency
in dealing with lists yet save at most a line or two of
coding? And what you imply here, that the fact of
having decorators works for PyObjC was considered in
the overall decision feels like putting the cart before
the horse.

Python is said to come with the batteries included, then
that's where new features should go, to the battery level
not into the wiring.

Istvan.

 

[Xavier Combelle]

Isn't that like saying, I avoid programming in general because it's
error-prone? I don't follow.

I don't agree with the comparison. Programming may be error prone, but
can you suggest something
to replace it? In the other side, there is different way to avoid
threads: make a heavy process, doing a
single event loop. You should have a really special and hard work to use
them routinely.

 

[Dominic]

threads: make a heavy process, doing a

single event loop.

Well, often chaining generators is good enough!

Another -- probably not so good solution is to
do an interface specification and generate
code with it. "Active Tasks" could be
automatically _inverted_ , depending on your choice.
Michael Jackson (hee, not the singer) has
done this in the 70s in COBOL. Maybe
you have heard of JSD or JSP ;-)
(JSP (Jackson Structured Programming) and
JSD (Jackson System Development))
In JSD you model entities as processes. (CSP)
(Yes it is OOP analysis ;-) )
Afterwards you map processes too processors.
Probably you have only one, so all
processes but one must be inverted.
However a scheduler solution is
also possible.

Maybe someone comes up with a @decorator
solution, which does invert active
tasks on the fly

Ciao,
Dominic

 

[Dominic]

Yes, I'm thorougly annoyed with creeping featurism.

Well, you need not use them in your own code.
Define and enforce a subset like it is common in ADA.

However I am not sure myself were decorators will
takes us, they'll certainly(?) have a major influence.
Maybe it is a bit like generator-introduction.
They seemed strange at first and now I think
they are natural and help to simplify coding patterns.

Some postings about @decorators remind me of that
old farmer who only eats stuff he knows already.


Ciao,
Dominic

 

[Anthony Baxter]

Just because someone sits down and codes it does not mean it
should be added to the language. Features are forever, you
cannot just can't eliminate them in the next version.

The discussion on decorators has been going on for over 2 years
on python-dev. There is a broad concensus that they're a useful
feature, and are not _just_ being added for PyObjC.

As far as "fixing" the GIL - well, no-one on Python-Dev has obviously
found it to be a problem for them. I certainly can't recall a time in the
last 5 years where the GIL has been a problem in any application I've
written (including a number of heavily threaded applications). Coding
threaded applications in Python does require _knowledge_ of the GIL,
true, but there's nearly always ways to get around it.

As a couple of people have said already - if it's truly a problem for you,
you're able to do something about it. Either look into how it could be
fixed, or hell, pay someone else to work on it for you. No-one is going
to work on something merely because you assert that it is important.
Heck, put together a decent case for _why_ it is a problem.

Greg's free-threading patches for Python 1.4 were there for years.
The interest in them was so low that no-one was even interested
in keeping them up-to-date with current Python (which would have
been a much smaller task than writing them from scratch).

One final point is that with the various new implementations of
Python (Jython, IronPython, Python-on-Parrot, Python-on-smalltalk-VM,
Python-on-Scheme-VM) it's quite possible that one or more of
these new implementations won't have the GIL. You might look into
one of these implementations, and if it looks good to you, contribute!

This is not saying "fix it yourself". This is saying "the people who
work on Python, in their own spare time (almost no-one is paid
to hack on Python) don't consider that the GIL is an issue for
them". You can't just say "fix it!" and expect volunteers to do the
work for you.

Anthony

 

[Miklós]

Well, you need not use them in your own code.
Define and enforce a subset like it is common in ADA.

I've got a faint idea that this "subset" approach greatly contributes to the
unpopularity of Ada.
Once a core feature is in place, you just cannot insulate yourself from
that.

Some postings about @decorators remind me of that
old farmer who only eats stuff he knows already.

And perhaps he might stay healthy longer than that young one who eats each
and every new nutriment and colourful shiny berries. <wink>

Cheers,
Miklós

 

[Harald Massa]

Ville,

just to add some:

I guess this depends on the developer. I can't help lovin' stuff like
genexps that are coming in 2.4,

yeah, I am really looking forward to them. List comprehensions are a so
natural way to express some situations, and with the additional elegance
of "not having to lock memory" it will be even more phantastic.

Also ... the sets module, introduced with 2.3, I think. It is SO natural
- many, many problems can be formulated within sets-algebra (find all
customers who have X and Y ... just get the intersections of two sets).
And with 2.4 sets will be a builtin, that shows me the commitment of the
Python community that they will be supported for EVEN longer than the
usual many years of a stdlib module.

And: the decimals data type. When working in the financial industry, it
is UNBELIEVABLE PHANTASTIC to finally get a feature that helped COBOL
live THROUGH AGES. (while being the MOST UGLY chick on the block, COBOL
allways had VERY WEALTHY boy friends - not being cool, but having a big
car, big house...) - and now Python will be ONE OF THE FIRST really sexy
chicks of programming languages that will have a DECIMAL type!!

Than: datetime, introduced with 2.3. Yes, there was mx.datetime before.
Who has read a DBAPI-Database-Driver? "try: import mx.datetime ...
except: print some wired error" --- with datetime in the standard lib
there is no longer a need to download additional modules just to print
nice dates. GREAT!!!

I do not understand at all what's that decorators thingy, but I know that
girls often put some deco into flats and it really looks cosy after, so I
assume that decorators are not on the dark side of the source.

Maybe we should ask for a ternary decorator, because I learned that
"ternary" and "decorator" triggers high emotions within the Python
community.

I often struggle with encodings - and in growing versions of Python you
can see that also Python does sth. more with encodings - allowing unicode
in more and more places, growing "string types" instead , putting
Encoding-Tags within source code. That's a healty relationship with a
language, I feel, when the language and me are struggling with the same
problems.

So: I am really looking forward to Python 2.4, decorated or not, I will
take it.

Harald

 

[Ronald Oussoren]

The discussion on decorators has been going on for over 2 years
on python-dev. There is a broad concensus that they're a useful
feature, and are not _just_ being added for PyObjC.

I sure hope not [*]. I just happened to dumb enough to volunteer a
use-case for decorators in the last round of discussions, and somehow
this got misunderstood :-(

I do believe decorators are generally useful, as shown by various
examples shown on python-dev. The also happen to make live easier for
users of PyObjC.

Ronald

[*] I'm one of the primairy developers for PyObjC, don't interpreted
this a "-1" for decorators.

 

[Miklós]

Ville,
I do not understand at all what's that decorators thingy, but I know that
girls often put some deco into flats and it really looks cosy after, so I
assume that decorators are not on the dark side of the source.

The rest of the novelties in v2.4 are more or less fine with me, too.
But IMHO decorators are exactly Dark Code Wader stuff. Basically it's about
self-modifying code.

Maybe we should ask for a ternary decorator,

Great stuff for my original sarcastic post.

Cheers,
Miklós

 

[Robin Becker]

Ok, seems like we have @decorators. It's a nice tribute to Intercal. But
I'd prefer #decorators.

We have metaclasses for pleasure brain-melting. We have generators,
new-style classes and old-style classes. Lambdas are wonderful.
We do need real interfaces, not abused classes. Python still needs to have
inlined functions, macros and templates. Operator overloading is not enough,
one should be able to introduce new ones freely. There's a heavy demand for
terciary comparisons. Why don't we have a 'case' statement and 'do... repeat
()'?
We cannot do without general closures, of course. Hm, automatic garbage
collection is sometimes fine but explicite memory allocation/deallocation is
really unavoidable for efficient programs.
Wait, pointers are a must! Static typing should have been introduced a long
time ago. You just cannot do without absolute address variables.

Yes, I'm thorougly annoyed with creeping featurism.

Miklós

......

I'm not exactly against 'decorators' although it seems to me thay should
be called 'transformers' as it seems they act to transform the following
function definition.

On the other hand the proposed syntax is abominable.

Additionally there now seems to be at least two 'one and only obvious'
ways to handle these transformations.

@classmethod
def meth(...):
...

or
def meth(...):
...
meth = classmethod(meth)


As a final silliness (or perhaps miracle) this can apparently all be
done in python using code from PJ Eby so a standard decorate mechanism
could have gone into the library while the Gods argue.

Decision by exhaustion is surely not a good recipe.

 

[Harald Massa]

Miklós,

The rest of the novelties in v2.4 are more or less fine with me, too.
But IMHO decorators are exactly Dark Code Wader stuff. Basically it's
about self-modifying code.

So you are suggesting that THE SOURCE it is strong with this young padawan?
But I'm sure considered carefully the BDFL has before giving in, hasn't he?

I honestly ask that someone could explain to the less enlightened like me
what this decorator thingy really is. What does it make a correct step on
the path to enlightenment?

Harald

 

[Anthony Baxter]

As a final silliness (or perhaps miracle) this can apparently all be
done in python using code from PJ Eby so a standard decorate mechanism
could have gone into the library while the Gods argue.

PJE's implementation is pretty nasty. There's no way that would go into
a released version of Python. Guido's also said that he's not happy with
the approach.

Decision by exhaustion is surely not a good recipe.

It might be approaching exhaustion after the last few days, but before
then, it was more decision-by-exhaustive-argument. After 2+ years of
discussion, and no concensus, eventually it comes down to Guido to
decide which way feels best.

 

[Xavier Combelle]

As far as "fixing" the GIL - well, no-one on Python-Dev has obviously
found it to be a problem for them. I certainly can't recall a time in the
last 5 years where the GIL has been a problem in any application I've
written (including a number of heavily threaded applications). Coding
threaded applications in Python does require _knowledge_ of the GIL,
true, but there's nearly always ways to get around it.

For information is there some internet resources to understand what
exactly the GIL is about ?

 

[Anthony Baxter]

For information is there some internet resources to understand what
exactly the GIL is about ?

Plenty - try this google search:
http://www.google.com/search?q=site:docs.python.org+GIL
for a couple of simple ones, but there's plenty more - google for
"python GIL" (or "python free-threading" for patches that were
written against Python 1.4 to remove the single lock. They ended
up slowing down Python significantly.

Note that much has been written about the GIL. Much of the stuff
I've seen has been from people who obviously don't understand
it, but fear it anyway.

Anthony

 

[Dominic]

I've got a faint idea that this "subset" approach greatly contributes to the

unpopularity of Ada.

Well, this may be true. I do not know.
I have read some papers about the SPARK ADA subset.
It has been chosen so that they can prove
certain properties and be happy with most compilers.
For safety critical or embedded applications ADA
is not too bad. It is probably more often used
in this field than Python.
Maybe this can be changed for better.. ;-)

And perhaps he might stay healthy longer than that young one who eats each
and every new nutriment and colourful shiny berries. <wink>

Yes, there is some truth in here.
I knew you were going to write that. ;-)

Nikolaus Wirth must have had such an attitude
when he designed Oberon.
Aesthetically Oberon is a very fine language.
However if you peek into Oberon source
code you will often see that they have
used linked lists and rewritten the
code everywhere. Because they do not
have _generics_ and casting seems not
popular among Oberon programmers.
Purity comes with a price tag.

Nevertheless, I think Python has been
pretty feature complete and I am not
sure if @decorators are good or bad.
The addition of generators though
has helped me a lot.
I'll have to trust the Python
language designers since
nobody seems to have actual data
to support or discard @decorators.
As far as I know they are part
of C# and Java (recently).
Maybe some experience and data
has been gathered by them.

Ciao,
Dominic

 

[Miklós]

Yes, there is some truth in here.
I knew you were going to write that. ;-)

Sure thing I wrote that.

Nevertheless, I think Python has been
pretty feature complete and I am not
sure if @decorators are good or bad.

Decorators, the way things are now, will IMHO actively entice people to
abuse code.
A flawed analogy is computed goto's. We all know by now that 'goto' is evil
;-) but computed goto's (albeit made code flow 'more powerful')
only worsened reading the spaghetti code which results in the usage of
goto's.

As far as I know they are part
of C# and Java (recently).
Maybe some experience and data
has been gathered by them.

Dunno anything about C# but Java decorators are not that 'dynamic'. And IMHO
this is a good thing in this case.

Cheers,
Miklós

 

[Arthur]

Decorators, the way things are now, will IMHO actively entice people to
abuse code.

Undoubtedly we will begin to see the decorator equivalent of the
3-tier architecture embedded in a list comprehension that one comes
across from time to time.

Though I find it hard to get too upset about the addition of a feature
that accommodates a portion of the community, and that can be safely
ignored in writing one's own code if it serves no needs of one's own.

And while I guess that could be said of most, if not all new features
that have caused controversy, I somehow see this feature more
modularized and offset than the others.

Perhaps the fact that it is so appraently unPythonic is what
modularizes it effectly enough to be considered non-infectious.

One hopes.

Art

 

[Xavier Combelle]

Thank you for the address.this page is especially clear
I am quite agree with you.

Plenty - try this google search:
http://www.google.com/search?q=site:docs.python.org+GIL
for a couple of simple ones, but there's plenty more - google for
"python GIL" (or "python free-threading" for patches that were
written against Python 1.4 to remove the single lock. They ended
up slowing down Python significantly.

That is not really surprizing, the GIL is a very simple and very
efficient way to manage
multithreading. After reading, I can't undersand that other interpreted
languages don't do the same.
The use of GIL make that interpreter work as a single thread process.

Note that much has been written about the GIL. Much of the stuff
I've seen has been from people who obviously don't understand
it, but fear it anyway.

Why fear it ? There is no really problems: from a performance point of
view, it's a very light way
to limit the overhead of mutltithreading.. From developper point of
view, it seems to be just
a constraint to write C code which access to Python. Moreover, just
blocking IO, or code
wich do a lot of computing seems to worry about it. For blocking IO, I
wonder, if the main part
si not used by the development team of python and for long computing
task, a different heavy
process seems be better design. I am certainly wrong, but I don't see where.

 

[G. S. Hayes]

What really hurts is that I can't honestly tell Java programmers that

Python is a slam dunk compared to Java & the JVM. Java has no GIL,

Java has drunk the threads Kool-Aid, and Java programmers are very
likely to be thread-crazy because of the lack of a select equivalent
(until recently) and the lack of good access to processes via fork or
similar. Luckily, it's usually pretty easy to convince people of the
benefits of NOT overusing threads once they've made that mistake once.



I honestly think a big reason the GIL isn't a major issue to most
Pythonistas is because they tend to know that there's a better way to
approach most problems than using threads.

Isn't that like saying, I avoid programming in general because it's

error-prone? I don't follow.

Not really. You need to program to get things done, but presumably
you leverage all the work that's been done over the past 40+ years to
make that an easier, less error-prone task. It's like saying you
shouldn't program in assembly because it's error prone; sure, there
are some cases where it makes sense, but in general it's not a great
idea.



Threads are error-prone because they intentionally circumvent the
years of OS work that went into getting protected memory for us in the
first place. Even for tasks that aren't neatly divided into an event
loop, processes are available for parallel execution. 99% of the time
they're a better choice _except_ that two of the most dominant
programming platforms out there don't have a good (efficient and
featureful) implementation (Java and Windows). It's absolutely insane
that many Windows programmers believe that to use parallel processing
requires you throw out memory protection; the two concepts are very
distantly related. The decision to use threads vs. processes ought to
come down to whether you want to share ALL memory (and deal with the
locking and reentrancy issues that come along with that choice) or
only a specified subset of memory. When you view it from that
perspective, threads are only rarely the right choice.



Indeed, in all the years that I've been writing code I've never once
found a problem where I felt threads were the right approach--and that
includes work on high-performance servers with thousands of concurrent
users, large-scale data visualization applications that depend on the
results of many parallel searches, and realtime audio processing
applications with non-realtime visualization and UI components.
That's not to say there aren't limited problem domains where threads
are the right answer, but as a general construct they're wildly
overused. I'm frankly glad that Python doesn't encourage more of the
same.