Lisp mentality vs. Python mentality

[Paul Rubin]

Come on, you're just making stuff up. How the *hell* do you get
switching threads twice out of that?

I think I mis-read your suggestion. I thought you meant to query
another thread before acquiring a lock.

Anyway, the usual reason to want to kill a thread is that it's become
unresponsive for some reason, while holding onto some resource like a
lock or a bound socket. This would be in the category of fault
recovery from conditions that weren't expected ahead of time.

Another situation is when you want to just stop the program
altogether, similar to kill -9'ing it, with no attempt to shut down
gracefully. Kind of a global ctrl-C. Right now there is no way to do
that for a multithreaded program except by literally sending a process
kill, which is pretty ugly.

 

[Zamnedix]

In answering the recent question by Mark Tarver, I think I finally hit
on why Lisp programmers are the way they are (in particular, why they
are often so hostile to the "There should only be one obvious way to
do it" Zen).

Say you put this task to a Lisp and a Python programmer: Come up with
a good, generic, reusable way to compare two lists.  What are their
respective trains of thought?

Lisp programmer:

Well, there is a standard function called mismatch that does it, but I
can't recommend it.  First of all, you don't know where that
function's been.  Anyone and their mother could have worked on it, did
they have good, sound programming practice in mind when they wrote
it?  Of course not.  Let's be real here, we have to implement this by
hand.

(defun lists-are-equal (a b)
   (or (and (not a) (not b))
       (and (= (car a) (car b)) (lists-are-equal (cdr a) (cdr b))))

There, much better than the standard function, and better yet, it's in
the *absolute minimal form possible*.  There is no way to express list
comparison in a more reduced form.  It's almost erotic how awesome it
is.  I'm---whoa, ok, I'm getting a little excited now, settle down.
Well, come to think of it, that's really not that good.  First of all
it's a function.  I mean, it just sits there and does nothing till you
call it.  How boring is that?  It can't react to the current
situation.  Plus it compares all lists the same way, and that is
really inefficient.  Every list compare is a new problem.  Different
lists need different comparative strategies.  This function simply
won't do.  I need a macro that can intelligently compile the right
list compare methodology in.  For instance, if we want to compare two
lists that are known at compile time, we don't want to waste time
comparing them at runtime.  No, the macro must detect constant
arguments and special case them.  Good start.  Now, we have to
consider the conditions this comparison is being done under.  If the
user is passing the result of a sort to this macro, it's almost
certain that they are trying to see whether the lists have the same
elements.  We can do that a lot more efficiently with a countset.  So
let's have the macro check to see if the forms passed to it are all
sort calls.  Better yet, let's check for my own powerful sort macro.
Hmm.  Wait... I think my 4600-line sort macro already checks its
calling context to see if its results are being fed to a list
comparison.  I'll have to refactor that together with this macro.  Ok,
good, now I am sure other users will eventually want to customize list
comparison for their own use, after all every list comparison is
different and I can't possibly anticipate all of them.  A user needs
to be able to adapt to the situation, so it's vitally important to
create a plug-in infrastructure to give them that flexibility.  Now,
what about exceptions, there's a millions ways to deal with that...

...and so on until eyelids can no longer stay open....

Python programmer:

a == b.  Next question.

Carl Banks, who might be exaggerating

...a little.

Hahaha. I love it.

 

[Aahz]

As well I'd like to outline, that, IMO, your answer exhibits the
common attitude among pythonistas: everything should be done in one
true way, which is the best option (and that is how it's implemented
in the current version of the language).

Did you see my comment about Java? This particular issue has little to
do with Python. I won't disagree that what you're describing is
sometimes a problem in the Python community, but you're picking the
wrong issue to claim its relevance.

 

[Vsevolod]

Did you see my comment about Java? This particular issue has little to
do with Python. I won't disagree that what you're describing is
sometimes a problem in the Python community, but you're picking the
wrong issue to claim its relevance.

OK, I'm not a big expert in Python. That was just the thing, that was
important to me recently. I won't claim, that it's regularly a problem
with Python, although from reading Guido's blog I get that impression.
(Well, I understand, that's how BDFLs should behave
Yet there was no response to my point, that the original example was
not realistically depicting the Lisp world, while more characteristic
of the Python one.

Best regards,
Vsevolod

 

[Aahz]

Yet there was no response to my point, that the original example was
not realistically depicting the Lisp world, while more characteristic
of the Python one.

That's because there's no response to make; the original post was a joke,
and trying to have a serious discussion about it rarely excites people.

If you want to talk about Python and problems you're running into, you
should start a new thread.

 

[David Bolen]

"This should be used with caution: it is implementation-defined
whether the thread runs cleanup forms or releases its locks first."
This doesn't mean deprecated. It means: implementation-dependent. For
example in SBCL: "Terminate the thread identified by thread, by
causing it to run sb-ext:quit - the usual cleanup forms will be
evaluated". And it works fine.

I'm curious - do you know what happens if threading is implemented as
a native OS thread and it's stuck in an I/O operation that is blocked?
How does the Lisp interpreter/runtime gain control again in order to
execute the specified function? I guess on many POSIX-ish
environments, internally generating a SIGALRM to interrupt a system
operation might work, but it would likely have portability problems.

Or is that combination (native OS thread and/or externally blocking
I/O) prevented by the runtime somehow (perhaps by internally polling
what appears to code as blocking I/O)? But surely if there's an
access to OS routines, the risk of blocking must be present?

That scenario is really the only rationale use case I've run into for
wanting to kill a thread, since in other cases the thread can be
monitoring for an application defined way to shut down.

-- David

 

[Paul Rubin]

I'm curious - do you know what happens if threading is implemented as
a native OS thread and it's stuck in an I/O operation that is blocked?
How does the Lisp interpreter/runtime gain control again in order to
execute the specified function? I guess on many POSIX-ish
environments, internally generating a SIGALRM to interrupt a system
operation might work, but it would likely have portability problems.

Someone wrote a paper proposing some Python language extensions to
support asynchronous exceptions in Python threads. I don't have the
url handy but you can probably search for it.

 

[Dan Sommers]

I agree with your opinion about keeping the abstraction layers
shallow, but in my view high-order and helper functions do not comprise
a new abstraction layer. For example in Lisp, using map, reduce (fold),
or any other high-order function is just like using for, or while in a
normal imperative language.

If I hit a call to map or to reduce, I've hit the bottom: map and reduce
are defined by Lisp and not by the programmer.

And also I would like to point out that "hitting the language
sooner", means to know every function in the standard Python library
(which is by far uncomprehensible, its huge) and most of the times you
also need the documentation. And if we go this path, when debugging we
could use a smart IDE, which should show as tool-tip-text for function
names their documentation, and in this way all we have to do to
understand a particular function is just to point it out.

Although huge, at least the standard Python library is *bounded* (even if
only within a given release). The more I use it, the better I know it,
and there's probably an 80/20 rule in there somewhere (if I concentrate
on programs in my particular area(s) of interest, there may even be a
lurking 90/10 rule). Maybe I don't know the Standard Patterns well
enough (I still speak OO as a second language), but even when I see
something as apparently clear as add_listener(f), I have to wonder about
the specific semantics of your particular add_listener function.

 

[Vsevolod]

That's because there's no response to make; the original post was a joke,
and trying to have a serious discussion about it rarely excites people.

In every joke there's a grain of truth. And usenet is precisely for
that thing -- discussions. Even of stupid jokes.

If you want to talk about Python and problems you're running into, you
should start a new thread.

I'm not at that level of proficiency with the language. I believe most
of my technical problems are connected with lack of knowledge or
experience, not the language's features. While conceptual problems
seem futile to discuss. There's another saying: "when in Rome do as
the Romans do"

Best regards,
Vsevolod

 

[Vsevolod]

I'm curious - do you know what happens if threading is implemented as
a native OS thread and it's stuck in an I/O operation that is blocked?
How does the Lisp interpreter/runtime gain control again in order to
execute the specified function? I guess on many POSIX-ish
environments, internally generating a SIGALRM to interrupt a system
operation might work, but it would likely have portability problems.

We're arguing to the old argument, who knows better, what the
programmer wants: language implementor or the programmer himself.
AFAIK, Python community is on former side, while Lisp one -- on the
later. As always, there's no right answer.

Best regards,
Vsevolod

 

[Aahz]

I'm not at that level of proficiency with the language. I believe most
of my technical problems are connected with lack of knowledge or
experience, not the language's features. While conceptual problems
seem futile to discuss. There's another saying: "when in Rome do as
the Romans do"

c.l.py isn't just for the experts; if you have a question along the
lines of "here's something I do in Lisp, how do I do it in Python?",
you're entirely on-topic.

Also, most of us are happy to explain why Python does things in certain
ways -- as with Lisp, the more you wrap your mind into the Python Way,
the easier you'll find your work.

Note that neither of these is "why isn't Python more like Lisp?", that's
where you get into futility territory. ;-)

 

[Dan Sommers]

You truly don't know Lisp. *Everything* in Lisp can be _redefined_ and
if you can't do something conveniently that way, you can use a _macro_
to implement convenient new syntax for it.

Yes, I agree: Python and Lisp are extremely dynamic languages. I *can*
redefine map, reduce, +, and other operators and functions, but I know
better. When is the last time you examined someone else's code, and
asked them what their "map" function did (in Lisp or in Python)?

 

[David Bolen]

We're arguing to the old argument, who knows better, what the
programmer wants: language implementor or the programmer himself.
AFAIK, Python community is on former side, while Lisp one -- on the
later. As always, there's no right answer.

Note I wasn't trying to argue anything, I was actually interested in
how the behavior is handled in Lisp? Do you know how the Lisp
implementation of threads you spoke about handles this case?

E.g., can the Lisp implementation you are familiar with actually kill
such a thread blocked on an arbitrary external system or library call?

-- David

 

[John Nagle]

Note I wasn't trying to argue anything, I was actually interested in how the
behavior is handled in Lisp? Do you know how the Lisp implementation of
threads you spoke about handles this case?

E.g., can the Lisp implementation you are familiar with actually kill such a
thread blocked on an arbitrary external system or library call?

That's more of an OS thing.

Under QNX, the real-time operating system that does almost everything
by message passsing, all operations which block are not only interruptible,
but can be given a timeout. QNX has a well worked out system for interrupting
system calls and canceling threads in a sound way. If you really want to
see how this ought to work, look at QNX:

http://www.qnx.com/developers/docs/6.3.0SP3/neutrino/lib_ref/t/threadcancel.html

This sort of thing isn't used much. It's for emergencies, like "We just
lost the other CPU in the redundant pair. Cancel all nonessential threads
and take over control of the rolling mill NOW". Python doesn't do jobs
like that.

John Nagle

 

[Vsevolod]

Note I wasn't trying to argue anything, I was actually interested in
how the behavior is handled in Lisp? Do you know how the Lisp
implementation of threads you spoke about handles this case?

E.g., can the Lisp implementation you are familiar with actually kill
such a thread blocked on an arbitrary external system or library call?

-- David

The behavior up to some detail is described here: http://www.sbcl.org/manual/Threading.html

Overall, if this issue comes up important, it is possible to either
get the more detailed explanation from the developers, poke in the
source code or implement the needed behavior yourself. Basically, you
can't know the precise semantics and corner cases of all the language
and implementation constructs you use (especially, when speaking about
libraries, which threading definitely is both in lisp and python).

Moreover, in this case thread-killing semantics is largely defined by
the underlying system (if we are talking about native threads), and
those systems (like Unix) somehow happen to function , or can be
defined arbitrarily, if we consider green threads. So the answer, that
it can be only poorly specified, IMO, is just an indulgence from
implementing the operation.

Best regards,
Vsevolod