merits of Lisp vs Python

[Ken Tilton]

Rightly or wrongly, people fear...

So when people fear wrongly we burn whatever witches we must to reassure
them?

that Lisp's macros push Lisp closer to
that hypothetical anything-goes language than is healthy. Maybe that's a
problem of perception rather than a problem of fact, but you have to ask,
why do people perceive Lisp that way?

Could it be because of people like J Shrager who writes things like this?

"Can't you just expand the language via macros to create whatever facility
of this sort [major new features with new syntax] you need..."

The context was CLOS. Something that big needs new syntax. But what you
do not know is that the syntax is Lispy, so learning it is tantamount to
understanding new function names, as even Pythonistas must do (tho a few

(This thread, dated 8 Dec 2006 16:14:44 -0800)

Or Wolfram Fenske:

"All the interesting features that haven't originated from Lisp (e. g. OO
from Smalltalk) could in turn easily be implemented in Lisp with a couple
of macros."

Right, something big like an OO.

(This thread, dated 8 Dec 2006 23:38:02 -0800)

To someone outside of Lisp, that looks like "I can make Lisp look like any
language I like in just a few lines."

On the contrary, on the rare case when it happens (CLOS, my Cells
package) one is obliged by exactly the hobgoblins you fear to remain
true to lisp patterns when extending Lisp. And there are patterns in
macro usage as well:

(my-macro <optional arguments to macro expansion>
<my-macro-forms>+)

Your hands must be getting sore from banging that drum so hard and so
long -- has it ever occurred to you that good programmers concerned with
power do not obfuscate code? And now you have read a dozen messages from
serious Lispniks trying to tell you that that is not the case, and you
persist.

We love the chance to preach to the unsaved, so we are indebted to you
for the many chances to clarify, but something tells me I should check
the c.l.p archives to make sure I am not chatting away happily with the
village idiot.

And that implies that to read a Lisp
program, one might need to be able to read code that looks like Lisp, or
Smalltalk, or Prolog, or Fortran, or all of those, or whatever bizarre
syntax the developer wanted it to look like.

"I don't want to think, I just want to bang on de drum all day.
I don't want to learn, just want to bang on de drum all day."

My point isn't whether or not their claims are correct (a "couple" of
macros? really?) but that things like this feed the perception...

Ah, well at least you are copping to witch-burning. Thx.

And it isn't really that much comfort to be told that good Lisp
developers know not to do stupid things with macros.

Hmm, didn't we see that as a comforter for some powerful Python
mechanism earlier in this thread?

Sure. But in the real
world of programming, most developers aren't good developers, they are
merely average

Two questions: do you really voluntarily use libraries from crappy
developers? Second, you think a language can stop people from writing
bad code? Java tried, and look where that got the language. Elsewhere a
Pythonista (Mr. Rubin?) bemoaned this thread's appearance of Pythonistas
shrinking from power... could you two get together?


ken

--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon

 

[Ken Tilton]

Er, hang on a moment... how do you *know* when you've
got a mistake in indending? You must be visually
verifying the indentation... rather like one does
with Python code...

Absolutely, and you are not disagreeing with Mr. Atkins, tho you seem to
think you are.

But with Lisp one does not have to clean up the indentation manually
after thrashing away at ones code. As I type each right parens I eyeball
its partner as the editor highlights it to make sure I have not missed
anything, then give the re-indent command and eyeball the result. It
/is/ still possible to screw up because I work fast and loose, but only
very rarely do I end up with a parens in the wrong place, and then the
compiler finds something to whine about.

ken

--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon

 

[Steven D'Aprano]

Now, if you want to tell me that, despite all the talk, Lisp coders don't
actually create new syntax or mini-languages all that often, that they
just use macros as functions, then the question becomes: why do you need
macros then if you are just using them as functions? Why not use functions?

Macros let you write what amounts to functions that don't evaluate
their arguments. Think of the endless clpy wars over the ternary
conditional operator. [snip]
That is trivial to do with a macro

I know that. It was more of a rhetorical question -- Lispers are either
trying to emphasis the radical nature of what you can do with macros, or
understate it and make them seem just like functions.

but can't be done with a function.

*shrug*

You can't get to Sydney from Melbourne via Hobart by bicycle either. You
either change the requirements (why does it have to be by bicycle?) or the
path (why go through Hobart?) or change the infrastructure (build a long,
long bridge connecting Hobart to the mainland). That's three tactics for
solving the problem of "go from Sydney to Melbourne". It is just not true
that the only "correct" solution is to build the bridge.

In the case of the ternary operator, changing the language is like
building the bridge. It's *a* solution. Maybe it is even the best
solution. But it is ludicrous to say that "change the language" is the
only solution.

(I know you didn't say that -- but so many people seem to think that
adding new syntactic support for their pet feature is the only solution to
their problem, no matter what the problem. Those people would, I imagine,
wet themselves in excitement if Python ever got macros. "Now at last I can
make Python look just like the bastard love-child of Perl and Java!" *wink*)

 

[Steven D'Aprano]

How would that be a bad thing? Do you do a lot of programming without a
Python environment. But I love the wall of flak you are throwing up.

Actually, yes, sometimes it is useful to print code out and read it on the
train, or in the bath, without the advantage of syntax highlighting,
pretty-printing, parenthesis-balancing or code folding. Not necessarily as
pleasant as having all those things, but it is nice that working Python
code is, by definition, already formatted correctly for pretty printing.
Even if you're stuck on some god-forsaken Windows PC with just Notepad,
you can still read Python code.

Now, *writing* Python code with Notepad isn't as easy, but it is still
doable. How about Lisp code?

That's not a criticism of Lisp exactly, but a reminder to think about not
just what problem you're trying to solve, but what resources you will have
to solve it. If you *know* that you're going to need to edit code by
ssh-ing across an high-latency connection to a machine without Emacs, then
Lisp will probably not be the best solution.

The day has not yet arrived that nobody ever needs to edit code in a
plain, vanilla text editor.

 

[Paul Rubin]

Even if you're stuck on some god-forsaken Windows PC with just Notepad,
you can still read Python code.

Now, *writing* Python code with Notepad isn't as easy, but it is still
doable. How about Lisp code?

I've generally used IDLE when editing Python under Windows. It comes
with the Python distro after all.

Yes, very large systems have been developed with much worse editing
facilities than Notepad. Specifically, through the whole 1960's and
early 1970's, display terminals were a rarity and people usually
edited code (whether in Lisp or another language) on printing
terminals (usually noisy Teletypes). It gets worse than that. In the
very early days (the Lisp terms "car" and "cdr" are derived from names
of registers of the ancient IBM 709 mainframes) they used punch card
systems instead of printing terminals. It gets worse than THAT. One
of the important historical Lisp applications was James R. Slagle's
SAINT symbolic integration program on the IBM 704. Written on punched
cards. Except Slagle sadly became blind while writing this program,
so he did a fair amount of the development by feeling the holes in the
cards with his fingers to read them. I don't think anyone writes
Python that way ;-).

The day has not yet arrived that nobody ever needs to edit code in a
plain, vanilla text editor.

It's not impossible or terribly difficult to write Lisp that way, it's
just unpleasant, once you've gotten used to doing it with editors that
automatically indent and balance parens.

 

[Kirk Sluder]

On Sat, 09 Dec 2006 21:55:19 +0000, Kirk Sluder wrote:

Who says they do? All forms of abstraction have been criticized. Sometimes
the criticism is valid. Sometimes it warns against abuses of the
abstraction. Just because a feature is useful sometimes doesn't
necessarily mean the benefit outweighs the cost.

The primary focus of the criticism in this discussion has been that
macros are a bad from of abstraction compared to libraries, (and I'm
admittedly extending that to objects, functions and classes.)

But at least you know that foolib is a module or package. You know what
from and import do, and that can't change. And you know that bar is an
object with a method also called bar, it is being called, and the
argument is a string "somefile". Those things can't change. Imagine a
hypothetical language where those two lines could mean *anything*.

But as was pointed out earlier in the thread, those keywords can be
overshadowed by custom functions in python as well. The only reason
that we don't assume those two python lines could mean *anything* is
because those mechanisms are not well advertised, and most people
don't have a need to shadow "import."

Of course, it is possible to abstract away all of the lisp in such a
way that you have a completely new programming language. But is this
really a problem? Perl5 and bash are abstractions of C. Python has
been implemented on both C and java, and chunks of perl6 have been
implemented on top of Haskell. Completely new programming languages
and frameworks develop their own base of knowledge and expertise
separate from the parent language.

My point isn't whether or not their claims are correct (a "couple" of
macros? really?) but that things like this feed the perception that Lisp
is close to that hypothetical language where anything could be anything.
If anything could be anything, do you really know what (+ 1 2) means
without reading every line of code?

How do you know that operator has not been shadowed or overloaded in
python? In most cases what you have to do is trust both the
underlying implementation, and the author(s) of the libraries you
use.

Both python and lisp share mechanisms to protect namespaces is part
of the answer. So one way to protect myself is to run my code in
it's own namespace, and require explicit addressing of imported
functions and constructs. In this way I can be pretty certain that
(+ 1 2) uses the lisp primitives, and has not been shadowed, while
(SOMEPACKAGE:+ 1 2) should be treated with suspicion.

The process of writing lisp often involves access to the REPL, so I
can have lisp expand the definition of any expression:
CL-USER> (macroexpand '(+ 1 2))
(+ 1 2)
NIL
CL-USER> (macroexpand '(loop for i upto 100 collect i))
(BLOCK NIL
...<expansion snipped for brevity>...
)
T
CL-USER>

This tells me that + is not a macro in my current environment. While
loop is.

And at least one of the powerful aspects of lisp is than since lisp
programs are nested stets of s-expressions, you can design something
that walks through the code and highlights all macro calls, and
foreign functions that shadow native functions. I don't how
difficult this would be for languages such as python.

.... extended argument snipped ...

Or maybe it is only an advantage while Lisp programmers are a
self-selected group of above-average skill. Wait until fifty million VB
code monkeys start writing Lisp macros and maybe, just maybe, you'll wish
they were using a less powerful and more restrictive language.

Perhaps it's because I'm a social scientist and not a programmer by
training, but I find many arguments for *technical* solutions to
*human performance* problems to be rather weak as a general
practice. In some cases, using a very restrictive language may be
the best solution for the problem. However, there are plenty of
other ways around the problem that can be tailored to special needs.

In addition to sandboxing macros into packages and walking the code
to detect macros, you could run a custom lisp image that does not
permit DEFMACRO or enforces documentation on DEFMACRO. Or you could
rely on social mechanisms like only using libraries from trusted
sources, and better training of those code monkeys.

 

[Bill Atkins]

Fast. Very fast!

You hit it on the head. Interpreted langauges were the efficiency
breakthrough we've all been waiting for.

 

[Ken Tilton]

Now, if you want to tell me that, despite all the talk, Lisp coders don't
actually create new syntax or mini-languages all that often, that they
just use macros as functions, then the question becomes: why do you need
macros then if you are just using them as functions? Why not use functions?

Macros let you write what amounts to functions that don't evaluate
their arguments. Think of the endless clpy wars over the ternary
conditional operator.
[snip]

That is trivial to do with a macro

I know that. It was more of a rhetorical question -- Lispers are either
trying to emphasis the radical nature of what you can do with macros, or
understate it and make them seem just like functions.

Yep, both. The first is rare. CLOS is one, my Cells (ported this summer
to PyCells as part of SoC 2006) is another. The latter is the norm.

ken

--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon

 

[Ken Tilton]

The day has not yet arrived that nobody ever needs to edit code in a
plain, vanilla text editor.

Gee, 200kloc of Lisp and I have not got there yet. Keep banging that
drom, Steve.

ken

--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon

 

[Bill Atkins]

Actually, yes, sometimes it is useful to print code out and read it on the
train, or in the bath, without the advantage of syntax highlighting,
pretty-printing, parenthesis-balancing or code folding. Not necessarily as
pleasant as having all those things, but it is nice that working Python
code is, by definition, already formatted correctly for pretty printing.
Even if you're stuck on some god-forsaken Windows PC with just Notepad,
you can still read Python code.

Now, *writing* Python code with Notepad isn't as easy, but it is still
doable. How about Lisp code?

Of course we can read Lisp without IDE's; we look for indentation cues
just as you guys do.

That's not a criticism of Lisp exactly, but a reminder to think about not
just what problem you're trying to solve, but what resources you will have
to solve it. If you *know* that you're going to need to edit code by
ssh-ing across an high-latency connection to a machine without Emacs, then
Lisp will probably not be the best solution.

Oh no? Emacs lets you save and write files over an SSH connection,
even if you end up in the strange straw man situation you're
describing ("yeah, but can you edit Lisp if your keyboard doesn't have
paren keys?").

 

[Bill Atkins]

I don't even know what that means.

And yet you continue to post as if you know Common Lisp...

 

[Kirk Sluder]

So it is good that English restricts the expressiveness and power of the
syntax and grammar. While we're talking English, we can both understand
each other, and in fact people who redefine words and ignore the common
meaning of them are often covering weaknesses in their arguments.

Ohh, can the guy who does discourse analysis for a (meager) living
respond to this?

To start with, English does not restrict the expressiveness and
power of the syntax and grammar. People who use the English language
in specific communities and in specific forms of discourse do. The
key to how this happens occurs on another layer of how language
works which is almost always left out of internet discussions of
language: pragmatics.

Here is the way it works on usenet. You write, "...English
restricts the expressiveness and power of the syntax and grammar."
And I write, "You should probably do a bit of casual reading into
human linguistics before you make such silly statements."

In general however, there is this great mechanism embedded in human
communication called "feedback." Statements that are unclear or
misunderstood are met with a blank stare, "huh?" or "???" depending
on medium, mode and context variables.

The same goes for programming languages. Extra expressiveness comes at the
cost of reduced communication between programmers -- the code becomes
harder to read for those who haven't already learnt how to read it.

Well, extending this analogy to programming languages, the same
social mechanisms are in effect. In addition to the technical syntax
of computer languages, there is also a fair amount of pragmatics
involved. Lisp code is expected to conform to a lisp style, and
python code is expected to conform to a python style. Developments
that run counter to that style are more likely to be rejected than
developments that conform to that style.

http://www.python.org/doc/essays/styleguide.html
http://www.lisp.org/table/style.htm

 

[Steven D'Aprano]

You could maybe give another example: how would one realize something
like (memoize function) in Python?

By spending thirty seconds googling:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/325205
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/498110

If your needs are very simple, something as simple as this will do it:

def memoize(func):
def f(arg, _cache={}):
if _cache.has_key(arg):
return _cache[arg]
t = func(arg)
_cache[arg] = t
return t
return f

Or (defmethod name :after ..)?

I don't even know what that means. Would you like to translate?

 

[Ken Tilton]

If that's the best example of what macros can be used for, frankly I'm
unimpressed.

We're shocked.



ken

--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon

 

[Steven D'Aprano]

Well, of course there is. There are always going to be patterns in
the code you write that could be collapsed. Language has nothing to
do with it; Lisp without macros would still be susceptible to
boilerplate.

Here's a concrete example:

(let ((control-code (read-next-control-code connection)))
(ecase control-code
(+break+
(kill-connection connection)
(throw :broken-by-client))
(+status+
(send-status-summary connection))
((+noop+ +keep-alive+))))
;; +X+ indicates a constant

Eight lines, excluding the comment, and it doesn't handle the case where
control-code is not one of +break+, +status+, +noop+ or +keep-alive+,
although the ecase macro does. And how many lines of code is ecase?

All of that
boilerplate is handled by the macro. In Python, I would need to do
something like:

control_code = connection.read_next_control_code()
if control_code == +break+:
connection.kill()
throw blah
else if control_code == +status+:
connection.send_status_summary()
else if control_code == +noop+ || control_code == +keep_alive+:
else:
error "CONTROL_CODE fell through conditional cascade; was not one of +BREAK+, +STATUS+, +NOOP+, +KEEP_ALIVE+"

Your Python syntax is rather wonky, but that's incidental.

Nine lines, including handling the case where control_code is none of the
four constants. Ten if you add the "pass" statement that it actually
needs. And it is completely self-contained, with no external functions or
macros to understand.

Saving one line of code, at the expense of having another block of code to
write or understand -- is that really the best example of what macros are
used for in practice? You don't really save writing any boilerplate code,
except for else clause, unless you're claiming that "if" and "elif" is
boilerplate. Fine, you claim them as boilerplate. I'm going to claim all
those unnecessary brackets as boilerplate.

Yes, I know the parser needs them. But as so many people keep telling me,
once you've been writing Lisp code for a month, you don't even notice the
brackets. That makes them unnecessary for the developer, and therefore
something the computer should handle on its own. You've already split
expressions with whitespace, why should you have to use brackets as well?
That's just boilerplate.

Okay, I'm impressed that ecase can pick up on the four constants being
tested against, and feed their names (rather than merely their values)
into an error message. Python has nothing like that, and if you only have
three or four things to test against, *and* they have names, that could be
a useful thing to do. And if I've understood correctly, that's more a
feature of Lisp's symbols than of macros.

But if you're testing against fifty different values, well, is it really
useful for your error message to list all fifty names? Or do you have
another macro ecase-with-lots-of-tests?

And how does ecase handle the more general case of testing against
calculated objects rather than named constants? Or is there yet another
macro for that?

If that's the best example of what macros can be used for, frankly I'm
unimpressed. Yes, I can see some benefit. But I don't see that the benefit
is worth the added complexity. Maybe there are more complex tasks that
macros are better suited for.

 

[Rob Warnock]

+---------------
| >I think an editing program that balances parens automatically is near
| >indispensible for writing Lisp code. I can't stand writing Lisp
| >without Emacs.
|
| And that is why I will never write Lisp. I loathe Emacs.
+---------------

An appropriate version of "Vi" is sufficient. I prefer "nvi-1.79",
but have found that the "vim" at work is also adequate. Either will:

1. Flash matching parens on insert; jump back & forth between matching
parens on command ("%").

2. Deal with S-exprs as "objects" for shifting left/right, copying,
deleting, or pasting. [The shifting works best when you gloablly
"set sw=1" and then use "." to repeat the last shift.]

3. Minimal auto-indent, but "good enough" for my purposes [given #2].

That's pretty much all you need to code in Lisp. It's what *I* use.
So "I loathe Emacs" is *NOT* a believable excuse for avoiding Lisp...


-Rob

 

[Rob Warnock]

+---------------
| (If you want symbols to be case sensitive and default case to be lower
| then most Lisp's allow that, but the ANSI spec dosn't give a standard way.)
+---------------

What about (setf (readtable-case *readtable*) :invert)? That's in ANSI.


-Rob

 

[Steven D'Aprano]

As I type each right parens I eyeball
its partner as the editor highlights it to make sure I have not missed
anything,

Er, weren't you one of the people claiming that you don't notice parens
when you're reading or writing Lisp code?

Oh yes, there is this:

Subject: Re: merits of Lisp vs Python
From: Ken Tilton <[email protected]>
Newsgroups: comp.lang.lisp,comp.lang.python
Date: Sat, 09 Dec 2006 02:29:56 -0500

Yeah, you are, you just did not use it heads down for a month. The way
to tell if you spent enough time on Lisp is to look at Lisp code. If you
see any parentheses, you have not spent enough time. They disappear in a
month.
[end quote]

So, if you're still seeing parentheses, does that mean you've been coding
Lisp for less than a month?

 

[Steven D'Aprano]

It's not impossible or terribly difficult to write Lisp that way, it's
just unpleasant, once you've gotten used to doing it with editors that
automatically indent and balance parens.

Er, an editor that does that is not a plain, vanilla text editor.

 

[Ken Tilton]

Er, weren't you one of the people claiming that you don't notice parens
when you're reading or writing Lisp code?

Steve, you seem to be doing everything you can to make what is basically
a decent cultural exchange unpleasant, mostly by bellowing like some mad
rogue elephant over and over again about macros, now by trying to play
gotcha.

I said they disappear. They do. I do not see parens when I look at code.
The "watching of the matching" is only after furious thrashing during
refactoring, and even then it is unconscious most of the time because we
are usually also indenting as we go.

I presume you look to see if your indentation is correct while you are
indenting, with no more effort than I eyeball the highlighted parens.

You think that some gotcha is somehow going to miraculously reveal that
I do in fact have trouble with parentheses? Now /that/ would be dumb of you.

Lisp for less than a month?

I see you figured it our for yourself. Obviously I love parentheses and
would hate to program without them, so obviously your gotcha falls short.

I myself am simply delighted. I decided I should indeed check out if I
was up against the c.l.p village idiot. Nope, turns out you are the
Master of FUD:

http://www.itwire.com.au/content/view/7359/53/

"According to Steven D'Aprano, operations manager of longstanding open
source services firm Cybersource, Microsoft's new game of attempting to
strike fear into the heart of Linux users by invoking the spectre of
patent infringements may soon take an ominous new twist. Mr D'Aprano
believes that Microsoft may well use similar tactics employed by its
proxy the BSA six years ago when it raided the offices of guitar string
maker Ernie Ball with armed marshalls and found a few machines with
non-compliant software."

I love it, you turning FUD on the FUDster, MS. Good for you! A whole
article with nothing but your unsubstantiated guesses as to Microsoft
suing someone! I do not who impresses me more, you giving such an
interview or the reporter who printed it... no, you guys are allowed to
screw up, the editor should be shot.

Hey, Steve, save that crap for Microsoft, will you?

peace. ken


--
Algebra: http://www.tilton-technology.com/LispNycAlgebra1.htm

"Well, I've wrestled with reality for thirty-five
years, Doctor, and I'm happy to state I finally
won out over it." -- Elwood P. Dowd

"I'll say I'm losing my grip, and it feels terrific."
-- Smiling husband to scowling wife, New Yorker cartoon