merits of Lisp vs Python

[Alex Mizrahi]

(message (Hello 'Paul)
(you :wrote n '(11 Dec 2006 04:14:20 -0800))
(

??>> optimizing language. i think it's called trampolined style.
??>> example can be found in PAIP book: interpreter of Scheme is
implemented in

PR> This book doesn't seem to be online. But anyway I think you mean,
PR> compile the Scheme code into one giant CL function, which is no longer
PR> really a clean mapping of Scheme to CL, it's just writing an
PR> interpreter. And I think that interpreter has to do its own
PR> management of environments rather than letting the CL runtime do it.
PR> So basically it says CL is turing-complete and can do everything,
PR> which we already knew ;-).

point is that writting some custom interpreter in CL is very easy AND you
can still use all other features of CL AND communicate with other DSLs.

PR> the different macros being used. But I think I understand one part of
PR> what was confusing me before: your call/cc macros depend on a
PR> nonstandard feature of some CL implementations.

no, it's standard Common Lisp. it's an ARNESI library. it's very standard --
i was able to run non-modified code in Armed Bear Common Lisp implementation
(it's running on JVM), that is very new and does not fully support the
standard -- but nevertheless, CPS code runs out of the box on it.

PR> You can't write a call/cc macro in standard CL--you instead have to
PR> write something that transforms the entire program.

yes, i cannot write just call/cc -- i should wrap code into with-call/cc.
with-call/cc macro performs transformation of the code to CPS -- and then
call/cc is enabled.
i can enable call/cc for specific functions -- i define them with defun/cc
instead of defun. so, i can define both normal CL functions and call/cc
enabled functions. typically, call/cc is required only in a few pieces of
program where interrupts come, so it's OK.

if i'm going to support generators, i only need to run CPS tranformation on
the body of generator function -- that will tear it in yield points.

PR> I think even with TCO though, you still can't do coroutines with CL
PR> macros (but you can do them with real call/cc). The point is that you
PR> have to do something like TCO on calls that are not tail calls and
PR> actually have to return. You again have to transform the whole
PR> program, not just expand a call/cc macro locally inside functions that
PR> use it.

i think only functions that are coroutines should be fully transformed.

)
(With-best-regards '(Alex Mizrahi) :aka 'killer_storm)
"People who lust for the Feel of keys on their fingertips (c) Inity")

 

[=?ISO-8859-1?Q?Andr=E9_Thieme?=]

Eight lines, excluding the comment, and it doesn't handle the case where
control-code is not one of +break+, +status+, +noop+ or +keep-alive+,
although the ecase macro does. And how many lines of code is ecase?


Your Python syntax is rather wonky, but that's incidental.

Nine lines, including handling the case where control_code is none of the
four constants. Ten if you add the "pass" statement that it actually
needs. And it is completely self-contained, with no external functions or
macros to understand.

Counting source lines is not the important thing. Look at the complexity
and the number of tokens.
The Lisp code is easier, because it is more abstract. You are closer in
saying what you want.

Counting tokens:
(let ((control-code (read-next-control-code connection))) 4
(ecase control-code 2
(+break+ 1
(kill-connection connection) 2
(throw :broken-by-client)) 2
(+status+ 1
(send-status-summary connection)) 2
((+noop+ +keep-alive+)))) 2
======
16


control_code = connection.read_next_control_code() 4
if control_code == +break+: 4
connection.kill() 2
throw blah 2
else if control_code == +status+: 4
connection.send_status_summary() 2
else if control_code == +noop+ || control_code == +keep_alive+: 8
else: 1
error "CONTROL_CODE fell through conditional cascade; 2
was not one of +BREAK+, +STATUS+, +NOOP+, +KEEP_ALIVE+"
======
29
plus a "pass", would make 30 vs 16 "Brain-units".


André
--

 

[Paul Rubin]

PR> the different macros being used. But I think I understand one part of
PR> what was confusing me before: your call/cc macros depend on a
PR> nonstandard feature of some CL implementations.

no, it's standard Common Lisp. it's an ARNESI library. it's very standard --

The ARNESI doc at:

http://common-lisp.net/project/bese...onverting_a_Subset_of_Common_Lisp_to_CPS.html

begins:

By transforming common lisp code into CPS we allow a restricted form
of CALL/CC. While we use the name call/cc what we actually provide is
more like the shift/reset operators (where with-call/cc is reset and
call/cc is shift).

I don't know what shift/reset are but the above says that call/cc in
the Scheme sense is not really implemented. There are also a bunch of
limitations on what can be in the call/cc body, e.g. you can't use
unwind-protect.

Anyway, the nonstandard feature is tail call optimization, if you're
turning continuations into closures, and that's just for the simple
case of dealing with loops written in CPS. E.g.: if you write

(defun sum_to_n (n &optional (acc 0))
;; (sum_to_n n) computes sum of integers from 0 to n
(if (= n 0)
0
(sum_to_n (- 1 n) (+ n acc))))

This function is properly tail recursive (unless I messed up) so with

(sum_to_n 10000000000)

the CPS version is supposed to return the right answer if you wait
long enough. It is not allowed to overflow the stack. This is
guaranteed to work in Scheme but is not guaranteed in CL. And as I
think we agree, with coroutines it gets even worse.

 

[Paul Rubin]

(defun sum_to_n (n &optional (acc 0))
;; (sum_to_n n) computes sum of integers from 0 to n
(if (= n 0)
0
(sum_to_n (- 1 n) (+ n acc))))

Of course meant:

(defun sum_to_n (n &optional (acc 0))
;; (sum_to_n n) computes sum of integers from 0 to n
(if (= n 0)
acc
(sum_to_n (- 1 n) (+ n acc))))

I've made that same error several times in Haskell as well. I'm not
used to this.

 

[philip.armitage]

(e-mail address removed) ha escrito:

I cannot understand why. It is like if you claim that packaging things
in boxes is difficult to learn.

HTML and XML have more brackets than LISP (usually double) for
structuring data and everyone has learned HTML.

I think maybe you missed the point I was making.

To make it clearer I'm saying that the arguments that are being made
over and over again against Lisp in this thread have been the
antithesis of my experience since moving from Python to Lisp.

I just prefer personal experience to popular misconceptions

Phil

 

[=?ISO-8859-1?Q?Andr=E9_Thieme?=]

With Lisp macros, even that isn't guaranteed. Now, if Lispers would say
"Oh yes, macros give you great power, and with great power comes great
responsibility. Be careful."

Well, macros are one (big) thing that Lisp has and which many other
languages don't have. Their are other things too, and some of them are
in Python as well, which is a very nice scripting language.

Often macros save just some bits of code. Saving one loc is not much you
might say. But think about it the other way around.
How would you like it to call doodleShooble() each time before you use
the if statement? Of course you would not like it. The good thing about
Lisp is, that you can eliminate this pattern.

Apropos pattern.. most design patterns are not (very) visible in Lisp.
Many of them can be abstracted away with macros+functional programming.


André
--

 

[=?ISO-8859-1?Q?Andr=E9_Thieme?=]

In addition to macros that define classes or methods, a common macro
is the WITH-* macro, which sets up some kind of context, runs the body
inside that context, and then does some cleanup.

For example, my Lisp vendor, LispWorks, provides the macro MP:WITH-LOCK :

(mp:with-lock (*the-big-lock*)
(do-something-atomic)
(something-else)
(almost-odone))

The generated first seizes a process lock called *THE-BIG-LOCK*, runs
the code in the body and then releases the lock. I never have to
worry that I've taken a lock without releasing it, because LispWorks
has encoded that behaviour into MP:WITH-LOCK (and if they handn't,
this macro is trivial to write).

Now I can tell if I'm keeping a lock around too long because all the
code that appears inside this WITH-LOCK is all the code that I'm
locking. Further, the generated code is surrounded by an
UNWIND-PROTECT, which means that if an error is raised or anything
abnormal happens in the dynamic scope of the UNWIND-PROTECT, Lisp will
run cleanup code as it flees back up the stack. So if there is an
error in my code, it does not prevent other processes from seizing
that lock, because it will be released as the error is signaled.

Here is the expansion:

CL-USER 7 > (write (macroexpand '(mp:with-lock (*the-big-lock*)
(do-something-atomic)
(something-else)
(almost-odone)))
retty t :case :downcase)
(let ((#:g17553 *the-big-lock*))
(when (mprocess-lock #:g17553)
(unwind-protect
(progn (do-something-atomic) (something-else) (almost-odone))
(mp::in-process-unlock #:g17553))))

Now it happens what Graham described. The blub programmer will explain
that this is not needed, because he can do the same thing in blub.

In Python you could write a function withLock that takes a lock and a
function.
Then you can say
def throwAway001():
doSomethingAtomic()
somethingElse()
almostOdone()

withLock(theBigLock, throwAway001)


or maybe

withLock(theBigLock,
lambda ():
doSomethingAtomic()
somethingElse()
almostOdone())

So Lisp saves you the "lambda ():" or "def throwAway001():".
That is nice I would say. Why write it if the computer could do that for
you?


André
--

 

[=?ISO-8859-1?Q?Andr=E9_Thieme?=]

Macros let you write what amounts to functions that don't evaluate
their arguments. Think of the endless clpy wars over the ternary
conditional operator. You want to write something like

def ternary(test, iftrue, iffalse):
if test: return iftrue
else iffalse

but because of side effects, you don't want

a = cond(test, f(x), g(x))

to evaluate both f and g. That is trivial to do with a macro but
can't be done with a function.

I think you could do that with functional programming.
You can protect the evaluation by encapsulating the args in a function
object?


def f_Args(x):
return x

def g_Args(x):
return x


and then
a = cond(test, f, g, f_Args(x), g_Args(x))

if you adopt cond. But of course it is getting ugly.
So a macro can free you from this extra code.


André
--

 

[Bill Atkins]

Yes. I changed my opinion on advocating Python having macros in one
of our long threads on the subject. Maintainance counts.

Yes, it does, but that should take you to exactly the opposite
conclusion.

This is new to me. In fact, for the compiled languages available to me.
Using them *first* would be the difficult choice.

These are not real sentences, but if you're saying that compiled
languages make programming more difficult, then you're simply using
the wrong compiled languages. Lisp is a dynamic language that also
supports compilation to native code.

Unlike Lisp, Python does not have a ubiquitous compiler. It is
therefore
made to interface nicely with compiled languages. Other compiled

What on earth does this mean? You're saying that because Python
doesn't have a compiler, it can interface more easily to compiled
languages? That's nonsense.

Further, most Lisp implementations support an interface to C that
doesn't require you to write and compile C code in order to use C
extensions in Lisp. Can Python do the same more "nicely" than Lisp?

language users see the need for dynamic interpreted languages like
Python and maintain links Python such as the Boost Python C++
wrapper. IronPython for .NET, Jython for Java.
Lisp is its own interpreter and compiler, which should be a great
advantage, but only if you don't make the mistake of ignoring the
wealth of code out there that is written in other languages.

Um.

 

[Bill Atkins]

There's no way you could compile Python to efficient
machine code just by macro expansion. You'd also need
some very heavy-duty type inferencing.

When I used to use Ruby a lot, I believed this line that the Ruby
community fed itself (and apparently Python feeds itself as well):
Ruby/Python has to be interpreted because it's too dynamic. This is
wrong, of course. A compiler shifts a lot of decisions that an
interpreter would have to make at runtime to compile-time. There is
no reason a dynamic language can't enjoy this efficiency. On the
other hand, if Python is doing a hash lookup on every function call,
as Alex Mizrahi claims, compilation may not do much to smooth over
such awkwardness.

Python is extremely dynamic, even more so than Lisp.
That's why compiling Python is hard, not because it
doesn't have macros.

Uh huh. "More so than Lisp"? Just making stuff up now?

Despite its dynamism, Lisp is quite compilable. For example, I can
redefine classes, functions, macros, etc. at runtime and compiled code
referring to the old code will still work. You are conflating
dynamism with interpretedness, and that's incorrect.

 

[Bill Atkins]

When moving a set of statements in Python, you
are usually selecting a set of complete lines,
cutting them out and then pasting them in
between two other lines somewhere else.

You're missing Ken's point, which is that in Lisp an s-expression
represents a single concept - I can cut out the second form of an IF
and know that I'm cutting the entire test-form. I don't have to
choose the correct "set of complete lines" to correctly move code
around.

Having edited both Lisp and Python code fairly
extensively, I can't say that I find editing
Python code to be any more difficult or error
prone.

How extensively?

On the plus side, Python makes less demands on the
capabilities of the editor. All you really need
is block-shifting commands. Bracket matching is
handy for expressions but not vital, and you
certainly don't need bracket-based auto-indenting.

Oh, please. So we should restrict the power of the languages we
choose just to make sure that our code can be edited in Notepad?

 

[Juan R.]

(e-mail address removed) ha escrito:

I think maybe you missed the point I was making.

Yes i did, sorry

To make it clearer I'm saying that the arguments that are being made
over and over again against Lisp in this thread have been the
antithesis of my experience since moving from Python to Lisp.

I just prefer personal experience to popular misconceptions

I often count 'parentheses' used in other approaches.

E.g. the LISP-based

[HTML [@:XMLNS http://www.w3.org/1999/xhtml]
[HEAD
[TITLE Test page]]
[BODY]]

is SLiP (Python)

html(xmlns="http://www.w3.org/1999/xhtml"):
head():
title(): "Test page"
body():

LISP-based:
5 (
5 )
1 @
1 :

Python:
4 (
4 )
1 =
4 "
4 :

 

[Kay Schluehr]

Now, speaking as a scientist, permit me to make a small practical
suggestion: Why not just figure out a way to simplify some brand of
Python -- make parens (or whatever) optionally replace whitespace and
line breaks as syntax -- and then add a simple macro facility -- macros
are actually a very simple extension if you have homogenous syntax,
homogenizing your syntax to the point where macros are possible is the
hard part -- and just see what happens.

The problem is not so much to add a macro facility ( or source
transformer ) but to soften it and make the macro facility actually
*simple* to use. Note that you don't need to replace whitespaces. The
Python parser is LL(1) and whitespace is almost completely abstracted
away once the Python source was tokenized successfully ( there is
exactly one non-terminal in the Python grammar, the "suite" NT, that
uses INDENT and DEDENT as moral equivalents for left- and right parens
). Note also that a homogenous syntax is not that important when
analyzing parse trees ( on the contrary, the more different structures
the better ) but when synthesizing new ones by fitting different
fragments of them together.

There are two basic approaches. In one you start with a piece of source
code ( a template ) and enhance the source description slightly with
somewhat like template parameters that keep source fragments and expand
them. In this case you also need an unquoting mechanism or a way to
evaluate code within the template. The second approach acts with high
level wrappers of more low level source trees ( just like ASTs are high
level wrappers of concrete syntax trees in most languages ). In Python
one might even use operator overloading to hide the AST structure and
provide a more convenient syntax.

But this only describes the transformational aspect. The definitional
part is not less important. I try to consider a grammar description of
a full programming language as as script in an own little language (
actually it is and the grammars grammar is often just an EBNF grammar
description ). This however presumes that enhancing grammars to enhance
languages is a reasonable approach not just in a Lex-Yacc ( or ANTLR )
setting. The next question concerns compositionality of language
enhancements or composition of even completely independent language
definitions and transformers both on source and on binary level. While
this is not feasible in general without creating ambiguities, I believe
this problem can be reduced to ambiguity detection in the underlying
grammars.

One of two general things are
likely to happen: Either people will not use them, and they will
languish and die, and then at least you can say; "Been there, done
that" to the Lisp community. Or, more likely, the some subset of the
Python community will get it, and will figure out how useful they are,
although it might take some time. And then you might find yourself with
a wonderful new tool.

I think so too.

You might even get a compiler out of the deal, at
a pretty low cost, too! If you get macros, and get a compiler, I'm
pretty sure that you will have no problem winning over the Lisp
community, who would LOVE to have your extensive libraries, and that
you will probably be able to maintain and improve your flagging
position wrt Ruby (which, according to Matz, is more-or-less just Lisp
w/o macros.)

Just a moment ago you called programmers of other languages "flies"
which I found annoying and now you offer LOVE in big letters? I don't
even think the "competition" to Ruby matters. Once an easy to use
metaprogramming system could be done for Python it could be ported with
some adaptions to other languages with more "complicated syntax" ( non
LL(1) parsable ).

Kay

 

[Paddy]

conclusion.

I won't duplicate the arguments against macros made elsewhere in the
thread.

languages make programming more difficult, then you're simply using
the wrong compiled languages. Lisp is a dynamic language that also
supports compilation to native code.

Lisp was not a compiled language available to me, and even after my use
of Cadence Skill, I would not consider Lisp for writing an extension
unless Lisp had a library close to what I wanted, and there was a good
way to link
Python to the compiled Lisp code.

doesn't have a compiler, it can interface more easily to compiled
languages? That's nonsense.

No. I am saying that *because* it does not have a compiler, it has been
*made to* integrate nicely with compiled languages; and further, I am
saying that because some compiled language package maintainers see the
advantages of using dynamic languages, they support Python integration.

Further, most Lisp implementations support an interface to C that
doesn't require you to write and compile C code in order to use C
extensions in Lisp. Can Python do the same more "nicely" than Lisp?

Python does the same. It might well be nicer but I do not know how Lisp
does this.
http://docs.python.org/dev/lib/module-ctypes.html
http://www.boost.org/libs/python/doc/
http://www.python.org/pypi/pyobjc/1.3.5
(The last is used within Apple for some aspects of development).
The above list is not exhaustive

Um.

Yep.

- Paddy.

 

[Paul Rubin]

When I used to use Ruby a lot, I believed this line that the Ruby
community fed itself (and apparently Python feeds itself as well):
Ruby/Python has to be interpreted because it's too dynamic.

I don't think you can reasonably compile Python just by what we'd
usually call macro expansion. You need fancier compiler techniques.

Uh huh. "More so than Lisp"? Just making stuff up now?

Python is more dynamic than Lisp.

Despite its dynamism, Lisp is quite compilable.

Yes. Lisp is dynamic, but less so than Python. And not by
coincidence, Lisp is more compilable than Python.

For example, I can redefine classes, functions, macros, etc. at
runtime and compiled code referring to the old code will still work.
You are conflating dynamism with interpretedness, and that's
incorrect.

If you say foo.frob() in Python, that's supposed to look up 'frob' in
a dictionary hanging off of foo. You can modify the contents of this
dictionary any time you want. The Lisp equivalent would be some
generic function (frob foo) that you define with CLOS in the usual
way, but then there's some hashtable that lets you redefine frob at
any time by modifying it (i.e. just a normal hashtable that you poke
with setf, no special notification to the class system). This can
happen anywhere in the code, in another thread, or whatever. You can
even replace that hashtable with another hashtable. You can also
insert (at any time) a __getattr__ method into foo's class, that is a
user-supplied function that replaces the hash lookup.

This stuff is all quite hard to optimize the way CLOS can be
optimized. I'd like to hope Python tones down these aspects of its
dynamism as it continues to evolve.

 

[Marc 'BlackJack' Rintsch]

Once an easy to use metaprogramming system could be done for Python it
could be ported with some adaptions to other languages with more
"complicated syntax" ( non LL(1) parsable ).

FYI: Here's how Nemerle does macros: http://nemerle.org/Macros

I guess you can't really transform Nemerle into a completely different
language, but it is at least interesting to see such a feature in language
with a more complex syntax than Lisp.

Ciao,
Marc 'BlackJack' Rintsch

 

[Espen Vestre]

If you say foo.frob() in Python, that's supposed to look up 'frob' in
a dictionary hanging off of foo. You can modify the contents of this
dictionary any time you want.

You can redefine CLOS methods at run time any time you like, so this
doesn't make Python more /dynamic/ than CLOS. Maybe you should replace
"more dynamic" with "less managable", if that's what you mean?

 

[Jan Dries]

Oh, please. So we should restrict the power of the languages we
choose just to make sure that our code can be edited in Notepad?

Perhaps not. The use of a decent editor seems a fair requirement for any
language. But one of the things that I dislike about Java, .NET and to
some extent XML (XML Schema for instance), is that the only way to
really be productive in these languages/environments is to use tools
that generate or otherwise manage huge amounts of code for you based on
whatever GUI settings. If the language is so complex or verbose that you
can't really use it without a GUI tool, then why bother having a
language in the first place. Furthermore these tools are typically
expensive and to run comfortably they require more processing power and
memory than the lightweight ultra-portable type laptops that I like so
much can provide.

I can't speak about Lisp, but the great thing about Python, IMHO, is
that you can get quite far with not much more than Notepad. I find this
important because I find GUIs a very tedious and ineffective way to
describe whatever it is that I am trying to implement.

Perhaps I'm just getting old ...

Regards,
Jan

 

[Kay Schluehr]

FYI: Here's how Nemerle does macros: http://nemerle.org/Macros

I guess you can't really transform Nemerle into a completely different
language, but it is at least interesting to see such a feature in language
with a more complex syntax than Lisp.

Ciao,
Marc 'BlackJack' Rintsch

Hi Mark, there are quite a lot of meta programming systems ( MPS ) for
non Lispy languages ( O'Caml, Haskell, Java of course and also Dylan
as a member of the "Lisp family" with non homogenous syntax ). I hope I
will find time in the new year to review and compare them to the
grammar based approach I described in the grandparent post and follow
myself with "EasyExtend" for Python - which is *radical* and somewhat
in between a language specific MPS and Lex/Yacc. The idea to separate
the MPS from the host language but providing a multi-language framework
is somewhat complementary to that of PyPy that is a framework that
supports several backends for one language.

 

[Paul Rubin]

You can redefine CLOS methods at run time any time you like, so this
doesn't make Python more /dynamic/ than CLOS. Maybe you should replace
"more dynamic" with "less managable", if that's what you mean?

Can you redefine CLOS methods without calling CLOS functions that tell
the object system what to expect (so it can do things like update the
MRO cache)? I.e. can you redefine them by poking some random
dictionary? You can in Python. I don't claim that's a good thing.