Python syntax in Lisp and Scheme

[David Eppstein]

But let us consider cutting lines 6 and 7 and putting them
between lines 21 and 22. We get this:

15 while stack:
16 directory = stack.pop()
17 for file in os.listdir(directory):
18 fullname = os.path.join(directory, file)
19 files.append(fullname)
20 if os.path.isdir(fullname) and not os.path.islink(fullname):
21 stack.append(fullname)
6 print "- last accessed:", time.ctime(atime)
7 print "- last modified:", time.ctime(mtime)
22 return files

But it is unclear whether the intent was to be outside the while,
or outside the for, or part of the if.

[/QUOTE]

When I cut and paste code, I know what my intent is, and I use the
block-indent (shift-left or shift-right) features of my text editor to
achieve that intent.

 

[David Eppstein]

Ok so in Python a function can DEF another function in its body. I assume
this can be returned to the caller. When you have a nested DEF like that,
is the nested function's name globally visible?

Not by default, but it can be made to be via an appropriate "global"
declaration.

 

[Kaz Kylheku]

That is a case of bringing all of the function into the present
context so it can be mixed with the closures manually created there.

A macro controls how much material is inlined and how much isn't.

What I said here is silly because a HOF can also be structured to have
inlined and noninlined parts: an inlined skeleton function that calls
non-inlined functions.

A macro can have its own binary interface between the expanded
material and some material in a library that is associated with the
macro.

The macro decides what forms need to be made into a closure that is
passed to the library and which are not.

This is the real point; in a Lisp macro, material from parameters is
usually spun into closures precisely because inlining is not wanted;
it's a compromise which allows some remote function to call back into
the present lexical environment, allowing the macro to generate less
inlined bloat.

In functional languages with lazy evaluation, evey expression is
effectively a closure. When you call some function f with argument x,
that x is effectively a lambda function which returns the value of x
in its original environment.

When the argument to a function is an expression, it need not be
evaluated prior to the call; it may be delayed until the last possible
moment when its value is actually needed, and it may never be
evaluated at all.

So it seems as if higher order functions have the evaluation control
of macros. But delaying evaluation is not the same as control over
evaluation. Moreover, a macro is not just some mechanism for delaying
the evaluation of expressions; it's a mechanism for assigning an
arbitrary meaning to expressions.

A HOF can generate code only by interpolating parameters into a fixed
body. The meaning of everything stays the same. A HOF can trivially
replace a macro under two conditions:

1. The macro is used as an interface to hide lambdas from the user.
This is not necessary if the language has lazy evaluation. Every
expression is already a lambda, capable of being passed to a foreign
environment and evaluated there, with all the hooks to the original
environment.

2. The macro is used as a trivial inlining mechanism, which
substitutes expressions into a template. HOF inlining effectively does
the same thing. Since evaluation is lazy, there is little or no
semantic difference between evaluating parameters and calling a
function, or inserting the unevaluated expressions into the body and
inlining it.

In Common Lisp, we don't have the lazy evaluation semantics, so
certain macro uses that fall under 1. are essential. People who argue
against these always use HOF examples from another programming
language. But CL programmers don't want to switch to another
programming language. You can't switch programming languages while
holding everything else constant.

Certain macro uses in Lisp that seem to fall under 2. are also
essential; some macros that do nothing more than substitute their
arguments into some functional template nevertheless have imperative
semantics, such as conditional or repeated evaluation of some forms.
When Lisp programmers demonstrate these uses, again, they are
countered by examples from a different programming language. ``Gee, if
you only had virtual sequences (or whatever gadget), you wouldn't need
that imperative DO- stuff.'' But now you are no longer talking about
just HOF's, but HOF's plus virtual sequences.

In general, for every macro use, you could design a programming
language feature, such as higher order functions, lazy sequences and
whatnot, and then claim that the macro use is unnecessary when you
have that feature. The problem is that there is no end to this
progression. Macros can implement anything you want (arbitrary syntax
to semantics mapping), and that anything can always be given a name
and codified in some offshoot academic language under a coat of
hard-coded syntax, and you can then claim that you eliminated macros
once and for all for all situations that you (and by implication
everyone else) cares about.

 

[Alexey Dejneka]

... and now you'll have to clarify what you mean by "defun doesn't
nest" ...

The context of this was in accessing closures from function defs.
Maybe you can call defun from within another function, but it won't give
you a closure.[/QUOTE]

Really?

* (defun outer (x)
(defun inner ()
x))
OUTER
* (outer 3)
INNER
* (inner)
3
* (outer 11)
STYLE-WARNING: redefining INNER in DEFUN

INNER
* (inner)
11


--
Regards,
Alexey Dejneka

"Alas, the spheres of truth are less transparent than those of
illusion." -- L.E.J. Brouwer

 

[Raffael Cavallaro]

In functional languages, a lot of higher order functions are used for
iterating over some container datastructure. E.g. map,iter,foldl,zip,...

As long as the same idiom is not used over and over again, in the same
way, this is fine. Once you find that you are repeating the same idiom,
I think it's time to name that idiom descriptively, and make it a named
function or macro.

A lambda expression parameter of such functions plays the
same role as a body of a "while" or "for" loop in C.
Do you think that your criterions should be applied to blocks
of codes in C also?

The problem with C is that it lacks a true macro system (as opposed to a
simple token substitution system). This means that there will be cases
where there is a frequently used idiom that could be abstracted in a
lisp macro, but not in a C macro.

But, insofar as it is possible to abstract out repetition and unecessary
implementation detail, it should be done in any language that supports
named functions. Better still if the language supports macros for those
cases that would require them.

 

[Raffael Cavallaro]

Raffael Cavallaro

I don't know why but I feel like trying to summarize.

Nice summary.

Your position now appears to have
changed to state that lambdas are ok to use, but their use should be
restricted.

First, let me say that I'm not delusional enough to think that a few
Usenet posts by me are going to change the way anyone codes. I do find
Usenet most useful for clarifying my own thinking, however. Nothing
forces you to be precise more than having to defend, amplify, and
clarify your ideas in the light of other peoples comments. If that means
that my stated position appears to have changed, then fine. The whole
point of discussion is for people come away from it with a clearer idea
of what both they and others really mean to say. If one's ideas are
never modified in the slightest by discussion, then one isn't really
listening. I would say that my position is the same - I still have the
same tastes in source code style. However, I think that this back and
forth has made my views much more precise (I'm pretty sure that I've
never written up 2 explicit criteria for the appropriate use of
anonymous functions before, for example).That increased precision is a
good thing.

But when you're writing small functions and
everybody else is writing 300+ line functions you begin to wonder if
it is you that is doing something wrong. It is nice to see that other
people actually do think about how to write and structure good code.

You should dowload squeak (the open source Smalltalk implementation) and
browse the code The squeak browser, like most Smalltalk code browsers,
is quite nice. Realizing that the entire functionality of such a complex
dynamic environment is composed of an interconnected set of thousands of
methods, few of them more than a dozen lines in lenght, can be a kind of
revelation if one hasn't seen a whole system written in that style.

 

[Terry Reedy]

"Terry Reedy" <[email protected]> wrote in message

(e-mail address removed) writes:
Since having the correct amount of whitespace is *vital* to the
correct operation of a Python program, it seems that the task of
maintaining it is made that much more difficult because it is only
conspicuous by its absence.

That remembers me that when the languages had significant

spaces,
the

programming was done with forms, sheets of physical paper preprinted
with empty spaces:

[further idiocy snipped]

I don't know why you categorize it as idiocy.

The 'it' that I snipped was an ASCII mockup of a Fortran coding sheet
or something and the implied suggestion that we Pythoneers are
Neanderthals who should write our programs on such, instead of using
Python-aware computer editors. Very funny, not.

A month and two ago, some Lispers posted polite and intelligent
suggestions on c.l.py that Pythoneers not familiar with Lisp might
want to take a real look. I bit and finally read the used Winston and
Horn LISP book that I bought about 10 years ago. However, the recent
spate of garbage-dumping has lead me to question whether I should even
look at CL. It seems that c.l.lisp would not be a friendly place to
ask questions.

I, having no experience whatsoever with Python, find it hard to believe that
indentation is easy without "("s and "{"s. (Yes, yes, I know, it can't be so
hard. I've read the other posts, but I find it hard to believe

nonetheless.)

??? Don't you use indentation when writing Lisp?
It is even easier without adding redundant fences.
Learn something before 'sharing' such useless beliefs.

In the next paragraph you ask people not to be judgmental and here
you are doing it yourself.

I specifically suggested, and suggest, that Lispers, including you,
follow the posted suggestion of fellow Lispers that people not finally
judge the burdensomeness of a syntactic feature without actually using
it enough to experience the corresponding benefit, so one can make a
balanced judgment. Calling a stupid insult 'idiotic' is another
matter entirely ;-)

I don't know what you've meant by "white-space" here.

If you look more carefully at both my paragraph and the one near the
top copied from
Mr. prunesquallor's post, you might discover that 'whitespace', a
programming term dating back 30 years old or more, occurs in a phrase
quoted from Mr. P's post. So ask him.

Removing white-space from (1 2 3) gives me (123). I agree, very

different,

Which is exactly my point. Thank you for agreeing. This supports the
main point of my post, which is that attacking Python for having
critical whitespace is logically also an attack on Lisp for the same
reason.

but noone knowing the language would consciously do this, not even a

newbie.

Ditto for Python newbies and removing commas or whitespace where
needed.

....

In Lisp, apart from the whitespace-where-required, whitespace is
insignificant. You can write (1 2 3)
as well as (1 2 3) and get the same meaning.

And in Python, (1, 2, 3) == (1,2,3). So what?

So we Lispers don't need to use any forms.

Neither do we Pythoneers -- and it was idiotic to suggest otherwise.

Terry J. Reedy

 

[David Eppstein]

The context of this was in accessing closures from function defs.
Maybe you can call defun from within another function, but it won't give
you a closure.

Really?[/QUOTE]

Apparently I was mistaken, at least for modern lisps. I haven't
programmed in lisp for a long time; the most recent lisp manual I can
find at hand is a 1981 Chinual, which still seems to be using dynamic
scoping and requires an explicit function call to create a closure.

 

[Joe Marshall]

However, the recent spate of garbage-dumping has lead me to question
whether I should even look at CL. It seems that c.l.lisp would not
be a friendly place to ask questions.

I'll apologize for the posts of mine that drifted over into
comp.lang.python they were unintentional.

We're not deliberately unfriendly here in c.l.l, but about once or
twice a month someone comes into the group and makes claims about the
illegibility of parenthesis. We do get tired of this.

Of course it is as unreasonable to bitch and moan to python people
about whitespace in comp.lang.python. I've haven't intentionally
started any arguments in comp.lang.python about it, but I'm afraid
that I haven't been as vigilant in watching the followups as I should
have been.

If you look more carefully at both my paragraph and the one near the
top copied from Mr. prunesquallor's post, you might discover that
'whitespace', a programming term dating back 30 years old or more,
occurs in a phrase quoted from Mr. P's post. So ask him.

Whitespace in Lisp is defined as

whitespace n. 1. one or more characters that are either the graphic
character #\Space or else non-graphic characters such as #\Newline
that only move the print position. 2. a. n. the syntax type of a
character that is a token separator. b. adj. (of a character) having
the whitespace[2a] syntax type[2]. c. n. a whitespace[2b] character.

Other non-graphic characters usually include #\Tab, #\Form, #\Return,
#\Linefeed, and sometimes #\VT and #\NUL.

different,

Which is exactly my point. Thank you for agreeing. This supports the
main point of my post, which is that attacking Python for having
critical whitespace is logically also an attack on Lisp for the same
reason.

I was being imprecise. Since Python treats intertoken whitespace
essentially the same as Lisp, there is no logical argument that can
include one and exclude the other. My concern revolves around
delineating code blocks through creating common prefixes on the
associated lines, *and* that these prefixes are hard to see (although
it is *not* hard to see that they exist, nor hard to see if two
adjacent ones are the same or different, but that it *is* hard to see
exactly how many of them are there in deeply indented code).


I'll remove further followups to comp.lang.python

 

[Edi Weitz]

Apparently I was mistaken, at least for modern lisps. I haven't
programmed in lisp for a long time; the most recent lisp manual I
can find at hand is a 1981 Chinual, which still seems to be using
dynamic scoping and requires an explicit function call to create a
closure.

Fair enough. But then we should compare this Lisp to the state of
Python around 1981...

Edi.

 

[Tim Hochberg]

Fair enough. But then we should compare this Lisp to the state of
Python around 1981...

Nope, with python in 2010 -- at that point, Python will be about as old
as Lisp was in '81.

-tim

 

[Pascal Costanza]

> In article <[email protected]>,

car and cdr is that they _don't_ mean anything specific.

>
>
>
> gdee, you should read early lisp history ;-). car and cdr ha[d|ve] a
> very specific meaning

Yes, but noone (noone at all) refers to that meaning anymore. It's a
historical accident that doesn't really matter anymore when developing code.


Pascal

 

[Pascal Costanza]

> Because these hard-coded syntaxes are prettier than the extensible Lisp
> syntax full of parentheses.

Obviously, you haven't tried to actually program in Lisp. Getting used
to the syntax is a matter of one or two weeks at most of actual coding,
provided you use a decent editor that supports parenthesis highlighting
and automatic indentation.

Despising Lisp because of its syntax is like thinking that Japanese is a
too complicated language because it doesn't use latin characters.


Mind you, I had more or less exactly the same objections to Lisp's
syntax about one and a half year ago. Now, I don't see any language out
there that nearly provides the same level of usability than Lisp,
exactly because of its regular syntax "full of parentheses". It's far
prettier and easier to read than languages whose syntaxes are based on
some unreflected, purely aesthetical choices.


Pascal

 

[ketil+news]

the last paragraph, which you site above, stand in strange contrast to the
remainder of the post.

Well, the post he replied to, contained:

| No one is talking about need [to name functions], but about clarity
| of exposition.

| It is perfectly possible to program functionally in lisp, as I'm sure
| you know. It just makes code less readable to use _anonymous_ functions.

I interpreted this rather as a blanket statement, and one I happen to
disagree with. Perhaps Raffael meant that anonymous functions *can*
make code less readable *sometimes*, which we all seem to agree on so
vehemently?

A bit later, you say in <[email protected]>

| i am trying only to understand the implications of an argument
| which, at least as stated, rather unequivocally deprecates bindings.

-- a point of view I don't quite see where you picked up. It is
certainly not one I've seen advocated seriously by anybody. And, as
you say:

there is no need to overstate some elses position in order to, in the end,
make the same point.

-kzm

 

[David Mertz]

|it is *not* hard to see that they exist, nor hard to see if two
|adjacent ones are the same or different, but that it *is* hard to see
|exactly how many of them are there in deeply indented code).

Here's a quick rule that is pretty damn close to categorically true for
Python programming: If you use more than five levels of indent, you are
coding badly. Something is in desperate need of refactoring.

I did a scan of my Gnosis Utilities, which the tool SLOCCount' reports
as having about 7500 lines of Python code. Using a quick custom script,
I find that I use more than depth five a total of four times--all of
them within a class definition. Maybe that means I should do a little
refactoring, but even five levels is pretty unusual.

Visually distinguishing among four or five possible indent levels is
extremely easy. The nonsensical non-problem about indecipherable
indents just simply never happens in useful code.

Yours, David...

--
mertz@ | The specter of free information is haunting the `Net! All the
gnosis | powers of IP- and crypto-tyranny have entered into an unholy
..cx | alliance...ideas have nothing to lose but their chains. Unite
| against "intellectual property" and anti-privacy regimes!
-------------------------------------------------------------------------

 

[Alexander Schmolck]

EXACTLY! The issue is that the inconsistent state in lisp is rarely
a legal expression, but an inconsistent state in Python often is.

In case you misinterpreted the above: I meant in *both* python and lisp. The
likelihood of inconsistency in terms of a legal expression that doesn't do
what the superficial observer might think (which wasn't what I meant above) is
certainly considerably greater in lisp than in python (simply because python's
syntax places far more constraints on what a certain parse-tree can look like
as actual code).

I've also tried my best to show that for the examples you presented there is
no need to take an intermediate editing step in pyhthon that involves a
semantically valid but unintended state which could plausibly figure as a
source of errors.

'as

 

[Michele Dondi]

Well it certainly _can_ be formalized. (Have you any experience
with _axiomatic_ Euclidean geometry? Not as in Euclid - no pictures,

No, I have no experience with it. Just heard it exists: I presume
you're talking about the work of Hilbert... but then I'm not sure that
it provides a full formalization! It's clear though that it brings one
step forward towards formalization.

nothing that depends on knowing what lines and points really are,
everything follows strictly logically from explictly stated axioms.
Well, I have no experience with such a thing either, but I know
it exists.)

[should have read ahead...]

Whether the formal version would be totally incomprehensible
depends to a large extent on how sophisticated the formal
system being used is - surely if one wrote out a statement
of Euler's theorem in the language of set theory, with no
predicates except "is an element of" it would be totally
incomprehensible. Otoh in a better formal system, for
example allowing definitions, it could be just as comprehensible
as an English version. (Not that I see that this question has

In any case I'm sure too that *that particular proof* can be
formalized! But, even if I am under the impression that most proofs,
for some meaning of "most", would be affected by the concern expressed
above, somehow I feel like the effect with this one would be "one
order of magnitude" stronger, to say the least!

any relevance to the existence of alleged philosophical
inconsistencies that haven't been specified yet...)

In fact, it has no relevance to that matter: I thought that my
introduction should have made that clear. My comment is something I
happend to think about sometimes; your words just reminded me of it
and this seemed to be the right chance to talk about it. No more than
that, no more than a naive comment...


Michele
--

Comments should say _why_ something is being done.

Oh? My comments always say what _really_ should have happened.
- Tore Aursand on comp.lang.perl.misc

 

[Darius]

I thought it was obvious that I was not advocating eliminating anonymous
functions entirely, since I said they should be used inside named
function bodes.

To be completely clear, I'm advocating only using anonymous functions
when two conditions are met:

1. When the functionality provided is _unique_. If that which the
anonymous function is performing is also done elsewhere, especially if
it is done in several different places throughout the code, then that
anonymous function should be recast as a descriptively named function or
macro.

2. If it is necessary to show, at that _particular_ source location,
_how_ a piece of functionality is provided. If the reader doesn't need
to know _how_ things are implemented in that _particular_ source
location, we should use a descriptively named function or macro instead.

The reason people are attacking your posts is because the above has
NOTHING to do with anonymous functions. This advice should be
followed independent of anonymous functions.

1. When the functionality provided is _unique_. If that which the
code is performing is also done elsewhere, especially if
it is done in several different places throughout the code, then that
code should be recast as a descriptively named function or
macro.

2. If it is necessary to show, at that _particular_ source location,
_how_ a piece of functionality is provided. If the reader doesn't need
to know _how_ things are implemented in that _particular_ source
location, we should use a descriptively named function or macro
instead.

Further in response to 2, limiting the use of anonymous functions to
only named functions that are showing how things are implemented
doesn't limit them at all! You are virtually always in a named
function and you are always showing how to implement something. 2 is a
null statement. Nevertheless, what (I presume) you were intending
(something along the lines of 'create and use abstraction barriers'
again has nothing to do with anonymous functions or at least no
special attachment to them.

Finally, all this advice (generalized to include all code) is a)
obvious to anyone who has been programming for more than half a day,
and b) quite standard software engineering advice. The reason why
your position is being extended beyond your intent is simply because
people presumed from your statements of the obvious and focus on
anonymous functions that you were attempting to state something other
than the obvious. To end this post, I feel I should bring special
attention to the fact that people over 1.98 meters should be wary of
dangerous things.

 

[james anderson]

the last paragraph, which you site above, stand in strange contrast to the
remainder of the post.

Well, the post he replied to, contained:

| No one is talking about need [to name functions], but about clarity
| of exposition.

| It is perfectly possible to program functionally in lisp, as I'm sure
| you know. It just makes code less readable to use _anonymous_ functions.

I interpreted this rather as a blanket statement, and one I happen to
disagree with.

i suggest that the interpretation is not reasonable in the context of the
remainder of that message. i do not quote it here, and will spare one the
explication, but suggest, as one has done for me, that the interested reader
reexamine the text in question for their own edification.

Perhaps Raffael meant that anonymous functions *can*
make code less readable *sometimes*, which we all seem to agree on so
vehemently?

that is the reasonable interpretation of his position, expecially in light of
his later posts, but also given the original text.

A bit later, you say in <[email protected]>

| i am trying only to understand the implications of an argument
| which, at least as stated, rather unequivocally deprecates bindings.

to cite the most inexplicable paragraph from the post to which i was
referring, there is a passage

"Why do you insist on naming *functions*? You could equally well say that
every list should be named, so you would see its purpose rather than its
contents. Perhaps every number should be named, so you can see what it
represents rather than its value. You could say that each statement of
a compound statement should be moved to a separate function, so you can
see what it does by its name, not how it does it by its contents. It's
all equally absurd."

which proceeds from an unfounded attribution through equally unfounded
suppositions, to an absurd conclusion.

-- a point of view I don't quite see where you picked up. It is
certainly not one I've seen advocated seriously by anybody. And, as
you say:

given the rhetorical quality of the paragraph which i cite above, one might
suppose that the reader overlooked it.

....

 

[Raffael Cavallaro]

The reason people are attacking your posts is because the above has
NOTHING to do with anonymous functions. This advice should be
followed independent of anonymous functions.

You're misreading those who have argued against me. They seem to think
that this advice should _not_ be followed in the case of anonymous
functions. I.e., the anonymous function camp seem to think that
anonymous functions are such a wonderfully expressive tool that they are
more clear than _actual desriptive function names_.

I agree with you; this advice should be followed, period (well, it is
_my_ advice, after all). But advocates of a particular functional style
think it is perfectly alright to use the same unnamed functional idiom
over and over throughout source code, because functional abstractions
are so wonderfully expressive. They think it is just fine to expose the
implementation details in code locations where it is completely
unnecessary to know the implementation specifics.

How else can this be construed?

A name is an extra level of indirection. You must follow it to be
100% sure what the function means, or to understand what does it really
mean that it does what it's named after. The code also gets longer - not
only more verbose but the structure of the code gets more complex with
more interdependent parts. When you have lots of short functions, it's
harder to find them. There are many names to invent for the writer and
many names to rememner for a reader. Function headers are administrative
stuff, it's harder to find real code among abstractions being introduced
and used.

In other words, don't use names _at all_ if you can avoid them. Just
long, rambling tracts of code filled with anonymous functions. After
all, you'd just have to go look at the named function bodes anyway, just
to be _sure_ they did what they said they were doing. (I wonder if he
disassembles OS calls too, you know, just to be sure).

Really I personally think they are often just enamored of their own
functional l33tness, but you'll always have a hard time convincing
programmers that their unstated preference is to write something so
dense that only they and others equally gifted in code decipherment can
grasp it. Many programmers take it badly when you tell them that their
job should be much more about communicating intent to other human beings
than about being extremely clever. As a result of this clever agenda, an
unmaintainably large proportion of the code that has ever been written
is way too clever for its own good.