Joachim Durchholz said:
You're right, and made me rethink what's actually disturbing me about
macros.
Perhaps it's that I have to adapt to dual-mode (two-tier?) thinking:
I have to reason about both what the macros are doing and what the
software is doing. Alternatively, I could consider the macros as
"part of the language" and not reason about the macro code but about
their effects - in which case I have effectively augmented the
language by all macros that are in use.
I think you're right that you have to adopt a dual-mode of thinking.
When you're writing macros you're essentially extending the
compiler/language to recognize constructs that would otherwise be
meaningless.
Then, having written them, you use them as if they were in the
language all along. At one level this is no different from extending a
language by writing a function. That is, if you're working in a
language (like Common Lisp, Scheme, C, or--I imagine--any FPL) where
much of the "language" itself is implemented in terms of built-in
functions, if you write a new function you're extending the language
and after you've written it you can forget about the details of how it
works and just use it. The only difference between macros and
functions is that macros operate by generating code which then
performs actions as opposed to *being* code that performs actions. But
that level of indirection makes it easy to express things that would
otherwise be difficult (in my experience.)
Personally, I'd still prefer a compiler that's evaluating constant
expression.
Hmmm. If it will make you feel any better, macros are just fuctions
whose domain and range happens to be Lisp expressions. That happen to
be run by the compiler. So eventually the compiler is evaluating
constant expressions, just some of them were automatically derived
from the written source.
Is there anything that a macro does that can't be done by
preevaluating data structures that contain functions (or closures)?
At first glance, I'd say no, but then I don't know what macros are
used for in practice.
Well it depends whether you consider syntax to be "anything". I think
it was you who objected to one of my examples by saying, "that's just
syntactic sugar". Macros can (and many do) do large amount of
under-the-covers bookkeeping. For instance here are a few rules from a
grammar for a lexer for Java source code:
(defprod line-terminator () (/ #\newline (#\return (? #\newline))))
(defprod white-space () (/ #\space #\tab #\page line-terminator))
(defprod input () ((* input-element) (? #\Sub)))
(defprod input-element () (/ white-space comment token))
(defprod token () (/ identifier java-keyword literal separator operator))
DEFPROD is a macro that squirrels away the stuff on the right which is
an s-expy form of BNF. The rest of the grammar is more of the same. At
the bottom of the grammar file where the productions are diffined I
have this form:
(deflexer java-lexer (input)
(
tokens identifier java-keyword literal separator operator)))
That DEFLEXER call (another macro) expands into a single parsing
function built out of all the productions created by DEFPROD calls,
appropriately wired together and embedded into code that takes care of
the stepping through the input and gather up values, etc. And that
function is compiled into extremely efficient code because all the
intercommunication between productions goes through lexical variables.
And the file containing these calls to DEFPROD and DEFLEXER is legal
Lisp source which I can feed to the compiler and get native machine
code back.
So I don't know if that is "anything" or not. I don't know how I would
write such a thing in Haskell, et al. but I know this is a *lot*
cleaner than what *I'd* be able to do in Java, Perl, Python, or C.
Um, well, yes, there is one thing that macros can do: extending
syntax in ways that aren't part of the original language syntax.
E.g. replacing all those parentheses by indentation, or something
similar un-Lispish. (Extending syntax in such ways is a mistake
IMHO, but YMMV. Anyway, I'm more interested in the question if
there's any /semantics/ that can be done via macros but not via
compile-time evaluation.)
Actually, changing the syntax is--if one thinks one must--is really
done by read-macros which are quite different. But most Lispers agree
with you--there's just not enough benefit to changing the syntax to be
worth it. Except for occasionally making a new syntax for expressing
certain frequently created literal objects that otherwise would
require a much more verbose creation form. (Someone gave a great
example the other day in another thread of an airline reservation
system (Orbitz I think) that has a special kind of object used to
represent the three-letter airport codes. Since they wanted to always
have the same object representing a given airport they needed to
intern the objets with the TLA as the key. But rather than writing
(intern-airport-code "BOS") everywhere, they wrote a reader macro that
let them write: #!BOS. Since this was an incredibly common operation
in their system, it was worth a tiny bit of new syntax. But note,
again, that's not *changing* the syntax so much as extending it.)
I believe it's not DEFMACRO that's complicating things, it's the
macros that it allows (see above).
Fair enough. But do you object to the ability to write new functions
on the grounds that that just means you have a lot of new functions to
learn and that complicates things needlessly? That's obviously a
rhetorical question but I am actually curious why you find them
different, if you do.
Agreed on all accounts (except that I don't know how "multilingual"
Lispers really are *g*).
Well, there is the problem that once folks find Lisp they tend to stop
looking for better things because what could be better than Lisp. ;-)
But most Lispers take a fairly circuitous path to Lisp and hit a bunch
of other languages before they find it.
Does anybody have a keyword-style list of useful applications of the
macro facilities?
"99% of all cases" is a pretty good argument actually
It's just that Pascal doesn't (want to) believe that it's enough for
type checking. His problem, not mine...
Agreed. I'm getting more and more convinced that it's not language
size or KISS issues that's setting me off, it's that "two-tier
thinking" that I (perhaps mistakenly?) associate with macros.
The funny thing is to me, when you say "two-tier thinking" that
perfectly describes how I think about the process of making
abstractions. Regardless of the *kind* of abstraction one is creating,
one has to be facile at switching mental gears between *building* the
abstraction and *using* it. You are probably so used to doing this
when writing functions that you don't even notice the switch. But
because macros are a bit strange you *notice* the switching and it
annoys you. I suspect that anyone who's capable of building functional
abstractions would--if they actually used macros--quickly learn to
switch gears equally smoothly when writing and using macros.
I wouldn't want HM typing if it were just beautiful theory.
HM typing happens to be a beautiful theory. Things are getting less
beautiful once you interact with the Real World (TM), which is
stateful - OTOH, Real World is a mess, so don't expect computing to be
beautiful anymore when there is interaction with it *g*. What
surprised me is how much of a computation can be separated from such
interaction. With the proper framework, one can even describe
interaction patterns (which are themselves stateless), feed these
patterns to the framework, and watch in amazement how the execution
engine follows these patterns. It's the kind of abstractive facility
I've been yearning for decades...
Lisp could do this just as well. It's just not done because taking the
shortcut and doing stateful computations directly is so much easier.
(And I don't pretend that functional languages are doing this kind of
thing perfectly right now. I think the potential in these ideas is
just beginning to be exploited - and what's available is already quite
impressive.)
Actually I'm trying to understand macros and macro usage right now,
without having to learn all the details of CL (which would be a bit of
overkill - I know it might not be enough, but then my time is limited
so I'm doing my best within the available budget).
Sure. Cheers.
-Peter
