For Kenny Tilton: Why do I need macros revisited.

[Chris Reedy]

For everyone -

Apologies for the length of this message. If you don't want to look
at the long example, you can skip to the end of the message.

And for the Python gurus among you, if you can spare the time, I
would appreciate any comments (including words like evil and disgusting,
if you think they are applicable :-}) on the example here.

Kenny -

I asked my question about macros with some malice aforethought. This
is a topic I've thought about in the past.

I sympathize with Andrew on this. I had to study this and the following
code for awhile before I figured out what you are doing. However, I do
have an advantage, I know lisp and the macro system and just how
powerful the macro system is.

(defun get-cell (self slotname) ;; this fn does not need duplicating
(let ((sv (slot-value self slotname)))
(typecase sv
(function (funcall sv self))
(otherwise sv))))

(defmethod right ((self box)) ;; this needs duplicating for each slot
(get-cell box right))

But I just hide it all (and much more) in:

(defmodel box ()
((left :initarg :left :accessor left)
(right :initarg :right :accessor right)))

....using another macro:

(defmacro defmodel (class superclasses (&rest slots))
`(progn
(defclass ,class ,superclasses
,slots)
,@(mapcar (lambda (slot)
(destructuring-bind
(slotname &key initarg accessor)
slot
(declare (ignore slotname initarg))
`(defmethod ,accessor ((self ,class))
(get-cell self ',slotname))))
slots)))

Ok. The following is more lines of code than you have here. On the other
hand, I think it does about the same thing:

<Example>

class cellvalue(object):
def __init__(self):
self._fn = None
def _get_value(self, obj):
if hasattr(self, '_value'):
return self._value
else:
value = self._fn
if callable(self._fn):
value = value(obj)
self._value = value
return value
def _set_value(self, value):
self._value = value
def _del_value(self):
del self._value
def _set_fn(self, fn):
self._fn = fn
if hasattr(self, '_value'):
self._del_value()

class modelproperty(object):
def _init_cell(self, obj):
obj.__dict__[self.name] = cellvalue()
def __get__(self, obj, cls):
if obj is None:
return self
else:
return obj.__dict__[self.name]._get_value(obj)
def __set__(self, obj, val):
obj.__dict__[self.name]._set_value(val)
def __delete__(self, obj):
obj.__dict__[self.name]._del_value()
def setfn(self, obj, fn):
obj.__dict__[self.name]._set_fn(fn)
def setname(self, name):
self.name = name
self.__doc__ = 'Model Property '+str(name)

class modeltype(type):
def __init__(cls, name, bases, dict):
super(modeltype, cls).__init__(name, bases, dict)
modelprops = []
for attr, decl in dict.items():
if isinstance(decl, modelproperty):
decl.setname(attr)
modelprops.append(attr)
if modelprops:
__originit = getattr(cls, '__init__')
def _modeltype_init(self, *args, **kw):
for attr in modelprops:
getattr(self.__class__, attr)._init_cell(self)
if __originit is not None:
__originit(self, *args, **kw)
setattr(cls, '__init__', _modeltype_init)
.... __metaclass__ = modeltype
.... x = modelproperty()
.... def __init__(self, x=None):
.... self.__class__.x.setfn(self, x)17

I think that has most of the behavior you were looking for. As you can
see from the example, I leaned (I'm not sure leaned is a strong enough
work ) on the newer capabilities for metaclasses and descriptors.
(And learned a lot about exactly how they work by writing this up!)

</Example>

Having looked at the two pieces of code, the only thing that struck me
about how they're used is that the lambda expression needed in the
Python version is clunkier than the version in the Lisp version. On the
other hand, that might be addressed by a somewhat more elegant syntax in
Python for constant code blocks, something that's been mentioned by
others in recent messages.

Even though the Python setup (with two classes and a metaclass) is
longer than the Lisp version, I'm not sure it's any clunkier or harder
to understand.

So back to my original question, why do I want macros in Python?

Let me provide a candidate answer and rebuttal:

The real reason I want to do macros in Lisp is that they allow me to
easily write new custom languages. Would a sufficiently powerful macro
facility in Python allow me to do this? I suspect that the answer to
this question would only be yes if that included some sort of parser
generator facility.

Expanding on that: One of the nice things about Python (at least for
those like me who like the language) is the clean syntax. However, that
comes at a price: Needing a real parser to parse the language.

Lisp on the other hand has an extremely simple syntax that doesn't
require a real parser. That allows me to create whole new "syntaxes" in
lisp, since I'm not really changing the syntax, just changing how a
given set of S-expressions are interpreted.

On the other hand, if I'm going to go to the complexity of including a
parser generator so I can generate my own custom languages, it sounds to
me like I'm about to reproduce what the Perl/Parrot people are up to. (I
admit that I'd really like the time to look at what they're doing more
deeply.)

Which brings me back to my original question: Would a macro facility in
Python really buy me anything? And, in view of Alex's arguments, would
that benefit outweigh the potential significant costs in other areas?

Chris

 

[Kenny Tilton]

For everyone -

Apologies for the length of this message. If you don't want to look at
the long example, you can skip to the end of the message.

And for the Python gurus among you, if you can spare the time, I would
appreciate any comments (including words like evil and disgusting, if
you think they are applicable :-}) on the example here.

Kenny -

I asked my question about macros with some malice aforethought. This
is a topic I've thought about in the past.

I sympathize with Andrew on this. I had to study this and the following
code for awhile before I figured out what you are doing. However, I do
have an advantage, I know lisp and the macro system and just how
powerful the macro system is.

(defun get-cell (self slotname) ;; this fn does not need duplicating
(let ((sv (slot-value self slotname)))
(typecase sv
(function (funcall sv self))
(otherwise sv))))

(defmethod right ((self box)) ;; this needs duplicating for each slot
(get-cell box right))

But I just hide it all (and much more) in:

(defmodel box ()
((left :initarg :left :accessor left)
(right :initarg :right :accessor right)))

....using another macro:

(defmacro defmodel (class superclasses (&rest slots))
`(progn
(defclass ,class ,superclasses
,slots)
,@(mapcar (lambda (slot)
(destructuring-bind
(slotname &key initarg accessor)
slot
(declare (ignore slotname initarg))
`(defmethod ,accessor ((self ,class))
(get-cell self ',slotname))))
slots)))

Ok. The following is more lines of code than you have here. On the other
hand, I think it does about the same thing:

<Example>

class cellvalue(object):
def __init__(self):
self._fn = None
def _get_value(self, obj):
if hasattr(self, '_value'):
return self._value
else:
value = self._fn
if callable(self._fn):
value = value(obj)
self._value = value
return value
def _set_value(self, value):
self._value = value
def _del_value(self):
del self._value
def _set_fn(self, fn):
self._fn = fn
if hasattr(self, '_value'):
self._del_value()

class modelproperty(object):
def _init_cell(self, obj):
obj.__dict__[self.name] = cellvalue()
def __get__(self, obj, cls):
if obj is None:
return self
else:
return obj.__dict__[self.name]._get_value(obj)
def __set__(self, obj, val):
obj.__dict__[self.name]._set_value(val)
def __delete__(self, obj):
obj.__dict__[self.name]._del_value()
def setfn(self, obj, fn):
obj.__dict__[self.name]._set_fn(fn)
def setname(self, name):
self.name = name
self.__doc__ = 'Model Property '+str(name)

class modeltype(type):
def __init__(cls, name, bases, dict):
super(modeltype, cls).__init__(name, bases, dict)
modelprops = []
for attr, decl in dict.items():
if isinstance(decl, modelproperty):
decl.setname(attr)
modelprops.append(attr)
if modelprops:
__originit = getattr(cls, '__init__')
def _modeltype_init(self, *args, **kw):
for attr in modelprops:
getattr(self.__class__, attr)._init_cell(self)
if __originit is not None:
__originit(self, *args, **kw)
setattr(cls, '__init__', _modeltype_init)
... __metaclass__ = modeltype
... x = modelproperty()
... def __init__(self, x=None):
... self.__class__.x.setfn(self, x)17

I think that has most of the behavior you were looking for. As you can
see from the example, I leaned (I'm not sure leaned is a strong enough
work ) on the newer capabilities for metaclasses and descriptors.
(And learned a lot about exactly how they work by writing this up!)

<g> looks a lot like the code I was writing when I began a Python port
of my Cells project. I'll be open-sourcing the Lisp version soon, you
can do the Python port.

You are absolutely right. Metaclasses are killer. I am surprised
Pythonistas afraid of macros let them into the language! I actually had
a metaclass implementation of Cells until I decided to release the
source. The MOP is not part of the standard, and it shows across
implementations. Hell, MCL does not even expose a MOP.

</Example>

Having looked at the two pieces of code, the only thing that struck me
about how they're used is that the lambda expression needed in the
Python version is clunkier than the version in the Lisp version.

You will be delighted to know that one of Lisp priesthood sees no need
for macros since what they do can be done with lambdas. Not sure if that
is so where one is generating multiple top-level forms from one clause.
Me, I like hiding implementation details as mush as possible, and having
just one place to go when changing how something works.

As for the lambda being clunkier, un-hunh, and if you decide to change
the mechanism after a lot of them have been coded, perhaps passing a
second argument, here comes the mega-edit. Worse, what happens when the
toy hack reaches maturity and you get the final version of the macro:

this:
(c? (+ 10 (left self)))

expands to:
(make-c-dependent
:code '((+ 10 (left self)))
:rule (lambda (c &aux (self (c-model c)))
(+ 10 (left self))))

One funny thing is that even if one open codes that, the readers you all
are so worried about still do not know what C-DEPENDENT is.

Now picture a make-instance with five of those in a row. The text widget
font is a function of a user preference, all four dimensions are a
function of the font size and string length, the color may be a function
of sysntax judgment for highlighting, etc etc. Now you cannot see the
semantics for all the redundant, essentially meaningless wiring.

So back to my original question, why do I want macros in Python?

Hiding the wiring, where a function will not do because the argument is
your source code.

Let me provide a candidate answer and rebuttal:

The real reason I want to do macros in Lisp is that they allow me to
easily write new custom languages.

No, that is not a big reason. It is cool that one could do that readily
if one stuck to the basic sexpr notation, but my experience to date is
that I do not need a wholly new language as long as I can drop into new
syntax (c?...) periodically to get what I would have created in a wholly
new embedded language.

Which brings me back to my original question: Would a macro facility in
Python really buy me anything? And, in view of Alex's arguments, would
that benefit outweigh the potential significant costs in other areas?

Lispniks have not seen code become unreadable because of macros. That is
something you all (well, most) are imagining. You are almost positing it
as given. And it is simply not so. We have pointed out again and again
that macros are no harder to understand than functions or classes, but
no one addresses that point. Some say "I want to see all the legal
Pythion instructions". Except when it is in a function, I guess.
Strangely, quite a few of you have also conceded macros can leverage a
language.

Well, it seems like we have covered everything pretty thoroughly. Back
to RoboCup!

--

kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker

 

[Jacek Generowicz]

http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-3.html#%_sec_
Temp_4
] As Scheme became more widespread, local dialects began to
] diverge until students and researchers occasionally found it difficult
] to understand code written at other sites.

That has probably more to do with the fact that Scheme is
minimalistic. And if you have a minimalistic langugage, you end up
having to implement a lot of utilities before you can get on with what
you are actulally trying to achieve (or have your language
implementation provide them as extensions).

Schemers criticize Common Lisp for being too big. The consequence of
having a big language (a la CL, not a la C++) is that most things that
"everybody" needs are already in the language. In Scheme "everybody"
ends up re-inventing the things that "everybody" needs, separately,
leading to fragmentation. In CL, everybody agrees on how to implement
the things that "everybody" needs: you use the ones defined by the
standard. In this way, any macro-magic is restricted to the domain
specific corners of your application.

In summary, I reckon that any inter-Scheme incomprehensibility or
Scheme fragmentation results from starting with a language that
doesn't offer enough out of the box.

 

[Paul Foley]

Terry Reedy pointed out something from some Scheme documentation
found by Dave Kuhlman:
http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-3.html#%_sec_
Temp_4
] As Scheme became more widespread, local dialects began to
] diverge until students and researchers occasionally found it difficult
] to understand code written at other sites.

I cannot say if that's because of macros or other aspects of Scheme.

Scheme, as such, is barely a language. It's pretty useless on its own.
Most of the functionality of an actual Scheme /implementation/ is
extra-standard stuff (i.e., not Scheme), which, naturally enough,
every implementor does differently.

[Scheme didn't even have macros until R4RS (1991?), and then only as
an appendix...]

What about Terry's comment, based on the above URL:
) Reading the third referenced page on Macros, I notice that the amount
) of syntax definition for the macro sublanguage is as large as a
) substantial portion (one-third?) of that for core Python (if condensed
) to the same density). So, just by definitional bulk, having it in the
) language would not be a free ride.

Scheme macros are not the same as Lisp macros. Scheme macros are a
lot more complicated (and less useful; but most Scheme implementations
also offer Lisp-style macros...)


--
Just because we Lisp programmers are better than everyone else is no
excuse for us to be arrogant. -- Erann Gat

(setq reply-to
(concatenate 'string "Paul Foley " "<mycroft" '(#\@) "actrix.gen.nz>"))

 

[Terry Reedy]

Scheme, as such, is barely a language. It's pretty useless on its own.
Most of the functionality of an actual Scheme /implementation/ is
extra-standard stuff (i.e., not Scheme), which, naturally enough,
every implementor does differently.

Core Python, while useful on it own, is also pretty sparse. So I
gather it would be something like having multiple Python
implementations (of comparable popularity) with different sets of
builtins and overlapping but incompatible 'standard' libraries. The
Python community is fortunate to have avoided that.

....

Scheme macros are not the same as Lisp macros. Scheme macros are a
lot more complicated (and less useful; but most Scheme implementations
also offer Lisp-style macros...)

Please excuse my confusing of Scheme with Lisp. I am pleased to find
that my difficulty in reading the Scheme macro stuff wasn't just me.
But note that there have been proposals that if Python were to have a
macro facility, it should be 'hygenic', which I presume means like
Scheme macros.

Until some advocate of a Python macro facility adds more detail to the
proposal beyound 'Lisp/Scheme-like maco facility', it is really hard
for someone like me, who has used neither, to have much of any
concrete idea of what is being proposed.

Terry J. Reedy

 

[David Mertz]

|I've seen David Mertz' xml_pickle code which makes good use of
|[metaclasses], but don't yet follow how it works.

I'm afraid I must insist here that the use of metaclasses in
gnosis.xml.pickle is utterly trivial, and just for show. You'll
have to look elsewhere for "good use" of them .

Yours, David...

 

[Kenny Tilton]

Kenny Tilton:



What about Terry's comment, based on the above URL:
) Reading the third referenced page on Macros, I notice that the amount
) of syntax definition for the macro sublanguage is as large as a
) substantial portion (one-third?) of that for core Python (if condensed
) to the same density). So, just by definitional bulk, having it in the
) language would not be a free ride.

Oy. This comment is about understanding how to write macros. We are
talking about whether:

(with-open-file (f "my.data" <...options...>)
(do-fun-things-with f))

....is harder to understand than:

(open-file-and-do "my.data"
<...options...>
:do-this (lambda (f) (do-fun-things-with f)))

btw, /writing/ macros takes a while to learn, or it did for me anyway,
because one has moved to a different space than a programmer normally
occupies, somewhere inside the compiler you might say. So some of the
time you are typing in code for run-time and some of the time you are
typing in code that runs at compile time to produce the code that runs
at runtime. And there's some weird syntax, with backquotes and coomas
and ampersands. Great fun, once it is second nature, but till then...the
neat thing is realizing you can put in debug statements that execute
while the macro is being expanded, not at run time. And you can have a
bug in the expanding code such that evaluating the form dies before it

Leverage a language ... for what? And for whom?

The keystone macro of the RoboCup client I plan to share with soccer
hooligans everywhere is DEFTASK:

(deftask <taskname> &key achieved subtasks wait-on-subtasks?
attempt retry-when)

achieved is a rule for deciding when the task has been achieved.

subtasks are subtasks to be completed (achieved) before the task at hand
is attempted, unless...

wait-on-subtasks? is nil, in which case the task can decide on its won
when to check if it has alreadt been achieved (you'll need an example).

attempt is code which generates a command to the soccer server,
hopefully leading to task achievement

retry-when tells the engine when to try again if not yet achieved. eg,
don't turn again to look for the ball until we have gotten a new "see"
message.

Now that's pretty slim doc, but there will be more and examples galore
(and an on-line help desk <g>), but some points:

No, no one looking at my code has ever seen deftask before. But if they
have done any lisp at all, they will know I have built a little task
solving engine for them. They will know because I used standard Lisp
terminology, the "def" prefix. They will understand all the keywords,
but immediately scream for documentation because keywords do not a
user's guide make. They will also see that the style is declarative,
which is meant to be helpful (and is) but they will sweat bullets until
they have a solid feel for how the engine works inside ("why did a
completed subtask start getting solved again"). I'll have to come clean
on the guts of the engine.

Or, as some of the folks on our team have done, they can macroexpand any
deftask form. They can look at the source of deftask as well, but that
can be harder because the final output is kinda disguised by all the
code which builds the final output. Once they see the astonishing
output, they will likely decide just to have me provide a functional
spec so they do not have to worry about the engine internals. Like I do.

They just write:

(deftask head-for-ball () ;; literally set body heading for ball
()
:achieved (bwhen (ball (^ball))
(< (+ .neck. (seen-dir ball)) 10))
:subtasks (list (mktask find-ball))
:attempt (bwhen (ball (^ball))
(let ((correction (+ .neck. (seen-dir ball))))
(ptrc nil "correcting" correction) ;;debug macro!
`(turn ,correction))))

The unadvertised synatx above (the two () forms near the beginning, are
a giveaway that deftask expands into the familiar (to lispniks) defclass
form. ie, deftask is implemented as defclass. Again, I make an effort to
keep the syntax lispy and recognizable and familiar and all that good
stuff. They can add slots to their task, have supertasks, etc etc.

So the defclass power is /leveraged/ to make deftask even better than i
could make it. Generic functions automatically can be dispatched by
soccer task, without me lifting a finger.

In the end, a bunch of people with just an interest in machine learning
and genetic algorithms and soccer can have some fun concnetrating just
on the ML and GA and football. Because I am providing a task-executing
"language" which deals with the soccer server for them.

As for what the above would look like without macros:

(defmodel head-for-ball (task)
()
default-initargs
:wait-on-subtasks-p t
:kids (c? (let ((task self) (self (player self)))
(mapcar (lambda (subtask)
(if (symbolp subtask)
(make-instance subtask layer self)
subtask))
(thekids (list (mktask find-ball))))))
:achieved-body (lambda (cells::c &aux (task (cells::c-model
cells::c)) (self (player task)))
(declare (ignorable self task))
(bwhen (ball (^ball))
(< (+ .neck. (seen-dir ball)) 10)))
:retry-when-body (lambda (cells::c &aux (task (cells::c-model
cells::c)) (self (player task)))
(declare (ignorable cells::c self task))
nil)
:attempt-body (lambda (cells::c &aux (task (cells::c-model
cells::c)) (self (player task)))
(declare (ignorable self task))
(bwhen (ball (^ball))
(let ((correction (+ .neck. (seen-dir ball))))
(ptrc nil "head-for-ball correcting" correction)
`(turn ,correction))))))

Not too bad, but then you have to expand defmodel.

--

kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker

 

[Jacek Generowicz]

Until some advocate of a Python macro facility adds more detail to the
proposal beyound 'Lisp/Scheme-like maco facility', it is really hard
for someone like me, who has used neither, to have much of any
concrete idea of what is being proposed.

Until some advocate of a Python macro facility adds more detail to the
proposal ... it is really hard for me to have much of any concrete
idea what is being proposed ... because Lisp-style macros rely on the
fact that Lisp code is expressed in a fundamental Lisp data structure,
in the form of its parse tree. This ain't the case in Python, so the
whole macro business suddenly gets more complicated by an order of
magnitude or two.

I just can't imagine what such a macro system would look like. Maybe
someone with deep knowledge of Dylan can enlighten us.

 

[David Mertz]

|I've seen David Mertz' xml_pickle code which makes good use of
|[metaclasses], but don't yet follow how it works.

I'm afraid I must insist here that the use of metaclasses in
gnosis.xml.pickle is utterly trivial, and just for show. You'll
have to look elsewhere for "good use" of them .

Yours, David...

 

[Michele Simionato]

You are absolutely right. Metaclasses are killer. I am surprised
Pythonistas afraid of macros let them into the language!

Me too. Actually they are quite *scaring*. In my post on
"PEP 312 (and thus 308) implemented with a black magic trick" actually
I argued that they were much *worse* than macros, since they are able
to change the *semantics* of the language, a much more dangerous
thing than simply changing the syntax. In a sense, they are deeply
unpythonic,
the exact opposite of "explicit is better than implicit".
Maybe it enters in "practicality beats purity", but this is stretching
the principle quite a bit. Same for descriptors and properties, which
still are much less dangerous.
I think metaclasses were intended not to be used, except from
developers
and very advanced users. The problem is that once you have a feature
in
a language, people will use it. I do like metaclasses, but I would be
scared
in debugging a large program written by others using them and with
poor ocumentation. Each time you see a class, you are never sure of
what it is
doing; when the bug happens, you are not sure if it is in the class or
in
the metaclass (this happened to me even with my own programs).
It is quite worrysome, actually. Fortunately, we are good people, here
on c.l.py, and I don't think anybody would mess things up, but it
could
*easily* be done.
Now, I trust the programmer and I do like the power and the freedom;
but still it strikes to me as a case of multiple personality to have
a language that at the surface is plain, clear, nicely indented, and
under the cover ... caveat emptor!

Let it be this way, anyway, and let us continue to advertise it as
a "clean" language. Let us stop people who want case statement, repeat
... until, ternary operators, braces, and the like, and let us under
the cover give people the freedom to program classes and altering at
will nearly everything at the semantics level. Let the other worry
about syntax ...

;-)

I actually had
a metaclass implementation of Cells until I decided to release the
source. The MOP is not part of the standard, and it shows across
implementations. Hell, MCL does not even expose a MOP.

Quite impressive. You are actually saying that you do prefer Python
over
Lisp on a problem of ultra-high level programming which should be
the rightful Lisp playhouse ... To be fair, the reason is not lack of
power by Lisp, but lack of standardization, still it is something that
makes me thinking ...

Michele Simionato, Ph. D.
(e-mail address removed)
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---

 

[Michele Simionato]

|I've seen David Mertz' xml_pickle code which makes good use of
|[metaclasses], but don't yet follow how it works.

I'm afraid I must insist here that the use of metaclasses in
gnosis.xml.pickle is utterly trivial, and just for show. You'll
have to look elsewhere for "good use" of them .

Yours, David...

For instance Psyco was using metaclasses to wrap the methods to
"speed-up", at least when I looked at version 0.4.
This was a quite clean usage, simple and effective.
When you need to wrap many methods in a class (for instance for
logging purposes or other reasons) a metaclass is most probably
the best solution you have at your disposal.

Michele Simionato, Ph. D.
(e-mail address removed)
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---

 

[Paul Foley]

Please excuse my confusing of Scheme with Lisp. I am pleased to find
that my difficulty in reading the Scheme macro stuff wasn't just me.
But note that there have been proposals that if Python were to have a
macro facility, it should be 'hygenic', which I presume means like
Scheme macros.

Until some advocate of a Python macro facility adds more detail to the
proposal beyound 'Lisp/Scheme-like maco facility', it is really hard
for someone like me, who has used neither, to have much of any
concrete idea of what is being proposed.

I don't know what others are proposing.

In Lisp, a macro is really just a function like any other (so people
arguing against macros who are not also against functions are clearly
insane!). It takes two arguments: a "form" (i.e., some Lisp code) and
an "environment" (a fairly useless opaque object), and returns more
Lisp code. So (a simplified version of) EVAL in a Lisp without macros
might look something like

(defun eval (form)
(typecase form
(symbol (symbol-value form))
(atom form)
(cons (apply (first form) (mapcar #'eval (rest form))))))

and to add macros, all that's necessary is

(defun macroexpand (form)
(if <form is a macro form>
(macroexpand (funcall (macro-function (first form)) form <environ>))
form))

(defun eval (form)
(let ((form (macroexpand form)))
(typecase form
(symbol (symbol-value form))
(atom form)
(cons (apply (first form) (mapcar #'eval (rest form)))))))


If you write a macro, LIST2, such that (LIST2 1 2 3 ...) expands into
(LIST (LIST 1) (LIST 2) (LIST 3) ...), and then you use it in a
function like

(defun foo (list)
(let ((x (list2 1 2 3)))
...))

in a Lisp-2 (i.e., a language with separate namespaces for variables
and functions, so you can have both a variable and a function with the
same name at the same time) there's no problem, but in a Lisp-1
(single shared namespace, like Scheme and Python) you'd have a big
problem: when the list2 macro is expanded, "list" is a variable -- the
argument to "foo" -- not the list-constructing function the
macro-writer expected.

So Scheme introduced the so-called "hygienic" macros [a bad name,
implying that non-hygienic macros are "dirty" and to be avoided;
"hygienic" macros are the ones I want to avoid!], where names used in
the macro expansion are in the scope where the macro was defined,
rather than the scope that's in effect where it's expanded, so that
"list" in the expansion refers to the list function, not the argument
to foo, and bindings made by the macro are not visible to code that
comes in from user code.

But Scheme macros are not just functions like Lisp macros, either. A
Scheme macro is defined as a set of patterns to be matched against
the code, and an associated rewrite rule which specifies the code to
use instead. [I don't see any reason why these rules need to be
written in a Lispy syntax...]

--
Just because we Lisp programmers are better than everyone else is no
excuse for us to be arrogant. -- Erann Gat

(setq reply-to
(concatenate 'string "Paul Foley " "<mycroft" '(#\@) "actrix.gen.nz>"))

 

[Kenny Tilton]

Me too. Actually they are quite *scaring*.

A few months back I started on a PyCells project, and of course was
having great fun down at the metaclass level. It was a little daunting
compared to the Lisp MOP, which just takes the same concepts of calls
and instance and does a conceptual-shift-up (new machine language
instruction?) so a class is now an instance (of a metaclass, which is
just another class really--the "meta" is only to keep one's head
straight). Python might be viewed as easier because it jumps over to
using internals such as that __dict__ thing (sorry if that is wrong, it
has been a while and I only played for a few weeks), so it there is none
of that class-being-an-instance confusion. Fun stuff, either way.

Quite impressive. You are actually saying that you do prefer Python
over
Lisp on a problem of ultra-high level programming which should be
the rightful Lisp playhouse ... To be fair, the reason is not lack of
power by Lisp, but lack of standardization, still it is something that
makes me thinking ...

I'd call it a draw. The CLOS MOP is a better playground, but Python has
a clear win in that there is only one Python. If I really need the MOP
for a project, I just pick a CL which exposes it. If I want to deliver a
package any lispnik can use, I better not use the MOP.

btw, I forgot the other reason for switching back to a macro-based
implementation: performance. It turns out ACL and I think other
implementations optimize method dispatch by (wait for it) memoizing
which method to invoke for which combo of arguments. A half-baked test
showed maybe a 30% slowdown of SLOT-VALUE because I had specialized
SLOT-VALUE-USING-CLASS, the GF that lies just under S-V.

You know, when I converted to the MOP for Cells I was all excited at how
great it would be to do it in a metaclass and make Cells almost
invisible. At the time I decided to switch back so I could share Cells,
I noticed that that arrow could point the other way: macros are so cool
that Cell semantics can be achieved /without/ tinkering with the object
model! You tell me, which is cooler? I honestly don't know. Certainly in
CL where the MOP did not make it into the standard (and the possible
loss of CLOS optimizations) put the burden on a metaclass approach to
prove a macro solution is not possible.

Cell internals without the metaclass is actually simpler, because now
slot-value is a backdoor. I had to set a flag under the metaclass
version so cell internals could actually get at a slot's value! On the
other hand, not having the backdoor may have been vital to marrying my
Cells metaclass to a persistent object metaclass. (How do you like that?
Multiple inheritance of metaclasses! <g>) Haven't tried ripping out the
metaclass of Cells from that project yet, so I do not know if I can.

Even if CLOS metaclasses were standard, kind of in the spirit of some of
this thread, I think I would like to avoid doing Cells by tinkering with
the object model. A hack that kinda disappears at run time (the macros
expand into vanilla CL) is probably going to play better with other
hacks, including not breaking optimizations. ANd it may be a Message
From God(tm) that Cells internals got simpler when I ripped out the
metaclass thing.

--

kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker

 

[Michele Simionato]

A few months back I started on a PyCells project, and of course was
having great fun down at the metaclass level. It was a little daunting
compared to the Lisp MOP, which just takes the same concepts of calls
and instance and does a conceptual-shift-up (new machine language
instruction?) so a class is now an instance (of a metaclass, which is
just another class really--the "meta" is only to keep one's head
straight). Python might be viewed as easier because it jumps over to
using internals such as that __dict__ thing (sorry if that is wrong, it
has been a while and I only played for a few weeks), so it there is none
of that class-being-an-instance confusion. Fun stuff, either way.

I don't follow you. In Python it is just the same, a class is just an
instance of a metaclass, a metaclass is just an instance of a meta-meta
class, etc. Metaclasses are classes, their simply have a different
type.__new__ method than object.__new__ so the instantiation syntax
looks a bit different.

The CLOS MOP is a better playground, but Python has
a clear win in that there is only one Python. If I really need the MOP
for a project, I just pick a CL which exposes it. If I want to deliver a
package any lispnik can use, I better not use the MOP.

Hear, hear, people who say CL is more standard than Python!

(How do you like that?
Multiple inheritance of metaclasses! <g>)

I need more multiple inheritance for metaclasses than for classes.
You maybe interested in this recipe of mine:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/204197

Even if CLOS metaclasses were standard, kind of in the spirit of some of
this thread, I think I would like to avoid doing Cells by tinkering with
the object model. A hack that kinda disappears at run time (the macros
expand into vanilla CL) is probably going to play better with other
hacks, including not breaking optimizations. ANd it may be a Message
From God(tm) that Cells internals got simpler when I ripped out the
metaclass thing.

As always, the most abstract solution is not necessarely the best
solution.

Michele Simionato, Ph. D.
(e-mail address removed)
http://www.phyast.pitt.edu/~micheles
--- Currently looking for a job ---

 

[Jacek Generowicz]

Hear, hear, people who say CL is more standard than Python!

It is. The MOP did not make it into the standard, which is a pity, but
does not detract from the fact that the standard exists.

Actulally, the MOP is a particularly thorny issue, in this respect, as
it is _almost_ standard.

 

[Kenny Tilton]

I don't follow you. In Python it is just the same, a class is just an
instance of a metaclass, a metaclass is just an instance of a meta-meta
class, etc. Metaclasses are classes, their simply have a different
type.__new__ method than object.__new__ so the instantiation syntax
looks a bit different.

Glad I said it had been a while! Maybe I was less confused on the
Python MOP because I had already done a CL MOP hack?

Hear, hear, people who say CL is more standard than Python!

I doubt they say precisely that, if standard means "the same across all
implementations".

What I would say is that CL stands at the end of the long road Python is
now headed down, with divergent branches (not a problem for Python)
brought together in a big, stabilizing spec.

Python is more like a volcanic island rising out of the sea, still being
formed, features still being debated. Heady days, being there at the
creation. Then again, "may you live in an interesting time" is a Chinese
curse.



--

kenny tilton
clinisys, inc
http://www.tilton-technology.com/
---------------------------------------------------------------
"Career highlights? I had two. I got an intentional walk from
Sandy Koufax and I got out of a rundown against the Mets."
-- Bob Uecker

 

[Donald 'Paddy' McCarthy]

Until some advocate of a Python macro facility adds more detail to the
proposal ... it is really hard for me to have much of any concrete
idea what is being proposed ... because Lisp-style macros rely on the
fact that Lisp code is expressed in a fundamental Lisp data structure,
in the form of its parse tree. This ain't the case in Python, so the
whole macro business suddenly gets more complicated by an order of
magnitude or two.

I just can't imagine what such a macro system would look like. Maybe
someone with deep knowledge of Dylan can enlighten us.

You might take a look at:
http://groups.google.com/[email protected]&rnum=12

This was my attempt at putting down some of the things that I'd like to
see in Python Macros/

Cheers, Pad.