A critic of Guido's blog on Python's lambda

[Alexander Schmolck]

A bug is a non-conformance to spec.

There is a world beyond specs, you know. If copies of allegro CL accidently
sent out death-threats to the US president on a weekly basis, because someone
at franz accidently or purposefully left in some pranky debugging code the
fact that this behaviour would likely neither violate the ansi spec nor any
other specs that ACL officially purports to adhere to wouldn't make it any
less of a bug (or help to pacify your customers).

Kenny's statement was specifically about Common Lisp

No Kenny's statement was about contrasting the way something is done in python
and the way something is done in common lisp (with the implication that the
latter is preferable). Of course the way something is done in common lisp is
almost tautologically in closer agreement with the ansi common lisp spec than
the way it is done in python, so agreement with the clhs is not a useful
criterion when talking about design features and misfeatures when contrasting
languages.

I thought it would have been pretty obvious that I was talking about language
design features and language design misfeatures (Indeed the infamously post
hoc, "It's a feature, not a bug" I was obviously alluding too doesn't make
much sense in a world were everything is tightly specified, because in it
nothing is post-hoc).

, which has a spec.

Bah -- so does fortran. But scheme also has operational semantics.

Now, what was your rationale for it _being_ a bug?

I just don't think the way special variable binding (or variable binding in
general[1]) is handled in common lisp is particularly well designed or
elegant.

Special variables and lexical variables have different semantics and using
convention and abusing[2] the declaration mechanism to differentiate between
special and lexical variables doesn't strike me as a great idea.

I can certainly think of problems that can occur because of it (E.g. ignoring
or messing up a special declaration somewhere; setf on a non-declared variable
anyone? There are also inconsistent conventions for naming (local) special
variables within the community (I've seen %x%, *x* and x)).

Thus I don't see having to use syntactically different binding and assignment
forms for special and lexical variables as inherently inferior.

But I might be wrong -- which is why was asking for the rationale of Kenny's
preference. I'd be even more interested in what you think (seriously; should
you consider it a design feature (for reasons other than backwards
compatiblity constraints), I'm pretty sure you would also give a justification
that would merrit consideration).

'as

Footnotes:
[1] The space of what I see as orthogonal features (parallel vs. serial
binding, single vs. multiple values and destructuring vs non-destructuring
etc.) is sliced in what appear to me pretty arbitrary, non-orthogonal and
annoying (esp. superfluous typing and indentation) ways in CL.

[2] Generally declarations don't change the meaning of an otherwise
well-defined program. The special declaration does. It's also a potential
source of errors as the declaration forces you to repeat yourself and to
pay attention to two places rather than one.

 

[Ken Tilton]

You seem to think that conflating special variable binding and lexical
variable binding is a feature and not a bug. What's your rationale?

Transparency. That is where power comes from. I did the same things with
Cells. Reading a slot with the usual Lisp reader method transparently
creates a dependency on the variable. To change a variable and have it
propagate throughout the datamodel, Just Change It.

Exposed wiring means more work and agonizing refactoring.

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

 

[Duane Rettig]

There is a world beyond specs, you know. If copies of allegro CL accidently
sent out death-threats to the US president on a weekly basis, because someone
at franz accidently or purposefully left in some pranky debugging code the
fact that this behaviour would likely neither violate the ansi spec nor any
other specs that ACL officially purports to adhere to wouldn't make it any
less of a bug (or help to pacify your customers).

It wouldn't be a bug in Allegro CL, because it would never happen in an Allegro CL
that hasn't been enhanced with some kind of program. And although that program
itself could have a bug whereby such a threat were accidental, I would tend not
to call it accidental, I would tend to call it expicit, and thus not a bug but
an intended consequence of such explicit programming.

My reason for responding to you in the first place was due to your poor use
of the often misused term "bug". You could have used many other words or
phrases to describe the situation, and I would have left any of those alone.

For example:

No Kenny's statement was about contrasting the way something is done in python
and the way something is done in common lisp (with the implication that the
latter is preferable). Of course the way something is done in common lisp is
almost tautologically in closer agreement with the ansi common lisp spec than
the way it is done in python, so agreement with the clhs is not a useful
criterion when talking about design features and misfeatures when contrasting
languages.

I thought it would have been pretty obvious that I was talking about language
design features and language design misfeatures (Indeed the infamously post
hoc, "It's a feature, not a bug" I was obviously alluding too doesn't make
much sense in a world were everything is tightly specified, because in it
nothing is post-hoc).

Whether it is preferable is a matter of opinion, and whether Kenny meant it
to infer preferability (I suspect so) or not has nothing to due with whether
it is a bug. Instead, you should call it a "design misfeature", which would
set the stage for a more cogent argumentation on the point, rather than on the
hyperbole. By the way, if you do call it a design misfeature, I would be
arguing against you, but that is another conversation.

, which has a spec.

Bah -- so does fortran. But scheme also has operational semantics.

Now, what was your rationale for it _being_ a bug?

I just don't think the way special variable binding (or variable binding in
general[1]) is handled in common lisp is particularly well designed or
elegant.

Then call it "misdesigned" or "inelegant".

Special variables and lexical variables have different semantics and using
convention and abusing[2] the declaration mechanism to differentiate between
special and lexical variables doesn't strike me as a great idea.

Then call it a "bad idea".

I can certainly think of problems that can occur because of it (E.g. ignoring
or messing up a special declaration somewhere; setf on a non-declared variable
anyone? There are also inconsistent conventions for naming (local) special
variables within the community (I've seen %x%, *x* and x)).

Then call it "not fully standardized or normative".

Thus I don't see having to use syntactically different binding and assignment
forms for special and lexical variables as inherently inferior.

Then call it "inherently inferior".

But I might be wrong -- which is why was asking for the rationale of Kenny's
preference.

But you _didn't_ ask him what rationale he had for his _preference_, you
asked him his rationale for considering it not a _bug_.

I'd be even more interested in what you think (seriously; should
you consider it a design feature (for reasons other than backwards
compatiblity constraints), I'm pretty sure you would also give a justification
that would merrit consideration).

Well, OK, let's change the conversation away from "bug"-ness and toward any of
the other negatives we discussed above. I actually doubt that I can provide
a justification in a small space without first understanding who you are
and from what background you are coming, so let me turn it around and ask
you instead to knock down a straw-man:

You seem to be saying that pure lexical transparency is always preferable
to statefulness (e.g. context). Can we make that leap? If not, set me
straight. If so, tell me: how do we programmatically model those situations
in life which are inherently contextual in nature, where you might get
a small piece of information and must make sense of it by drawing on
information that is _not_ given in that information, but is (globally,
if you will) "just known" by you? How about conversations in English?
And, by the way, how do you really know I'm writing to you in English, and
not some coded language that means something entirely different?

 

[jayessay]

You seem to think that conflating special variable binding and lexical
variable binding is a feature and not a bug. What's your rationale?

And the particularly ugly, crappy, half baked python emulation is what?
A feature? Right.


/Jon

 

[jayessay]

I think the point is that, with the variable actually being just a
string and with dedicated new explicit functions required as
"accessors", well, you could hack that up in any language with
dictionaries. It is the beginnings of an interpreter, not Python
itself even feigning special behavior.

Exactly. Of course this is going to be totally lost on the intended
audience...


/Jon

 

[jayessay]

Great -- so can I see some code? Can't be that difficult, it takes about 10-15
lines in python (and less in scheme).

Do you actually need the code to understand this relatively simple concept???

/Jon

 

[Alexander Schmolck]

My reason for responding to you in the first place was due to your poor use
of the often misused term "bug". You could have used many other words or
phrases to describe the situation, and I would have left any of those alone.

I'm happy to accept your terminology of bug (not conforming to a certain
specification) for the remainder of this discussion so that we can stop
quibbling over words.

[...]

Well, OK, let's change the conversation away from "bug"-ness and toward any of
the other negatives we discussed above. I actually doubt that I can provide
a justification in a small space without first understanding who you are
and from what background you are coming, so let me turn it around and ask
you instead to knock down a straw-man:

You seem to be saying that pure lexical transparency is always preferable
to statefulness (e.g. context).
No.

Can we make that leap? If not, set me straight.

I think that in most contexts lexical transparency is desirable so that
deviations from lexical transparency ought to be well motivated. I also
believe that a construct that is usually used to establish a lexically
transparent binding shouldn't be equally used for dynamic bindings so that it
isn't syntactically obvious what's going on. I've already provided some
reasons why CL's design wrt. binding special and lexical variables seems bad
to me. I don't think these reasons were terribly forceful but as I'm not aware
of any strong motivation why the current behaviour would be useful I'm
currently considering it a minor wart.

To make things more concrete: What would be the downside of, instead of doing
something like:

(let ((*x* ...)) [(declare (special *x*))] ...) ; where [X] denotes maybe X

doing any of the below:

a) using a different construct e.g. (fluid-let ((*x* ...)) ...) for binding
special variables
b) having to use *...* (or some other syntax) for special variables
c) using (let ((q (special *x*) (,a ,b ,@c)) (values 1 2 '(3 4 5 6)))
(list q ((lambda () (incf *x*))) a b c)) ; => (1 3 3 4 (5 6))

(It's getting late, but hopefully this makes some vague sense)

If so, tell me: how do we programmatically model those situations in life
which are inherently contextual in nature, where you might get a small piece
of information and must make sense of it by drawing on information that is
_not_ given in that information, but is (globally, if you will) "just known"
by you? How about conversations in English? And, by the way, how do you
really know I'm writing to you in English, and not some coded language that
means something entirely different?

We can skip that part.

'as

 

[Alexander Schmolck]

Do you actually need the code to understand this relatively simple concept???

Yes. I'd be genuinely curious to see how an implementation in Java, Pascal, C,
(or any other language that has little more than dictionaries) compares to
python and CL.

In my limited understanding I have trouble seeing how you'd do without either
unwind-protect/try-finally or reliable finalizers for starters.

'as

 

[Alexander Schmolck]

Transparency.

That's is circular. You might be right, but you failed to provide a rationale
and not just a restatement.

That is where power comes from. I did the same things with Cells. Reading a
slot with the usual Lisp reader method transparently creates a dependency on
the variable.

Let me see if I understand it right -- if an instance of class A has a ruled
slot a that reads an instance of class B's slot b then it is noted somewhere
that A's a depends on b?

To change a variable and have it propagate throughout the datamodel, Just
Change It.


Exposed wiring means more work and agonizing refactoring.

Well, you claim that in that instance python suffers from exposed wiring and I
claim that CL suffers from a (minor) booby trap. You can't typically safely
ignore whether a variable is special as a mere wiring detail or your code
won't work reliably (just as you can't typically safely ignore whether
something is rigged or not even if booby-trapness is pretty transparent) --
it's as simple as that (actually its a bit worse because the bug can be hard
to detect as lexical and special variables will result in the same behaviour
in many contexts).

So in the case of booby traps and special variables, I generally prefer some
exposed wiring (or strong visual clues) to transparency.

I'd like to see a demonstration that using the same binding syntax for special
and lexical variables buys you something apart from bugs.

'as

 

[Ken Tilton]

I will expand on my earlier "transparency" rationale with a further
rationale for transparency: I do not need no stinkin' rationale. A
special variable is still a variable. They should be set, read, and
bound (say, by "let") the same way as any other variable.

You need a rationale. It sounds as if you want some noisey syntax to
advertise the specialness. I do not think the Python community will
appreciate you messing up their pretty code.

You are right about one thing: specialness needs advertising. You know
what we do in Lisp? We obediently name special variables with bracketing
*s, like *this*. Too simple?

There is a world beyond specs, you know. If copies of allegro CL accidently
sent out death-threats to the US president on a weekly basis, because someone
at franz accidently or purposefully left in some pranky debugging code the
fact that this behaviour would likely neither violate the ansi spec nor any
other specs that ACL officially purports to adhere to wouldn't make it any
less of a bug (or help to pacify your customers).




No Kenny's statement was about contrasting the way something is done in python
and the way something is done in common lisp (with the implication that the
latter is preferable).

Close, but no. The question I was weighing in was "has Michele
replicated special variables?". My implication was, "Not yet -- can you
match the transparency?", and it was an honest question, I do not know.
Again, transparency is a qualitative difference.

I liked your solution better, btw, because it does minimize the noise.
For fun, you should call the class ** instead of special, so we end up
with: **.b = 42

We'll understand.

Of course the way something is done in common lisp is
almost tautologically in closer agreement with the ansi common lisp spec than
the way it is done in python, so agreement with the clhs is not a useful
criterion when talking about design features and misfeatures when contrasting
languages.

Again, no, it is not the spec, it is the highly-valued Python quality of
clean code. Also, the consistency of treating variables as variables,
regardless of some special/dynamic quality.

Some background. Lisp is a big language, and I am self taught and do not
like to read, grew up in Lisp in isolation. Not many Lispers in the
exercise yard. Discovered special variables only when we hired an old
hand who gently corrected a howler:

(let* ((old-x *x*))
(setf *x* 42)
....
(setf *x* old-x))

I still laugh at that. Anyway, as soon as I learned that, I was able to
make Cells syntax infinitely more transparent. And guess what? It also
made dependency identification automatic instead of cooperative, and
when I rebuilt a huge Cells-based app I discovered two or three cases
where I had neglected to publish a dependency.

It's a mystery, but somehow simpler syntax... oh, wait, this is
c.l.python, I am preaching to the choir.

I just don't think the way special variable binding (or variable binding in
general[1]) is handled in common lisp is particularly well designed or
elegant.

See above. There is nothing like a concrete experience of implementing a
hairy library like Cells /without/ leveraging specials and then
converting to specials. Talk about an Aha! experience. I mean, bugs ran
screaming from their nests simply because of the implementation change--
we call that A Message From God that the design has taken a correct turn.

Special variables and lexical variables have different semantics and using
convention and abusing[2] the declaration mechanism to differentiate between
special and lexical variables doesn't strike me as a great idea.

I know what you mean, but I like reading tea leaves, and I find it
fascinating that *this* somehow eliminates all ambiguity. Background:
don't know where I might find it, but I once saw a thread demonstrating
the astonishing confusion one could create with a special variable such
as a plain X (no *s). Absolutely mind-bogglingly confusing. Go back and
rename the special version *x*, and use *x* where you want to rebind it.
Result? Utterly lucid code. Scary, right?

I can certainly think of problems that can occur because of it (E.g. ignoring
or messing up a special declaration somewhere; setf on a non-declared variable
anyone?

Sh*t, you don't respond to compiler warnings? Don't blame CL for your
problems.

There are also inconsistent conventions for naming (local) special
variables within the community (I've seen %x%, *x* and x)).

OK, you are in flamewar mode, now you are just making things up.

Thus I don't see having to use syntactically different binding and assignment
forms for special and lexical variables as inherently inferior.

DUDE! They are both variables! Why the hell /should/ the syntax be
different? "Oh, these are /cross-training/ sneakers. I'll wear them on
my hands." Hunh?




kenny

 

[Ken Tilton]

I thought special variables meant dynamic binding, i.e.

(defvar *x* 1)
(defun f ()
(print *x*) ;; -> 2
(let ((*x* 3))
(g)))
(defun g ()
(print *x*)) ;; - > 3

That was normal behavior in most Lisps before Scheme popularlized
lexical binding. IMO it was mostly an implementation convenience hack
since it was implemented with a very efficient shallow binding cell.
That Common Lisp adapted Scheme's lexical bindings was considered a
big sign of CL's couthness. So I'm a little confused about what Ken
Tilton is getting at.

Paul, there is no conflict between your example and mine, but I can see
why you think mine does not demonstrate dynamic binding: I did not
demonstrate the binding applying across a function call.

What might be even more entertaining would be a nested dynamic binding
with the same function called at different levels and before and after
each binding.

I just had the sense that this chat was between folks who fully grokked
special vars. Sorr if I threw you a curve.

kenny

 

[Ken Tilton]

Everything else responded to separately, but...

I'd like to see a demonstration that using the same binding syntax for special
and lexical variables buys you something apart from bugs.

Buys me something? Why do I have to sell simplicity, transparency, and
clean syntax on c.l.python?

kenny

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

 

[Duane Rettig]

I think that in most contexts lexical transparency is desirable so that
deviations from lexical transparency ought to be well motivated. I also
believe that a construct that is usually used to establish a lexically
transparent binding shouldn't be equally used for dynamic bindings so that it
isn't syntactically obvious what's going on. I've already provided some
reasons why CL's design wrt. binding special and lexical variables seems bad
to me. I don't think these reasons were terribly forceful but as I'm not aware
of any strong motivation why the current behaviour would be useful I'm
currently considering it a minor wart.

To make things more concrete: What would be the downside of, instead of doing
something like:

(let ((*x* ...)) [(declare (special *x*))] ...) ; where [X] denotes maybe X

Let's start with this. You seem to be saying that the above construct is inferior
to the alternatives you are about to suggest. Why? And since you are adding
an optional form, let's break it down into its separate situations:

1. (let ((*x* ...)) (declare (special *x*)) ...)

Here there is no question about the specialness of *x*; it is textually
obvious what the binding is - that it is not a lexical binding but a special
binding.

2. (let ((*x* ...)) ...)

[where there is no special declaration for *x* within the form]

Here, the issue is that it is not obvious that *x* is special (in this case,
it would have to already be a dynamic variable (what we internally call
"globally special"), because a special declaration within a lexical context
does not affect inner bindings. Perhaps this form is the one you are really
having trouble with.

doing any of the below:

a) using a different construct e.g. (fluid-let ((*x* ...)) ...) for binding
special variables

Unless you also _remove_ the #2 case above, this seems no diferent than writing
a macro for the #1 case, above.

b) having to use *...* (or some other syntax) for special variables

In fact, the spec does suggest precisely this (see
http://www.franz.com/support/documentation/8.0/ansicl/dictentr/defparam.htm,
in the Notes section), and to the extent that programmers obey the suggestion,
the textual prompting is present in the name.

c) using (let ((q (special *x*) (,a ,b ,@c)) (values 1 2 '(3 4 5 6)))
(list q ((lambda () (incf *x*))) a b c)) ; => (1 3 3 4 (5 6))

(It's getting late, but hopefully this makes some vague sense)

Well, sort of; this seems simply like a sometimes-fluid-let, whose syntax could
easily be established by a macro (with destructurings whose form is (special X)
could be specially [sic] treated.

Now if in the above example you would have trouble with (a) and/or (c)
based on the absence of a "lexical" declaration (i.e. one that would undo
the effect of a globally special declaration), thus guaranteeing that a
fluid-let or a "sometimes-fluid-let" would work, you should know that while
I was working on the Environments Access module I theorized and demonstrated
that such a declaration could be easily done within a conforming Common Lisp.
I leave you with that demonstration here (though it really is only for
demonstration purposes only; I don't necessarily propose that CL should add
a lexical declaration to the language):

[This only works on Allegro CL 8.0]:

CL-USER(1): (defvar pie pi)
PIE
CL-USER(2): (compile (defun circ (rad) (* pie rad rad)))
CIRC
NIL
NIL
CL-USER(3): (circ 10)
314.1592653589793d0
CL-USER(4): (compile (defun foo (x) (let ((pie 22/7)) (circ x))))
FOO
NIL
NIL
CL-USER(5): (foo 10)
2200/7
CL-USER(6): (float *)
314.2857
CL-USER(7): (sys:define-declaration sys::lexical (&rest vars)
nil
:variable
(lambda (declaration env)
(declare (ignore env))
(let* ((spec '(lexical t))
(res (mapcar #'(lambda (x) (cons x spec))
(cdr declaration))))
(values :variable res))))
SYSTEM::LEXICAL
CL-USER(8): (compile (defun foo (x) (let ((pie 22/7)) (declare (sys::lexical pie)) (circ x))))
; While compiling FOO:
Warning: Variable PIE is never used.
FOO
T
NIL
CL-USER(9): (foo 10)
314.1592653589793d0
CL-USER(10):

 

[Marcin 'Qrczak' Kowalczyk]

I think the point is that, with the variable actually being just
a string and with dedicated new explicit functions required as
"accessors", well, you could hack that up in any language with
dictionaries. It is the beginnings of an interpreter, not Python
itself even feigning special behavior.

If the semantics and the global structure of the code is right, only
you don't like the local concrete syntax, then the complaint is at
most as justified as complaints against Lisp parentheses.

 

[Marcin 'Qrczak' Kowalczyk]

I'd like to see a demonstration that using the same binding syntax
for special and lexical variables buys you something apart from bugs.

There are 3 fundamental operations related to plain mutable variables:

A1. Making a new mutable variable with an initial value.
A2. Getting the current value.
A3. Setting the new value.

and 4 operations related to dynamically scoped variables:

B1. Making a new dynamic variable with an initial value.
B2. Getting the current value.
B3. Setting the new value.
B4. Local rebinding with a new initial value.

If you don't ever use B4, dynamic variables behave exactly like plain
variables. For this reason I see no point in distinguishing A2 from B2,
or A3 from B3. Dynamic variables are a pure extension of plain variables
by providing an additional operation.

Distinguishing the syntax of A1 and B1 is natural: somehow it must be
indicated what kind of variable is created.

Mutability is orthogonal to dynamic scoping. It makes sense to have a
variable which is like a plain variable but without A3, and a variable
which is like a dynamic variable but without B3, although it doesn't
provide anything new, only allows to express more constraints with a
potential for optimization. I won't consider them here.

Common Lisp does something weird: it uses the same syntax for A1 and B4,
where the meaning is distinguished by a special declaration. Here is
its syntax:

Directly named plain variables:
A1. (let ((name value)) body) and other forms
A2. name
A3. (setq name value), (setf name value)

First-class dynamic variables:
B1. (gensym)
B2. (symbol-value variable)
B3. (set variable value), (setf (symbol-value variable) value)
B4. (progv `(variable) `(value) body)

Directly named dynamic variables:
B1. (defvar name value), (defparameter name value)
B2. name
B3. (setq name value), (setf name value)
B4. (let ((name value)) body) and other forms

Dynamic variables in Lisp come in two flavors: first-class variables
and directly named variables. Directly named variables are always
global. You can convert a direct name to a first-class variable by
(quote name).

Plain variables have only the directly named flavor and they are
always local. You can emulate the first-class flavor by wrapping a
variable in a pair of closures or a closure with dual getting/setting
interface (needs a helper macro in order to be convenient). You can
emulate a global plain variable by wrapping a dynamic variable in a
symbol macro, ignoring its potential for local rebinding. You can
emulate creation of a new first-class variable by using a dynamic
variable and ignoring its potential for local rebinding, but this
can't be used to refer to an existing directly named plain variable.

In order to create a plain variable, you must be sure that its name is
not already used by a dynamic variable in the same scope.

So any essential functionality is possible to obtain, but the syntax
is very irregular.

 

[Lasse Rasinen]

ps. flaming aside, PyCells really would be amazingly good for Python. And
so Google. (Now your job is on the line. <g>) k

Here's something I wrote this week, mostly as a mental exercise ;-)
The whole code is available at <http://www.iki.fi/~lrasinen/cells.py>,
I'll include a test example below. Feel free to flame away ;-)

(As for background: I like CL better as a language, but I also like Python
a lot. However, I was employed for 3 years as a developer and maintainer
in a Python data mining application, so I'm more fluent in Python than CL.)

The code is mostly based on Kenny's descriptions of Cells in the following
messages:
<[email protected]>
<[email protected]>
<[email protected]>

In addition, I have looked at the CL source code briefly, but I'm not sure
if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class (however, if you define a rule by hand in the __init__
function, it'll work also. I think; haven't tested).

I can possibly be persuaded to fix bugs in the code and/or to implement
new features ;-)

Features:
- Tracks changes to input cells dynamically (normal attributes are not tracked)
- Callbacks for changes (see caveats)
- Requires Python 2.4 for the decorator syntax (@stuff)
- Should calculate a cell only once per change (haven't tested ;-)

Caveats:
- The input cell callbacks are not called with the class instance
as the first argument, while the rule cell callback are. This
is mostly due to laziness.
- There is no cycle detection. If you write cyclic dependencies, you lose.
- There is very little error checking.

Example follows:

def x_callback(oldval, newval):
print "x changed: %s => %s" % (oldval, newval)

class Test(cellular):
def __init__(self):
self.x = InputCell(10, callback=x_callback)

def y_callback(self, oldval, newval):
print "y changed: %s => %s" %(oldval, newval)

def a_callback(self, oldval, newval):
print "a changed: %s => %s" %(oldval, newval)

def g_callback(self, oldval, newval):
print "g changed: %s => %s" %(oldval, newval)

@rule(callback=y_callback)
def y(self):
return self.x ** 2

@rule(callback=a_callback)
def a(self):
return self.y + self.x

@rule(callback=g_callback)
def g(self):
if self.x % 2 == 0:
return self.y
else:
return self.a


$ python cells.py

y changed: __main__.unbound => 100
a changed: __main__.unbound => 110
g changed: __main__.unbound => 100
=============
x changed: 10 => 4
y changed: 100 => 16
a changed: 110 => 20
g changed: 100 => 16
=============
x changed: 4 => 5
y changed: 16 => 25
a changed: 20 => 30
g changed: 16 => 30

 

[Ken Tilton]

Here's something I wrote this week, mostly as a mental exercise ;-)

It's fun, right? But what you have is a complete wreck.

The whole code is available at <http://www.iki.fi/~lrasinen/cells.py>,
I'll include a test example below. Feel free to flame away ;-)

(As for background: I like CL better as a language, but I also like Python
a lot. However, I was employed for 3 years as a developer and maintainer
in a Python data mining application, so I'm more fluent in Python than CL.)

The code is mostly based on Kenny's descriptions of Cells in the following
messages:
<[email protected]>
<[email protected]>
<[email protected]>

In addition, I have looked at the CL source code briefly, but I'm not sure
if any concepts have survived to the Python version. Since Python's object
model is sufficiently different, the system is based on rules being
defined per-class...

That will be a total disaster for PyCells, if true. But I do not think
it is. You just need a constructor that takes some slot initializers,
and initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Trust me, you lose a vast amount of power unless different instances of
the same class can have different rules for the same slot.

... (however, if you define a rule by hand in the __init__
function, it'll work also. I think; haven't tested).

I can possibly be persuaded to fix bugs in the code and/or to implement
new features ;-)

PyCells looks like it will be a project for SoC2006, so you may as well
relax. But I understand if you want to keep going, it is great fun. btw,
I have met more than a few people who had done something like Cells
independently, and there are many full-blown similar implementations

Features:
- Tracks changes to input cells dynamically (normal attributes are not tracked)

Ha! All your rules depend on the input cell itself! How about A depends
on B depends on C?

- Callbacks for changes (see caveats)
- Requires Python 2.4 for the decorator syntax (@stuff)
- Should calculate a cell only once per change (haven't tested ;-)

Quite hard to test deliberately, but it happens "in nature". But it will
not happen until you do A->B->C. Once you have /that/ working, make A
the input, then have B and C both use A. But also have B use C, and
jiggle things around until A happens to think it should update B first,
then C. What happens is that B runs and uses C, but C has not been
updated yet. C is inconsistent with A, but is being used to calculate a
new value for B which does see the new value of A. Mismatch! B will get
sorted out in a moment when C gets recalculated and tells B to calculate
a second time, but meanwhile after the first recalculation of B the
on-change callback for that got invoked, missiles were launched, and
Moscow has been destroyed.

Caveats:
- The input cell callbacks are not called with the class instance
as the first argument, while the rule cell callback are. This
is mostly due to laziness.

And unacceptable!

have fun.

kenny

ps. In the getattr for any Cell-mediated slot, look to see if "parent"
is non-nil. If so, set up a dependency. k

--
Cells: http://common-lisp.net/project/cells/

"Have you ever been in a relationship?"
Attorney for Mary Winkler, confessed killer of her
minister husband, when asked if the couple had
marital problems.

 

[Lasse Rasinen]

That will be a total disaster for PyCells, if true. But I do not think it
is. You just need a constructor that takes some slot initializers, and
initialize the slots to one of: a normal value; an InputCell itself
initialized with a starting value, if only nil; or a RuledCell itself
initialized with a lambda.

Hmm, just tried it:
.... def __init__(self):
.... self.a = cells.InputCell(10)
.... self.b = cells.RuleCell(lambda self: self.a+1, self, None)
11

So it does work out-of-the-box ;-)

PyCells looks like it will be a project for SoC2006, so you may as well
relax.

You really want to start a SoC project on something that takes about two
weeks from an average Python programmer? What does the guy do for the rest
of the summer?

(I think I spent 4-5 hours on this actually sitting on the computer,
sandwiched between remodeling and cleaning and work. The rest of the two
weeks would be making it more robust ;-)

Ha! All your rules depend on the input cell itself! How about A depends on
B depends on C?

Oops. I'm sorry for the inaccurate terminology. They depend only the cells
they use as inputs (their "children"), and not only on InputCells.

(I use the parent-child terminology because as English is not my native
language, I had trouble remembering which depend* variable was which ;-)

Quite hard to test deliberately, but it happens "in nature". But it will
not happen until you do A->B->C. Once you have /that/ working, make A the
input, then have B and C both use A. But also have B use C, and jiggle
things around until A happens to think it should update B first, then C.
What happens is that B runs and uses C, but C has not been updated yet. C
is inconsistent with A, but is being used to calculate a new value for B
which does see the new value of A. Mismatch! B will get sorted out in a
moment when C gets recalculated and tells B to calculate a second time,
but meanwhile after the first recalculation of B the on-change callback
for that got invoked, missiles were launched, and Moscow has been
destroyed.

If you check the testcase, you'll see there are such dependencies, and all
the callbacks fire just once (and in dependency-related order).

Furthermore, the timestamp mechanism SHOULD take care of those (if the
cell is older than its children, it gets recalculated before it will
provide any data, and thus C will get recalculated before B uses it.

ps. In the getattr for any Cell-mediated slot, look to see if "parent" is
non-nil. If so, set up a dependency. k

Already done, see BaseCell.value() ;-)

 

[Ken Tilton]

Hmm, just tried it:



... def __init__(self):
... self.a = cells.InputCell(10)
... self.b = cells.RuleCell(lambda self: self.a+1, self, None)



11

So it does work out-of-the-box ;-)

So why exactly did you say that the differences in the object model made
it impossible? I was really stunned by that claim. And you sounded so
confident. What went wrong there? It was trivial, right? How did you
miss that?

You really want to start a SoC project on something that takes about two
weeks ...

You sound so confident.

Do you know the deliverables? I know you do not know Cells. You say you
looked at the code -- it does not show. I can also tell you have not
done much serious programming, or you would know that twelve weeks is
more like twelve minutes than three months.

A new test suite, documentation (a first, there is none now), a full
port of Cells in all their ten years of sophisticated evolution and
variety (no, not your stupid pet trick), and as a demo project an entire
cells-driven GUI, probably a port of my new Celtk (+ Cells Tk) work, all
in a language without macros, without special variables, with a more
modest OO system, and limited first class functions... oh, I think we'll
keep him busy.

Now since you are such a genius, maybe you can help with something.
Trust me on this: this is one place where macros would be able to hide a
ton of implementation wiring it does no one any good to look at, and
actually turns into a maintenance nightmare whenever Cells might get
revised.

Is there any experiemntal macro package out there for Python? Maybe a
preprocessor, at least? Or are there ways to actually hack Python to
extend the syntax? My honest guess is that Cells will port readily to
Python but leave everyone very interested in finding some way to hide
implementation boilerplate. Got anything on that?

kenny

 

[Ben]

Nothing you have described sounds that complicated, and you never come
up with concrete objections to other peoples code (apart that it took
10 years to write in Lisp, so it must be really hard)

Why are you running a SoC project for PyCells if you dislike the
language so much. People who do like Python can implement it if they
need it (which I haven't seen any good examples that they do)

Please don't force a student to create a macro system just to port a
system to Python, as it won't really be python then. Use Pythonic
methodology instead. There are already plenty of ways to hide
complicated functionality, just not necessarily the way you want to do
it.

Cheers,
Ben