Python's "only one way to do it" philosophy isn't good?

D

Douglas Alan

Michele Simionato said:
You should really be using pychecker (as well as Emacs autocompletion
feature ...):

I *do* use Emacs's autocompletion, but sometimes these sorts of bugs
creep in anyway. (E.g., sometimes I autocomplete in the wrong variable!)
~$ pychecker -v x.py
Processing x...

Warnings...

x.py:4: Variable (longVarableName) not used

[I know you will not be satisfied with this, but pychecker is really
useful,

Okay, I'll check out PyChecker and PyLint, though I'm sure they will
annoy the hell out of me. They're probably less annoying than
spending all day tracking down some stupid bug.
since it catches many other errors that no amount of
macroprogramming would evere remove].

And likewise, good macro programming can solve some problems that no
amount of linting could ever solve.

|>oug
 
P

Paul Rubin

Douglas Alan said:
And likewise, good macro programming can solve some problems that no
amount of linting could ever solve.

I think Lisp is more needful of macros than other languages, because
its underlying primitives are too, well, primitive. You have to write
all the abstractions yourself. Python has built-in abstractions for a
few container types like lists and dicts, and now a new and more
general one (iterators), so it's the next level up. Haskell abstracts
the concept of containers to something called monads, so operations
like loops and list comprehensions fall out automatically (it took me
a while to realize that--Haskell listcomps weren't a bright new idea
someone thought of adding to an otherwise complete language: they were
already inherently present in the list monad operations and their
current syntax is just minor sugaring and is actually restricted on
purpose to make the error messages less confusing).

So, a bunch of stuff one needs macros to do conveniently in Lisp, can
be done with Python's built-in syntax. And a bunch of stuff that
Python could use macros for, are easily done in Haskell using delayed
evaluation and monads. And Haskell is starting to grow its own macro
system (templates) but that's probably a sign that an even higher
level language (maybe with dependent types or something) would make
the templates unnecessary.
 
D

Douglas Alan

I think Lisp is more needful of macros than other languages, because
its underlying primitives are too, well, primitive. You have to write
all the abstractions yourself.

Well, not really beause you typically use Common Lisp with CLOS and a
class library. If you ask me, the more things that can (elegantly) be
moved out of the core language and into a standard library, the
better.
Python has built-in abstractions for a few container types like
lists and dicts, and now a new and more general one (iterators), so
it's the next level up.

Common Lisp has had all these things for ages.
And a bunch of stuff that Python could use macros for, are easily
done in Haskell using delayed evaluation and monads. And Haskell is
starting to grow its own macro system (templates) but that's
probably a sign that an even higher level language (maybe with
dependent types or something) would make the templates unnecessary.

Alas, I can't comment too much on Haskell, as, although I am familiar
with it to some extent, I am far from proficient in it. Don't worry
-- it's on my to-do list.

I think that first I'd like to take Gerry Sussman's new graduate
class, first, though, and I'll find out how it can all be done in
Scheme.

|>oug
 
A

Alexander Schmolck

Douglas Alan said:
Common Lisp has had all these things for ages.

Rubbish. Do you actually know any common lisp?

There is precisely no way to express

for x in xs:
blah(x)

or
x = xs[key]

in either scheme or CL, which is a major defect of both language (although
there has been a recent and limited proposal for sequence iteration by c.
rhodes which is implemented as an experimental extension in sbcl). This is
stuff even C++, which is about the lowest-level language anyone uses for
general purpose programming these days has been able to express for decades
(modulo foreach syntax).

In a decent scheme it's easy enough to define your own collection
class/iteration protocol, which does allow you to do something like the above,
but of course only for container abstractions that you have some control over
yourself. Even in this limited sense you can forget about doing this in CL in
a way that meshes nicely with the existing primitives (inter alia because of
spurious inconsistencies between e.g. sequence and hash-access and
under-specification of the exception hierachy) and anything as expressive as
generators/coroutines in CL with reasonable effort and performance which won't
even allow you to write LOOPs over custom container types (the nonstandard
ITERATE package has limited support for this).

'as
 
D

Douglas Alan

Rubbish. Do you actually know any common lisp?

Yes, though it's been quite a while, and it was mostly on Lisp
Machines, which, at the time, Common Lisp was still being
standardized, and so Lisp Machine "Chine Nual" Lisp wasn't quite
Common Lisp compliant at the time. Also, Lisp Machine Lisp had a lot
of features, such as stack groups, that weren't put into Common Lisp.
Also, my experience predates CLOS, as at the time Lisp Machines used
Flavors.

Most of my Lisp experience is actually in MacLisp (and Ulisp and
Proto, neither of which you've likely heard of). MacLisp was an
immediate precursor of Common Lisp, and didn't have a standard object
system at all (I rolled one myself for my applications), but it had
the Loop macro and if I recall correctly, the MacLisp Loop macro
(which was nearly identical to the Chine Nual Loop macro, which I
thought was ported rather unsullied for Common Lisp). In any case,
IIRC, there were hooks in the Loop macro for dealing with iterators
and I actually used this for providing an iterator-like interface to
generators (for Lisp Machines) that I coded up with macros and stack
groups.

It may be that these hooks didn't make it into the Common Lisp Loop
macro, or that my memory of what was provided by the macro is a little
off. What's not off, is that it was really easy to implement these
things, and it wasn't like I was some sort of Lisp guru -- I was just
an undergraduate student.

I will certainly admit that Lisp programmers at the time were (and
likely still are) much more enamored of mapping functions than of
iterators. Mapping functions certainly get the job done as elegantly
as iterators most of the time, although I would agree that they are
not quite so general. Of course, using generators, I was easily able
to make a converter that would take a mapping function and return a
corresponding iterator.

Scheme, on, the other hand, at least by idiom, has computation
"streams", and streams are equivalent to iterators.
There is precisely no way to express

for x in xs:
blah(x)

The canonical way to do this in Lisp would be something like:

(mapcar (lambda (x) (blah x))
xs)

Though there would (at least in MacLisp) be a differently named
mapping function for each sequence type, which makes things a bit less
convenient, as you have to know the name of the mapping function
for each type.
or
x = xs[key]

I'm not sure what you are asserting? That Common Lisp doesn't have
hash tables? That's certainly not the case. Or that it doesn't
provide standard generic functions for accessing them, so you can
provide your own dictionaries that are implemented differently and
then use exactly the same interface? The latter I would believe, as
that would be one of my criticisms of Lisp -- although it's pretty
cool that you can load whatever object system you would like (CLOS
being by far the most common), it also means that the core language
itself is a bit deficient in OO terms.

This problem would be significantly mitigated by defining new
standards for such things in terms of CLOS, but unfortunately
standards change unbearably slowly. There are certainly many
implementations of Lisp that solve these issues, but they have a hard
time achieving wide adoption. A language like Python, which is
defined by its implementation, rather than by a standard, can move
much more quickly. This debate though is really one more of
what is the best model for language definition, rather than one on
what the ideal language is like.

|>oug
 
P

Paul Rubin

Douglas Alan said:
I will certainly admit that Lisp programmers at the time were (and
likely still are) much more enamored of mapping functions than of
iterators. Mapping functions certainly get the job done as elegantly
as iterators most of the time, although I would agree that they are
not quite so general.

In the Maclisp era functions like mapcar worked on lists, and
generated equally long lists in memory. It was sort of before my time
but I have the impression that Maclisp was completely dynamically
scoped and as such, it couldn't cleanly make anything like generators
(since it had no way to make lexical closures).
Scheme, on, the other hand, at least by idiom, has computation
"streams", and streams are equivalent to iterators.

No not really, they (in SICP) are at best more like class instances
with a method that mutates some state. There's nothing like a yield
statement in the idiom. You could do it with call/cc but SICP just
uses ordinary closures to implement streams.
The canonical way to do this in Lisp would be something like:
(mapcar (lambda (x) (blah x)) xs)

At least you could spare our eyesight by writing that as
(mapcar #'blah xs) ;-). The point is that mapcar (as the name
implies) advances down a list using cdr, i.e. it only operates
on lists, not general iterators or streams or whatever.
x = xs[key]

I'm not sure what you are asserting? That Common Lisp doesn't have
hash tables? That's certainly not the case. Or that it doesn't
provide standard generic functions for accessing them

The latter. Of course there are getf/setf, but those are necessarily
macros.
A language like Python, which is defined by its implementation,
rather than by a standard, can move much more quickly. This debate
though is really one more of what is the best model for language
definition, rather than one on what the ideal language is like.

Python is not Perl and it has in principle always been defined by its
reference manual, though until fairly recently it's fostered a style
of relying on various ugly CPython artifacts like the reference
counting GC.

Lisp accumulated a lot of cruft over the decades and it kept some
baggage that it really could have done without. I don't think
Python's designers learned nearly as much from Lisp as they could
have, and Python has suffered because of it. Lisp still has an
awesome beauty in both the CL and Scheme incarnations. But it's like
listening to Elvis music--even if it can still get you dancing, at the
end of the day it's still a reflection of a bygone culture.
 
D

Douglas Alan

In the Maclisp era functions like mapcar worked on lists, and
generated equally long lists in memory.

I'm aware, but there were various different mapping functions. "map",
as opposed to "mapcar" didn't return any values at all, and so you had
to rely on side effects with it.
It was sort of before my time but I have the impression that Maclisp
was completely dynamically scoped and as such,

Yes, that's right.
it couldn't cleanly make anything like generators (since it had no
way to make lexical closures).

That's right, generators would have been quite difficult to do in
MacLisp. But a Lisp Machine (with stack groups) could have done them,
and did, with or without closures.
No not really, they (in SICP) are at best more like class instances
with a method that mutates some state. There's nothing like a yield
statement in the idiom.

Right -- I wrote "iterators", not "generators".
You could do it with call/cc but SICP just uses ordinary closures to
implement streams.

Yes, that's right.
At least you could spare our eyesight by writing that as
(mapcar #'blah xs) ;-).

Good point! But I just love lambda -- even when I'm just using it as
a NOP.... (Also I couldn't remember the syntax for accessing the
function property of a symbol in MacLisp.)
The point is that mapcar (as the name implies) advances down a list
using cdr, i.e. it only operates on lists, not general iterators or
streams or whatever.

Right, but each sequence type had it's own corresponding mapping
fuctions.
x = xs[key]
I'm not sure what you are asserting? That Common Lisp doesn't have
hash tables? That's certainly not the case. Or that it doesn't
provide standard generic functions for accessing them
The latter. Of course there are getf/setf, but those are necessarily
macros.

Right. OO on primitive data types is kind of hard in a non OO
language. So, when writing an application in MacLisp, or Lisp Machine
lisp, I might have had to spend a bit of time writing an application
framework that provided the OO features I needed. This was not
particularly hard to do in Lisp, but surely not nearly as nice as if
they had standardized such things. This would not be particularly
difficult to do, other than the getting everyone to agree on just what
the interfaces should be. But Lisp programmers, are of course, just
as recalcitrant as Python programmers.
Python is not Perl and it has in principle always been defined by its
reference manual,

And in Python's case, the reference manual is just an incomplete
description of the features offered by the implementation, and people
revel in features that are not yet in the reference manual.
though until fairly recently it's fostered a style of relying on
various ugly CPython artifacts like the reference counting GC.

That's not ugly. The fact that CPython has a reference-counting GC
makes the lifetime of object predictable, which means that like in
C++, and unlike in Java, you can use destructors to good effect. This
is one of the huge boons of C++. The predictability of lifespan makes
the language more expressive and powerful. The move to deprecate
relying on this feature in Python is a bad thing, if you ask me, and
removes one of the advantages that Python had over Lisp.
Lisp accumulated a lot of cruft over the decades and it kept some
baggage that it really could have done without.

Indeed -- true of most languages. Of course, there have been quite a
few Lisp dialects that have been cleaned up in quite a few ways (e.g.,
Dylan), but they, of course, have a hard time achieving any
significant traction.
I don't think Python's designers learned nearly as much from Lisp as
they could have, and Python has suffered because of it.
Amen.

Lisp still has an awesome beauty in both the CL and Scheme
incarnations.
Indeed.

But it's like listening to Elvis music--even if it can still get you
dancing, at the end of the day it's still a reflection of a bygone
culture.

Lisp is more like The Beatles.

And it's not bygone -- it's just nichified. Lisp is forever -- you'll
see.

|>oug
 
P

Paul Rubin

Douglas Alan said:
I'm aware, but there were various different mapping functions. "map",
as opposed to "mapcar" didn't return any values at all, and so you had
to rely on side effects with it.

The thing is there was no standard way in Maclisp to write something
like Python's "count" function and map over it. This could be done in
Scheme with streams, of course.
Right -- I wrote "iterators", not "generators".

Python iterators (the __iter__ methods on classes) are written with
yield statements as often as not.
Right, but each sequence type had it's own corresponding mapping fuctions.

Precisely, I think that's what Alexander was trying to get across, Lisp
didn't have a uniform interface for traversing different types of sequence.
they had standardized such things. This would not be particularly
difficult to do, other than the getting everyone to agree on just what
the interfaces should be. But Lisp programmers, are of course, just
as recalcitrant as Python programmers.

Python programmers tend to accept what the language gives them and use
it and not try to subvert it too much. I don't say that is good or bad.
And in Python's case, the reference manual is just an incomplete
description of the features offered by the implementation, and people
revel in features that are not yet in the reference manual.

No I don't think so, unless you count some things that are in accepted
PEP's and therefore can be considered part of the reference docs, even
though they haven't yet been merged into the manual.
That's not ugly. The fact that CPython has a reference-counting GC
makes the lifetime of object predictable, which means that like in
C++, and unlike in Java, you can use destructors to good effect. This
is one of the huge boons of C++. The predictability of lifespan makes
the language more expressive and powerful. The move to deprecate
relying on this feature in Python is a bad thing, if you ask me, and
removes one of the advantages that Python had over Lisp.

No that's wrong, C++ has no GC at all, reference counting or
otherwise, so its destructors only run when the object is manually
released or goes out of scope. The compiler normally doesn't attempt
lifetime analysis and it would probably be against the rules to free
an object as soon as it became inaccessible anyway. Python (as of
2.5) does that using the new "with" statement, which finally makes it
possible to escape from that losing GC-dependent idiom. The "with"
statement handles most cases that C++ destructors normally handle.

Python object lifetimes are in fact NOT predictable because the ref
counting doesn't (and can't) pick up cyclic structure. Occasionally a
cyclic GC comes along and frees up cyclic garbage, so some destructors
don't get run til then. Of course you can manually organize your code
so that stuff with destructors don't land in cyclic structures, but
now you don't really have automatic GC any more, you have (partially)
manual storage management. And the refcounts are a performance pig in
multithreaded code, because of how often they have to be incremented
and updated. That's why CPython has the notorious GIL (a giant lock
around the whole interpreter that stops more than one interpreter
thread from being active at a time), because putting locks on the
refcounts (someone tried in the late 90's) to allow multi-cpu
parallelism slows the interpreter to a crawl.

Meanwhile 4-core x86 cpu's are shipping on the desktop, and network
servers not dependent on the complex x86 architecture are using
16-core MIPS processors (www.movidis.com). Python is taking a beating
all the time because of its inability to use parallel cpu's, and it's
only going to get worse unless/until PyPy fixes the situation. And
that means serious GC instead of ref counting.
And it's not bygone -- it's just nichified. Lisp is forever -- you'll see.

Lisp may always be around in some tiny niche but its use as a
large-scale systems development language has stopped making sense.

If you want to see something really pathetic, hang out on
comp.lang.forth sometime. It's just amazing how unaware the
inhabitants there are of how irrelevant their language has become.
Lisp isn't that far gone yet, but it's getting more and more like that.
 
A

Andy Freeman

Precisely, I think that's what Alexander was trying to get across, Lisp
didn't have a uniform interface for traversing different types of sequence.

And he's wrong, at least as far as common lisp is concerned - map does
exactly that.

http://www.lispworks.com/documentation/HyperSpec/Body/f_map.htm

Map doesn't work on generators or iterators because they're not part
of the common lisp spec, but if someone implemented them as a library,
said library could easily include a map that handled them as well.
 
A

Andy Freeman

Map doesn't work on generators or iterators because they're not part
of the common lisp spec, but if someone implemented them as a library,
said library could easily include a map that handled them as well.

Note that this is is a consequence of something that Python does
better than lisp. Far more parts of python are defined in terms of
named operations which are data-type independent. As a result, they
work on things that the implementor (or spec) never considered.

That said, it's no big deal for a lisp program that needed an enhanced
map that also understands iterators and generators to use it.

Compare that with what a programmer using Python 2.4 has to do if
she'd like the functionality provided by 2.5's with statement. Yes,
with is "just syntax", but it's extremely useful syntax, syntax that
can be easily implemented with lisp-style macros.
 
P

Paul Rubin

Andy Freeman said:
And he's wrong, at least as far as common lisp is concerned - map does
exactly that.

http://www.lispworks.com/documentation/HyperSpec/Body/f_map.htm

"sequence" there just means vectors and lists.
Map doesn't work on generators or iterators because they're not part
of the common lisp spec, but if someone implemented them as a library,
said library could easily include a map that handled them as well.

Right, more scattered special purpose kludges instead of a powerful
uniform interface.
 
P

Paul Rubin

Andy Freeman said:
Compare that with what a programmer using Python 2.4 has to do if
she'd like the functionality provided by 2.5's with statement. Yes,
with is "just syntax", but it's extremely useful syntax, syntax that
can be easily implemented with lisp-style macros.

Not really. The with statement's binding targets all have to support
the protocol, which means a lot of different libraries need redesign.
You can't do that with macros. Macros can handle some narrow special
cases such as file-like objects, handled in Python with
contextlib.closing.

That said, the with statement was missing from Python for much too
long, since users were happy to rely on reference counting.
 
A

Andy Freeman

Not really.

Yes really, as the relevant PEP shows. The "it works like" pseudo-
code is very close to how it would be defined with lisp-style macros.
The with statement's binding targets all have to support
the protocol, which means a lot of different libraries need redesign.

That's a different problem, and it's reasonably solvable for anyone
who wants to use the roll-your-own with while writing an application
running under 2.4. (You just add the relevant methods to the
appropriate classes.)

The big obstacle is the syntax of the with-statement. There's no way
to define it in python with user-code.
 
G

Graham Breed

Douglas Alan wote:
Graham Breed said:
Another way is to decorate functions with their local variables:
from strict import my
@my("item")
... def f(x=1, y=2.5, z=[1,2,4]):
... x = float(x)
... w = float(y)
... return [item+x-y for item in z]

Well, I suppose that's a bit better than the previous suggestion, but
(1) it breaks the style rule of not declaring variables until you need
them, and (2) it doesn't catch double initialization.

(1) is a style rule that many style guides explicitly violate. What
is (2) and why would it be a problem?

A better way that I think is fine syntactically would be

from strict import norebind, set
@norebind
def f(x=1, y=2.5, z=[1.2.4]):
set(x=float(x))
set(w=float(y))
return [item+x-y for item in z]

It won't work because the Python semantics don't allow a function to
alter a nested namespace. Or for a decorator to get at the locals of
the function it's decorating. It's an example of Python restricting
flexibility, certainly.
No, the best way to catch false rebindings is to have the computers
catch such errors for you. That's what you pay them for.

How does the computer know which rebindings are false unless you tell
it?
And how do I easily do that? And how do I know if I even need to in
the face of sometimes subtle bugs?

In UNIX, you do it by putting this line in a batch file:

egrep -H 'rebound' $* | egrep -v '^[^:]+:[[:space:]]*([.[:alnum:]]+)
[[:space:]]*=(|.*[^."])\<\1\>'

You don't know you need to do it, of course. Like you wouldn't know
you needed to use the let and set macros if that were possible.
Automated checks are only useful for problems you know you might have.
They're not that uncommon, either.

The 300-odd line file I happened to have open had no examples of the
form x = f(x). There was one rebinding of an argument, such as:

if something is None:
something = default_value

but that's not the case you were worried about. If you've decided it
does worry you after all there may be a decorator/function pattern
that can check that no new variables have been declared up to a
certain point.

I also checked a 400-odd file which has one rebinding that the search
caught. And also this line:

m, n = n, m%n

which isn't of the form I was searching for. Neither would the set()
solution above be valid, or the substitution below. I'm sure it can
be done with regular expressions, but they'd get complicated. The
best way would be to use a parser, but unfortunately I don't
understand the current Python grammar for assignments. I'd certainly
be interested to see how your proposed macros would handle this kind
of thing.

This is important because the Python syntax is complicated enough that
you have to be careful playing around with it. Getting macros to work
the way you want with results acceptable to the general community
looks like a huge viper pit to me. That may be why you're being so
vague about the implementation, and why no macro advocates have
managed to get a PEP together. A preprocessor that can read in
modified Python syntax and output some form of real Python might do
what you want. It's something you could work on as a third-party
extension and it should be able to do anything macros can.


That aside, the short code sample I give below does have a rebinding
of exactly the form you were worried about. It's still idiomatic for
text substitutions and so code with a lot of text substitutions will
likely have a lot of rebindings. You could give each substituted text
a different name. I think that makes some sense because if you're
changing the text you should give it a name to reflect the changes.
But it's still error prone: you might use the wrong (valid) name
subsequently. Better is to check for unused variables.
I've certainly had it happen to me on several occasions, and sometimes
they've been hard to find as I might not even see the mispeling even
if I read the code 20 times.

With vim, all you have to do is go to the relevant line and type ^* to
check that the two names are really the same. I see you use Emacs but
I'm sure that has an equivalent.
(Like the time I spent all day trying to figure out why my assembly
code wasn't working when I was a student and finally I decided to ask
the TA for help, and while talking him through my code so that he
could tell me what I was doing wrong, I finally noticed the "rO" where
there was supposed to be an "r0". It's amazing how useful a TA can
be, while doing nothing at all!)

Assembly then, not Python.
Maybe PyLint is better than Lint for C was (hated it!), but my idea of
RAD does not include wading through piles of useless warning messages
looking for the needle warning in the warning haystack. Or running
any other programs in the midst of my code, run, code, run, ..., loop.

Well, that's the choice you have I'm afraid. Either the computer
nannies you about spurious bugs in your code or you miss genuine
bugs. I don't see why it makes a difference for an external tool to
do the nannying. Checking for unused variables is the obvious way to
check for spelling mistakes in Python.
As it is, I code in Python the way that a normal Python programmer
would, and when I have a bug, I track it down through sometimes
painstaking debugging as a normal Python programmer would. Just as
any other normal Python programmer, I would not use the alternatives
suggested so far, as I'd find them cumbersome and inelegant. I'd
prefer not to have been bit by the bugs to begin with. Consequently,
I'd use let and set statements, if they were provided (or if I could
implement them), just as I have the equivalents to let and set in
every other programming language that I commonly program in other than
Python.

Except that, unlike a normal Python programmer, you don't want to use
Python syntax.

I have, out of interest, written a Python script to convert from a
Python-like language with let and set into real Python. It's really
not that difficult. It's full of holes, of course, but I'm not
convinced that macros would be any different. Here it is:

import sys, re, os

general_match = r"(?m)^(\s*)%s(\s+)(\w+)(\s*=)"
general_insert = r"\1assert '\3' %s in locals()\n\1\3\4"

let_spec = general_match % "let", general_insert % "not"
set_spec = general_match % "set", general_insert % ""
escape_spec = general_match % r"\\([ls]et)", r"\1\2\3\4\5"

inputfile = sys.argv[1]
outputfile = os.path.splitext(inputfile)[0] + os.path.extsep + 'py'

code = open(inputfile).read()

for match, insert in let_spec, set_spec, escape_spec:
code = re.sub(match, insert, code) # rebound

open(outputfile, 'w').write(code)


Graham
 
D

Douglas Alan

Not really. The with statement's binding targets all have to support
the protocol, which means a lot of different libraries need redesign.
You can't do that with macros.

But that's a library issue, not a language issue. The technology
exists completely within Lisp to accomplish these things, and most
Lisp programmers even know how to do this, as application frameworks
in Lisp often do this kind. The problem is getting anything put into
the standard. Standardizing committees just suck.

I just saw a presentation today on the Boost library for C++. This
project started because the standard library for C++ is woefully
inadequate for today's programming needs, but any chance of getting
big additions into the standard library will take 5-10 years.
Apparently this is true for all computer language standards. And even
then, the new standard will be seriously lacking, because it is
usually based on armchair thinking rather than real-world usage.

So the Boost guys are making a defacto standard (or so they hope)
library for C++ that has more of the stuff you want, and then when the
standardizing committees get around to revising the actual standard,
the new standard will already be in wide use, meaning they just have
to sign off on it (and perhaps suggest a few tweaks).

Alas, the Lisp standards are stuck in this sort of morass, even while
many implementations do all the right things.

Python doesn't have this problem because it operates like Boost to
begin with, rather than having a zillion implementations tracking some
slow moving standard that then mandates things that might be nearly
impossible to implement, while leaving out much of what people need.

But then again, neither do many dialects of Lisp, which are developed
more or less like Python is. But then they aren't standards
compliant, and so they don't receive wide adoption.
Macros can handle some narrow special cases such as file-like
objects, handled in Python with contextlib.closing.

Macros handle the language part of things in Lisp perfectly well in
this regard. But you are right -- they certainly can't make
standardizing committees do the right thing.

|>oug
 
D

Douglas Alan

The thing is there was no standard way in Maclisp to write something
like Python's "count" function and map over it. This could be done in
Scheme with streams, of course.

I'm not sure that you can blame MacLisp for not being object-oriented.
The idea hadn't even been invented yet when MacLisp was implemented
(unless you count Simula). If someone went to make an OO version of
MacLisp, I'm sure they'd get all this more or less right, and people
have certainly implemented dialects of Lisp that are consistently OO.
Python iterators (the __iter__ methods on classes) are written with
yield statements as often as not.

I certainly agree that iterators can be implemented with generators,
but generators are a language feature that are impossible to provide
without deep language support, while iterators are just an OO
interface that any OO language can provide. Though without a good
macro facility the syntax to use them may not be so nice.
No that's wrong, C++ has no GC at all, reference counting or
otherwise, so its destructors only run when the object is manually
released or goes out of scope.

Right, but implementing generic reference-counted smart pointers is
about a page of code in C++, and nearly every large C++ application
I've seen uses such things.
Python (as of 2.5) does that using the new "with" statement, which
finally makes it possible to escape from that losing GC-dependent
idiom. The "with" statement handles most cases that C++ destructors
normally handle.

Gee, that's back to the future with 1975 Lisp technology. Destructors
are a much better model for dealing with such things (see not *all*
good ideas come from Lisp -- a few come from C++) and I am dismayed
that Python is deprecating their use in favor of explicit resource
management. Explicit resource management means needlessly verbose
code and more opportunity for resource leaks.

The C++ folks feel so strongly about this, that they refuse to provide
"finally", and insist instead that you use destructors and RAII to do
resource deallocation. Personally, I think that's taking things a bit
too far, but I'd rather it be that way than lose the usefulness of
destructors and have to use "when" or "finally" to explicitly
deallocate resources.
Python object lifetimes are in fact NOT predictable because the ref
counting doesn't (and can't) pick up cyclic structure.

Right, but that doesn't mean that 99.9% of the time, the programmer
can't immediately tell that cycles aren't going to be an issue.

I love having a *real* garbage collector, but I've also dealt with C++
programs that are 100,000+ lines long and I wrote plenty of Python
code before it had a real garbage collector, and I never had any
problem with cyclic data structures causing leaks. Cycles are really
not all that common, and when they do occur, it's usually not very
difficult to figure out where to add a few lines to a destructor to
break the cycle.
And the refcounts are a performance pig in multithreaded code,
because of how often they have to be incremented and updated.

I'm willing to pay the performance penalty to have the advantage of
not having to use constructs like "when".

Also, I'm not convinced that it has to be a huge performance hit.
Some Lisp implementations had a 1,2,3, many (or something like that)
reference-counter for reclaiming short-lived objects. This bypassed
the real GC and was considered a performance optimization. (It was
probably on a Lisp Machine, though, where they had special hardware to
help.)
That's why CPython has the notorious GIL (a giant lock around the
whole interpreter that stops more than one interpreter thread from
being active at a time), because putting locks on the refcounts
(someone tried in the late 90's) to allow multi-cpu parallelism
slows the interpreter to a crawl.

All due to the ref-counter? I find this really hard to believe.
People write multi-threaded code all the time in C++ and also use
smart pointers at the same time. I'm sure they have to be a bit
careful, but they certainly don't require a GIL.

I *would* believe that getting rid of the GIL will require some
massive hacking on the Python interpreter, though, and when doing that
it may be significantly easier to switch to having only a real GC than
having two different kinds of automatic memory management.

I vote, though, for putting in that extra work -- compatibility with
Jython be damned.
Lisp may always be around in some tiny niche but its use as a
large-scale systems development language has stopped making sense.

It still makes perfect sense for AI research. I'm not sure that
Lisp's market share counts as "tiny". It's certainly not huge, at
only 0.669% according to the TIOBE metric, but that's still the 15th
most popular language and ahead of Cobol, Fortran, Matlab, IDL, R, and
many other languages that are still in wide use. (Cobol is probably
still around for legacy reasons, but that's not true for the other
languages I mentioned.)
If you want to see something really pathetic, hang out on
comp.lang.forth sometime. It's just amazing how unaware the
inhabitants there are of how irrelevant their language has become.
Lisp isn't that far gone yet, but it's getting more and more like that.

Forth, eh. A chaque son gout, but I'd be willing to bet that most
Forth hackers don't believe that Forth is going to make a huge
resurgence and take over the world. And it still has something of a
place as the core of Postscript and maybe in some embedded systems.

Re Lisp, though, there used to be a joke (which turned out to be
false), which went, "I don't know what the most popular programming
language will be in 20 years, but it will be called 'Fortran'". In
reality, I don't know what the most popular language will be called 20
years from now, but it will *be* Lisp.

|>oug
 
D

Dennis Lee Bieber

Re Lisp, though, there used to be a joke (which turned out to be
false), which went, "I don't know what the most popular programming
language will be in 20 years, but it will be called 'Fortran'". In
reality, I don't know what the most popular language will be called 20
years from now, but it will *be* Lisp.
Call me when LISP looks like FORTRAN... (I spent 22 years working
FORTRAN IV and FORTRAN 77 -- hence my emphasis on the capitalized
FORTRAN rather than the 90/95 decree that the language, in general and
future is "Fortran")


So far, all this thread on "macros" has shown me is a desire to
create a meta-Python... A language for creating incompatible DIALECTS of
Python.

What happens when two individuals release "libraries" using these
proposed macros -- and have implement conflicting macros using the same
identifiers -- and you try to use both libraries in one application?

Maybe I'm isolated -- I've only encountered one meta language that
made sense... The Xerox "Meta-Symbol" assembler -- in which the native
instruction set for the Sigma series CPUs was /not/ "native" (it was an
"include" file of symbol definitions)... Meta-Symbol, at the core, was a
language for defining assembly languages (out of boredom one day, I
scribbled the Intel 8080 assembly statements as Meta-Symbol definitions
-- Meta-Symbol defined means to reference fields in the label, command,
and address fields, and how many bits those fields took... )
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/
 
P

Paul Rubin

Dennis Lee Bieber said:
What happens when two individuals release "libraries" using these
proposed macros -- and have implement conflicting macros using the same
identifiers -- and you try to use both libraries in one application?

Something like the current situation with Python web frameworks ;)
 
P

Paul Rubin

Douglas Alan said:
I'm not sure that you can blame MacLisp for not being object-oriented.
The idea hadn't even been invented yet when MacLisp was implemented
(unless you count Simula). If someone went to make an OO version of
MacLisp, I'm sure they'd get all this more or less right, and people
have certainly implemented dialects of Lisp that are consistently OO.

count() has nothing to with OO, it's just the infinite stream 1,2,3,...
which can be implemented as a Scheme closure the obvious way.
Right, but implementing generic reference-counted smart pointers is
about a page of code in C++, and nearly every large C++ application
I've seen uses such things.

That's because C++ has no GC.
Gee, that's back to the future with 1975 Lisp technology. Destructors
are a much better model for dealing with such things (see not *all*
good ideas come from Lisp -- a few come from C++) and I am dismayed
that Python is deprecating their use in favor of explicit resource
management. Explicit resource management means needlessly verbose
code and more opportunity for resource leaks.

And relying on refcounts to free a resource at a particular time is
precisely explicit resource management. What if something makes a
copy of the pointer that you didn't keep track of? The whole idea of
so-called smart pointers is to not have to keep track of them after
all. Anything like destructors are not in the spirit of GC at all.
The idea of GC is to be invisible, so the language semantics can be
defined as if all objects stay around forever. GC should only reclaim
something if there is no way to know that it is gone. For stuff like
file handles, you can tell whether they are gone or not, for example
by trying to unmount the file system. Therefore they should not be
managed by GC.
Also, I'm not convinced that it has to be a huge performance hit.
Some Lisp implementations had a 1,2,3, many (or something like that)
reference-counter for reclaiming short-lived objects. This bypassed
the real GC and was considered a performance optimization. (It was
probably on a Lisp Machine, though, where they had special hardware to
help.)

That is a common technique and it's usually done with just one bit.
All due to the ref-counter? I find this really hard to believe.
People write multi-threaded code all the time in C++ and also use
smart pointers at the same time. I'm sure they have to be a bit
careful, but they certainly don't require a GIL.

Heap allocation in C++ is a relatively heavyweight process that's used
sort of sparingly. C++ code normally uses a combination of static
allocation (fixed objects and globals), stack allocation (automatic
variables), immediate values (fixnums), and (when necessary) heap
allocation. In CPython, *everything* is on the heap, even small
integers, so saying "x = 3" has to bump the refcount for the integer 3.
There is a LOT more refcount bashing in CPython than there would be
in something like a Boost application.
It still makes perfect sense for AI research.

I think most AI research is being done in other languages nowadays.
There are some legacy Lisp systems around and some die-hards but
I have the impression newer stuff is being done in ML or Haskell.

I personally use Emacs Lisp every day and I think Hedgehog Lisp (a
tiny functional Lisp dialect intended for embedded platforms like cell
phones--the runtime is just 20 kbytes) is a very cool piece of code.
But using CL for new, large system development just seems crazy today.
Re Lisp, though, there used to be a joke (which turned out to be
false), which went, "I don't know what the most popular programming
language will be in 20 years, but it will be called 'Fortran'". In
reality, I don't know what the most popular language will be called 20
years from now, but it will *be* Lisp.

Well, they say APL is a perfect crystal--if you add anything to it, it
becomes flaws; while Lisp is a ball of mud--you can throw in anything
you want and it's still Lisp. However I don't believe for an instant
that large system development in 2027 will be done in anything like CL.

See:

http://www.cs.princeton.edu/~dpw/popl/06/Tim-POPL.ppt
http://www.st.cs.uni-sb.de/edu/seminare/2005/advanced-fp/docs/sweeny.pdf

(both are the same presentation, the links are the original Powerpoint
version and a pdf conversion) for a game developer discussing his
experiences with a 500 KLOC C++ program, describing where he thinks
things are going. I find it compelling. Mostly he wants a
dependently typed, pure functional language with explicit escapes to
impurity in places, with a "lenient" evaluation strategy, i.e. sort of
a softcore version of Haskell with less polymorphism but with
dependent types for stuff like sized arrays. This is in order to get
concurrency and parallelism automatically without dealing with threads
and locks, which of course are even worse than malloc/free in terms of
programmer insanity and pitfalls.

Everyone is running dual core x86's on their desktop and 4-core cpu's
are starting to appear, but the x86 is so complex that 4 cores per
package seems to be the maximum at the moment. But some non-x86
designs are sporting as many as 16 cores on a chip (www.movidis.com)
in general purpose computers, to say nothing of specialized ones like
the Cell processor. We have run out of megahertz and we are going
multicore. And if we don't start adopting languages that support
parallelism sanely, we'll be left in the dust by those languages, just
as Python (or Lisp or Java or whatever) have left non-GC languages in
the dust.
 

Members online

No members online now.

Forum statistics

Threads
474,444
Messages
2,571,709
Members
48,796
Latest member
Greg L.
Top