BIG successes of Lisp (was ...)

B

Bjorn Pettersen

Right... all along until CPython introduces a more elaborate
gc scheme.

.... which is highly unlikely to happen without preservation of reference
counting semantics for files... google if you're _really_ interested :-/
Note that reference-counting has problems with cyclic
references; probably not something that will bite you in the case of
open files, but definitely a problem you need to be aware of.

(a) true (by definition of cycle/ref.counting?)
(b) no relevance to CPython (which uses a generational collector
to reclaim reference cycles [with the regular finalizer problems]).
(c) why are open file objects special (are you saying objects with no
internal references are less likely to be reachable from a cycle,
or that there is something intrinsically special about (open?) files)?

-- bjorn
 
R

Raymond Wiker

Bjorn Pettersen said:
Note that reference-counting has problems with cyclic
references; probably not something that will bite you in the case of
open files, but definitely a problem you need to be aware of.

(a) true (by definition of cycle/ref.counting?)
(b) no relevance to CPython (which uses a generational collector
to reclaim reference cycles [with the regular finalizer problems]).

I didn't know that (obviously :)
(c) why are open file objects special (are you saying objects with no
internal references are less likely to be reachable from a cycle,
or that there is something intrinsically special about (open?) files)?

I cannot think of any scenarios where open files would be
involved in a cycle, that's all. That doesn't mean that it's impossible.

--
Raymond Wiker Mail: (e-mail address removed)
Senior Software Engineer Web: http://www.fast.no/
Fast Search & Transfer ASA Phone: +47 23 01 11 60
P.O. Box 1677 Vika Fax: +47 35 54 87 99
NO-0120 Oslo, NORWAY Mob: +47 48 01 11 60

Try FAST Search: http://alltheweb.com/
 
B

Bjorn Pettersen

I cannot think of any scenarios where open files would be
involved in a cycle, that's all. That doesn't mean that it's
impossible.

Ok. Btw., creating such a cycle is easy,

a = [open('foo')]
b = [a]
a.append(b)

(i.e. any file that's reachable from a cycle). IIRC, A commonly used
'markup' library up until recently kept open file handles for all inline
images as part of the internal object state -- although I'd perhaps call
that more of a bug than a scenario? <wink>

-- bjorn
 
I

Ingvar Mattsson

Bjorn Pettersen said:
Note that reference-counting has problems with cyclic
references; probably not something that will bite you in the case of
open files, but definitely a problem you need to be aware of.

(a) true (by definition of cycle/ref.counting?)
(b) no relevance to CPython (which uses a generational collector
to reclaim reference cycles [with the regular finalizer problems]).
(c) why are open file objects special (are you saying objects with no
internal references are less likely to be reachable from a cycle,
or that there is something intrinsically special about (open?) files)?

Open file handles is a "scarce resource". It wasn't uncommon to have
an upper limit of 32, including strin, stdout and stderr, in older
unix environments. These days, 256 or 1024 seems to be more common
(default) upper limits. usually not a problem, but if one does a lot
of "open, write" and leave it for the GC to clean up, one *may* hit
resource starvation problems one wouldn't have hit, had teh file
handles been clsoed in a timely fashion.

//Ingvar
 
A

Anton Vredegoor

One thing that has become clear in neuroscience is that almost all
(perhaps literally all) parts and functions of the brain benefit from
learning. Explicit memory is quite distinct from other memory
processes - it serves the conscious mind in a way that other memory
processes do not.

At a more extreme level, entire memories can be fabricated. The harder
you search for memories, the more they are filled in by made up stuff.
And as mentioned elsewhere, the brain is quite willing to invent
rationalisations for things where it cannot provide a real reason. Add
a psychiatrist prompting and providing hints as to the expected form
of the 'memory' and hey presto!

Many people seem to have an obsession with a kind of mystic view of
consciousness. Go through the list of things that people raise as
being part of consciousness, and judge it entirely by that list, and
it becomes just another set of cognitive functions - working memory,
primarily - combined with the rather obvious fact that you can't have
a useful understanding of the world unless you have a useful
understanding of your impact on it.

Sorry that I had to cut your long (but excellent) post to comment only
on a small subset of it. I want to reply only to the part that
concerns memory and consciousness, because I feel that you are having
a problem with these while the answer is already present in other
parts of your post.

According to Merleau-Ponty there is something to "the whole is more
than the sum of the parts" and other gestalt-like consciousness
mysticism. He explains this as the effect that having a sense of
direction and purpose gives each of the components that are involved a
new meaning such that they all operate coherently.

For example a primate can use a stick as an instrument, but a human
can use an instrument to build other instruments, thereby potentially
transforming the meanings of *all* objects in the environment, and
thereby finally being able to rearrange the environment itself or
gaining the powers to transport to some other place with such
efficient methods that the effect is virtually the same as rearranging
the environment.

My understanding of this point is that this effect is also seen very
prominently in the way we remember things. But contrary to your
position I see this rearranging of the constituent parts of our memory
-in a certain sense the basis of our consciousness- not as something
"fabricated" or "rationalized" (or in some other semi-pejorative
terms).

The way we structure our past is a positive accomplishment and we
should value it as such instead of insisting that there is only one
past and that any deviation from the "standard" (who's standard
anyway?) is showing some tendency to "evade problems" or gain unlawful
advantages over others.

In fact there is no fixed past at all, and the past that we have now
can be changed very easily, without using time-machines even. The
complete reality whether individual or shared is just a figment of our
collective or individual imagination and is therefore subject to
change without warning.

Ok, let me try to back up these bold claims with some trivial examples
of how our individual and collective pasts can be changed or are
changed or have been changed.

First a little "Gedankenexperiment", suppose you have been luckily
married for over twenty years and now I'm showing you a photograph
(supposedly of the kind that can't be forged) of your partner in a
compromising situation with someone else. I can back up this data with
several other facts, accumulated over the years, each of which you
have no prior knowledge of. Does this change your past, or just your
image of the past, so that it now comes closer to the "real" past?

If one looks at this example carefully one might wonder if this newer,
more "real", past is not also a figment of our imagination, and so on.

Another example, suppose someone on clpy who's first name is Alex came
into the possession of some convincing information linking corporate
culture and social factors to the divergence of programming styles and
data formats, in fact proving without a shred of doubt to him that
social factors are for 95% or more responsible for the generation of
divergent cultures of programming (with the accompanying dialects of
programming languages) through the process of "math-envy". This would
lead to some interesting new perspectives on macros being responsible
for the diverging of communities of programmers, making them into
mechanisms rather than sources. This could lead to a new world view
where from now on not macros would be avoided, but rather a certain
way of competition of programmers which would be identified as
unhealthy for the community as a whole. Would this change the past?
Certainly the whole macro-wars history of clpy would gain a new
interpretation.

Next consider the event of the twin tower airplane attack. Suppose it
was finally publicly acknowledged that there where no foreign nations
involved in the attack but that it was just a rather small group of
individuals acting on their own, backed by some very rich capitalist
but still only a small group of people acting in their own selfish
interest. Also no weapons of mass destruction could be found in Iraq,
Hussein had no connection with Osama bin Laden, and so on. This would
make the American attacks on two countries into war crimes and would
lay open speculations about the oil-industry being behind these so
called anti terrorism activities. Taking civil rights away from the
citizens of the united states could be another motivating factor for
certain powerful groups in the American government. Of course this is
all nonsense and pure speculation, but a quick view of what is still
written in Japanese history books about world war two, or an
investigation of what Iraqi school children had to learn about Saddam
Hussein will quickly disperse any doubts about what is possible in the
ways of completely misrepresenting collective memory.

Another example, imagine you had been suffering from a stomach ulcer
that was incurable and had taken drugs for years in order to reduce
the acidity of your stomach content. Someday someone comes along and
mentions that it's all caused by a bacterium called helicobacter and
you could be cured in two weeks. If you believed this person, would
this change your personal past?

There *is* no fixed past. What is possible is to make a past that is
purposeful, makes one strong and directed and flexible and open to
change. We should start seeing our pasts as an accomplishment and an
asset, and -realizing the moldability of it all- begin healthy
cooperations with other people to synchronize our pasts, incorporating
all the little "facts" of all people into a grand unifying theory in
which all these past episodes have a new meaning, and where everything
can find its place.

If you still think that's not magic, it's probably better to keep it
that way in order not to compromise the wonderful life that is
theoretically possible, given these observations.

Anton
 
S

Stephen Horne

There *is* no fixed past.

So you are saying that if a psychiatrist convinces his patient that
she was abused by her parents as a child, even though it never
occurred, to the point that she can remember it - then it must be
true!

I'm not confused on this - you are.

Sorry, but you are not keeping a clear distinction between "the past"
and "knowledge of the past". If memory is altered then it doesn't
change the past, only the (now incorrect) memory.

This sounds a lot like extreme cultural relativism. But I'm afraid
that if you drive your car past a village of people with no knowledge
of cars, while they may believe that your car is propelled by a demon
under the hood, in reality the engine will still be there.

Perception is not reality. It is only a (potentially flawed)
representation of reality. But reality is still real. And perception
*is* tied to reality as well as it can be by the simple pragmatic
principle of evolution - if our ancestors had arbitrary perceptions
which were detached from reality, they could not have survived and had
children.

You'll find many people who claim otherwise, of course. But the limits
of perception are actually already better understood than most people
realise. On the whole, they are pretty much the limits of information
processing.
 
R

Rainer Deyke

Raymond said:
Right... all along until CPython introduces a more elaborate
gc scheme.

Note that reference-counting has problems with cyclic
references; probably not something that will bite you in the case of
open files, but definitely a problem you need to be aware of.

I'm all for more elaborate gc schemes. In particular, I want one that gives
me guaranteed immediate destructors even in the presence of reference
cycles. And I want to language specification to guarantee it.

Having to explicitly close files is something I'd expect in C or assembly,
but it has no place in high-level languages. And "with-open-file" is too
limiting. Let the computer do the work, not the programmer.
 
F

Frode Vatvedt Fjeld

Rainer Deyke said:
I'm all for more elaborate gc schemes. In particular, I want one
that gives me guaranteed immediate destructors even in the presence
of reference cycles. And I want to language specification to
guarantee it.

Having to explicitly close files is something I'd expect in C or
assembly, but it has no place in high-level languages. And
"with-open-file" is too limiting. Let the computer do the work, not
the programmer.

This is the problem with languages that are part excellent and part
work in progress (read: crap). You learn to trust the language from
using the excellent parts (which tend to be the basic things you first
encounter), and when you eventually stumble upon the not-so-great
parts you trust those parts, too. It's extremely unfortunate, IMHO.
 
J

Jon S. Anthony

Rainer Deyke said:
I'm all for more elaborate gc schemes. In particular, I want one
that gives me guaranteed immediate destructors even in the presence
of reference cycles. And I want to language specification to
guarantee it.

It's difficult to even put a requirement for "gc" in a language
_specification_ let alone one that _guarantees_ this sort of thing.

Having to explicitly close files is something I'd expect in C or
assembly, but it has no place in high-level languages. And
"with-open-file" is too limiting. Let the computer do the work, not
the programmer.

What's limiting about it? Or are you saying you want the computer (or
I suppose the language spec) to somehow also open your files for you
as well? What would that even mean? If you don't require that, then
you have to explicitly ask for the file to be opened. Well, that's
what with-open-file is.

/Jon
 
A

Anton Vredegoor

Stephen Horne said:
Perception is not reality. It is only a (potentially flawed)
representation of reality. But reality is still real. And perception
*is* tied to reality as well as it can be by the simple pragmatic
principle of evolution - if our ancestors had arbitrary perceptions
which were detached from reality, they could not have survived and had
children.

If we had the same perceptions about reality as our ancestors, we
wouldn't reproduce very successfully now, because the world has
changed as a result of their reproductive success.

Anton
 
R

Rainer Deyke

Jon said:
It's difficult to even put a requirement for "gc" in a language
_specification_ let alone one that _guarantees_ this sort of thing.



What's limiting about it?

Unless I've completely misunderstood the concept, "with-open-file" requires
me to structure my program such that the file is opened at the beginning and
closed at the end of a block of code. In the absence of coroutines or
threads (both of which would significantly complicate the structure of my
code), this implies that the lifetime of open files is strictly
hierarchical: if file 'a' is opened while file 'b' is open, then file 'b'
must be closed before file 'a' is closed. More generally, it implies that
the file is opened at the beginning of a natural code block (instead of, for
example, on first use) and is closed at the end of the same block (instead
of, for example, when it is no longer needed).
 
T

Thomas F. Burdick

Bjorn Pettersen said:
... which is highly unlikely to happen without preservation of reference
counting semantics for files... google if you're _really_ interested :-/

I hope you put assertion in your code so that it won't run under
Jython, because that's the kind of insidious bug that would be
*aweful* to find the hard way.

(I really don't understand why you wouldn't just want at least real
closures, so you can just use call_with_open_file, and not have to
worry about what GC you're using when *opening* *files*)

--
/|_ .-----------------------.
,' .\ / | No to Imperialist war |
,--' _,' | Wage class war! |
/ / `-----------------------'
( -. |
| ) |
(`-. '--.)
`. )----'
 
P

Peter Seibel

Rainer Deyke said:
Unless I've completely misunderstood the concept, "with-open-file"
requires me to structure my program such that the file is opened at
the beginning and closed at the end of a block of code.

I don't know about "completely" but you've certainly misunderstood an
important point. Yes, WITH-OPEN-FILE attempts to open a file, executes
a block of code with the open stream bound to a variable, and then
closes the stream when the block is exited, even if something goes
wrong such as a condition being signaled.

But--and this is the bit I think you may have missed--if that's *not*
what you want you don't use WITH-OPEN-FILE. Lisp also provides OPEN
and CLOSE, that act as you'd expect and allow you to explicitly
control the lifetime of the file stream.

Which gets us back to Pascal's original point--WITH-OPEN-FILE is a
macro that abstracts a set of operations in a concise way. (In
addition to ensuring that the file is closed, it also allows us to
easily express what to do in certain situations while trying to open
the file--what to do if the file already exists or doesn't exist,
etc.)

-Peter
 
J

Jon S. Anthony

Rainer Deyke said:
Unless I've completely misunderstood the concept, "with-open-file" requires
me to structure my program such that the file is opened at the beginning and
closed at the end of a block of code.

No it does not require this - it is what you use when this is the
natural expression for what you would want.
In the absence of coroutines or threads (both of which would
significantly complicate the structure of my code), this implies
that the lifetime of open files is strictly hierarchical: if file
'a' is opened while file 'b' is open, then file 'b' must be closed
before file 'a' is closed.

No, even within the with-open-file of b you could close a whenever or
wherever you wanted. The with-open-file for a just guarantees certain
things when you leave the scope.
More generally, it implies that the file is opened at the beginning
of a natural code block

Well, the block that it itself opens.

and is closed at the end of the same block .

Is guaranteed to be closed. You could close it before this if that
made sense.

(instead of, for example, on first use) ... (instead of, for
example, when it is no longer needed)

OK. If your code needs to be more flexible in this, you would use
OPEN and/or CLOSE. If you had a particular open/close paradigm in
mind you could then write an abstraction (macro) akin to
with-open-file which provides behavior like what you say here and it
would generate the calls to open and close. I'm sure there are
scenarios where this might not be "reasonably doable".

/Jon
 
R

Robin Becker

Stephen Horne said:
.......
Perception is not reality. It is only a (potentially flawed)
representation of reality. But reality is still real. And perception
*is* tied to reality as well as it can be by the simple pragmatic
principle of evolution - if our ancestors had arbitrary perceptions
which were detached from reality, they could not have survived and had
children.
......
It is a commonplace of developmental psychology that the persistence of
objects is learned by children at some relatively young age (normally 3
months as I recall). I assume that children learn the persistence of
hidden objects by some statistical mechanism ie if it happens often
enough it must be true (setting up enough neural connections etc).

Would the reality of children subjected to a world where hidden objects
were somehow randomly 'disappeared' be more or less objective then that
of normal children.

Unlucky experimental cats brought up in vertical stripe worlds were
completely unable to perceive horizontals so their later reality was
apparently filled with invisible and mysterious objects. I can't
remember if they could do better by rotating their heads, but even that
would be quite weird.

I think it unwise to make strong statements about reality when we know
so little about it. Apparently now the universe is 90-95% stuff we don't
know anything about and we only found that out in the last 10 years.
 
R

Rainer Deyke

Peter said:
But--and this is the bit I think you may have missed--if that's *not*
what you want you don't use WITH-OPEN-FILE. Lisp also provides OPEN
and CLOSE, that act as you'd expect and allow you to explicitly
control the lifetime of the file stream.

Which leads us back to having to manually close files.

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?
 
M

Matthew Danish

[...] and is closed at the end of the same block (instead of, for
example, when it is no longer needed).

The reasoning behind WITH-OPEN-FILE is that file streams are resources
that should be allocated with dynamic extent, not indefinite. To make
that more concrete, it means that file streams should be closed
immediately after they are no longer being used (much like local
variables in C are immediately destroyed when the function returns).

I have seen debates before that argue that memory should be the only
resource managed by GC, and the opposite view that every resource should
be managable by GC (I don't think this is quite as feasible, but you can
look up the discussions on google, a year ago on c.l.l or so).

But WITH-OPEN-FILE doesn't prevent you from, say, having a file open the
entire time your program is running. Some people pointed out that you
can use OPEN and CLOSE, but also this would work fine:

(defvar *log-stream*) ;; an unbound special variable

(defun log (fmt-control &rest args)
(apply 'format *log-stream* fmt-control args))

(defun start (&key (log-file "log"))
(with-open-file (*log-stream* log-file
:direction :eek:utput
:if-exists :append)
;; *log-stream* is bound until this body exits
(call-other-functions)))
 
M

Matthew Danish

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

I already discussed this in another post.
What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?

One major difference between C++ and Lisp/Python is that C++ lacks
closures. This means that local variables do not have to be considered
for allocation on the heap. So saying

{
ofstream f;
f.open ("output");
...
// I presume ofstream dtor closes file? I'm a bit rusty
}

is okay in C++ to obtain a similar effect as WITH-OPEN-FILE, though
I am not sure how well it does when it comes to abnormal conditions.

In Common Lisp, and presumably now Python, lexical scope combined with
higher-order functions implies that variables may be captured in a
closure and have indefinite extent. In CL, also, the language does not
define memory management; leaving open the possibility for a better
solution than GC. CL's answer to the handling of dynamic-extent
resources is WITH-OPEN-FILE, and also the other operators from which
WITH-OPEN-FILE is defined (such as UNWIND-PROTECT).

One final question: do you expect C++ to clean up dynamically allocated
ofstream objects, automatically?
 
M

Marcin 'Qrczak' Kowalczyk

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

What I want to open a file and have it close automatically when I am done
with it. I can do that in C++. Why can't I do it in Python?

In C++ you must choose between a local variable for the stream (which is
equivalent to with-open-file) or dynamically allocated object (which is
equivalent to open & close, where close is spelled delete).

You can't say that you want the C++ way and you don't want explicit open &
close and with-open-file, because they are equivalent! They differ only in
syntax details, but the style of usage and limitations are the same.

The only Python's problem is that there is no nice way to pass an
anonymous function to with-open-file, and try:finally: doesn't look so
nice either. I'm not defending Python's lack of large anonymous functions
but the interface of open, close & with-open-file in general, irrespective
of the programming language.

In particular you shouldn't wish for GC to find unused files because it's
impossible without sacrificing other GC properties. No languages does that
except misused CPython (you shouldn't rely on that).
 
J

Joe Marshall

Rainer Deyke said:
Which leads us back to having to manually close files.

I DON'T want to manually close files. I DON'T want to deal with the
limitations of with-open-file. And, here's the important bit, I DON'T WANT
TO COMBINE OR CHOOSE BETWEEN THESE TWO METHODS, BOTH OF WHICH ARE FLAWED.

Let me see if I get this:

1. You don't want to tell the computer when you are done with the
file.

2. But the computer shouldn't wait until it can prove you are done
to close the file.

3. And you definitely are opposed to the idea of letting the
computer figure it out in general, but giving it hints when
it is important that it be prompt.

Do you want to tell the computer the file name, or should it read your
mind on that as well?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,774
Messages
2,569,599
Members
45,170
Latest member
Andrew1609
Top