does python have useless destructors?

[David Turner]

Not really. What you're doing is what I'd call "virtual stack" by
virtue of the fact that the heap objects are being managed by stack
objects.

Having read this through a second time, I'm not sure that you
understood the C++ code I posted. So here is an equivalent in Python:

class File:
def __init__(self, name):
self.fh = open(name, "r")
def __del__(self):
self.fh.close()

file_list = []
file_list.append(File(file_to_compile))
while len(file_list):
f = file_list[len(file_list)-1]
t = Token(f)
if t == Token.EOF:
file_list.pop()
else:
parse(t)


No stack objects in sight, yet this code is semantically equivalent to
the C++ code.

Don't let red herrings like std::stack confuse you .

Regards
David Turner

 

[Marcin 'Qrczak' Kowalczyk]

So what I'll do is write a PEP to fix the exception handling semantics
in CPython, and hope that pressure from users who discover how much
easier it is to write RAII -style code will eventually introduce
reference counting to Jython and friends.

Jython will not switch to reference counting, and programs relying on
destructors run immediately are still broken, relying on an implementation
detail of CPython.

 

[Manlio Perillo]

[...]

Hi.
Since __del__ isn't really 'useful', *maybe* a better solution is to
add another special method for classes, ad example a __finalize__
method.
Such method, if present, *will* be called during stack unwinding.
So, in __finalize__ one can release 'external' resources.

class file:
__init__(self, ...): ...

__finalize__(self): self.close()

...


Another useful addition could be to add a 'block' statement:

a_file = '...'

block:
f = open(a_file)
f.write(...)


As an example, block can be supposed to be equivalent to:

block:
statement

=>

def __block()
statement
__block()



Regards Manlio Perillo

 

[Marcin 'Qrczak' Kowalczyk]

Since __del__ isn't really 'useful', *maybe* a better solution is to
add another special method for classes, ad example a __finalize__
method.
Such method, if present, *will* be called during stack unwinding.

Called for all objects referred to by local variables, even those which
are also referred to by other variables? It does not make sense. For
example if you pass a file to a function, and the function temporarily
stores the file in its local variable, it would close it.

Called for objects which don't have other references? It's unimplementable
efficiently, and in a way which breaks easy integration with the garbage
collected host language.

 

[Michael Hudson]

[...]

Hi.
Since __del__ isn't really 'useful', *maybe* a better solution is to
add another special method for classes, ad example a __finalize__
method.

Yes! I'm not sure __finalize__ is really the best name, but that's
for another day.

Such method, if present, *will* be called during stack unwinding.
So, in __finalize__ one can release 'external' resources.

class file:
__init__(self, ...): ...

__finalize__(self): self.close()

...


Another useful addition could be to add a 'block' statement:

a_file = '...'

block:
f = open(a_file)
f.write(...)


As an example, block can be supposed to be equivalent to:

block:
statement

=>

def __block()
statement
__block()

I would urge everyone participating in this thread to read PEP 310,
the email conversation linked therein and (optional) *understand* it.

Hmm, good sigmonster

Cheers,
mwh

 

[David Turner]

For those of us not up on such things, could you explain the differences?

Destruction is what happens to objects in C++. C++ does not manage
your memory for you, so destruction happens at very specific times:

1. For stack-based objects, when the object "goes out of scope".
2. For heap-based objects, when the object is explicitly destroyed
with the delete operator.

Finalization is what happens to objects in Java. Java manages your
memory for you, so finalization happens at very specific times:

1. When the object is garbage collected.

The difficulty here is of course that garbage collection is
unpredictable, and finalization may not happen at all.

So essentially, when we say "destruction", we mean an operation that
is called clean up the object, where the programmer has some sort of
control over when it happens; when we say "finalization" we mean an
operation that is called to clean up the object, where the programmer
has no control over when it happens.

It's perfectly possible to have both destruction and finalization.
Here is one way of doing it in Python:

-----
class Both:
def __init__(self):
self.refcount = 0
def copy(self):
self.refcount += 1
return self
def unref(self):
self.refcount -= 1
if self.refcount == 0:
self.destruct()
def destruct(self):
print "destruction"
def __del__(self):
print "finalization"

if __name__ == "__main__":
x = Both().copy()
y = x.copy()
x.unref()
y.unref() # "destruction" printed at this point

# "finalization" probably printed here
-----

Hmm... I wonder if it would be possible to override the "="
operator...?

Regards
David Turner

 

[Marcin 'Qrczak' Kowalczyk]

Hmm... I wonder if it would be possible to override the "="
operator...?

No, and it would not be enough anyway. You would have to catch places
when an argument is passed to a function, or is returned, or when a local
variable goes out of scope, or when an object is put in a container,
or perhaps others that I forgot. It's impractical.

 

[Michael P. Soulier]

That is not true: the data is not lost. The file is closed eventually
(e.g. when Python exits), in which case the data is flushed to disk.

On Unix it is. I am unsure about recent versions of windows, but back
when I was coding pascal on Win95, I found out the hard way that if the
process exits without closing the descriptor, whatever is in the buffer
is lost. Welcome to windows.

Mike

 

[Marcin 'Qrczak' Kowalczyk]

On Unix it is. I am unsure about recent versions of windows, but back
when I was coding pascal on Win95, I found out the hard way that if the
process exits without closing the descriptor, whatever is in the buffer
is lost. Welcome to windows.

It doesn't depend on the OS but on the language or compiler. Borland
Pascal didn't flush files when the process exited, the OS could do nothing
about it because the data was in process' internal buffers.

 

[Aahz]

Hmm... I wonder if it would be possible to override the "="
operator...?

This comment indicates that you don't understand how Python works. In
Python, ``=`` is a *statement*, not an operator. This is more confusing
now than it used to be because of the augmented assignment operators
(that can only be used in a statement context). In any event, you can't
override ``=``.

 

[Donn Cave]

Yes! I'm not sure __finalize__ is really the best name, but that's
for another day.

Couldn't the name be __del__? Given the opportunity to have
both, and the assurance that __finalize__ will be called and
__del__ might not, what functionality would you leave in __del__?

I would urge everyone participating in this thread to read PEP 310,
the email conversation linked therein and (optional) *understand* it.

It seems to be superficially similar to finalization, but so
constrained that it's architecturally inconsequential - I mean,
it's by definition interchangeable with a try/finally construct,
so there isn't any potential code architecture where you can say
`couldn't do this without with'.

I guess there isn't much point in proposing improvements for
finalization and __del__, as long as there's a commitment to
support Python on a platform without support for finalization
like Java. But never one to be deterred by pointlessness,
suppose __finalize__ were a flag, instead of a method. It
has two functions: 1. declare that the object's __del__
method should be called when it's logically unreferenced -
either no references, or only referenced as part of a cycle
or traceback. 2. Serve as the extra reference count that's
needed for this, so __del__ will only be called once regardless
of further reference decrements, cycle analysis etc. Default
would be no __finalize__, but it should probably be added to
some class/types, e.g., fileobject.

Donn Cave, (e-mail address removed)

 

[Marcin 'Qrczak' Kowalczyk]

But never one to be deterred by pointlessness,
suppose __finalize__ were a flag, instead of a method. It
has two functions: 1. declare that the object's __del__
method should be called when it's logically unreferenced -
either no references, or only referenced as part of a cycle
or traceback. 2. Serve as the extra reference count that's
needed for this, so __del__ will only be called once regardless
of further reference decrements, cycle analysis etc.

I will repeat: it's unimplementable efficiently when
Python runtime is hosted by a language with non-refcount GC.

 

[Slawomir Nowaczyk]

On 15 Jun 2004 11:38:48 -0400
(e-mail address removed) (Aahz) wrote:

#> In article <[email protected]>,

#>> Hmm... I wonder if it would be possible to override the "="
#>> operator...?

#> This comment indicates that you don't understand how Python works. In
#> Python, ``=`` is a *statement*, not an operator.

Well, technically, "a=1" is a statement (Reference manual, 6.3
Assignment statements). The symbol "=" itself is a 'grammar
delimiter' (Reference manual, 2.6 Delimiters), which is neither
operator nor statement.

Nevertheless, there is no way to override "=".

--
Best wishes,
Slawomir Nowaczyk
( (e-mail address removed) )

Zawinski's Law: "Every program attempts to expand until it can read mail.
Those programs which cannot so expand are replaced by ones which can."

 

[Manlio Perillo]

Couldn't the name be __del__? Given the opportunity to have
both, and the assurance that __finalize__ will be called and
__del__ might not, what functionality would you leave in __del__?

Since objects construction is a two phase operation, maybe also the
destruction can be realized in two phases:
__new__ -> __init__ -> ... -> __deinit__ -> __del__

I guess there isn't much point in proposing improvements for
finalization and __del__, as long as there's a commitment to
support Python on a platform without support for finalization
like Java. But never one to be deterred by pointlessness,
suppose __finalize__ were a flag, instead of a method. It
has two functions: 1. declare that the object's __del__
method should be called when it's logically unreferenced -
either no references, or only referenced as part of a cycle
or traceback. 2. Serve as the extra reference count that's
needed for this, so __del__ will only be called once regardless
of further reference decrements, cycle analysis etc. Default
would be no __finalize__, but it should probably be added to
some class/types, e.g., fileobject.

Another solution would be to have an attribute for every classes:
del_func.
del_func by default is None and it is called during stack unwinding.
(Even if there are references to the object)

Programmer that want to use the C++ RAII pattern can do:
obj.del_func = obj.__del__



Regards Manlio Perillo

 

[Michael Hudson]

Couldn't the name be __del__?

As I think I said in one of the emails in the thread linked to from
PEP 310, life would be much easier if it wasn't.

Given the opportunity to have both, and the assurance that
__finalize__ will be called and __del__ might not, what
functionality would you leave in __del__?

None at all! This is my cunning plan...

It seems to be superficially similar to finalization,

OK, I've found this thread pretty hard to follow. What is
"finalization" in context?

but so constrained that it's architecturally inconsequential - I
mean, it's by definition interchangeable with a try/finally
construct, so there isn't any potential code architecture where you
can say `couldn't do this without with'.

Indeed.

Cheers,
mwh

 

[David Turner]

I will repeat: it's unimplementable efficiently when
Python runtime is hosted by a language with non-refcount GC.

So are we to take it that efficiency considerations are a serious
impediment to a potentially valuable safety feature?

Regards
David Turner

 

[Isaac To]

David> So are we to take it that efficiency considerations are a serious
David> impediment to a potentially valuable safety feature?

I tried to think in exactly this way a couple of days ago. But soon I
concluded that we already got what we can possibly get: CPython is actually
garbage collected with reference counting, dealing with loops as a special
case. There the Python runtime implements everything, and as such it can
dictate a slower but perhaps more predictable method as garbage collection.
Jython runtime do not try to manage objects, and leave it to the Java
runtime. There is no way to implement reference counting there, unless one
want to give up the possibility of using Java objects in Jython and vice
versa. And still we cannot rule out a possibility that we will have our
compiled counterpart (Psyco?) finding that reference counting is a
perfermance bottleneck, and in that case it is very legitimate for it to
change reference counting method to something else like copy collection. So
for program to be run in CPython only, one may find it convenient to assume
that reference counting is used. But for any portable code, one must be
more careful---exactly our current situation. We need no more discussion
about how to collect garbages. On the other hand, PEP 310 discusses
something completely unrelated, which still should attract some eyeballs.

Regards,
Isaac.

 

[Donn Cave]

So are we to take it that efficiency considerations are a serious
impediment to a potentially valuable safety feature?

Gross efficiency considerations in the Java implementation,
specifically.

Donn Cave, (e-mail address removed)

 

[Donn Cave]

As I think I said in one of the emails in the thread linked to from
PEP 310, life would be much easier if it wasn't.


None at all! This is my cunning plan...

You're too deep for me.

OK, I've found this thread pretty hard to follow. What is
"finalization" in context?

Operational definition would be `call __del__ or its C equivalent',
at the effective end of the object's lifetime. That's the way
I understand it - notify object that its time is up, let it do
its final things.

Donn Cave, (e-mail address removed)

 

[Michael Hudson]

Operational definition would be `call __del__ or its C equivalent',
at the effective end of the object's lifetime.

OK. I claim you can't really have that, and that you don't really
need it anyway. The idea behind PEP 310 is to acheive the ends of
RAII in C++ by different means.

What else do you want to use __del__ methods for?

Cheers,
mwh