what's the point of rpython?

[Paul Rubin]

Now that doesn't make sense at all. What is all this mutable stuff
you have to go through, and what does it have to do with the GIL-less
implementation?

foo.bar() has to look up bar in foo's attribute dictionary.

 

[Bryan Olson]

Correct, and, getting back to the point, an instance itself would not
require a mutex. The dict would need it, of course.

The dict is part of the object and some important slots are mutable.
What's more, if your point was to do away with the GIL without changing
Python semantics nor requiring heaping masses of locking, I fear you've
not fully grasped the problem.

Languages such as Java, C++, and C# do not require nearly as much
locking as Python because they are not nearly as dynamic. Consider how a
method is invoked. Java / C++ / C# can always resolve the method with no
locking; the data they need is fixed at link time. Python is much more
dynamic. A demo:


from __future__ import print_function

# A simple class hierarchy:

class Foo (object):
title = "Mr. Foo"

def identify(self):
print("I'm called", self.title)

class Bar (Foo):
title = "Ms. Bar"

class Jafo (Bar):
title = "Major Jafo"

dude = Jafo()


# Searches 5 dicts to find the function to call:

dude.identify()


# Class dicts are mutable:

def id(self):
print("I'm still called", self.title)

Jafo.identify = id
dude.identify()


# An object's class can change:

dude.__class__ = Bar
dude.identify()


# A class's base classes can change:

class Fu (object):
def identify(self):
print("Call me", self.title)

Bar.__bases__ = (Fu,)
dude.identify()


Result:I'm called Major Jafo
I'm still called Major Jafo
I'm called Ms. Bar
Call me Ms. Bar

In that first simple call of dude.identify(), Python looked up "dude" in
the module's (mutable) dict to find the object. Then it looked in
object's (mutable) dict, and did not find "identify". So it looked at
the object's (mutable) __class__ slot, and in that class's (mutable)
dict. It still did not find "identify", so it looked in the class's
(mutable) __bases__ slot, following Python's depth-first "object
protocol" and thus looking in what other (mutable) class dicts and
(mutable) __bases__ slots were required.

An object's __dict__ slot is *not* mutable; thus we could gain some
efficiency by protecting the object and its dict with the same lock. I
do not see a major win in Mr. Banks' point that we do not need to lock
the object, just its dict.

 

[Paul Rubin]

An object's __dict__ slot is *not* mutable; thus we could gain some
efficiency by protecting the object and its dict with the same lock. I
do not see a major win in Mr. Banks' point that we do not need to lock
the object, just its dict.

If the dict contents don't change often, maybe we could use an
STM-like approach to eliminate locks when reading. That would of
course require rework to just about every C function that accesses
Python objects.

 

[Carl Banks]

The dict is part of the object and some important slots are mutable.
What's more, if your point was to do away with the GIL without changing
Python semantics nor requiring heaping masses of locking, I fear you've
not fully grasped the problem.

If that's what you think I thought, I fear you haven't read anything
I've written.

[snip]

An object's __dict__ slot is *not* mutable; thus we could gain some
efficiency by protecting the object and its dict with the same lock. I
do not see a major win in Mr. Banks' point that we do not need to lock
the object, just its dict.

I'm not sure where you got the idea that I was claiming this was a
major win. I'm not sure where you got the idea that I claimed that
having to lock all mutable objects wouldn't be slow. For Pete's sake,
you followed up to a post where I *agreed* that it would be slow.


Carl Banks

 

[Bryan Olson]

Of course class instances aren't immutable types: they're not even
types.

Class instances my or may not be types, but that has nothing to do with
any point at issue here. I'm saying that class instances are usually,
mutable, contrary to your claim, "class instances are usually immutable".

Let me suggest that there is a distinction between an
immutable type and an immutable object.

Let me further suggest that Python's documentation is entirely clear:
instances of immutable types are immutable objects. Instances of mutable
types are generally mutable objects. For example, tuple is an immutable
type, and thus tuples are immutable; list is a mutable type, and thus
lists are mutable.

Immutable types are what you are talking about: it means that the type
provides usable mutator methods. (Whether they mutate the object
itself or some associated object doesn't matter.) Immutable objects
are a different thing: it means the object cannot change in memory.

Classes in Python are mutable types, usually. Class instances are
(except for the refcount) immutable objects, usually.

There's where we disagree. I assert that class instances are usually
mutable objects.

We usually talk about mutability of types, but mutability of objects
is appropriate for discussion as well. So I can't really agree with
your assessment that I wrong to call class instances immutable objects
aside from refcounts.

That confusion disappears once one grasps that instances of immutable
types are immutable objects.

BTW, here's a minor brain bender: immutable types are mutable objects.

Some brains are too easily bent. Python is one of the many
object-oriented languages that reifies types as run-time objects. I see
no point in going through Python's immutable types to examine if there
is any way to mutate the corresponding type objects.

Now that doesn't make sense at all. What is all this mutable stuff
you have to go through, and what does it have to do with the GIL-less
implementation? Can you explain further? Or are you just saying
it'll be slow.

I elaborated at some length in another strand of this thread.

 

[Carl Banks]

There's where we disagree. I assert that class instances are usually
mutable objects.

Nope, you're dead wrong, nothing more to it. The bits of a class
instance never change. The __dict__ is a mutable object. The class
instance itself isn't. It's not reasonable to call an object whose
bits can't change a mutable obect.

Anyway, all you're doing is distracting attention from my claim that
instance objects wouldn't need to be locked. They wouldn't, no matter
how mutable you insist these objects whose bits would never change
are.

Some brains are too easily bent.

[Snip attempt to take this comment seriously]

And some brains are so stodgy they can't even take a lighthearted
comment lightheartedly.


Carl Banks

 

[Hendrik van Rooyen]

Or you can argue that even when an argument is repeated indefinitely it
doesn't make it suddenly right.

This is no good.
It's a well known fact that anything I tell you three times is true.

To demonstrate:

Tim Rowe's post earlier in this thread was the funniest one in a long time.
Tim Rowe's post earlier in this thread was the funniest one in a long time.
Tim Rowe's post earlier in this thread was the funniest one in a long time.

see - it's true!

- Hendrik

 

[Hrvoje Niksic]

Nope, you're dead wrong, nothing more to it. The bits of a class
instance never change. The __dict__ is a mutable object. The class
instance itself isn't. It's not reasonable to call an object whose
bits can't change a mutable obect.

The "bits" of class instances can very well change.

class X(object): pass ....
x = X()
d = x.__dict__
x.__dict__ = {}
map(id, [d, x.__dict__])

[170329876, 170330012]

The Python cookbook even describes patterns that depend on this
operation working. Class instance's contents can also change if
__slots__ is in use, when its __class__ is assigned to (admittedly the
latter being a rare operation, but still).

Anyway, all you're doing is distracting attention from my claim that
instance objects wouldn't need to be locked. They wouldn't, no
matter how mutable you insist these objects whose bits would never
change are.

Only if you're not implementing Python, but another language that
doesn't support __slots__ and assignment to instance.__dict__.

 

[Gabriel Genellina]

Nope, you're dead wrong, nothing more to it. The bits of a class
instance never change. The __dict__ is a mutable object. The class
instance itself isn't. It's not reasonable to call an object whose
bits can't change a mutable obect.

Anyway, all you're doing is distracting attention from my claim that
instance objects wouldn't need to be locked. They wouldn't, no matter
how mutable you insist these objects whose bits would never change
are.

Me too, I don't get what you mean. Consider a list instance, it contains a
count of allocated elements, and a pointer to some memory block. They
change when the list is resized. This counts as "mutable" to me. I really
don't understand your claim.

 

[Paul Rubin]

Not only registered at the beginning of the function, but also (since
CPython uses C, not C++) explicitly unregistered at every point of
exit from the function. Emacs implements these as macros called GCPRO
and UNGCPRO, and they're very easy to get wrong. In a way, they are
even worse than the current Python INCREF/DECREF.

That's a fairly natural style in Lisp implementation and it is not
that difficult to code in. I've hacked inside Emacs and have written
another interpreter with a similar setup; it's certainly easier than
keeping track of refcounts in my experience. For one thing, you can
raise exceptions anywhere you want, and the stack unwind can clean up
the gc protection, but it can't know nearly as easily which refcounts
to adjust, unless you record all the increfs the same way as the
GCPROs.

 

[Steve Holden]

Of course class instances aren't immutable types: they're not even
types. Let me suggest that there is a distinction between an
immutable type and an immutable object.

Immutable types are what you are talking about: it means that the type
provides usable mutator methods. (Whether they mutate the object
itself or some associated object doesn't matter.) Immutable objects
are a different thing: it means the object cannot change in memory.

Classes in Python are mutable types, usually. Class instances are
(except for the refcount) immutable objects, usually.

We usually talk about mutability of types, but mutability of objects
is appropriate for discussion as well. So I can't really agree with
your assessment that I wrong to call class instances immutable objects
aside from refcounts.

BTW, here's a minor brain bender: immutable types are mutable objects.



Now that doesn't make sense at all. What is all this mutable stuff
you have to go through, and what does it have to do with the GIL-less
implementation? Can you explain further? Or are you just saying
it'll be slow.

OK, so we have recently discussed whether objects are values, whether
function arguments are passed by reference, whether names are
references, and now we are, I suspect, about to have a huge further
discussion on the meaning of "immutable".

Sometimes I start to find this eternal pedantry a little tedious. I
suspect it's time I once more dropped out of c.l.py for a while.

regards
Steve

 

[Carl Banks]

En Sat, 24 Jan 2009 06:06:02 -0200, Carl Banks <[email protected]>  
escribió:






Me too, I don't get what you mean. Consider a list instance, it contains a  
count of allocated elements, and a pointer to some memory block. They  
change when the list is resized. This counts as "mutable" to me. I really  
don't understand your claim.

Yeah, yeah, I know that, and in the bickering that ensued some aspects
of the original context were lost. I should really not have been
pulled into Bryan's strawman over the definition of immutable, since
it's just a label, I oughtn't give a damn what it's called, I only
care what it does. I didn't handle this repartee very well.

Anyway, it goes back to the original vision for a mark-and-sweep
Python language as I presented what seems like a long time ago.

I presented the type system that had three base metatypes instead of
the one base metatype we have now: immutable_type, mutable_type, and
mutable_dict_type. The default metatype for Python classes would be
mutable_dict_type, which is a type wherein the object itself would be
mutable but it would still have all the mutator methods __init__,
__setattr__, etc., but they could only act on the __dict__.
mutable_dict_types would not be allowed to define any slots, and
__dict__ wouldn't be reassignable. (However, it seems reasonable to
allow the base tp_new to accept a dict argument.)

OTOTH, list's metatype would be mutable_type, so the type object
itself would be mutable.

Bryan claimed that that would be a very different language from
Python, apparently because it hadn't occurred to him that by-and-
large, the instance itself doesn't change, only the dict does.
Perhaps Bryan was thinking of __dict__'s reassignability (that
certainly didn't occur to me); if he was I apologize for my snideness.

HAVING SAID THAT, I still still say what I proposed would not be a
radically different language from Python. A little different, of
course. Much slower, almost certainly.


Carl Banks

 

[Carl Banks]

Only if you're not implementing Python, but another language that
doesn't support __slots__ and assignment to instance.__dict__.

I am only going to say all Python types prior to 3.0 support classes
without __slots__, so while I agree that this would be a different
language, it wouldn't necessarily be "not Python".

(Python, of course, is what GvR says Python is, and he isn't going to
say that the language I presented is Python. No worries there!
I'm only saying that it is conceivably similar enough to be a
different version of Python. It would be a different language in the
same way that Python 2.6 is a different language from Python 3.0.)

Incidentally, the proposal does allow slots to be defined, but only
for actual mutable types, not for ordinary class instances.


Carl Banks

 

[Carl Banks]

I am only going to say all Python types prior to 3.0 support classes
without __slots__,

I made a mistake, and I don't want to risk confusion at this point.

"all Python ***versions** prior to 3.0"

and I am talking about old-style classes, of course. Prior to 2.2 no
classes at all supported slots.


Carl Banks

 

[Carl Banks]

The default metatype for Python classes would be
mutable_dict_type, which is a type wherein the object itself would be
mutable but it would still have all the mutator methods __init__,
__setattr__, etc., but they could only act on the __dict__.

Not wanting to risk confusion.

"The default metatype for Python classes would be mutable_dict_type,
which is a type wherein the object itself would be ***immutable*** but
it would still have all the mutator methods __init__, __setattr__,
etc., but they could only act on the __dict__."


Carl Banks

 

[Bryan Olson]

If the dict contents don't change often, maybe we could use an
STM-like approach to eliminate locks when reading. That would of
course require rework to just about every C function that accesses
Python objects.

I'm a fan of lock-free data structure and software transactional memory,
but I'm also a realist. Heck, I'm one of this group's outspoken
advocates of threaded architectures. Theoretical breakthroughs will
happen, but in real world of today, threads are great but GIL-less
Python is a loser.

Wherever Python is going, let's recognize that a scripting language that
rocks is better than any other kind of language that sucks.

 

[Steve Holden]

I'm a fan of lock-free data structure and software transactional memory,
but I'm also a realist. Heck, I'm one of this group's outspoken
advocates of threaded architectures. Theoretical breakthroughs will
happen, but in real world of today, threads are great but GIL-less
Python is a loser.

Wherever Python is going, let's recognize that a scripting language that
rocks is better than any other kind of language that sucks.

Guido, IIRC, has said that he's against any GIL-removal policy that
lowers performance on single-processor systems. Personally I'd be happy
if there were an *alternative* multi-processor implementation that was
slower for single-processor architectures and faster for
multi-processor, but I'm not about to start developing it.

regards
Steve

 

[Paul Rubin]

I'm a fan of lock-free data structure and software transactional
memory, but I'm also a realist. Heck, I'm one of this group's
outspoken advocates of threaded architectures. Theoretical
breakthroughs will happen, but in real world of today, threads are
great but GIL-less Python is a loser.

GIL-less Python (i.e. Jython) already exists and beats CPython in
performance a lot of the time, including on single processors.
Whether the GIL can be eliminated from CPython without massive rework
to every extension module ever written is a separate question, of
course. Jython can be viewed a proof of concept.

 

[Steve Holden]

GIL-less Python (i.e. Jython) already exists and beats CPython in
performance a lot of the time, including on single processors.
Whether the GIL can be eliminated from CPython without massive rework
to every extension module ever written is a separate question, of
course. Jython can be viewed a proof of concept.

<nods>. I think probably the GIL will never be extracted successfully.

Also IronPython and PyPy (though the latter only in concept for now, I
believe). Even Guido admits that CPython doesn't necessarily represent
the dominant future strain ...

regards
Steve

 

[Rhamphoryncus]

<nods>. I think probably the GIL will never be extracted successfully.

Also IronPython and PyPy (though the latter only in concept for now, I
believe). Even Guido admits that CPython doesn't necessarily represent
the dominant future strain ...

IMO it's possible to rewrite only the core while keeping the refcount
API for external compatibility, but a tracing GC API in portable C is
hideous. Enough to make me want to find or make a better
implementation language.