Why are so many built-in types inheritable?

[Fabiano Sidler]

Hi folks!

For debugging purposes I tried this:

--- snip ---
def foo(): pass
function = type(foo)

class PrintingFunction(function):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print args, kwargs
return function.__call__(self, args, kwargs)

class DebugMeta(type):
def __new__(self, name, bases, dict):
for name in dict:
if type(dict[name]) is function:
dict[name] = PrintingFunction(dict[name])

--- snap ---

Now I tought I were able to let all maethod of classes with DebugMeta as
metaclass print out their arguments. But I got the following sad error:

TypeError: Error when calling the metaclass bases
type 'function' is not an acceptable base type

That's awful, isn't it?
What could I do to get the above code working? (No, I disliked to re-
implement <type 'function'> without this unpleasant behaviour in Python.)

Greetings,
F. Sidler

 

[Fabiano Sidler]

Hello? Or, is there any obvious reason for this behaviour I don't see?

 

[Kent Johnson]

Hi folks!

For debugging purposes I tried this:

--- snip ---
def foo(): pass
function = type(foo)

class PrintingFunction(function):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print args, kwargs
return function.__call__(self, args, kwargs)

class DebugMeta(type):
def __new__(self, name, bases, dict):
for name in dict:
if type(dict[name]) is function:
dict[name] = PrintingFunction(dict[name])

--- snap ---

Now I tought I were able to let all maethod of classes with DebugMeta as
metaclass print out their arguments. But I got the following sad error:

TypeError: Error when calling the metaclass bases
type 'function' is not an acceptable base type

That's awful, isn't it?
What could I do to get the above code working? (No, I disliked to re-
implement <type 'function'> without this unpleasant behaviour in Python.)

You could do this with a simple decorator:
http://wiki.python.org/moin/PythonDecoratorLibrary#head-d4ce77c6d6e75aad25baf982f6fec0ff4b3653f4

or I think your class PrintingFunction would work as
class PrintingFunction(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print args, kwargs
return self.func(*args, **kwargs)

Kent

 

[Fabiano Sidler]

You could do this with a simple decorator:
http://wiki.python.org/moin/PythonDecoratorLibrary#head-d4ce77c6d6e75aad25baf982f6fec0ff4b3653f4

or I think your class PrintingFunction would work as
class PrintingFunction(object):
def __init__(self, func):
self.func = func
def __call__(self, *args, **kwargs):
print args, kwargs
return self.func(*args, **kwargs)

The problem with this is that the func_code attribute would contain
the code of PrintingFunction instead of func. What I wanted to do, is
to keep the original behaviour, i.e. set the variable __metaclass__ to
DebugMeta and so get debug output, without changing a function and
getting the original function's code object by the func_code
attribute, not PrintigFunction's one. That's why I *must* inherit from
<type 'function'>.

Greetings,
F. Sidler

 

[Ziga Seilnacht]

Fabiano Sidler wrote:

[snipped]

The problem with this is that the func_code attribute would contain
the code of PrintingFunction instead of func. What I wanted to do, is
to keep the original behaviour, i.e. set the variable __metaclass__ to
DebugMeta and so get debug output, without changing a function and
getting the original function's code object by the func_code
attribute, not PrintigFunction's one. That's why I *must* inherit from
<type 'function'>.

No, you don't have to:
.... def __init__(self, func):
.... object.__setattr__(self, 'func', func)
.... def __get__(self, obj, objtype):
.... return new.instancemethod(self, obj, objtype)
.... def __call__(self, *args, **namedargs):
.... print args, namedargs
.... func = object.__getattribute__(self, 'func')
.... return func(*args, **namedargs)
.... def __getattribute__(self, name):
.... func = object.__getattribute__(self, 'func')
.... return getattr(func, name)
.... def __setattr__(self, name, value):
.... func = object.__getattribute__(self, 'func')
.... setattr(func, name, value)
.... def __delattr__(self, name):
.... func = object.__getattribute__(self, 'func')
.... delattr(func, name)
........ def __new__(meta, name, bases, dict):
.... for name, obj in dict.iteritems():
.... if isinstance(obj, types.FunctionType):
.... dict[name] = DebugFunction(obj)
.... return type.__new__(meta, name, bases, dict)
........ __metaclass__ = DebugMeta
.... def spam(self, *args, **namedargs):
.... """Spam spam spam spam. Lovely spam! Wonderful spam!"""
.... pass
....<code object spam at ..., file "<stdin>", line 3>

Greetings,
F. Sidler

Ziga

 

[Fabiano Sidler]

No, you don't have to:

Okay, but I'd prefer!

[a lot of python code]

That's what I wanted to avoid. Additionally, the possibility to do it
this way doesn't make it reasonable that <type 'function'> is
inheritable. Are there any reasons for that?

Greetings,
F.Sidler

 

[Fabiano Sidler]

I really wanted to learn the reason for this, nothing else!

Greetings,
F. Sidler

 

[Georg Brandl]

I really wanted to learn the reason for this, nothing else!

I suspect performance reasons. Can't give you details but function
is used so often that it deserves special treatment.

Georg

 

[Antoon Pardon]

I suspect performance reasons. Can't give you details but function
is used so often that it deserves special treatment.

I would find this a bit odd. I think integers, tuples and lists
are used just as often if not more and they aren't treated special.

I for one would gladly treat some performance for the ability
to subclass slices.

class islice(slice):
...

doesn't work


And if I just write

class islice:

def __init__(self, start, stop, step):
self.start = start
self.stop = stop
self.step = step

then the following doesn't work:

lst = range(20)
sl = islice(2,6,None)
lst[sl]


So much for ducktyping.

 

[Georg Brandl]

I would find this a bit odd. I think integers, tuples and lists
are used just as often if not more and they aren't treated special.

Well, for integers, tuples and lists you can at least give useful
use cases for subclassing.

I now looked in the code, and the ability to subclass is given by a
flag in the type object. This is there in list or int, but not in function.

A little performance is saved by the fact that PyFunction_Check, which
verifies that an object is indeed a function, doesn't have to check for
subtypes.

So, if you want to make functions or slices subclassable, ask on python-dev!

Georg

 

[Michele Simionato]

I cannot find the reference now, but I remember Tim Peters saying some
time ago that the only
reason why FunctionType is not subclassable is that nobody bothered to
write a patch for it.

Michele Simionato

 

[Antoon Pardon]

I cannot find the reference now, but I remember Tim Peters saying some
time ago that the only
reason why FunctionType is not subclassable is that nobody bothered to
write a patch for it.

The question then is, why is there a need for such a patch?

I mean when types and classes became unified and ints, lists ... became
subclassable one would think that in all those new code that had to be
written, it wouldn't have been that difficult to see to it that all
types became subclassable. I find it hard to believe that in the
unification period the decision to make one type subclassable and
an other not was made solely on the basis that a patch was submitted
for the first but not for the other.

 

[Georg Brandl]

The question then is, why is there a need for such a patch?

I mean when types and classes became unified and ints, lists ... became
subclassable one would think that in all those new code that had to be
written, it wouldn't have been that difficult to see to it that all
types became subclassable. I find it hard to believe that in the
unification period the decision to make one type subclassable and
an other not was made solely on the basis that a patch was submitted
for the first but not for the other.

It's not that hard to understand, is it? Whoever made the builtin types new-
style types didn't add the BASETYPE flag to function or slice. Apparently
he thought it wasn't worth the effort as he couldn't imagine a use case for it.

So, when someone had liked them to be subclassable, he'd have written a patch.

Georg

 

[Antoon Pardon]

It's not that hard to understand, is it?

That depends.

Whoever made the builtin types new-
style types didn't add the BASETYPE flag to function or slice. Apparently
he thought it wasn't worth the effort as he couldn't imagine a use case for it.

Well that looks somewhat short sighted to me. It is also why python
seems to throws so many surprises at people.

My impression is that quite frequently people come here with a question
about why something doesn't work, that normally could be expected to
work.

The reason why it doesn't work then seems to boil down to the
developpers not taking the trouble of implementing something
in general but only for the cases for which they could imagine
a use case. Which means that when someone comes up with a use
case later he is stuck.

I know about practicality beating purity, but purity has it
practical aspects too. If the python people had been willing
to work a bit more at purity, that would have been a lot
of more practical for those who found something not working
as expected, although they had no reason to suspect so.

 

[Georg Brandl]

Well that looks somewhat short sighted to me. It is also why python
seems to throws so many surprises at people.

My impression is that quite frequently people come here with a question
about why something doesn't work, that normally could be expected to
work.

The reason why it doesn't work then seems to boil down to the
developpers not taking the trouble of implementing something
in general but only for the cases for which they could imagine
a use case. Which means that when someone comes up with a use
case later he is stuck.

I think you're overgeneralizing here. Do you have other examples of
such a strategy resulting in something that doesn't work although
it should?

Nota bene: Often developers run into a limitation that is the result
of a deliberate design choice, such as "why aren't strings mutable?"

I know about practicality beating purity, but purity has it
practical aspects too. If the python people had been willing
to work a bit more at purity, that would have been a lot
of more practical for those who found something not working
as expected, although they had no reason to suspect so.

I've told you already: if a developer wants a feature not currently
implemented, he/she can
- ask on python-dev why
- submit a feature request
- submit a patch

If he/she's not able to do one of these, he/she can at least convince some
other Python developer if the use case is strong enough.

Georg

 

[Antoon Pardon]

I think you're overgeneralizing here. Do you have other examples of
such a strategy resulting in something that doesn't work although
it should?

That is a very subjective question. I'm sure some will think
there is no reason why subclassing slices or functions should
work and so will not even consider this as something that
doesn't work but should.

But I will give you one example.

Consider the following:

lst[3:7:2]

What does it do? It constructs a slice object which is then used
as an index in lst.

So why doesn't this work:

fun(3:7:2)

What is wrong with expecting that a slice object would be constructed
here which would then be used as an argument for the function call?

Nota bene: Often developers run into a limitation that is the result
of a deliberate design choice, such as "why aren't strings mutable?"

Well that is fine, but in this case I haven't seen such a design
choice explained. On the contrary the only thing that I have
heard in this case is that is wasn't implemeted because noone
submitted a patch. So it seems hardly the result of a deliberate
design choice in this case.

I've told you already: if a developer wants a feature not currently
implemented, he/she can
- ask on python-dev why
- submit a feature request
- submit a patch

If he/she's not able to do one of these, he/she can at least convince some
other Python developer if the use case is strong enough.

Yes you told this already, and it ignores completely the point
I am trying to make. There is a point here beside convincing
the devolopers to implement this.

 

[Georg Brandl]

I think you're overgeneralizing here. Do you have other examples of
such a strategy resulting in something that doesn't work although
it should?

That is a very subjective question. I'm sure some will think
there is no reason why subclassing slices or functions should
work and so will not even consider this as something that
doesn't work but should.

But I will give you one example.

Consider the following:

lst[3:7:2]

What does it do? It constructs a slice object which is then used
as an index in lst.

Which wasn't true in older versions of Python. lst[x:y] was a special
syntax construct calling a special method __getslice__. Slice objects
were merely introduced to simplify index handling.

So why doesn't this work:

fun(3:7:2)

What is wrong with expecting that a slice object would be constructed
here which would then be used as an argument for the function call?

Point taken, now that slicing creates slice objects this would be
consistent. IIRC, there was indeed a PEP suggesting to introduce a range
literal which would have made this possible.

Well that is fine, but in this case I haven't seen such a design
choice explained. On the contrary the only thing that I have
heard in this case is that is wasn't implemeted because noone
submitted a patch. So it seems hardly the result of a deliberate
design choice in this case.


Yes you told this already, and it ignores completely the point
I am trying to make. There is a point here beside convincing
the devolopers to implement this.

Being? I mean, developer time isn't available en masse, and an overly
strict view on purity might sometimes actually prevent a feature being
implemented.

Georg

 

[Antoon Pardon]

Being? I mean, developer time isn't available en masse, and an overly
strict view on purity might sometimes actually prevent a feature being
implemented.

That there are different reasons why something is not implemented.
Something not implemented can be the result of a design choice.
This is how we want the language to look like, as a result something
like that will never be implemeted.

Or it can be the result of time constraints, yes we think this
should be implemented but unless someone else does it, this
is item 367 on out todo list.

Or it may be the result of an oversigth or something totally
different.


Before I'm considering going to py-dev and bother them with
an idea of mine, I would at least like to know that the
idea would be accepted as a good design choice within the
python philosophy.

So when argueing about good/bad design it doesn't help if
you already point to the next step.