super in Python 3 and variadic arguments

[Marco Buttu]

Given this class:
.... def afoo(*args):
.... print(args)

in Python 3 we can write the following class:
.... def bfoo(*args):
.... super(B, args[0]).afoo(*args[1:])
....(<__main__.B object at 0x7f5b3bde48d0>, 1, 2, 3)


without giving arguments to super, in this way:
.... def bfoo(self, *args):
.... super().afoo(*args)
........ def bfoo(*args):
.... super().afoo(*args[1:])
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in bfoo
RuntimeError: super(): no arguments

How come?

 

[Ned Batchelder]

Given this class:
... def afoo(*args):
... print(args)

in Python 3 we can write the following class:
... def bfoo(*args):
... super(B, args[0]).afoo(*args[1:])
...(<__main__.B object at 0x7f5b3bde48d0>, 1, 2, 3)


without giving arguments to super, in this way:
... def bfoo(self, *args):
... super().afoo(*args)
...... def bfoo(*args):
... super().afoo(*args[1:])
...Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in bfoo
RuntimeError: super(): no arguments

How come?

The no-args super() call inspects the calling environment to determine
the class and self. "self" is the first local name stored in
frame.f_code.co_localsplus, but *args doesn't put "args" into that entry
of the code object. Basically, super() is looking for the first regular
argument in the function.

--Ned.

 

[Marco Buttu]

class B(A):

... def bfoo(*args):
... super().afoo(*args[1:])
...

B().bfoo(1, 2, 3)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in bfoo
RuntimeError: super(): no arguments

How come?

The no-args super() call inspects the calling environment to determine
the class and self. "self" is the first local name stored in
frame.f_code.co_localsplus, but *args doesn't put "args" into that entry
of the code object

But is it a bug or the behavior we want? The first (implicit) argument
is stored as expected as the first one in the args tuple, and the args
tuple is inserted as expected in frame.f_locals:
.... def bfoo(*args):
.... frame = inspect.currentframe()
.... for obj, value in frame.f_locals.items():
.... print(obj, value, sep=' --> ')
.... # super().afoo(*args[1:])
....args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3)
frame --> <frame object at 0x7f28cad4b240>

So, why does not super use it?

 

[Ned Batchelder]

class B(A):
... def bfoo(*args):
... super().afoo(*args[1:])
...
B().bfoo(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in bfoo
RuntimeError: super(): no arguments

How come?

The no-args super() call inspects the calling environment to determine
the class and self. "self" is the first local name stored in
frame.f_code.co_localsplus, but *args doesn't put "args" into that entry
of the code object

But is it a bug or the behavior we want? The first (implicit) argument
is stored as expected as the first one in the args tuple, and the args
tuple is inserted as expected in frame.f_locals:
... def bfoo(*args):
... frame = inspect.currentframe()
... for obj, value in frame.f_locals.items():
... print(obj, value, sep=' --> ')
... # super().afoo(*args[1:])
...args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3)
frame --> <frame object at 0x7f28cad4b240>

So, why does not super use it?

I haven't seen the discussion that decided the behavior of super(), but
I'd guess that if you reported this as a bug, it would be closed as
wontfix, because: 1) the use case you describe isn't something people
actually write, 2) it would add to the complexity of super() to support
it, and 3) there's a simple way to write your code that does work:

class B(A):
def bfoo(self, *args):
super().afoo(*args)

(though it's a bit odd to call afoo from bfoo.)

Python has never claimed the kind of purity that makes everything work
in a totally simple consistent way. super() with no args is a kind of
hack to begin with. It involves a special case in the compiler (so that
using the name "super" as a function call will act as if you had
accessed the name "__class__" so that super can find it later), and
inspecting the stack frame during execution.

It's an interesting case of the Zen of Python. It violates one
("explicit is better than implicit"), but only because of another one
("practicality beats purity"). super(MyClass, self) in Python 2 is the
kind of brain-bender that so many people get wrong at first, that it's
helped plenty of people to do the arguments implicitly, even if there
are oddball edge cases that it doesn't seem to handle properly.

--Ned.

 

[Marco Buttu]

import inspect
class B(A):

... def bfoo(*args):
... frame = inspect.currentframe()
... for obj, value in frame.f_locals.items():
... print(obj, value, sep=' --> ')
... # super().afoo(*args[1:])
...

B().bfoo(1, 2, 3)

args --> (<__main__.B object at 0x7f28c960a590>, 1, 2, 3)
frame --> <frame object at 0x7f28cad4b240>

So, why does not super use it?

Python has never claimed the kind of purity that makes everything work
in a totally simple consistent way. super() with no args is a kind of
hack to begin with. It involves a special case in the compiler (so that
using the name "super" as a function call will act as if you had
accessed the name "__class__" so that super can find it later), and
inspecting the stack frame during execution.

It seems reasonable

It's an interesting case of the Zen of Python. It violates one
("explicit is better than implicit"), but only because of another one
("practicality beats purity"). super(MyClass, self) in Python 2 is the
kind of brain-bender that so many people get wrong at first, that it's
helped plenty of people to do the arguments implicitly, even if there
are oddball edge cases that it doesn't seem to handle properly.

--Ned.

Thanks for the comprehensive answer

 

[Steven D'Aprano]

super() with no args is a kind of hack to begin with. It involves a
special case in the compiler (so that using the name "super" as a
function call will act as if you had accessed the name "__class__" so
that super can find it later), and inspecting the stack frame during
execution.

super() with no arguments is *completely* a hack[1], and one where GvR
has said "Never again!" if I remember correctly. I don't think he regrets
allowing the super compile-time magic, just that it really is magic and
he doesn't want to make a habit of it.

One of the side-effects of this being a hack is that this doesn't work:

class X(Y):
def method(self, arg):
f = super
f().method(arg)





[1] Which is not necessarily a bad thing!

 

[Ian Kelly]

One of the side-effects of this being a hack is that this doesn't work:

class X(Y):
def method(self, arg):
f = super
f().method(arg)

Actually, that works just fine. The compiler sees that super is
accessed within the method and creates the closure necessary to make
it work. This does fail, however:

f = super
class X(Y):
def method(self, arg):
f().method(arg)

 

[Steven D'Aprano]

Actually, that works just fine. The compiler sees that super is
accessed within the method and creates the closure necessary to make it
work. This does fail, however:

f = super
class X(Y):
def method(self, arg):
f().method(arg)

Ah, that's the one! Thanks for the correction.

I'll now go and write "I will always test my code snippets before
posting" on the blackboard one hundred times.

 

[Chris Angelico]

I'll now go and write "I will always test my code snippets before
posting" on the blackboard one hundred times.

print("I will always test my code snippets before posting\n"*100)

ChrisA

PS. Irony would be having a bug in that because I didn't test it. I
almost didn't, but remembered just before hitting Send.

 

[Marco Buttu]

super() with no arguments is*completely* a hack[1], and one where GvR
has said "Never again!" if I remember correctly. I don't think he regrets
allowing the super compile-time magic, just that it really is magic and
he doesn't want to make a habit of it.
....
[1] Which is not necessarily a bad thing!

Thanks a lot for this anecdote

 

[Marco Buttu]

Actually, that works just fine. The compiler sees that super is
accessed within the method and creates the closure necessary to make
it work. This does fail, however:

f = super
class X(Y):
def method(self, arg):
f().method(arg)

Very interesting! Thanks