default value in __init__

[bearophileHUGS]

Chris Rebert:

Although primitive and likely somewhat flawed, you may find the
statistics in the "Compatibility Issues" section ofhttp://mail.python.org/pipermail/python-3000/2007-February/005704.html
to be of interest.

I am quite glad to see that I am not the only one that cares for such
topic. And I think Guido is wrong here, but I can also see there's
little hope in fixing this Python wart. We'll probably have to see and
use another language to see this problem/bug fixed in some way (and
maybe other issues added, no language is perfect). Python is now
mature, being 15+ years old, so it's also rigid. Computer languages,
just like living species, resist change, and usually you need a new
language/species to fix some larger warts/bugs/design problems.
Python3 fixes tons of problems of Python2.x, but it's almost a new
language

Bye,
bearophile

 

[Steven D'Aprano]

Can you give an example of how useful it is? Something worth the pain of
newbies tripping over it every week?

Did you read the effbot's explanation in the link above? He gives two
examples, memoization and binding of locals.

The second example is especially interesting, because that's also a
Gotcha for newbies (not just noobs either...), and the solution to that
specific gotcha is Python's use of define-time binding of default values.

callbacks = [None]*4
for i in xrange(len(callbacks)):

.... callbacks = lambda s: '%d %s' % (i, s)
........ print cb('string')
....
3 string
3 string
3 string
3 string


Newbies get confused by this almost as often as by the default value
semantics, but the simplest solution to this gotcha is to use Python's
default values:
.... callbacks = lambda s, i=i: '%d %s' % (i, s)
........ print cb('string')
....
0 string
1 string
2 string
3 string


If Python re-evaluated the default value i=i at runtime, the above would
break.

 

[Aaron \Castironpi\ Brady]

Aaron "Castironpi" Brady wrote:
[about how default argument behavior should, in his opinion, be changed]
Say what you like. The language is as it is by choice. Were it, for some
reason, to change we would then be receiving posts every week that
didn't understand the *new* behavior.
Sometimes people just have to learn to confirm with reality instead of
requiring reality to confirm with their preconceptions. This is one such
case.
regards
 Steve

I am not convinced it should either stay or go, but it's hard to argue
one way or the other about something so deeply entrenched.  However,
what are your thoughts, whatever the default behavior is, on a
decorator that provides the alternative?  That is, a decorator that
either reevaluates default arguments each time when the language
evaluates them once, or a decorator that evaluates arguments once,
when the languages evaluates them each time?
That is not obvious and I don't know of any empirical evidence that
entails it.  Hard to search the standard library for that figure.

Although primitive and likely somewhat flawed, you may find the
statistics in the "Compatibility Issues" section ofhttp://mail.python.org/pipermail/python-3000/2007-February/005704.html
to be of interest.

Cheers,
Chris

I remember, I've seen it before. Are you proposing that the number of
posts we'd receive about this feature is proportional to its frequency
of usage?

 

[Aaron \Castironpi\ Brady]

Did you read the effbot's explanation in the link above? He gives two
examples, memoization and binding of locals.

The second example is especially interesting, because that's also a
Gotcha for newbies (not just noobs either...), and the solution to that
specific gotcha is Python's use of define-time binding of default values.
snip
Newbies get confused by this almost as often as by the default value
semantics, but the simplest solution to this gotcha is to use Python's
default values:

snip

Both of those are shorthand notations and there are alternative ways
to express both of them. The alternatives might even be more literal,
that is, less idiomatic, in the language's tokens' semantics.

For the first one, effbot says: 'You could use a global variable
containing a dictionary instead of the default value; it’s a matter of
taste'. You could also use an attribute of the function, which is an
entry in func_dict.

For the second one, it is more verbose, but you can add an enclosing
lambda or def expression, and call it on the spot. I'll be honest:
those are plusses, that is pros of the decision, but they aren't that
strong.

If Python re-evaluated the default value i=i at runtime, the above would
break.

Not with a mere extra lambda. The fact that a syntax is an
opportunity to have a behavior does not imply that it should have
one. The fact that newbies ask about these semantics doesn't imply
that they'd ask about another one just as much. The fact that these
semantics have these two uses, doesn't imply that the others don't
have more. Immutable defaults behave identically in both.

 

[Steven D'Aprano]

On Thu, 16 Oct 2008 12:18:49 -0700, Aaron \"Castironpi\" Brady wrote:

[snip]

Not with a mere extra lambda.

Not so. It has nothing to do with lambda, lambda just happens to be a
convenient example. Here's the code I demonstrated:
.... callbacks = lambda s, i=i: '%d %s' % (i, s)
........ print cb('string')
....
0 string
1 string
2 string
3 string


At the end of the first loop, i == 3. If the default value i=i was re-
evaluated each time the function was called, then i would get the value 3
each time, which is the same behaviour you get from the version with this:

callbacks = lambda s: '%d %s' % (i, s)

Worse, because you're now relying on i as a global, it's subject to
strange and mysterious bugs if you later change i and then call the
callback.

The fact that a syntax is an opportunity
to have a behavior does not imply that it should have one. The fact
that newbies ask about these semantics doesn't imply that they'd ask
about another one just as much. The fact that these semantics have
these two uses, doesn't imply that the others don't have more.

Nowhere did I say that the one logically implies the other. I was asked
for examples of how the current behaviour is useful, not to prove that
the current behaviour logically follows from first principles. If you
want to use a programming language where function default values are re-
evaluated at runtime, you know where to find them.

By the way, for the record I myself has found that behaviour useful on
occasion. But that's easy to do with current Python:


def spam(x, y=None):
if y is None:
# re-evaluate the default value at runtime
y = get_some_other_value()
return x + y

So if you want that behaviour, you can get it. But if Python's semantics
changed, then how would you implement today's semantics where the default
is evaluated once only? I don't think you can. So Python's current
semantics allows the behaviour you want, but in a slightly inconvenient
form; but the re-evaluate-at-runtime semantics would prevent the
behaviour I want completely.

Immutable defaults behave identically in both.

Not quite. To have immutable defaults behave identically, you would need
to remove at least one more feature of Python: the ability to set a
default value to an arbitrary expression, not just a literal.

Why do you need to do this? This toy example demonstrates the problem if
you don't:

yy = 3 # immutable value bound to the name yy
def spam(x, y=yy-1):
return x + y


will have the *expression* yy-1 re-evaluated when they call the function.
That means that even though 2 is immutable, you can no longer rely on the
default value being 2, or even existing at all. (What if I del yy at some
point, then call the function?)

So now to get the behaviour you desire, you not only have to change the
way Python functions are implemented (and that will have a real and
significant performance cost), but you also have to change the parser to
only allow literals as default values.

Note that there is absolutely nothing wrong with using an expression when
setting default values. But you have to prohibit it, or else introduce
unexpected behaviour which will trip up not just noobs but everybody. And
then you'll have noobs writing in weekly asking why they can't write
"def foo(x, y=10**6)" instead of y=10000000.

 

[Aaron \Castironpi\ Brady]

On Thu, 16 Oct 2008 12:18:49 -0700, Aaron \"Castironpi\" Brady wrote:

[snip]

Not with a mere extra lambda.

Not so. It has nothing to do with lambda, lambda just happens to be a
convenient example. Here's the code I demonstrated:

...     callbacks = lambda s, i=i: '%d %s' % (i, s)
...>>> for cb in callbacks:

...     print cb('string')
...
0 string
1 string
2 string
3 string

At the end of the first loop, i == 3. If the default value i=i was re-
evaluated each time the function was called, then i would get the value 3
each time, which is the same behaviour you get from the version with this:

callbacks = lambda s: '%d %s' % (i, s)

Worse, because you're now relying on i as a global, it's subject to
strange and mysterious bugs if you later change i and then call the
callback.

The fact that a syntax is an opportunity
to have a behavior does not imply that it should have one.  The fact
that newbies ask about these semantics doesn't imply that they'd ask
about another one just as much.  The fact that these semantics have
these two uses, doesn't imply that the others don't have more.

Nowhere did I say that the one logically implies the other. I was asked
for examples of how the current behaviour is useful, not to prove that
the current behaviour logically follows from first principles. If you
want to use a programming language where function default values are re-
evaluated at runtime, you know where to find them.

By the way, for the record I myself has found that behaviour useful on
occasion. But that's easy to do with current Python:

def spam(x, y=None):
    if y is None:
        # re-evaluate the default value at runtime
        y = get_some_other_value()
    return x + y

So if you want that behaviour, you can get it. But if Python's semantics
changed, then how would you implement today's semantics where the default
is evaluated once only? I don't think you can. So Python's current
semantics allows the behaviour you want, but in a slightly inconvenient
form; but the re-evaluate-at-runtime semantics would prevent the
behaviour I want completely.

Immutable defaults behave identically in both.

Not quite. To have immutable defaults behave identically, you would need
to remove at least one more feature of Python: the ability to set a
default value to an arbitrary expression, not just a literal.

Why do you need to do this? This toy example demonstrates the problem if
you don't:

yy = 3  # immutable value bound to the name yy
def spam(x, y=yy-1):
    return x + y

will have the *expression* yy-1 re-evaluated when they call the function.
That means that even though 2 is immutable, you can no longer rely on the
default value being 2, or even existing at all. (What if I del yy at some
point, then call the function?)

So now to get the behaviour you desire, you not only have to change the
way Python functions are implemented (and that will have a real and
significant performance cost), but you also have to change the parser to
only allow literals as default values.

Note that there is absolutely nothing wrong with using an expression when
setting default values. But you have to prohibit it, or else introduce
unexpected behaviour which will trip up not just noobs but everybody. And
then you'll have noobs writing in weekly asking why they can't write
"def foo(x, y=10**6)" instead of y=10000000.

You're correct. I overstated the fact.

Not quite. To have immutable defaults behave identically, you would need

Immutable literal defaults behave identically in both. Obviously
semantics would change if semantics change.

Not so. It has nothing to do with lambda, lambda just happens to be a
convenient example. Here's the code I demonstrated:

snip

What I stated is true, but I may not have stated it verbosely enough.

callbacks= [ None ]* 3
for i in range( len( callbacks ) ):

.... callbacks[ i ]= ( lambda i: ( lambda s: '%d %s' % (i, s) ) )
( i )
........ print cb('string')
....
0 string
1 string
2 string

One extra lambda, called on the spot, as I stated. The 'callbacks'
items do not rely on 'i' as a global; they contain a cell that refers
to the contents of 'i' at the time they're called, that is, a entry in
unique namespaces. When 'i' is rebound the next time through the loop
or later, or deleted, they still have their values of it.

Of course, if you're iterating over a collection of mutables, the
namespaces get references to those. Then, if you mutate your iterator
variable later, as opposed to rebind it, the namespace which contains
it in a cell will see that change; they are one in the same object.
Yours does that too.

Nowhere did I say that the one logically implies the other. I was asked
for examples of how the current behaviour is useful, not to prove that
the current behaviour logically follows from first principles. If you
want to use a programming language where function default values are re-
evaluated at runtime, you know where to find them.

I'm almost getting the feeling that you're being defensive/protective
of Python of one of its quirks, as though that was important to its
identity, and you just couldn't look on it the same without it. (A
pop t.v. character exclaimed, "I can't believe you caved!") I'm not
sure to what extent pride factors in to stubbornness like that. Is it
a selection criteria, such as you have to like quirks to be friends
with Pythoners? No offense or anything; we're all proud of Python.

With, adopting the term, 'per-call evaluations', expressions
participate at their own risk. If they refer to a variable that isn't
in their namespace closure, the evaulation of them fails. It's true
whether you do it by hand:

def f( a= None ):
if a is None:
a= i #i non-local

Or it occurs in the language's definition.

... not to prove that
the current behaviour logically follows from first principles.

Well, does it? I jest. It's not clear either one does.

Incidentally, you could combine the two issues to get a persistent
reference to the non-local variable you want to use in a default
value, copying it into the namespace; but you just have to mutate it
via its location when you do.

I think a decorator would be a good compromise. Python keeps its
definition-time evaluation, and newbies, or whoever needs them, are
still able to use per-call evaluation.

yy = 3 # immutable value bound to the name yy
def spam(x, y=yy-1):
return x + y

yy= 3
@calltime_default( y= yy- 1 ) #might need quotes
def spam(x, y):
return x + y

Then if you delete yy, 'spam' fails correctly. I don't really see an
argument that it hurts the language (or the standard library) any...
provided its possible without syntax support.

I'm not so sure that weeding out newcomers by inattentiveness isn't
unconstructive or unbeneficial to either group, vets or newcomers.
You might argue that the same individuals would just ask just as
inattentive questions later on... but at least vets would have some
variety, and Python would have more fans. It's not clear that the
feature is doing them the favor of gauging for them the level of
attentiveness writing Python requires. I'll point out though as an
aside, that harmless introversion can be mistaken for actual elitism.

 

[Lawrence D'Oliveiro]

We already get people asking why code like this doesn't return 3:

fns = [ lambda: x for x in range(10) ]
fns[3]()

9

... making this change to default arguments would mean the
solution usually proposed to the function scoping question above would no
longer work:

fns = [ lambda y=x: y for x in range(10) ]
fns[3]()

3

The right solution, of course, is

fns = [(lambda x : lambda : x)(x) for x in range(10)]

 

[Steven D'Aprano]

We already get people asking why code like this doesn't return 3:

fns = [ lambda: x for x in range(10) ] fns[3]()

9

... making this change to default arguments would mean the solution
usually proposed to the function scoping question above would no longer
work:

fns = [ lambda y=x: y for x in range(10) ] fns[3]()

3

The right solution, of course, is

fns = [(lambda x : lambda : x)(x) for x in range(10)]

Only if by "right solution" you mean "excessively verbose, confusing, and
the sort of thing that makes even supporters of lambda cringe".

Yes yes, it's just a factory function written with lambdas. It's still
ugly and exactly the sort of thing that gives ammunition to lambda-
haters. Unlike the solution given by Duncan, which is understandable to
any newbie who has learned about default values and lambda, your solution
requires an understanding of higher-level functions (functions that
return functions, for anyone who doesn't recognise the term) that most
newbies won't have.

And while I don't much care for premature optimization, I will point out
that creating a factory function just to call it once then throw it away
is very wasteful, and that waste is demonstrated by the running time
being more than double that of Duncan's solution:

timeit.Timer('[ lambda y=x: y for x in range(10) ]').repeat() [7.6332600116729736, 6.9825620651245117, 7.0891578197479248]
timeit.Timer('[(lambda x : lambda : x)(x) for x in range(10)]').repeat()

[18.984915971755981, 17.808281898498535, 18.432481050491333]

 

[Aaron \Castironpi\ Brady]

We already get people asking why code like this doesn't return 3:
fns = [ lambda: x for x in range(10) ] fns[3]()
9
... making this change to default arguments would mean the solution
usually proposed to the function scoping question above would no longer
work:
fns = [ lambda y=x: y for x in range(10) ] fns[3]()
3

The right solution, of course, is

    fns = [(lambda x : lambda : x)(x) for x in range(10)]

Only if by "right solution" you mean "excessively verbose, confusing, and
the sort of thing that makes even supporters of lambda cringe".

Yes yes, it's just a factory function written with lambdas. It's still
ugly and exactly the sort of thing that gives ammunition to lambda-
haters. Unlike the solution given by Duncan, which is understandable to
any newbie who has learned about default values and lambda, your solution
requires an understanding of higher-level functions (functions that
return functions, for anyone who doesn't recognise the term) that most
newbies won't have.

And while I don't much care for premature optimization, I will point out
that creating a factory function just to call it once then throw it away
is very wasteful, and that waste is demonstrated by the running time
being more than double that of Duncan's solution:

timeit.Timer('[ lambda y=x: y for x in range(10) ]').repeat()

[7.6332600116729736, 6.9825620651245117, 7.0891578197479248]>>> timeit.Timer('[(lambda x : lambda : x)(x) for x in range(10)]').repeat()

[18.984915971755981, 17.808281898498535, 18.432481050491333]

No, there's a difference in meaning. One creates a function that is
called with 0 arguments. The other creates a function that can be
called with 0 or 1 arguments. The purpose of a parameter is something
that the caller can supply, but doesn't have to. It is not for
internal-use-only items. Nested namespaces and object attributes can
be.

 

[Lawrence D'Oliveiro]

In message

The purpose of a parameter is something that the caller can supply, but
doesn't have to. It is not for internal-use-only items.

Exactly!

 

[Lawrence D'Oliveiro]

The right solution, of course, is

fns = [(lambda x : lambda : x)(x) for x in range(10)]

Only if by "right solution" you mean "excessively verbose, confusing, and
the sort of thing that makes even supporters of lambda cringe".

Yes yes, it's just a factory function written with lambdas. It's still
ugly and exactly the sort of thing that gives ammunition to lambda-
haters.

It's NOT ugly. It's EXACTLY the right sort of thing you do with
lambda-expressions, going right back to Church.

It's no more ugly than, say, parentheses in Lisp.

 

[Steven D'Aprano]

In message


Exactly!

Says who?

Using arguments for internal-use-only is a perfectly acceptable example
of practicality beating purity.

 

[Aaron Brady]

Says who?

Using arguments for internal-use-only is a perfectly acceptable example
of practicality beating purity.

That's a stretch.

 

[Steven D'Aprano]

That's a stretch.

It's a standard Python idiom used by the standard library.

 

[Aaron Brady]

It's a standard Python idiom used by the standard library.

It's a compromise solution, where the other compromises are about as
good.

....Except for the confused newbies. But why should they pick this
feature to ignore the documentation for? It trades off intuitiveness
for convenience.

 

[Paul McGuire]

No. That's exactly the point! Basic Python is so transparent that
you can start using it without reading anything, just looking at
a few examples. _Because_ of that it's their responsibility to
ensure that if you look at a few examples you then have a complete
understanding of the language.

I agree, Python really does strive to be intuitive and easy-to-learn.
So the oddity of the behavior of "optional_list_arg=[]" is a recurring
surprise to those who jump first and read documentation later.
Besides the tutorials, reference docs, and FAQs, there are also some
web pages with titles like "Python Gotchas" and "Common Mistakes in
Python" that usually tread this ground too.

In particular default parameters should work the way the user
expects! The fact that different users will expect different
things here is no excuse...

Are you being sarcastic? Short of "import mindreading", I don't know
how Python would know which behavior a given user would expect. Maybe
instead of a "code smell", this particular Python wart is a "design
smell".

What is surprising is that Python cannot discriminate between this:
y = 100
def f(a,x=y):
print a+x101

and this:

def f(a,x=[]):
print a+len(x)
x.append(a)

f(1) 1
f(1) 2
f(1,[1,2,3]) 4

Is x supposed to be a default arg or a "static" arg (in the sense of a
static var within a function as one finds in C)?

-- Paul

 

[Chris Rebert]

No. That's exactly the point! Basic Python is so transparent that
you can start using it without reading anything, just looking at
a few examples. _Because_ of that it's their responsibility to
ensure that if you look at a few examples you then have a complete
understanding of the language.

I agree, Python really does strive to be intuitive and easy-to-learn.
So the oddity of the behavior of "optional_list_arg=[]" is a recurring
surprise to those who jump first and read documentation later.
Besides the tutorials, reference docs, and FAQs, there are also some
web pages with titles like "Python Gotchas" and "Common Mistakes in
Python" that usually tread this ground too.

Specifically:
http://www.onlamp.com/pub/a/python/2004/02/05/learn_python.html?page=2
http://www.ferg.org/projects/python_gotchas.html#contents_item_6
http://zephyrfalcon.org/labs/python_pitfalls.html

Cheers,
Chris
--
Follow the path of the Iguana...
http://rebertia.com

In particular default parameters should work the way the user
expects! The fact that different users will expect different
things here is no excuse...

Are you being sarcastic? Short of "import mindreading", I don't know
how Python would know which behavior a given user would expect. Maybe
instead of a "code smell", this particular Python wart is a "design
smell".

What is surprising is that Python cannot discriminate between this:
y = 100
def f(a,x=y):
print a+x101

and this:

def f(a,x=[]):
print a+len(x)
x.append(a)

f(1) 1
f(1) 2
f(1,[1,2,3]) 4

Is x supposed to be a default arg or a "static" arg (in the sense of a
static var within a function as one finds in C)?

-- Paul

 

[Steven D'Aprano]

Are you being sarcastic?

Yes, David was being sarcastic. Or possibly ironic. Satirical? One of
those humour things. Whatever it was, I think you're the second person
who missed it.

Short of "import mindreading", I don't know
how Python would know which behavior a given user would expect.

Exactly.

Besides, Guido has a time machine, and apparently antigravity is being
added to the standard library, so I don't see why we can't have
mindreading too.

 

[Bruno Desthuilliers]

David C. Ullrich a écrit :

(snip)

Erm, I think maybe your irony detector needs a little calibration...

Possibly, yes...

[...]
In particular default parameters should work the way the user
expects! The fact that different users will expect different
things here is no excuse...

I was worried someone might not realize I was being sarcastic,
which is why I threw in this obvious impossibility

If different users expect different - mostly incompatible - things, how
would it be possible to have it working "the way the user expect" ?

but I guess it wasn't enough.

Obviously not - at least for me. OTHO, I've seen peoples very seriously
asking for such obvious impossibilities.

And the answer is, of course, 42.