just another default argument value gotcha

[Frank Bechmann]

Eventually most of you will not learn much from this because it's just
another event in the 'default argument value gotcha' series, but
because it cost me some hours yesterday to spot this 'error' in a
famous python tool I thought it might still help other people to save
some time.

I tried to use some method which was documented to write to
'sys.stdout' per default so that changing 'sys.stdout' to bind to
another object should allow to get grip on the method's output. but
that didn't work - reason was that that 'sys.stdout' was used as
default argument value for the method. The following code describes it
better then my english can do, so as it seems for me, one should make
sure to use the 'f2' variant.

import StringIO, sys

def f1(msg, out=sys.stdout):
out.write("%s\n" % msg)

def f2(msg, out=None):
if not out:
out = sys.stdout
out.write("%s\n" % msg)

if __name__ == "__main__":
buf = sys.stdout = StringIO.StringIO()
f1("calling f1")
f2("calling f2")
data = buf.getvalue()
buf.close()
sys.stdout = sys.__stdout__
print "=========\n", data

results in:

calling f1 <-- written directly by 'f1'
========== => redirection didn't work
calling f2 <-- written from data stored in 'buf'


running the following greps on this machine brought only 2 possible
spots, but that doesn't mean too much since this is our gaming
computer with only a very basic python installation; eventually
someone with a real huge package list would bring up more hits:

$ find . -name '*.py' -exec grep -H '\bdef\b.*\(.*\=sys.stdout.*\)[
\t]*:' \{} \;$ find . -name '*.py' -exec grep -H '\bdef\b.*\(.*\=sys.stderr.*\)[
\t]*:' \{} \;descriptions=1, verbosity=1):

 

[Michael Hudson]

Eventually most of you will not learn much from this because it's just
another event in the 'default argument value gotcha' series, but
because it cost me some hours yesterday to spot this 'error' in a
famous python tool I thought it might still help other people to save
some time.

I tried to use some method which was documented to write to
'sys.stdout' per default so that changing 'sys.stdout' to bind to
another object should allow to get grip on the method's output. but
that didn't work - reason was that that 'sys.stdout' was used as
default argument value for the method.

FWIW, I think this is precisely the reason that "print >>None, ..."
prints to sys.stdout. It lets you write

The following code describes it better then my english can do, so as
it seems for me, one should make sure to use the 'f2' variant.

import StringIO, sys

def f1(msg, out=sys.stdout):
out.write("%s\n" % msg)

def f2(msg, out=None):
if not out:
out = sys.stdout
out.write("%s\n" % msg)

either of these as

def f3(msg, out=None):
print >>out, msg

Cheers,
mwh

 

[Terry Reedy]

Eventually most of you will not learn much from this because it's just
another event in the 'default argument value gotcha' series, but

There are two issues: when the default value is determined and whether it
is mutable. The concern here is timing.

I tried to use some method which was documented to write to
'sys.stdout' per default

The meaning of the expression 'sys.stdout' may depend on when it is
evaluated.

so that changing 'sys.stdout' to bind to
another object should allow to get grip on the method's output.

If you change the binding before the evaulation, you will affect the
result; otherwise not and your inference is incorrect.

but that didn't work - reason was that that 'sys.stdout' was used as
default argument value for the method.

Without the doc quoted, I can't tell whether it was misleading or if you
misread it. If the doc said 'default argument value' then it was exactly
right.

The following code describes it better then my english can do,
so as it seems for me, one should make sure to use the 'f2' variant.

The code to use is the one that gives the result you want. To give a
function parameter a default *value*, evaluated once and good for all
function calls, write the expression yielding the value (a constant object)
in the definition-time header as in

import StringIO, sys

def f1(msg, out=sys.stdout):
out.write("%s\n" % msg)

To me, and apparently to GvR, this is what 'default value' means. If you
want a backup *expression*, conditionally evaluated at each function call
(and therefore potentially yielding different objects in different function
calls), write the expression in the run-time body as in

def f2(msg, out=None):
if not out:
out = sys.stdout
out.write("%s\n" % msg)

In this case, I see passing an value for param 'out' via the
runtime-evaluated global attribute sys.stdout as an alternate (implicit)
arg passing mechanism, one that is fairly common. Simplifying f2 gives

def f3(msg):
out = sys.stdout
out.write(("%s\n" % msg)

The added complexity of f2 gives one the option of passing 'out' either
directly in the call or indirectly via the global. I would only use it if
one needed and were going to use the added flexibility. The authors of the
method you used did not think it necessary.

Another issue your problem touches on is the ambiguity of 'builtin' names
like 'sys' and 'sys.stdout'. When the docs use such names, they sometimes
to usually to always mean the objects originally bound to the name on
interpreter startup, ignoring any possible later rebindings within a
program execution. The reader is expected to interprete the text
immediately using the default, pre-startup bindings. For example, in
'functions return None by default', 'None' means the singleton NoneType
object, not its runtime binding. Similarly, 'sys.stdout' is sometimes used
to refer specifically to its original rather than 'current' binding. I
would tend to assume this for library modules unless the doc, the code, or
experiment showed otherwise.

Terry J. Reedy

 

[Frank Bechmann]

thx for that tip, it worked as predicted.

nevertheless it seems to be an error if some function/method uses the
'sys.stdout' default value but *not* the 'print >>out' syntax. and
this in turn means that my simple grep error finder doesn't help too
much, you still have to look at the code of the matching
functions/methods.