python function parameters, debugging, comments, etc.

[Chris Friesen]

I've got a fair bit of programming experience (mostly kernel/POSIX stuff in C). I'm fairly new to python though, and was hoping for some advice.

Given the fact that function parameters do not specify types, when you're looking at someone else's code how the heck do you know what is expected for a given argument? (Especially in a nontrivial system where the parameter is just passed on to some other function and may not be evaluated for several nested function calls.)

Is the recommendation to have comments for each function describing the expected args?

I was trying to debug some stuff that someone else wrote. It turned out that the problem was in code like this:



def rebuild_instance(self, context, instance, image, ...)
request_spec = scheduler_utils.build_request_spec(context, image, [instance])
...stuff...
other_function(...,image,...)


where build_request_spec looks like:

def build_request_spec(ctxt, image, instances):
...etc...


and it took me a while to realize that rebuild_instance() was being passed the image ID (basically just a string), and other_function() was expecting the image ID, but build_request_spec() was expecting the actual image dictionary.

It also took me a while to realize that that build_request_spec() was expecting a list of instances, while rebuild_instance() was passing in a single instance. That one is already fixed in the above code.


So what's the recommended way of dealing with stuff like this in larger projects with many developers?

Thanks,
Chris

 

[Rotwang]

I've got a fair bit of programming experience (mostly kernel/POSIX stuff in C). I'm fairly new to python though, and was hoping for some advice.

Given the fact that function parameters do not specify types, when you're looking at someone else's code how the heck do you know what is expected for a given argument? (Especially in a nontrivial system where the parameter is just passed on to some other function and may not be evaluated for several nested function calls.)

Is the recommendation to have comments for each function describing the expected args?

[...]

In the Python community, one of the programming styles that is
encouraged is "duck-typing". What this means is that rather than writing
functions that check whether arguments passed to that function are of a
specific type, the function should simply use any methods of those
arguments it requires; that way the function will still work if passed
an argument whose type is a custom type defined by the user which has
the right interface so that the function body still makes sense (if it
quacks like a duck, then the function might as well treat it like a
duck). If a user passes an argument which doesn't have the right methods
then the function will fail, but the traceback that the interpreter
provides will often have enough information to make it clear why it failed.

(see http://docs.python.org/3/glossary.html#term-duck-typing )


So the upside of duck-typing is clear. But as you've already discovered,
so is the downside: Python's dynamic nature means that there's no way
for the interpreter to know what kind of arguments a function will
accept, and so a user of any function relies on the function having
clear documentation. There are several ways to document a function;
apart from comments, functions also have docstrings, which will be
displayed, along with the function's signature, when you call
help(function). A docstring is a string literal which occurs as the
first statement of a function definition, like this:

def foo(x, y = 2):
'''This function takes an argument x, which should be iterable, and
a function y, which should be a numeric type. It does nothing.'''
pass


If I call help(foo), I get this:

Help on function foo in module __main__:

foo(x, y=2)
This function takes an argument x, which should be iterable, and
a function y, which should be a numeric type. It does nothing.


In Python 3.0 and later, functions can also have annotations; they have
no semantics in the language itself but third-party modules can use them
if they choose to do so. They look like this:

def foo(x: str, y: int = 2, z: 'Hello' = None) -> tuple:
return a, b, c

For more about annotations, see here:

http://www.python.org/dev/peps/pep-3107/


So the short answer is that Python gives you several methods for making
it clear what kind of arguments the functions you define should be
passed, but unfortunately you'll likely encounter functions written by
people who made no use of those methods. On the plus side, Python's
exception reporting is good, so if in doubt just try using a function in
the interactive interpreter and see what happens (with the usual caveats
about using untrusted code, obviously).

 

[Oscar Benjamin]

So the upside of duck-typing is clear. But as you've already discovered, so
is the downside: Python's dynamic nature means that there's no way for the
interpreter to know what kind of arguments a function will accept, and so a
user of any function relies on the function having clear documentation.

It is still necessary to document the arguments of functions in
explicitly typed languages. Knowing that you need a list of strings
does not mean that you know what the function expects of the values of
the strings and what it will try to do with them.

When you see something like
int atoi (const char * str);
you know that it takes a string and returns an integer. However the
function name does not clearly indicate any purpose. What kind of
string should I pass in? Is the returned value an error code or a
value generated from the string (it's actually both). Even if you know
that the function parses strings representing integers there are still
many different formats for representing numbers as strings. Should the
string be in a locale-dependent format? What kind of text encoding is
it using (utf-8 maybe)? Should the characters represent an integer in
decimal format or hex, octal, binary or something else?

Inspecting types can be a quick way to gain information about the
meaning of arguments and return values but it is not something that
you should be relying on to replace documentation.


Oscar

 

[Rotwang]

So the upside of duck-typing is clear. But as you've already discovered, so
is the downside: Python's dynamic nature means that there's no way for the
interpreter to know what kind of arguments a function will accept, and so a
user of any function relies on the function having clear documentation.

It is still necessary to document the arguments of functions in
explicitly typed languages. Knowing that you need a list of strings
does not mean that you know what the function expects of the values of
the strings and what it will try to do with them.

[...]

Well, yes. I didn't intend to suggest otherwise.