Python's idiom for function overloads

[Frans Englich]

Hello,

Since Python doesn't have static typing, how is the same result as traditional
function overloads results in acheived? With function overloads the
"selection of code path depending on data type" is transparent and automatic
since the typing system figure out what goes to what.

But in Python, when one wants to be able to pass different data types into a
single "entry point" for functionality, how is that best done? To in a
function do an if statement with the type() function?


Cheers,

Frans

 

[Philippe Fremy]

Hi Frans,

Since Python doesn't have static typing, how is the same result as traditional
function overloads results in acheived?

With dynamic typing obviously.

You can not reproduce the C++ overload idiom but you can get something
close with manual type testing.

> To in a
> function do an if statement with the type() function?

I am not aware of any other method.

def a( arg1 ):
if type(arg1) == types.IntType: return aWithInt(arg1)
if type(arg1) == types.ListType: return aWithList(arg1)
...

As you see, it is a bit tedious sometimes.

If you want to juggle with completely different signatures, you have to
play with variable argument lists. But I have found in my experience
that the best way to get close to the C++ idiom, while improving
readbility, is by using kwparams:

def a(**kwparams):
if kwparams.has_key('argInt'): aWithInt(kwparams['argInt'])
if kwparams.has_key('argString'): aWithString(kwparams['argString'])

The parsing code is the same, but the intent of the user is better
expressed and you can catch misuse in a better fashion:
if kwparams.has_key('argInt') and kwparams.has_key('argString'):
print "You stupid moron, a can be used only with string or int but not
both at the same time!"
sys.exit(1)

The example speaks better in a real case. Imagine a processPixmap function:
processPixmap( pixmap=QPixmap(...) )
processPixmap( filename='my_pixmap.png' )
processPixmap( buffer=my_pixmap_string_content )

It works actually even better with multiple arguments.

regards,

Philippe

 

[Steven Bethard]

But in Python, when one wants to be able to pass different data types into a
single "entry point" for functionality, how is that best done? To in a
function do an if statement with the type() function?

It often depends a lot on the specific use case... Do you have a
particular example in mind?

Steve

 

[F. GEIGER]

Since Python doesn't have static typing, how is the same result as
traditional
The more you program in Python, the less you are missing it.

As Philippe already said, use objects that support the protocol or decide
what to do with it after having checked its type. I do that, if I have to,
like so:

def doIt(arg):
if type(arg) == type([]):
map(doIt, arg)
else:
# Do it on a scalar type
# ...
return result

HTH
Franz GEIGER

Hi Frans,

Since Python doesn't have static typing, how is the same result as traditional
function overloads results in acheived?

With dynamic typing obviously.

You can not reproduce the C++ overload idiom but you can get something
close with manual type testing.

To in a
function do an if statement with the type() function?

I am not aware of any other method.

def a( arg1 ):
if type(arg1) == types.IntType: return aWithInt(arg1)
if type(arg1) == types.ListType: return aWithList(arg1)
...

As you see, it is a bit tedious sometimes.

If you want to juggle with completely different signatures, you have to
play with variable argument lists. But I have found in my experience
that the best way to get close to the C++ idiom, while improving
readbility, is by using kwparams:

def a(**kwparams):
if kwparams.has_key('argInt'): aWithInt(kwparams['argInt'])
if kwparams.has_key('argString'): aWithString(kwparams['argString'])

The parsing code is the same, but the intent of the user is better
expressed and you can catch misuse in a better fashion:
if kwparams.has_key('argInt') and kwparams.has_key('argString'):
print "You stupid moron, a can be used only with string or int but not
both at the same time!"
sys.exit(1)

The example speaks better in a real case. Imagine a processPixmap function:
processPixmap( pixmap=QPixmap(...) )
processPixmap( filename='my_pixmap.png' )
processPixmap( buffer=my_pixmap_string_content )

It works actually even better with multiple arguments.

regards,

Philippe

 

[Jacek Generowicz]

Hi Frans,



With dynamic typing obviously.

You can not reproduce the C++ overload idiom

Of course you can. Use a multimethod of some sort.

The canonical answer to the OQ, of course, includes things like
"Consider whether you really want/need to do this" and "duck typing".

Frequently, in Python, code which checks for types, rather than
checking for features, ends up being excessively restrictive and
insufficiently general.

 

[Jacek Generowicz]

As Philippe already said, use objects that support the protocol or decide
what to do with it after having checked its type. I do that, if I have to,
like so:

1 def doIt(arg):
2 if type(arg) == type([]):
3 map(doIt, arg)
4 else:
5 # Do it on a scalar type
6 # ...
7 return result

Now, consider that line 3 would execute very happily if the type of
arg were

1) list,
2) str,
3) tuple,
4) dict,
5) file,
6) any other built-in iterable or sequence,
7) any useer-defined iterable or sequence,
8) a subclass of any of the above,
9) god knows what else ...

.... yet in line 2 you have ensured that the whole function will not
work properly for any of the listed types other than the first.

You could make the code much more general by doing it like this:

try:
map(doIt, arg)
except TypeError:
...


The important thing to note is that the actual types of Python objects
are usually not very interesting or important at all. What is much
more important is what the object is able to do: what messages it
understands, what protocols it supports.

Hiding some code behind a type-check, in Python, is quite frequently
the wrong thing to do.

 

[Philippe Fremy]

Frequently, in Python, code which checks for types, rather than
checking for features, ends up being excessively restrictive and
insufficiently general.

That's true, but checking for the exact features that are going to be
needed by your code is usually quite complicated and painful to
maintain. At the beginning, you need only a sequence. In the end, you
need a mutable sequence that supports slicing, so you could go by
requiring a list as well.

Enforcing types also brings the benefit that the program is more
deterministic. In my experience, there is a lot more benefits to have an
object whose type is clearly identified than to have function that
accepts generic objects.

I would go as far as saying that variables should have immutable types.
It is more restricting that what python provides currently, but leads to
clearer programming practice: the intent of the developer shows up in
the type he uses.

Of course, with languages such as OCaml, you get both of it: liberal
typing with enforced consistency.

While Python's dynamic typing liberty is enjoyable, I think it harms
when you start to work on big projects. It prevents you to put many
safety assumptions which might bite you back.

regards,

Philippe

 

[Jacek Generowicz]

Enforcing types also brings the benefit that the program is more
deterministic. In my experience, there is a lot more benefits to have
an object whose type is clearly identified than to have function that
accepts generic objects.


I would go as far as saying that variables should have immutable
types.

If you really believe this, then I would go as far as saying that you
would be happier in a language other than Python.

It is more restricting that what python provides currently, but
leads to clearer programming practice: the intent of the developer
shows up in the type he uses.

Not in my opinion or experience.

To each his own, and vice versa.

 

[F. Petitjean]

Le Tue, 01 Feb 2005 12:10:47 +0100, Philippe Fremy a écrit :

snip

Enforcing types also brings the benefit that the program is more
deterministic. In my experience, there is a lot more benefits to have an
object whose type is clearly identified than to have function that
accepts generic objects.

If you insist to always have only clearly identified types of variables
you will use the Hungarian notation :
bool bIsDir = blah
const char *lpszMessage = "It's a directory";

I would go as far as saying that variables should have immutable types.
It is more restricting that what python provides currently, but leads to
clearer programming practice: the intent of the developer shows up in
the type he uses.

clearer programming practice which goes with unreadable code ?

While Python's dynamic typing liberty is enjoyable, I think it harms
when you start to work on big projects. It prevents you to put many
safety assumptions which might bite you back.

Python is strongly typed (at run-time). You can get a traceback sure but
it is rather difficult to get unreliable results.

 

[Frans Englich]

It often depends a lot on the specific use case... Do you have a
particular example in mind?

I did have a specific scenario, but it blurred into a general wondering about
typing. I think my problem was, and still is, that I don't think in Python
terms but try to force other idioms. Thinking in static typic terms when
Python goes in the opposite direction clearly shows friction is created.


The replies were interesting,

Frans

 

[Jorgen Grahn]

But in Python, when one wants to be able to pass different data types into a
single "entry point" for functionality, how is that best done? To in a
function do an if statement with the type() function?

Have a look at this older thread on this subject:

Subject: Re: Does python support multi prototype.
Message-ID: <[email protected]>

/Jorgen

 

[Simo Melenius]

You can not reproduce the C++ overload idiom but you can get something
close with manual type testing.


I am not aware of any other method.

def a( arg1 ):
if type(arg1) == types.IntType: return aWithInt(arg1)
if type(arg1) == types.ListType: return aWithList(arg1)
...

Or:

def a_overloader (arg1):
return my_typed_a_func_dict[type (arg1)] (arg1)

Next I'd put my hands in automating the creation of these wrappers and
the my_typed_a_func_dict map based on my implementation written so
far. Then I'd think of parameterizing on any arbitrary destructuring
of arguments like def foo ((a,b), c) and also on classes to which
instance methods are bound to. At this point, a preprocessor might be
handy to avoid seeing all the internals after which things would
probably start looking sick enough to either switch languages or
thinking of the right problem first and then come up with a pythonic
solution to _that_.


br,
S

 

[Stephen Thorne]

You can not reproduce the C++ overload idiom but you can get something
close with manual type testing.


I am not aware of any other method.

def a( arg1 ):
if type(arg1) == types.IntType: return aWithInt(arg1)
if type(arg1) == types.ListType: return aWithList(arg1)
...

Or:

def a_overloader (arg1):
return my_typed_a_func_dict[type (arg1)] (arg1)

Next I'd put my hands in automating the creation of these wrappers and
the my_typed_a_func_dict map based on my implementation written so
far. Then I'd think of parameterizing on any arbitrary destructuring
of arguments like def foo ((a,b), c) and also on classes to which
instance methods are bound to. At this point, a preprocessor might be
handy to avoid seeing all the internals after which things would
probably start looking sick enough to either switch languages or
thinking of the right problem first and then come up with a pythonic
solution to _that_.

Pah.

Use a decorator.

Code is available here:
Jack Diederich posted a good multimethod snippet here, in response to
my own clumsy effort.

http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/5b50153e3fb84862
http://tinyurl.com/4awat

type(arg1) == types.IntType
def multi_type(t):
return multi(lambda x:type(x) == t)

@multi_type(types.IntType)
def foo(x):
return someOperationOnInt(x)

@multi_type(types.FloatType)
def foo(x):
return someOperationOnFloat(x)

Regards,
Stephen Thorne.

"Just because it is possible doesn't make a good idea *wink*",
-Jack Diederich, Aug 2004