Checking Signature of Function Parameter

[Travis Parks]

I am trying to write an algorithms library in Python. Most of the
functions will accept functions as parameters. For instance, there is
a function called any:

def any(source, predicate):
for item in source:
if predicate(item):
return true;
return false;

There are some things I want to make sure of. 1) I want to make sure
that source is iterable. 2) More importantly, I want to make sure that
predicate is callable, accepting a thing, returning a bool.

This is what I have so far:

if source is None: raise ValueError("")
if not isinstanceof(source, collections.iterable): raise TypeError("")
if not callable(predicate): raise TypeError("")

The idea here is to check for issues up front. In some of the
algorithms, I will be using iterators, so bad arguments might not
result in a runtime error until long after the calls are made. For
instance, I might implement a filter method like this:

def where(source, predicate):
for item in source:
if predicate(item):
yield item

Here, an error will be delayed until the first item is pulled from the
source. Of course, I realize that functions don't really have return
types. Good thing is that virtually everything evaluates to a boolean.
I am more concerned with the number of parameters.

Finally, can I use decorators to automatically perform these checks,
instead of hogging the top of all my methods?

 

[Chris Angelico]

if source is None: raise ValueError("")
if not isinstanceof(source, collections.iterable): raise TypeError("")
if not callable(predicate): raise TypeError("")

Easier: Just ignore the possibilities of failure and carry on with
your code. If the source isn't iterable, you'll get an error raised by
the for loop. If the predicate's not callable, you'll get an error
raised when you try to call it. The only consideration you might need
to deal with is that the predicate's not callable, and only if you're
worried that consuming something from your source would be a problem
(which it won't be with the normal iterables - strings, lists, etc,
etc). Otherwise, just let the exceptions be raised!

ChrisA

 

[Travis Parks]

Easier: Just ignore the possibilities of failure and carry on with
your code. If the source isn't iterable, you'll get an error raised by
the for loop. If the predicate's not callable, you'll get an error
raised when you try to call it. The only consideration you might need
to deal with is that the predicate's not callable, and only if you're
worried that consuming something from your source would be a problem
(which it won't be with the normal iterables - strings, lists, etc,
etc). Otherwise, just let the exceptions be raised!

ChrisA

I guess my concern is mostly with the delayed exceptions. It is hard
to find the source of an error when it doesn't happen immediately. I
am writing this library so all of the calls can be chained together
(composed). If this nesting gets really deep, finding the source is
hard to do, even with a good debugger.

Maybe I should give up on it, like you said. I am still familiarizing
myself with the paradigm. I want to make sure I am developing code
that is consistent with the industry standards.

 

[Chris Angelico]

Maybe I should give up on it, like you said. I am still familiarizing
myself with the paradigm. I want to make sure I am developing code
that is consistent with the industry standards.

In Python, the industry standard is "easier to ask forgiveness than
permission" - that is, let the exceptions happen. It's not worth the
hassle of error-checking when the result of not checking an error is
exactly the same thing.

ChrisA

 

[Ian Kelly]

I am trying to write an algorithms library in Python. Most of the
functions will accept functions as parameters. For instance, there is
a function called any:

def any(source, predicate):
   for item in source:
       if predicate(item):
           return true;
   return false;

Perhaps not the best name, since there is already a built-in called
"any" that would be masked by this. Using the built-in, "any(source,
predicate)" would be written as "any(predicate(x) for x in source)"

I guess my concern is mostly with the delayed exceptions. It is hard
to find the source of an error when it doesn't happen immediately. I
am writing this library so all of the calls can be chained together
(composed). If this nesting gets really deep, finding the source is
hard to do, even with a good debugger.

Agreed that there are cases where it is useful to do this. But there
is no delayed execution in the example you've given, so the exceptions
will happen immediately (or at least, within the same stack frame).
The stack traces will still come from the "any" function and will look
basically the same as the stack traces you'll get from raising the
exceptions by hand.

HTH,
Ian

 

[Chris Rebert]

I am trying to write an algorithms library in Python. Most of the
functions will accept functions as parameters. For instance, there is
a function called any:

def any(source, predicate):
   for item in source:
       if predicate(item):
           return true;
   return false;

There are some things I want to make sure of. 1) I want to make sure
that source is iterable. 2) More importantly, I want to make sure that
predicate is callable, accepting a thing, returning a bool.

I am more concerned with the number of parameters.

That can be introspected using the `inspect` module:
http://docs.python.org/library/inspect.html#inspect.getargspec

Finally, can I use decorators to automatically perform these checks,
instead of hogging the top of all my methods?

Certainly. Although, as others have said, the cost-benefit ratio of
adding code to perform such somewhat-redundant checks might make it
not worth the trouble.

Cheers,
Chris

 

[Nobody]

More importantly, I want to make sure that
predicate is callable, accepting a thing, returning a bool.

The "callable" part is do-able, the rest isn't.

The predicate may accept an arbitrary set of arguments via the "*args"
and/or "**kwargs" syntax, and pass these on to some other function.
Exactly *which* function may be the result of an arbitrarily complex
expression. Or it may not even call another function, but just use the
arbitrary set of arguments in an arbitrarily complex manner.

IOW, determining in advance what will or won't work is actually impossible.

 

[Travis Parks]

The "callable" part is do-able, the rest isn't.

The predicate may accept an arbitrary set of arguments via the "*args"
and/or "**kwargs" syntax, and pass these on to some other function.
Exactly *which* function may be the result of an arbitrarily complex
expression. Or it may not even call another function, but just use the
arbitrary set of arguments in an arbitrarily complex manner.

IOW, determining in advance what will or won't work is actually impossible.

Thanks for everyone's input. I decided that I will put some basic
checks up front, like "is it None", "is it Iterable" and "is it
callable". Other than that, I am letting things slide. Asking for
forgiveness is always easier anyway.

Just so everyone knows, I am defining these methods inside a class
called IterableExtender:

class IterableExtender(collections.Iterable):...

I wanted to allow for calls like this:

extend(range(0, 1000)).map(lambda x: x * x).where(lambda x: x % 2 ==
0).first(lambda x: x % 7 == 0)

It allows me to compose method calls similarly to LINQ in C#. I think
this looks better than:

first(where(map(range(0, 1000), lambda x: x * x, lambda x: x % 2 == 0,
lambda x : x % 7 == 0)))

Internally to the class, there are "private" static methods taking raw
inputs and performing no checks. The public instance methods are
responsible for checking input arguments and wrapping results.

Eventually, I will start working on algorithms that work on
MutableSequences, but for now I am just doing Iterables. This is
turning out to be a great learning experience.

 

[Ian Kelly]

I wanted to allow for calls like this:

extend(range(0, 1000)).map(lambda x: x * x).where(lambda x: x % 2 ==
0).first(lambda x: x % 7 == 0)

It allows me to compose method calls similarly to LINQ in C#. I think
this looks better than:

first(where(map(range(0, 1000), lambda x: x * x, lambda x: x % 2 == 0,
lambda x : x % 7 == 0)))

FWIW, I would be inclined to write that in Python like this:

def first(iterable):
try:
return next(iter(iterable))
except StopIteration:
raise ValueError("iterable was empty")

squares = (x * x for x in range(0, 1000))
first(x for x in squares if x % 14 == 0)

It does a bit too much to comfortably be a one-liner no matter which
way you write it, so I split it into two.

Cheers,
Ian

 

[Travis Parks]

FWIW, I would be inclined to write that in Python like this:

def first(iterable):
    try:
        return next(iter(iterable))
    except StopIteration:
        raise ValueError("iterable was empty")

squares = (x * x for x in range(0, 1000))
first(x for x in squares if x % 14 == 0)

Python's comprehensions make the need for many of the methods I am
writing unnecessary. Which probably explains why no ones really
bothered to write one before.

The only problem I have above is either the composition causes complex
method calls first(where(map(range(..., it requires complex
comprehensions or it requires breaking the code into steps. Even my
approach has problems, such as the overhead of carrying an invisible
wrapper around.

It does a bit too much to comfortably be a one-liner no matter which
way you write it, so I split it into two.

Cheers,
Ian

Yeah. I have already seen a lot of better ways of writing my code
based solely on your example. I didn't know about iter as a built-in
function. I have been calling __iter__ directly. I also need to think
more about whether methods like "where" and "map" are going to be
beneficial. The good thing is that someone will be able to use my
wrapper in any context where an Iterable can be used. It will allow
someone to switch between styles on the fly. I'm still not convinced
that this library is going to be very "pythony".

I wrote a post a few days ago about how I know the syntax and
libraries fairly well, but I don't have the "philosophy". I haven't
seen a lot of tricks and I am never sure what is the "norm" in Python.
I am sure if an experienced Python programmer looked at my code,
they'd immediately know I was missing a few things.

 

[Ethan Furman]

I wrote a post a few days ago about how I know the syntax and
libraries fairly well, but I don't have the "philosophy". I haven't
seen a lot of tricks and I am never sure what is the "norm" in Python.
I am sure if an experienced Python programmer looked at my code,
they'd immediately know I was missing a few things.

The best thing to do now is pick something and run with it. (Sounds
like you have.) Expect to redesign and reimplement three or four times
as you get a feel for what's pythonic. And have fun!

~Ethan~