dynamically generated runtime methods & reflection

Discussion in 'Python' started by Jay Loden, Jun 14, 2007.

  1. Jay Loden

    Jay Loden Guest

    Hi all,

    First, apologies if anyone gets this twice, but it took me quite a
    while to figure out that Python.org is evidently rejecting all mail
    from my mail server because I don't have reverse DNS configured.

    I'm not even sure how to phrase this question properly or the right
    terminology on this so bear with me. What I'm looking to find out is

    a) is there a functionality in Python where I can call a method I have
    not defined ahead of time, and based on the method name, change the
    functionality of the method at runtime?

    b) if not, what is the "Pythonic" approach to the problem outlined
    below? Any recommendations on how to approach the problem differently
    are welcome.

    I've googled and read my Python reference pretty extensively and I've
    found some hints but nothing that really answered my questions, so
    here I am :) I did figure out that you can overload __getattr__ in
    a clas to define a new method at runtime, but I got stuck when I
    couldn't figure out how to have a method know what name it was
    originally called with. That's the basic question, see below for the
    context I'm asking the question in and *why* I want to do the above


    The software product I work on has a socket-based API that can be
    accessed directly via telnet & manually typing commands, or by a set
    of Perl modules that wrap the socket functionality. I am looking to
    write a set of Python modules that match the same functionality.
    Basically you can telnet to a port and type something like

    item.list "param1=value1", "param2=value2"

    And it will return the results (if any) to you in a given format along
    with a response code, e.g. 200 OK or 400 ERROR. The Perl modules just
    wrap this so that instead of the above, you can do something like

    $MySocketServer = new SocketServer(Host=>'127.0.0.'1, Port=>'9999');

    if (! $MySocketServer->ListItem(itemName=>$item_name)) {
    print "failed to retrieve list";
    print "reason: " . $MySocketServer->GetErrorMsg();

    The ListItem() method handles the work of communicating across the
    socket, waiting for the response, and determine the success/failure
    based on the return code. The part where it gets interesting is that
    our Perl modules don't actually have a ListItem() method. Instead,
    there is a generalized "_api_func()" method that looks at the name of
    the method you called (using Autoload I believe, for any Perlheads out
    there), and bases the action taken on the name.

    In the example above, for instance, it sees that you called
    ListItem(), so it transforms that to item.list and makes the
    parameters of the method the parameters that are passed to item.list
    via the socket server. Then it takes the response back from the socket
    to determine the function return code. If there are any results such
    as a list of items, they are then able to be accessed through a
    reference to a hash of the results, and there is a GetResultRef
    accessor function. In essence, our Perl modules are an incredibly
    lightweight, simplistic wrapper for the socket-based API, and almost
    all of the work is offloaded to the API implementation server-side.

    I welcome any suggestions, feedback, sample code, documentation,
    whitepapers etc. The only thing I don't want to have to do is go and
    define a separate method for every single one of the possible
    permutations of socket commands if I can avoid it!
    Jay Loden, Jun 14, 2007
    1. Advertisements

  2. Yes. Implement a __getattr__ method on your class (which you mention).
    Ahh, so you want to pass the method name to the method that you are
    returning to be called. No problem.
    .... def __getattr__(self, name):
    .... return functools.partial(self.ActualMethod, name)
    .... def ActualMethod(self, name, *args, **kwargs):
    .... #handle *args and **kwargs based on name!
    .... print name, args, kwargs

    - Josiah
    Josiah Carlson, Jun 14, 2007
    1. Advertisements

  3. Jay Loden

    Jay Loden Guest

    Thanks, this is exactly what I was looking for! For some reason functools didn't even show up at all during Google searches...must have just had the wrong search terms.

    Jay Loden, Jun 14, 2007
  4. Well, the particular operation is typically called 'currying a
    function', and unless you know what to look for, it isn't very easy to
    make happen.

    On the other hand, it is also relatively easy to implement by hand if

    - Josiah
    Josiah Carlson, Jun 14, 2007
  5. Replace "make happen" to "discover in the standard library".

    - Josiah
    Josiah Carlson, Jun 15, 2007
  6. Josiah Carlson a écrit :
    it's not 'currying' but 'partial application'.

    Currying is somehow the reverse of partial : it's a way of building a
    multiple-args function from single-args functions.
    Bruno Desthuilliers, Jun 15, 2007
  7. Wikipedia says "currying or Schönfinkelisation[1] is the technique of
    transforming a function that takes multiple arguments into a function
    that takes a single argument" -- and FWIW I agree with Wikipedia in this
    case; the reverse (going from single-arg to multiple-args) would be
    "uncurrying", though I don't think I've ever used that term myself.

    functools.partial's name may be more precise (because it can, e.g., go
    from a function taking 3 arguments to one taking 2 -- not just from N
    down to 1) but your 'pedantic' remark seems pedantically wrong:).

    Alex Martelli, Jun 15, 2007
  8. Alex Martelli a écrit :
    cf below...
    The definition commonly agreed upon (in the FP world at least) is that
    currying is the process that "build" ("emulate", whatever...)
    multiple-args functions in a context that only supports single-arg
    functions (ie: ML, Haskell), so that (using Python syntax):

    f(x, y, z)

    would really be in fact (and under the hood):


    where f(x) returns a function closing over(x) and taking y, itself
    returning a function closing over x and y and taking z...

    Talking about this:
    currying (...) reduces multiple-argument
    functions to single-argument functions only (Schoenfinkel,

    So while *very closely* related to partial application, it's not exactly
    the same thing.

    FWIW, you can also have a look here:
    I don't - and I'm not the only one:

    I had mistakenly learned that curry was a form of generalized partial
    application from the paper : Function Currying in Scheme by Jeffrey A.
    and the Wikipedia entry (I should have known better), however I was
    mildly reprimanded for making this novice mistake in a recent paper
    submission to ICFP
    Note that I didn't say "going from", but "building". The context is a
    functional language where there's *no* such thing as "multiple args"

    Re-reading, I can agree that my choice of words may have been a bit
    poor, specially wrt/ the word "reverse". What I meant here was that
    partial application is used in the context of multiple-args function to
    'build' a function taking n-x arguments from a function taking n
    arguments, while currying is used in the context of single-arg functions
    to "emulate" multiple-args functions.
    I certainly won't pretend knowing more about CS that you do, but given
    the definition of currying in pep-0309 - which exactly matches the
    definition I first learned when toying with Haskell -, I maintain my
    pedantic remark until proven wrong - which BTW should not be overly
    difficult if it happened to be the case !-)

    As a last word, the original version of pep-0309 was named "curry", and
    has been corrected since:

    It isn't function currying, but partial application. Hence the name is
    now proposed to be partial().
    Bruno Desthuilliers, Jun 15, 2007
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.