Class introspection and dynamically determining function arguments

Discussion in 'Python' started by Mark English, Jan 20, 2005.

  1. Mark English

    Mark English Guest

    I'd like to write a Tkinter app which, given a class, pops up a
    window(s) with fields for each "attribute" of that class. The user could
    enter values for the attributes and on closing the window would be
    returned an instance of the class. The actual application I'm interested
    in writing would either have simple type attributes (int, string, etc.),
    or attributes using types already defined in a c-extension, although I'd
    prefer not to restrict the functionality to these requirements.

    The only way I can imagine to do this is to create an instance of the
    class in question, and then start poking around in its attributes
    dictionary (initially just using dir). So firstly, if there is instead a
    way to do this without creating an instance I'd be interested.

    Secondly, the code won't know exactly how to initialise the class
    instance used to determinte the attributes. Do I need to make it a
    prerequesite that all instances can be created with no arguments ?
    Should I force/allow the user to pass an instance instead of a class ?
    Should I be using inspect.getargspec on the class __init__ method and
    then a loop with a try and a lot of except clauses, or is there a nicer
    way to do this ? Presumably the pickling code knows how do
    serialise/deserialise class instances but I'm not sure how I'd use this
    without already having a class instance to hand.

    Lastly, does such an app already exist ?

    Thanks for any help.


    -----------------------------------------------------------------------
    The information contained in this e-mail is confidential and solely
    for the intended addressee(s). Unauthorised reproduction, disclosure,
    modification, and/or distribution of this email may be unlawful. If you
    have received this email in error, please notify the sender immediately
    and delete it from your system. The views expressed in this message
    do not necessarily reflect those of LIFFE Holdings Plc or any of its subsidiary companies.
    -----------------------------------------------------------------------
     
    Mark English, Jan 20, 2005
    #1
    1. Advertising

  2. Mark English wrote:

    > The only way I can imagine to do this is to create an instance of the
    > class in question, and then start poking around in its attributes
    > dictionary (initially just using dir). So firstly, if there is instead a
    > way to do this without creating an instance I'd be interested.


    This is the only way to go, as python has no attribute declarations as
    static compiled languages have. But of course not all classes may feature
    default constructors so that creating an instance is impossible. Or you
    create unwanted sideeffects - think of the notorious class
    DeleteMyHarddisk....

    As youself already mentioned that maybe you have to impose certain
    prerequisites, you maybe want to extend this to the point where for each
    class you want to make dynamically instantiatable you need some
    declaration. This of course depends on your desired application - whatfor
    is it planned?

    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Jan 20, 2005
    #2
    1. Advertising

  3. Mark English

    Nick Coghlan Guest

    Re: Class introspection and dynamically determiningfunction arguments

    Diez B. Roggisch wrote:
    > Mark English wrote:
    > As youself already mentioned that maybe you have to impose certain
    > prerequisites, you maybe want to extend this to the point where for each
    > class you want to make dynamically instantiatable you need some
    > declaration. This of course depends on your desired application - whatfor
    > is it planned?


    If this only has to work for classes created for the purpose (rather than for an
    arbitrary class):

    Py> class Buildable(object):
    .... __slots__ = ["x", "y"]
    .... def __init__(self, **kwds):
    .... super(Buildable, self).__init__(self, **kwds)
    .... for attr in Buildable.__slots__:
    .... setattr(self, attr, kwds[attr])
    ....
    Py> b = Buildable(x = 1 , y = 2)
    Py> b.x
    1
    Py> b.y
    2

    (Note that you don't *have* to use slots, you can use a non-special class
    attribute if you don't want the other side effects)

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
     
    Nick Coghlan, Jan 20, 2005
    #3
  4. Re: Class introspection and dynamically determining function arguments

    Nick Coghlan wrote:
    >
    > If this only has to work for classes created for the purpose (rather than
    > for an arbitrary class):
    >


    Certainly a step into the direction I meant - but still missing type
    declarations. And that's what at least I'd like to see - as otherwise you
    don't know what kind of editing widget to use for a property.
    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Jan 20, 2005
    #4
  5. Mark English

    Nick Coghlan Guest

    Re: Class introspection and dynamicallydetermining function arguments

    Diez B. Roggisch wrote:
    > Nick Coghlan wrote:
    >
    >>If this only has to work for classes created for the purpose (rather than
    >>for an arbitrary class):
    >>

    >
    >
    > Certainly a step into the direction I meant - but still missing type
    > declarations. And that's what at least I'd like to see - as otherwise you
    > don't know what kind of editing widget to use for a property.


    Hmm, true. You really need a name:type dict to define each class that is going
    to be generated.

    Perhaps the simplest way is to require all such classes to have a "getExample"
    class method that produces a fully populated example instance (making it a class
    method means that you shouldn't need to override it in a subclass if you don't
    change the signature of __init__).

    Then the widget generator doesn't need to care about *how* that default example
    gets generated, and can be something simple like:

    build_widget(name, data_type):
    ...

    build_widget_list(cls):
    example = cls.getExample()
    widgets = []
    for attr, value in example.__dict__:
    widgets.append(build_widget(attr, type(value)))
    return widgets

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
     
    Nick Coghlan, Jan 21, 2005
    #5
  6. On Thu, 20 Jan 2005 11:24:12 -0000, "Mark English" <> wrote:

    >I'd like to write a Tkinter app which, given a class, pops up a
    >window(s) with fields for each "attribute" of that class. The user could
    >enter values for the attributes and on closing the window would be
    >returned an instance of the class. The actual application I'm interested
    >in writing would either have simple type attributes (int, string, etc.),
    >or attributes using types already defined in a c-extension, although I'd
    >prefer not to restrict the functionality to these requirements.

    My first reaction is that if you can limit the problem, it could be done fairly simply.
    [... having read to the end by now, I see you were thinking of inspecting the __init__
    method, so you are close to what I was thinking, but maybe this will be useful anyway,
    so since it's done, I'll post it ;-) ]

    E.g., if the attributes of interest are all set by simple attribute assignment in
    the __init__ method of the class, using __init__ arguments with default arguments,
    then you could inspect the __init__ function for parameter names and infer types
    from the default arguments.

    If the instance attribute name are bound in the __new__ method, you could still conceivably
    do something if you had a similar known signature, but the for unrestricted general case
    your Tkinter app would have to supply too much information than can't be inferred or be known
    by the user. E.g., how would a Tkinter app (or any other) know what to ask the user to supply
    to set the attributes of

    class WhatEver:
    def __init__(self, attr):
    self.attr = attr

    ?

    >
    >The only way I can imagine to do this is to create an instance of the
    >class in question, and then start poking around in its attributes
    >dictionary (initially just using dir). So firstly, if there is instead a
    >way to do this without creating an instance I'd be interested.

    As above. IMO you will have to restrict the problem. But perhaps that is
    no so bad.

    >
    >Secondly, the code won't know exactly how to initialise the class
    >instance used to determinte the attributes. Do I need to make it a
    >prerequesite that all instances can be created with no arguments ?

    I think that is a byproduct of the approach I suggested (i.e., all args
    having default values for type inference), but there's probably other
    ways.

    >Should I force/allow the user to pass an instance instead of a class ?

    I would say no. Especially considering possible side effects.

    >Should I be using inspect.getargspec on the class __init__ method and
    >then a loop with a try and a lot of except clauses, or is there a nicer

    I don't know if you need a _lot_ of except clauses, but yes, at least one,
    and a loop to allow retrying after typos etc.

    >way to do this ? Presumably the pickling code knows how do
    >serialise/deserialise class instances but I'm not sure how I'd use this
    >without already having a class instance to hand.
    >
    >Lastly, does such an app already exist ?

    Don't know. Wouldn't be surprised.

    >
    >Thanks for any help.


    This shows you are probably close to solving your own problem with inspect.getargspec:

    >>> class C(object):

    ... def __init__(self, a=1, b='bee', c=1.2):
    ... self.a = a
    ... self.b = b
    ... self.c = c
    ...
    >>> inspect.getargspec(C.__init__)

    (['self', 'a', 'b', 'c'], None, None, (1, 'bee', 1.2))
    >>> print inspect.getargspec.__doc__

    Get the names and default values of a function's arguments.

    A tuple of four things is returned: (args, varargs, varkw, defaults).
    'args' is a list of the argument names (it may contain nested lists).
    'varargs' and 'varkw' are the names of the * and ** arguments or None.
    'defaults' is an n-tuple of the default values of the last n arguments.

    >>> args,_,_,defaults = inspect.getargspec(C.__init__)
    >>> values = []


    Obviously you need an inner loop for correcting typos etc., and try/except to catch relevant
    errors, but I'm just typing interactively here, so her goes:

    >>> for name, defv in zip(args[-len(defaults):], defaults):

    ... vstr = raw_input('Please enter a string suitable for %s(s): '%type(defv).__name__)
    ... values.append(type(defv)(vstr))
    ...
    Please enter a string suitable for int(s): 123
    Please enter a string suitable for str(s): 'this is a string'
    Please enter a string suitable for float(s): 1.5
    >>> values

    [123, "'this is a string'", 1.5]
    >>> c = C(*values)
    >>> c.a

    123
    >>> c.b

    "'this is a string'"
    >>> c.c

    1.5
    >>> cdef = C()
    >>> cdef.a, cdef.b, cdef.c

    (1, 'bee', 1.2)

    I guess the single quotes on the str(s) entry shows that all the characters get entered ;-)
    You might want to think about whether to process backslashes as escapes in your input. E.g.,

    >>> raw_input('Enter some escape example: ')

    Enter some escape example: Not an newline: \n -- nor '\n' ;-)
    "Not an newline: \\n -- nor '\\n' ;-)"
    >>> print raw_input('Enter some escape example: ')

    Enter some escape example: Not an newline: \n -- nor '\n' ;-)
    Not an newline: \n -- nor '\n' ;-)

    BTW, You can avoid importing inspect and get the data of

    >>> inspect.getargspec(C.__init__)

    (['self', 'a', 'b', 'c'], None, None, (1, 'bee', 1.2))

    that you want by looking at the unbound method C.__init__
    (though im_func may be deprecated eventually?):

    >>> C.__init__.im_func.func_code.co_varnames

    ('self', 'a', 'b', 'c')
    >>> C.__init__.im_func.func_defaults

    (1, 'bee', 1.2)

    or by getting the __init__ function as such, by avoiding the attribute
    access that makes it and unbound or bound method

    >>> C.__dict__['__init__'].func_code.co_varnames

    ('self', 'a', 'b', 'c')
    >>> C.__dict__['__init__'].func_defaults

    (1, 'bee', 1.2)

    Regards,
    Bengt Richter
     
    Bengt Richter, Jan 22, 2005
    #6
  7. Re: Class introspection and dynamically determiningfunction arguments


    >On Thu, 20 Jan 2005 11:24:12 -0000, "Mark English" <> wrote:
    >
    >
    >
    >>I'd like to write a Tkinter app which, given a class, pops up a
    >>window(s) with fields for each "attribute" of that class. The user could
    >>enter values for the attributes and on closing the window would be
    >>returned an instance of the class. The actual application I'm interested
    >>in writing would either have simple type attributes (int, string, etc.),
    >>or attributes using types already defined in a c-extension, although I'd
    >>prefer not to restrict the functionality to these requirements.
    >>
    >>

    Hmm, I missed the original post, but I'll jump in anyway:

    This sounds a heck of a lot like a property-editing system. When
    creating a property-modeled system, the best approach is normally to
    use something that actually models the properties, rather than
    trying to guess at the metadata involved by poking around in an
    arbitrarily structured object.

    My BasicProperty system allows for this kind of interaction
    (property-sheets) using wxPython (rather than Tkinter) when using
    wxoo. You declare classes as having a set of data-properties (which
    can have defaults or not, constraints or not, restricted data-types
    or not, friendly names or not, documentation or not). Normally you
    create these classes as subclasses of a class that knows how to
    automatically assign __init__ parameters to properties, and knows
    how to tell (e.g.) wxoo about the properties of the class.

    Those same property classes also allow for editing properties of
    database rows in PyTable, but that isn't likely relevant to your
    case. We've also used them internally to create a rather large
    web-based property-editing mechanism (applied to such databases),
    but again, not really relevant to the case at hand.

    Anyway, if you aren't interested in BasicProperty for this task; another
    project on which I work, PyDispatcher provides fairly robust mechanism
    (called robustApply) for providing a set of possible arguments and using
    inspect to pick out which names match the parameters for a function in
    order to pass them in to the function/method/callable object. That
    said, doing this for __init__'s with attribute values from an object's
    dictionary doesn't really seem like the proper way to approach the problem.

    Good luck,
    Mike

    ________________________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://www.vrplumber.com
    http://blog.vrplumber.com
     
    Mike C. Fletcher, Jan 22, 2005
    #7
  8. On Fri, 21 Jan 2005 20:23:58 -0500, "Mike C. Fletcher" <> wrote:

    >
    >>On Thu, 20 Jan 2005 11:24:12 -0000, "Mark English" <> wrote:
    >>
    >>
    >>
    >>>I'd like to write a Tkinter app which, given a class, pops up a
    >>>window(s) with fields for each "attribute" of that class. The user could
    >>>enter values for the attributes and on closing the window would be
    >>>returned an instance of the class. The actual application I'm interested
    >>>in writing would either have simple type attributes (int, string, etc.),
    >>>or attributes using types already defined in a c-extension, although I'd
    >>>prefer not to restrict the functionality to these requirements.
    >>>
    >>>

    >Hmm, I missed the original post, but I'll jump in anyway:
    >
    > This sounds a heck of a lot like a property-editing system. When
    > creating a property-modeled system, the best approach is normally to
    > use something that actually models the properties, rather than
    > trying to guess at the metadata involved by poking around in an
    > arbitrarily structured object.

    I agree that "poking around in an arbitrarily structured object" is not
    a likely road to satisfaction, but sometimes the arbitrary can be taken out
    an something pretty simple can be done ;-)

    OTOH, I probably should have mentioned that there are ways to view these
    kinds of problems from a longer perspective. E.g., I googled for
    "model view controller" and found a nice wiki page at

    http://wact.sourceforge.net/index.php/ModelViewController

    that may be worth reading for the OP, just for ideas. There is interesting discussion
    of many MVC-related issues, but I don't know anything about the associated project.

    >
    > My BasicProperty system allows for this kind of interaction
    > (property-sheets) using wxPython (rather than Tkinter) when using
    > wxoo. You declare classes as having a set of data-properties (which
    > can have defaults or not, constraints or not, restricted data-types
    > or not, friendly names or not, documentation or not). Normally you
    > create these classes as subclasses of a class that knows how to
    > automatically assign __init__ parameters to properties, and knows
    > how to tell (e.g.) wxoo about the properties of the class.

    Does the BasicProperty base class effectively register itself as an observer
    of subclass properties and automatically update widgets etc., a la Delphi
    data-driven visual components? I've thought of doing a light-weight form
    extension class that would use a text (maybe CSV) definition to control
    contruction, and easy programmatic manipulation by python of the definition
    parameters, like a stripped-down version of the text view of Delphi forms.
    It could also be done via Tkinter, to prototype it. It would be interesting
    to allow dragging widgets and edges around in Tkinter and round-trip the parameter
    changes automatically into the text representation. A little (well, ok, a fair amount ;-)
    further and you'd have a drag-n-drop GUI design tool. But don't hold your breath ;-)

    >
    > Those same property classes also allow for editing properties of
    > database rows in PyTable, but that isn't likely relevant to your
    > case. We've also used them internally to create a rather large
    > web-based property-editing mechanism (applied to such databases),
    > but again, not really relevant to the case at hand.

    Who knows, the OP may only be revealing his concerns about a small part of
    his great tapestry ;-)

    >
    >Anyway, if you aren't interested in BasicProperty for this task; another
    >project on which I work, PyDispatcher provides fairly robust mechanism
    >(called robustApply) for providing a set of possible arguments and using
    >inspect to pick out which names match the parameters for a function in
    >order to pass them in to the function/method/callable object. That
    >said, doing this for __init__'s with attribute values from an object's
    >dictionary doesn't really seem like the proper way to approach the problem.

    Sounds like a workaround for parameter passing that maybe should have been
    keyword-based?

    Regards,
    Bengt Richter
     
    Bengt Richter, Jan 22, 2005
    #8
  9. Re: Class introspection and dynamically determining function arguments

    Diez B. Roggisch <> wrote:

    > Nick Coghlan wrote:
    > >
    > > If this only has to work for classes created for the purpose (rather than
    > > for an arbitrary class):

    >
    > Certainly a step into the direction I meant - but still missing type
    > declarations. And that's what at least I'd like to see - as otherwise you
    > don't know what kind of editing widget to use for a property.


    Though it may be overkill for your needs, you'll be interested in
    Enthought's "Traits", I think; see, for example,
    <http://python.fyxm.net/pycon/papers/traits.html>. Facilitating such
    presentation tasks (no doubt including editing) appears to be a major
    driving force for Traits.


    Alex
     
    Alex Martelli, Jan 22, 2005
    #9
  10. Re: Class introspection and dynamically determiningfunction arguments

    Bengt Richter wrote:

    >On Fri, 21 Jan 2005 20:23:58 -0500, "Mike C. Fletcher" <> wrote:
    >
    >
    >
    >>>On Thu, 20 Jan 2005 11:24:12 -0000, "Mark English" <> wrote:
    >>>
    >>>

    ....

    >Does the BasicProperty base class effectively register itself as an observer
    >of subclass properties and automatically update widgets etc., a la Delphi
    >data-driven visual components? I've thought of doing a light-weight form
    >extension class that would use a text (maybe CSV) definition to control
    >contruction, and easy programmatic manipulation by python of the definition
    >parameters, like a stripped-down version of the text view of Delphi forms.
    >It could also be done via Tkinter, to prototype it. It would be interesting
    >to allow dragging widgets and edges around in Tkinter and round-trip the parameter
    >changes automatically into the text representation. A little (well, ok, a fair amount ;-)
    >further and you'd have a drag-n-drop GUI design tool. But don't hold your breath ;-)
    >
    >

    BasicProperty itself doesn't register as an observable/observer,
    BasicProperty is the lowest-level of the software stack, so it allows
    you to override and provide notification (e.g. using PyDispatcher) on
    property-setting. ConflictSolver (old project to create a room
    scheduler) used that to do automatic updating of widgets in the wxPython
    UI based on Model changes (though I don't remember if it was
    per-property or per-object). My goal for the wxoo project was to
    provide hooks in the wxPython GUI designers for dropping in property
    sheets and/or property-aware controls such that you would have the
    equivalent of "data aware" controls in VB or Access (keeping in mind
    that BasicProperty properties can also represent fields in database rows).

    Aside:

    The VRML97 field class in OpenGLContext does notifications for every
    set (using PyDispatcher), btw. It's a little more limited in its
    scope (focus on 3D data-types), but the effect is what allows the
    scenegraph to cache and then rebuild its internal rendering
    structures with very low overhead.

    ....

    >>Anyway, if you aren't interested in BasicProperty for this task; another
    >>project on which I work, PyDispatcher provides fairly robust mechanism
    >>(called robustApply) for providing a set of possible arguments and using
    >>inspect to pick out which names match the parameters for a function in
    >>order to pass them in to the function/method/callable object. That
    >>said, doing this for __init__'s with attribute values from an object's
    >>dictionary doesn't really seem like the proper way to approach the problem.
    >>
    >>

    >Sounds like a workaround for parameter passing that maybe should have been
    >keyword-based?
    >
    >

    Not as such, that is, not a workaround, and it shouldn't be keyword
    based ;) .

    The problem with using keyword-based passing is that every method needs
    to be written with this awareness of the keyword-handling structures.
    You spread pointless implementation details throughout your codebase.
    PyDispatcher lets you write very natural functions for dealing with
    events without having every function use **named parameters.

    I've now written quite a few such systems, and I'm currently balanced
    between two approaches; that taken in PyDispatcher (define only natural
    parameters, have the system figure out how to deliver them to you), and
    that taken in OpenGLContext (define an event-class hierarchy which
    encapsulates all information about the events).

    The PyDispatcher approach is nice in that it makes simple things very
    simple. You want access to the "lastValue" parameter in the
    "environment" of the event and nothing else, you define your function
    like so:

    def handler( lastValue ):
    print 'got last value', lastValue

    which works very nicely when you're early in the development of a
    system, or are linking multiple systems. There's no need to do all
    sorts of extra work defining event hierarchies, you can often leave
    given handlers entirely alone during refactoring if they aren't dealing
    with the changed properties in the event-environment.

    The OpenGLContext approach is more appropriate when you have a large
    system (such as OpenGLContext), where defining an event class is a
    trivial task compared to the total system expenditure. It allows for
    such things as putting methods on the event objects to make debugging
    easy, and providing common functionality. It starts to show it's worth
    when you start needing to reason about the phenomena of events
    themselves, rather than just about the phenomena the events represent
    (e.g. when you need to cache, delay, or reorder events).

    The named argument passing approach has the disadvantage that every
    function must be written with knowledge of that use of named arguments:

    def handler( lastValue, **named ):
    print 'got last value', lastValue

    when using such systems in the past I've often wound up with errors deep
    in an application where some seldom-called callback didn't have a
    **named parameter, so the system would abort (well, log an error) trying
    to call it.

    Is it a huge overhead? No. But its unnecessary if you've got a tool
    that takes care of that "implementation-level" detail for you. Compared
    to robustApply, the difference is pretty minimal. If you define a
    **named parameter, for instance, you will get the remainder of the
    robustApply environment passed in.

    In another lifetime, you could imagine these systems being written as
    wrappers that added a nested scope around the definition of the function
    such that the scope forwarded unresolved name references to the
    environment of the event (non-lexical scoping), but that would lack the
    explicitness of passing in the arguments from the environment and would,
    I suppose be evil in a quite non-trivial way.

    Peace, and have fun,
    Mike

    ________________________________________________
    Mike C. Fletcher
    Designer, VR Plumber, Coder
    http://www.vrplumber.com
    http://blog.vrplumber.com
     
    Mike C. Fletcher, Jan 22, 2005
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Stephen Nesbitt
    Replies:
    0
    Views:
    284
    Stephen Nesbitt
    Nov 22, 2004
  2. jmborr
    Replies:
    1
    Views:
    453
    Stargaming
    Nov 3, 2007
  3. Yuv
    Replies:
    5
    Views:
    483
    Chris Withers
    Nov 14, 2009
  4. Stefan Arentz

    Introspection on method arguments

    Stefan Arentz, Nov 5, 2004, in forum: Ruby
    Replies:
    1
    Views:
    103
    Robert Klemme
    Nov 5, 2004
  5. Daniel Berger
    Replies:
    5
    Views:
    119
    Mauricio Fernandez
    Feb 13, 2007
Loading...

Share This Page