Class introspection and dynamically determining function arguments

[Mark English]

I'd like to write a Tkinter app which, given a class, pops up a
window(s) with fields for each "attribute" of that class. The user could
enter values for the attributes and on closing the window would be
returned an instance of the class. The actual application I'm interested
in writing would either have simple type attributes (int, string, etc.),
or attributes using types already defined in a c-extension, although I'd
prefer not to restrict the functionality to these requirements.

The only way I can imagine to do this is to create an instance of the
class in question, and then start poking around in its attributes
dictionary (initially just using dir). So firstly, if there is instead a
way to do this without creating an instance I'd be interested.

Secondly, the code won't know exactly how to initialise the class
instance used to determinte the attributes. Do I need to make it a
prerequesite that all instances can be created with no arguments ?
Should I force/allow the user to pass an instance instead of a class ?
Should I be using inspect.getargspec on the class __init__ method and
then a loop with a try and a lot of except clauses, or is there a nicer
way to do this ? Presumably the pickling code knows how do
serialise/deserialise class instances but I'm not sure how I'd use this
without already having a class instance to hand.

Lastly, does such an app already exist ?

Thanks for any help.


-----------------------------------------------------------------------
The information contained in this e-mail is confidential and solely
for the intended addressee(s). Unauthorised reproduction, disclosure,
modification, and/or distribution of this email may be unlawful. If you
have received this email in error, please notify the sender immediately
and delete it from your system. The views expressed in this message
do not necessarily reflect those of LIFFE Holdings Plc or any of its subsidiary companies.
-----------------------------------------------------------------------

 

[Diez B. Roggisch]

The only way I can imagine to do this is to create an instance of the
class in question, and then start poking around in its attributes
dictionary (initially just using dir). So firstly, if there is instead a
way to do this without creating an instance I'd be interested.

This is the only way to go, as python has no attribute declarations as
static compiled languages have. But of course not all classes may feature
default constructors so that creating an instance is impossible. Or you
create unwanted sideeffects - think of the notorious class
DeleteMyHarddisk....

As youself already mentioned that maybe you have to impose certain
prerequisites, you maybe want to extend this to the point where for each
class you want to make dynamically instantiatable you need some
declaration. This of course depends on your desired application - whatfor
is it planned?

 

[Nick Coghlan]

Mark English wrote:
As youself already mentioned that maybe you have to impose certain
prerequisites, you maybe want to extend this to the point where for each
class you want to make dynamically instantiatable you need some
declaration. This of course depends on your desired application - whatfor
is it planned?

If this only has to work for classes created for the purpose (rather than for an
arbitrary class):

Py> class Buildable(object):
.... __slots__ = ["x", "y"]
.... def __init__(self, **kwds):
.... super(Buildable, self).__init__(self, **kwds)
.... for attr in Buildable.__slots__:
.... setattr(self, attr, kwds[attr])
....
Py> b = Buildable(x = 1 , y = 2)
Py> b.x
1
Py> b.y
2

(Note that you don't *have* to use slots, you can use a non-special class
attribute if you don't want the other side effects)

Cheers,
Nick.

 

[Diez B. Roggisch]

If this only has to work for classes created for the purpose (rather than
for an arbitrary class):

Certainly a step into the direction I meant - but still missing type
declarations. And that's what at least I'd like to see - as otherwise you
don't know what kind of editing widget to use for a property.

 

[Nick Coghlan]

Certainly a step into the direction I meant - but still missing type
declarations. And that's what at least I'd like to see - as otherwise you
don't know what kind of editing widget to use for a property.

Hmm, true. You really need a name:type dict to define each class that is going
to be generated.

Perhaps the simplest way is to require all such classes to have a "getExample"
class method that produces a fully populated example instance (making it a class
method means that you shouldn't need to override it in a subclass if you don't
change the signature of __init__).

Then the widget generator doesn't need to care about *how* that default example
gets generated, and can be something simple like:

build_widget(name, data_type):
...

build_widget_list(cls):
example = cls.getExample()
widgets = []
for attr, value in example.__dict__:
widgets.append(build_widget(attr, type(value)))
return widgets

Cheers,
Nick.

 

[Bengt Richter]

I'd like to write a Tkinter app which, given a class, pops up a
window(s) with fields for each "attribute" of that class. The user could
enter values for the attributes and on closing the window would be
returned an instance of the class. The actual application I'm interested
in writing would either have simple type attributes (int, string, etc.),
or attributes using types already defined in a c-extension, although I'd
prefer not to restrict the functionality to these requirements.

My first reaction is that if you can limit the problem, it could be done fairly simply.
[... having read to the end by now, I see you were thinking of inspecting the __init__
method, so you are close to what I was thinking, but maybe this will be useful anyway,
so since it's done, I'll post it ;-) ]

E.g., if the attributes of interest are all set by simple attribute assignment in
the __init__ method of the class, using __init__ arguments with default arguments,
then you could inspect the __init__ function for parameter names and infer types
from the default arguments.

If the instance attribute name are bound in the __new__ method, you could still conceivably
do something if you had a similar known signature, but the for unrestricted general case
your Tkinter app would have to supply too much information than can't be inferred or be known
by the user. E.g., how would a Tkinter app (or any other) know what to ask the user to supply
to set the attributes of

class WhatEver:
def __init__(self, attr):
self.attr = attr

?

The only way I can imagine to do this is to create an instance of the
class in question, and then start poking around in its attributes
dictionary (initially just using dir). So firstly, if there is instead a
way to do this without creating an instance I'd be interested.

As above. IMO you will have to restrict the problem. But perhaps that is
no so bad.

Secondly, the code won't know exactly how to initialise the class
instance used to determinte the attributes. Do I need to make it a
prerequesite that all instances can be created with no arguments ?

I think that is a byproduct of the approach I suggested (i.e., all args
having default values for type inference), but there's probably other
ways.

Should I force/allow the user to pass an instance instead of a class ?

I would say no. Especially considering possible side effects.

Should I be using inspect.getargspec on the class __init__ method and
then a loop with a try and a lot of except clauses, or is there a nicer

I don't know if you need a _lot_ of except clauses, but yes, at least one,
and a loop to allow retrying after typos etc.

way to do this ? Presumably the pickling code knows how do
serialise/deserialise class instances but I'm not sure how I'd use this
without already having a class instance to hand.

Lastly, does such an app already exist ?

Don't know. Wouldn't be surprised.

Thanks for any help.

This shows you are probably close to solving your own problem with inspect.getargspec:
... def __init__(self, a=1, b='bee', c=1.2):
... self.a = a
... self.b = b
... self.c = c
...

>>> inspect.getargspec(C.__init__) (['self', 'a', 'b', 'c'], None, None, (1, 'bee', 1.2))
>>> print inspect.getargspec.__doc__

Get the names and default values of a function's arguments.

A tuple of four things is returned: (args, varargs, varkw, defaults).
'args' is a list of the argument names (it may contain nested lists).
'varargs' and 'varkw' are the names of the * and ** arguments or None.
'defaults' is an n-tuple of the default values of the last n arguments.

>>> args,_,_,defaults = inspect.getargspec(C.__init__)
>>> values = []

Obviously you need an inner loop for correcting typos etc., and try/except to catch relevant
errors, but I'm just typing interactively here, so her goes:

>>> for name, defv in zip(args[-len(defaults):], defaults):

... vstr = raw_input('Please enter a string suitable for %s(s): '%type(defv).__name__)
... values.append(type(defv)(vstr))
...
Please enter a string suitable for int(s): 123
Please enter a string suitable for str(s): 'this is a string'
Please enter a string suitable for float(s): 1.5

>>> values [123, "'this is a string'", 1.5]
>>> c = C(*values)
>>> c.a 123
>>> c.b "'this is a string'"
>>> c.c 1.5
>>> cdef = C()
>>> cdef.a, cdef.b, cdef.c

(1, 'bee', 1.2)

I guess the single quotes on the str(s) entry shows that all the characters get entered ;-)
You might want to think about whether to process backslashes as escapes in your input. E.g.,
Enter some escape example: Not an newline: \n -- nor '\n' ;-)
"Not an newline: \\n -- nor '\\n' ;-)" Enter some escape example: Not an newline: \n -- nor '\n' ;-)
Not an newline: \n -- nor '\n' ;-)

BTW, You can avoid importing inspect and get the data of
(['self', 'a', 'b', 'c'], None, None, (1, 'bee', 1.2))

that you want by looking at the unbound method C.__init__
(though im_func may be deprecated eventually?):
(1, 'bee', 1.2)

or by getting the __init__ function as such, by avoiding the attribute
access that makes it and unbound or bound method

>>> C.__dict__['__init__'].func_code.co_varnames ('self', 'a', 'b', 'c')
>>> C.__dict__['__init__'].func_defaults

(1, 'bee', 1.2)

Regards,
Bengt Richter

 

[Mike C. Fletcher]

Hmm, I missed the original post, but I'll jump in anyway:

This sounds a heck of a lot like a property-editing system. When
creating a property-modeled system, the best approach is normally to
use something that actually models the properties, rather than
trying to guess at the metadata involved by poking around in an
arbitrarily structured object.

My BasicProperty system allows for this kind of interaction
(property-sheets) using wxPython (rather than Tkinter) when using
wxoo. You declare classes as having a set of data-properties (which
can have defaults or not, constraints or not, restricted data-types
or not, friendly names or not, documentation or not). Normally you
create these classes as subclasses of a class that knows how to
automatically assign __init__ parameters to properties, and knows
how to tell (e.g.) wxoo about the properties of the class.

Those same property classes also allow for editing properties of
database rows in PyTable, but that isn't likely relevant to your
case. We've also used them internally to create a rather large
web-based property-editing mechanism (applied to such databases),
but again, not really relevant to the case at hand.

Anyway, if you aren't interested in BasicProperty for this task; another
project on which I work, PyDispatcher provides fairly robust mechanism
(called robustApply) for providing a set of possible arguments and using
inspect to pick out which names match the parameters for a function in
order to pass them in to the function/method/callable object. That
said, doing this for __init__'s with attribute values from an object's
dictionary doesn't really seem like the proper way to approach the problem.

Good luck,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com

 

[Bengt Richter]

Hmm, I missed the original post, but I'll jump in anyway:

This sounds a heck of a lot like a property-editing system. When
creating a property-modeled system, the best approach is normally to
use something that actually models the properties, rather than
trying to guess at the metadata involved by poking around in an
arbitrarily structured object.

I agree that "poking around in an arbitrarily structured object" is not
a likely road to satisfaction, but sometimes the arbitrary can be taken out
an something pretty simple can be done ;-)

OTOH, I probably should have mentioned that there are ways to view these
kinds of problems from a longer perspective. E.g., I googled for
"model view controller" and found a nice wiki page at

http://wact.sourceforge.net/index.php/ModelViewController

that may be worth reading for the OP, just for ideas. There is interesting discussion
of many MVC-related issues, but I don't know anything about the associated project.

My BasicProperty system allows for this kind of interaction
(property-sheets) using wxPython (rather than Tkinter) when using
wxoo. You declare classes as having a set of data-properties (which
can have defaults or not, constraints or not, restricted data-types
or not, friendly names or not, documentation or not). Normally you
create these classes as subclasses of a class that knows how to
automatically assign __init__ parameters to properties, and knows
how to tell (e.g.) wxoo about the properties of the class.

Does the BasicProperty base class effectively register itself as an observer
of subclass properties and automatically update widgets etc., a la Delphi
data-driven visual components? I've thought of doing a light-weight form
extension class that would use a text (maybe CSV) definition to control
contruction, and easy programmatic manipulation by python of the definition
parameters, like a stripped-down version of the text view of Delphi forms.
It could also be done via Tkinter, to prototype it. It would be interesting
to allow dragging widgets and edges around in Tkinter and round-trip the parameter
changes automatically into the text representation. A little (well, ok, a fair amount ;-)
further and you'd have a drag-n-drop GUI design tool. But don't hold your breath ;-)

Those same property classes also allow for editing properties of
database rows in PyTable, but that isn't likely relevant to your
case. We've also used them internally to create a rather large
web-based property-editing mechanism (applied to such databases),
but again, not really relevant to the case at hand.

Who knows, the OP may only be revealing his concerns about a small part of
his great tapestry ;-)

Anyway, if you aren't interested in BasicProperty for this task; another
project on which I work, PyDispatcher provides fairly robust mechanism
(called robustApply) for providing a set of possible arguments and using
inspect to pick out which names match the parameters for a function in
order to pass them in to the function/method/callable object. That
said, doing this for __init__'s with attribute values from an object's
dictionary doesn't really seem like the proper way to approach the problem.

Sounds like a workaround for parameter passing that maybe should have been
keyword-based?

Regards,
Bengt Richter

 

[Alex Martelli]

Certainly a step into the direction I meant - but still missing type
declarations. And that's what at least I'd like to see - as otherwise you
don't know what kind of editing widget to use for a property.

Though it may be overkill for your needs, you'll be interested in
Enthought's "Traits", I think; see, for example,
<http://python.fyxm.net/pycon/papers/traits.html>. Facilitating such
presentation tasks (no doubt including editing) appears to be a major
driving force for Traits.


Alex

 

[Mike C. Fletcher]

....

Does the BasicProperty base class effectively register itself as an observer
of subclass properties and automatically update widgets etc., a la Delphi
data-driven visual components? I've thought of doing a light-weight form
extension class that would use a text (maybe CSV) definition to control
contruction, and easy programmatic manipulation by python of the definition
parameters, like a stripped-down version of the text view of Delphi forms.
It could also be done via Tkinter, to prototype it. It would be interesting
to allow dragging widgets and edges around in Tkinter and round-trip the parameter
changes automatically into the text representation. A little (well, ok, a fair amount ;-)
further and you'd have a drag-n-drop GUI design tool. But don't hold your breath ;-)

BasicProperty itself doesn't register as an observable/observer,
BasicProperty is the lowest-level of the software stack, so it allows
you to override and provide notification (e.g. using PyDispatcher) on
property-setting. ConflictSolver (old project to create a room
scheduler) used that to do automatic updating of widgets in the wxPython
UI based on Model changes (though I don't remember if it was
per-property or per-object). My goal for the wxoo project was to
provide hooks in the wxPython GUI designers for dropping in property
sheets and/or property-aware controls such that you would have the
equivalent of "data aware" controls in VB or Access (keeping in mind
that BasicProperty properties can also represent fields in database rows).

Aside:

The VRML97 field class in OpenGLContext does notifications for every
set (using PyDispatcher), btw. It's a little more limited in its
scope (focus on 3D data-types), but the effect is what allows the
scenegraph to cache and then rebuild its internal rendering
structures with very low overhead.

....

Sounds like a workaround for parameter passing that maybe should have been
keyword-based?

Not as such, that is, not a workaround, and it shouldn't be keyword
based .

The problem with using keyword-based passing is that every method needs
to be written with this awareness of the keyword-handling structures.
You spread pointless implementation details throughout your codebase.
PyDispatcher lets you write very natural functions for dealing with
events without having every function use **named parameters.

I've now written quite a few such systems, and I'm currently balanced
between two approaches; that taken in PyDispatcher (define only natural
parameters, have the system figure out how to deliver them to you), and
that taken in OpenGLContext (define an event-class hierarchy which
encapsulates all information about the events).

The PyDispatcher approach is nice in that it makes simple things very
simple. You want access to the "lastValue" parameter in the
"environment" of the event and nothing else, you define your function
like so:

def handler( lastValue ):
print 'got last value', lastValue

which works very nicely when you're early in the development of a
system, or are linking multiple systems. There's no need to do all
sorts of extra work defining event hierarchies, you can often leave
given handlers entirely alone during refactoring if they aren't dealing
with the changed properties in the event-environment.

The OpenGLContext approach is more appropriate when you have a large
system (such as OpenGLContext), where defining an event class is a
trivial task compared to the total system expenditure. It allows for
such things as putting methods on the event objects to make debugging
easy, and providing common functionality. It starts to show it's worth
when you start needing to reason about the phenomena of events
themselves, rather than just about the phenomena the events represent
(e.g. when you need to cache, delay, or reorder events).

The named argument passing approach has the disadvantage that every
function must be written with knowledge of that use of named arguments:

def handler( lastValue, **named ):
print 'got last value', lastValue

when using such systems in the past I've often wound up with errors deep
in an application where some seldom-called callback didn't have a
**named parameter, so the system would abort (well, log an error) trying
to call it.

Is it a huge overhead? No. But its unnecessary if you've got a tool
that takes care of that "implementation-level" detail for you. Compared
to robustApply, the difference is pretty minimal. If you define a
**named parameter, for instance, you will get the remainder of the
robustApply environment passed in.

In another lifetime, you could imagine these systems being written as
wrappers that added a nested scope around the definition of the function
such that the scope forwarded unresolved name references to the
environment of the event (non-lexical scoping), but that would lack the
explicitness of passing in the arguments from the environment and would,
I suppose be evil in a quite non-trivial way.

Peace, and have fun,
Mike

________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com