Create classes at runtnime

Discussion in 'Python' started by Marc Aymerich, Feb 4, 2011.

  1. Hi!
    I need to create a pretty complex class at runtime. something like
    this one:

    (note: "...." means that the number of attributes can be variable)

    class VirtualUserLimitForm(ModelForm):
    swap_limit = forms.CharField(max_length=100,
    initial=monitor1.default_limit)
    memory_limit = forms.CharField(max_length=100,
    initial=monitor2.default_limit)
    ...

    class Meta:
    model = model

    def __init__(self, *args, **kwargs):
    super(VirtualUserLimitForm, self).__init__(*args, **kwargs)
    if 'instance' in kwargs:
    self.fields['swap_limit'].initial =
    kwargs['instance'].monitoring.filter(monitor=monitor1)[0].current
    self.fields['memory_limit'].initial =
    kwargs['instance'].monitoring.filter(monitor=monitor2)[0].current
    ...

    I can generate all the needed code as string and then use exec(), but
    it seems ugly to me. I'm wondering if there is another way more
    elegant to do that? metaclasses maybe? What is your recommendation?

    Thanks!!
    Marc Aymerich, Feb 4, 2011
    #1
    1. Advertising

  2. Marc Aymerich

    Peter Otten Guest

    Marc Aymerich wrote:

    > I need to create a pretty complex class at runtime. something like
    > this one:


    I have a hunch that you've never heard the famous Kernighan quote:

    "Everyone knows that debugging is twice as hard as writing a program in the
    first place. So if you're as clever as you can be when you write it, how
    will you ever debug it?"

    Or that if you've heard it you don't heed it.

    > (note: "...." means that the number of attributes can be variable)
    >
    > class VirtualUserLimitForm(ModelForm):
    > swap_limit = forms.CharField(max_length=100,
    > initial=monitor1.default_limit)
    > memory_limit = forms.CharField(max_length=100,
    > initial=monitor2.default_limit)
    > ...
    >
    > class Meta:
    > model = model
    >
    > def __init__(self, *args, **kwargs):
    > super(VirtualUserLimitForm, self).__init__(*args, **kwargs)
    > if 'instance' in kwargs:
    > self.fields['swap_limit'].initial =
    > kwargs['instance'].monitoring.filter(monitor=monitor1)[0].current
    > self.fields['memory_limit'].initial =
    > kwargs['instance'].monitoring.filter(monitor=monitor2)[0].current
    > ...
    >
    > I can generate all the needed code as string and then use exec(), but
    > it seems ugly to me. I'm wondering if there is another way more
    > elegant to do that? metaclasses maybe?


    The metaclass does indeed take a dictionary argument where you can provide
    class attributes, e. g.:

    >>> C = type("C", (), dict(a=1, b=lambda self: 42))
    >>> C().a, C().b()

    (1, 42)

    > What is your recommendation?


    Find something that is simple and robust. Something else.

    Peter
    Peter Otten, Feb 4, 2011
    #2
    1. Advertising

  3. Marc Aymerich

    Carl Banks Guest

    On Feb 4, 1:32 pm, Peter Otten <> wrote:
    > Marc Aymerich wrote:
    > > I need to create a pretty complex class at runtime. something like
    > > this one:

    >
    > I have a hunch that you've never heard the famous Kernighan quote:
    >
    > "Everyone knows that debugging is twice as hard as writing a program in the
    > first place. So if you're as clever as you can be when you write it, how
    > will you ever debug it?"
    >
    > Or that if you've heard it you don't heed it.


    [snip]

    > > What is your recommendation?

    >
    > Find something that is simple and robust. Something else.



    I've done things like this before, and I would say the Kernigan quote
    is not appropriate here, and the implication that isn't not simple and
    robust is incorrect. It's repetitive code like the OP posted that's
    complex and flimsy. Repetitive code is the exact opposite of the DRY
    principle, and there's a reason DRY is one of the most often cited
    principles here.

    Using advanced techniques like metaclasses to exploit the similarity
    to reduce or eliminate the repetitiveness can, if done well, simplify
    code. By a lot.

    Advanced is not the same thing as complex.


    In this particular case I'd say the recommendation to do something
    else is a good one, but it's not true in general.


    Carl Banks
    Carl Banks, Feb 4, 2011
    #3
  4. Marc Aymerich

    Carl Banks Guest

    On Feb 4, 11:17 am, Marc Aymerich <> wrote:
    > Hi!
    > I need to create a pretty complex class at runtime. something like
    > this one:
    >
    > (note: "...." means that the number of attributes can be variable)
    >
    > class VirtualUserLimitForm(ModelForm):
    >     swap_limit = forms.CharField(max_length=100,
    > initial=monitor1.default_limit)
    >     memory_limit = forms.CharField(max_length=100,
    > initial=monitor2.default_limit)
    >     ...
    >
    >     class Meta:
    >         model = model
    >
    >     def __init__(self, *args, **kwargs):
    >         super(VirtualUserLimitForm, self).__init__(*args, **kwargs)
    >         if 'instance' in kwargs:
    >             self.fields['swap_limit'].initial =
    > kwargs['instance'].monitoring.filter(monitor=monitor1)[0].current
    >             self.fields['memory_limit'].initial =
    > kwargs['instance'].monitoring.filter(monitor=monitor2)[0].current
    >             ...
    >
    > I can generate all the needed code as string and then use exec(), but
    > it seems ugly to me. I'm wondering if there is another way more
    > elegant to do that?  metaclasses maybe? What is your recommendation?


    I'd recommend against using metaclasses (in the normal way) with
    Django ORM, since it (and pretty much all ORMs) already makes heavy
    use of metaclasses, and I doubt those metaclasses were designed to
    share metaclassing duties.

    At a minimum you'd have to derive your own metaclasses from Djangos,
    and that's not likely to work out well at all.

    The right way to do this might be to reorganize your database. It
    seems like you have all kinds of tables with different sets of
    columns, and the tables themselves are dynamic? Like if you get some
    data somewhere and it doesn't fit into an existing schema you make a
    new table for it? If that's the case then you'd be better off,
    perhaps, to reorganize the tables into some kind of association list.
    Put the "relational" aspect to use. (Left as an exercise for now but
    if you want pointers feel free to follow up.)

    But maybe you have no control of the tables; you simply have a lot of
    them and want to cut down on repetition. Then I think a factory
    function that builds a dict and passes it to type() would be
    reasonable. See the documentation of type() for explanation of
    arguments.

    def create_user_form(name,fields,_model):
    class Meta:
    model = _model
    dct = { 'Meta': Meta }
    for field in fields:
    dct[field] = forms.CharField(max_length=100,
    > initial=monitor1.default_limit)

    return type(name,(ModelForm,),dct)


    Carl Banks
    Carl Banks, Feb 4, 2011
    #4
  5. Marc Aymerich

    Ian Guest

    On Feb 4, 2:32 pm, Peter Otten <> wrote:
    > > (note: "...." means that the number of attributes can be variable)

    >
    > > class VirtualUserLimitForm(ModelForm):
    > >     swap_limit = forms.CharField(max_length=100,
    > > initial=monitor1.default_limit)
    > >     memory_limit = forms.CharField(max_length=100,
    > > initial=monitor2.default_limit)
    > >     ...

    >
    > >     class Meta:
    > >         model = model

    >
    > >     def __init__(self, *args, **kwargs):
    > >         super(VirtualUserLimitForm, self).__init__(*args, **kwargs)
    > >         if 'instance' in kwargs:
    > >             self.fields['swap_limit'].initial =
    > > kwargs['instance'].monitoring.filter(monitor=monitor1)[0].current
    > >             self.fields['memory_limit'].initial =
    > > kwargs['instance'].monitoring.filter(monitor=monitor2)[0].current
    > >             ...

    >
    > > I can generate all the needed code as string and then use exec(), but
    > > it seems ugly to me. I'm wondering if there is another way more
    > > elegant to do that?  metaclasses maybe?

    >
    > The metaclass does indeed take a dictionary argument where you can provide
    > class attributes, e. g.:
    >
    > >>> C = type("C", (), dict(a=1, b=lambda self: 42))
    > >>> C().a, C().b()

    >
    > (1, 42)


    This approach will basically work, but since you're subclassing the
    Django ModelForm, note that it already has its own metaclass:

    >>> django.forms.ModelForm.__class__

    <class 'django.forms.models.ModelFormMetaclass'>

    So you will need to either subclass ModelFormMetaclass for your
    metaclass and override the __init__ method, or modify what Peter
    posted like so:

    >>> C = django.forms.ModelForm.__class__("VirtualUserLimitForm", (django.forms.ModelForm,), dict(a=1, b=lambda self: 42))


    Cheers,
    Ian
    Ian, Feb 4, 2011
    #5
  6. On Fri, 04 Feb 2011 11:17:39 -0800, Marc Aymerich wrote:

    > Hi!
    > I need to create a pretty complex class at runtime. something like this
    > one:
    >
    > (note: "...." means that the number of attributes can be variable)
    >
    > class VirtualUserLimitForm(ModelForm):
    > swap_limit = forms.CharField(max_length=100,
    > initial=monitor1.default_limit)
    > memory_limit = forms.CharField(max_length=100,
    > initial=monitor2.default_limit)
    > ...


    [...]
    > I can generate all the needed code as string and then use exec(), but it
    > seems ugly to me. I'm wondering if there is another way more elegant to
    > do that? metaclasses maybe? What is your recommendation?


    Just add in the class attributes you want after creating the class.

    class VirtualUserLimitForm(ModelForm):
    pass


    f = forms.CharField # alias for brevity
    for name, value in [
    ('swap_limit', f(max_length=100, initial=monitor1.default_limit)),
    ('memory_limit', f(max_length=100, initial=monitor2.default_limit)),
    ('spam', 'spam spam spam spam'),
    ('eggs', 'fried sunny side up'),
    ]:
    setattr(VirtualUserLimitForm, name, value)


    There's probably a name for this sort of technique, and an entire chapter
    in "Design Patterns For People With Too Much Time On Their Hands" (15th
    edition), but I don't remember it :)



    --
    Steven
    Steven D'Aprano, Feb 5, 2011
    #6
  7. Marc Aymerich

    Ian Guest

    On Feb 4, 5:49 pm, Steven D'Aprano <steve
    > wrote:
    > Just add in the class attributes you want after creating the class.
    >
    > class VirtualUserLimitForm(ModelForm):
    >     pass
    >
    > f = forms.CharField  # alias for brevity
    > for name, value in [
    >     ('swap_limit', f(max_length=100, initial=monitor1.default_limit)),
    >     ('memory_limit', f(max_length=100, initial=monitor2.default_limit)),
    >     ('spam', 'spam spam spam spam'),
    >     ('eggs', 'fried sunny side up'),
    >     ]:
    >     setattr(VirtualUserLimitForm, name, value)


    This would probably cause some problems with the Django metaclass. It
    inspects the class dictionary for fields at the time the class in
    constructed, and so it would miss any fields added after construction
    time.
    Ian, Feb 5, 2011
    #7
  8. On Fri, 04 Feb 2011 17:19:23 -0800, Ian wrote:

    > On Feb 4, 5:49 pm, Steven D'Aprano <steve
    > > wrote:
    >> Just add in the class attributes you want after creating the class.

    [...]
    > This would probably cause some problems with the Django metaclass. It
    > inspects the class dictionary for fields at the time the class in
    > constructed, and so it would miss any fields added after construction
    > time.



    What Django metaclass? The OP didn't say anything about using Django.



    --
    Steven
    Steven D'Aprano, Feb 5, 2011
    #8
  9. Marc Aymerich

    Chris Rebert Guest

    On Fri, Feb 4, 2011 at 6:10 PM, Steven D'Aprano
    <> wrote:
    > On Fri, 04 Feb 2011 17:19:23 -0800, Ian wrote:
    >> On Feb 4, 5:49 pm, Steven D'Aprano <steve
    >> > wrote:
    >>> Just add in the class attributes you want after creating the class.

    > [...]
    >> This would probably cause some problems with the Django metaclass.  It
    >> inspects the class dictionary for fields at the time the class in
    >> constructed, and so it would miss any fields added after construction
    >> time.

    >
    > What Django metaclass? The OP didn't say anything about using Django.


    His code mentions "forms.CharField" and declares a nested class named
    "Meta"; this is indicative of Django.

    Cheers,
    Chris
    --
    http://blog.rebertia.com
    Chris Rebert, Feb 5, 2011
    #9
  10. Thank you all for the answers!

    I'll try to give you the context in which I need to generate classes
    like this.

    I'm developing a reusable control panel for an ISP. I have several
    modules (VirtualUser, SystemUser, VPS, VirtualHost, ...) and they all
    share the fact that you can have limits on resource consumption. for
    example.

    VPS: has limits on swap, memory, cpu and disk.
    SystemUser: has limits on disk and traffic.

    As all modules share this property of being limited I have decided to
    split the limit functionality in a separate module (called resources).
    Resources are associated dinamically with (vps, systemuser ...)
    through a Monitor class. So it's up to you decide what limits you are
    interested to put in the VPS or in the SystemUser ...

    So when you attach the "disk limit" to SystemUser model is necessary
    to create a web form for SystemUser a bit different than if you decide
    to have disk and traffic limits. That is what my posted code is
    supposed to do.

    Do you think that I am on the wrong way designing the control panel
    like this?

    Btw, I've seen several interesting ideas to create the class
    dinamically (with class factorys, MetaClass, dictionaries ..), but I
    have not yet clear whether this will be usefull for create the
    __init__ function at runtime too. Any clue on this?

    Many, many thanks!! :)
    Marc
    Marc Aymerich, Feb 5, 2011
    #10
  11. Marc Aymerich

    Peter Otten Guest

    Carl Banks wrote:

    > On Feb 4, 1:32 pm, Peter Otten <> wrote:
    >> Marc Aymerich wrote:
    >> > I need to create a pretty complex class at runtime. something like
    >> > this one:

    >>
    >> I have a hunch that you've never heard the famous Kernighan quote:
    >>
    >> "Everyone knows that debugging is twice as hard as writing a program in
    >> the first place. So if you're as clever as you can be when you write it,
    >> how will you ever debug it?"
    >>
    >> Or that if you've heard it you don't heed it.

    >
    > [snip]
    >
    >> > What is your recommendation?

    >>
    >> Find something that is simple and robust. Something else.

    >
    >
    > I've done things like this before, and I would say the Kernigan quote
    > is not appropriate here, and the implication that isn't not simple and
    > robust is incorrect. It's repetitive code like the OP posted that's
    > complex and flimsy. Repetitive code is the exact opposite of the DRY
    > principle, and there's a reason DRY is one of the most often cited
    > principles here.
    >
    > Using advanced techniques like metaclasses to exploit the similarity
    > to reduce or eliminate the repetitiveness can, if done well, simplify
    > code. By a lot.
    >
    > Advanced is not the same thing as complex.
    >
    >
    > In this particular case I'd say the recommendation to do something
    > else is a good one, but it's not true in general.


    The trouble with metaclasses is that they are not completely transparent.
    Instead of two levels (class and instance) you have to consider three.
    Steven D'Aprano has involuntarily given an example of what may go wrong.

    Marc has asked another question with a problem that could be attacked with a
    metaclass before. Had he been comfortable with metaclasses by now we
    wouldn't have seen the above post. So he is operating at the limits of his
    current understanding of Python in an area where it is not unlikely that
    he'll run into unexpected side effects of the django database wrapper.

    I think a reminder that he has to debug the whole thing is in order ;)

    Peter
    Peter Otten, Feb 5, 2011
    #11
  12. On Feb 5, 1:06 pm, Marc Aymerich <> wrote:
    > Thank you all for the answers!
    >
    > I'll try to give you the context in which I need to generate classes
    > like this.
    >
    > I'm developing a reusable control panel for an ISP. I have several
    > modules (VirtualUser, SystemUser, VPS, VirtualHost, ...) and they all
    > share the fact that you can have limits on resource consumption. for
    > example.
    >
    > VPS: has limits on swap, memory, cpu and disk.
    > SystemUser: has limits on disk and traffic.
    >
    > As all modules share this property of being limited I have decided to
    > split the limit functionality in a separate module (called resources).
    > Resources are associated dinamically with (vps, systemuser ...)
    > through a Monitor class. So it's up to you decide what limits you are
    > interested to put in the VPS or in the SystemUser ...
    >
    > So when you attach the "disk limit" to SystemUser model is necessary
    > to create a web form for SystemUser a bit different than if you decide
    > to have disk and traffic limits. That is what my posted code is
    > supposed to do.
    >
    > Do you think that I am on the wrong way designing the control panel
    > like this?
    >
    > Btw, I've seen several interesting ideas to create the class
    > dinamically (with class factorys, MetaClass, dictionaries ..), but I
    > have not yet clear whether this will be usefull for create the
    > __init__ function at runtime too. Any clue on this?
    >


    this seems to work :)

    def makeLimitForm(name, monitors, _model):

    class Meta:
    model = _model
    dct = { 'Meta': Meta }

    for monitor in monitors:
    field_name = monitor.resource + "_limit"
    dct[field_name] = forms.CharField(max_length=100,
    initial=monitor.default_limit)

    def __init__(self, *args, **kwargs):
    ModelForm.__init__(self, *args, **kwargs)
    if 'instance' in kwargs:
    for monitor in monitors:
    field_name = monitor.resource + "_limit"
    print monitor
    self.fields[field_name].initial =
    kwargs['instance'].monitoring.filter(monitor=monitor)[0].current

    dct['__init__'] = __init__

    return type(name,(ModelForm,),dct)
    Marc Aymerich, Feb 5, 2011
    #12
  13. Marc Aymerich

    Peter Otten Guest

    Marc Aymerich wrote:

    > On Feb 5, 1:06 pm, Marc Aymerich <> wrote:
    >> Thank you all for the answers!
    >>
    >> I'll try to give you the context in which I need to generate classes
    >> like this.
    >>
    >> I'm developing a reusable control panel for an ISP. I have several
    >> modules (VirtualUser, SystemUser, VPS, VirtualHost, ...) and they all
    >> share the fact that you can have limits on resource consumption. for
    >> example.
    >>
    >> VPS: has limits on swap, memory, cpu and disk.
    >> SystemUser: has limits on disk and traffic.
    >>
    >> As all modules share this property of being limited I have decided to
    >> split the limit functionality in a separate module (called resources).
    >> Resources are associated dinamically with (vps, systemuser ...)
    >> through a Monitor class. So it's up to you decide what limits you are
    >> interested to put in the VPS or in the SystemUser ...
    >>
    >> So when you attach the "disk limit" to SystemUser model is necessary
    >> to create a web form for SystemUser a bit different than if you decide
    >> to have disk and traffic limits. That is what my posted code is
    >> supposed to do.
    >>
    >> Do you think that I am on the wrong way designing the control panel
    >> like this?
    >>
    >> Btw, I've seen several interesting ideas to create the class
    >> dinamically (with class factorys, MetaClass, dictionaries ..), but I
    >> have not yet clear whether this will be usefull for create the
    >> __init__ function at runtime too. Any clue on this?
    >>

    >
    > this seems to work :)
    >
    > def makeLimitForm(name, monitors, _model):
    >
    > class Meta:
    > model = _model
    > dct = { 'Meta': Meta }
    >
    > for monitor in monitors:
    > field_name = monitor.resource + "_limit"
    > dct[field_name] = forms.CharField(max_length=100,
    > initial=monitor.default_limit)
    >
    > def __init__(self, *args, **kwargs):
    > ModelForm.__init__(self, *args, **kwargs)
    > if 'instance' in kwargs:
    > for monitor in monitors:
    > field_name = monitor.resource + "_limit"
    > print monitor
    > self.fields[field_name].initial =
    > kwargs['instance'].monitoring.filter(monitor=monitor)[0].current
    >
    > dct['__init__'] = __init__
    >
    > return type(name,(ModelForm,),dct)


    I've looked around to see whether Django offers an API for your usecase, but
    only found

    http://code.djangoproject.com/wiki/DynamicModels

    which seems to describe what you are already doing.
    Peter Otten, Feb 6, 2011
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. David

    Classes within classes

    David, Jul 21, 2005, in forum: ASP .Net
    Replies:
    2
    Views:
    4,926
    David
    Jul 22, 2005
  2. lonelyplanet999
    Replies:
    1
    Views:
    2,198
    VisionSet
    Nov 13, 2003
  3. Carfield Yim
    Replies:
    1
    Views:
    1,396
    Andrew Thompson
    May 31, 2004
  4. Razvan
    Replies:
    11
    Views:
    895
    Andrew Thompson
    Jul 17, 2004
  5. Razvan
    Replies:
    5
    Views:
    11,279
    Dale King
    Jul 27, 2004
Loading...

Share This Page