[rfc] An object that creates (nested) attributes automatically onassignment

Discussion in 'Python' started by Edd, Apr 11, 2009.

  1. Edd

    Edd Guest

    Hi folks,

    I'd like to use Python itself as the configuration language for my
    Python application. I'd like the user to be able to write something
    like this in their config file(s):

    cfg.laser.on = True
    cfg.laser.colour = 'blue'
    cfg.discombobulated.vegetables = ['carrots', 'broccoli']
    # ...

    To this end, I've created a class that appears to allow instance
    variables to be created on the fly. In other words, I can to the
    following to read a config file:

    cfg = Config()
    execfile(filename, {'cfg', cfg}, {})

    However, I think my implementation of the Config class is a little
    crappy. I'd really appreciate the critical eye of a pro. Here's the
    sauce:

    class Config(object):
    def __init__(self, sealed=False):
    def seal():
    for v in self._attribs.values():
    if isinstance(v, self.__class__): v.seal()
    del self.__dict__['seal']

    d = {'_attribs': {}, '_a2p': None}
    if not sealed: d['seal'] = seal

    self.__dict__.update(d)

    def __getattr__(self, key):
    if not key in self._attribs:
    d = Config(sealed='seal' not in self.__dict__)
    def add2parent():
    self._attribs[key] = d
    if self._a2p:
    self._a2p()
    self._a2p = None

    # if anything is assigned to an attribute of d,
    # make sure that d is recorded as an attribute of this
    Config
    d._a2p = add2parent
    return d
    else:
    return self._attribs[key]

    def __setattr__(self, key, value):
    if key in self.__dict__:
    self.__dict__[key] = value
    else:
    if not 'seal' in self.__dict__:
    clsname = self.__class__.__name__
    raise AttributeError("'%s' object attribute '%s'
    is read-only (object is sealed)" % (clsname, key))
    self.__dict__['_attribs'][key] = value
    if self._a2p:
    self._a2p()
    self._a2p = None

    def __delattr__(self, key):
    if key in self.__dict__:
    clsname = self.__class__.__name__
    raise AttributeError("can't delete '%s' object
    attribute '%s' as it is used for book-keeping!" % (clsname, key))
    else:
    if key in self._attribs:
    del self._attribs[key]

    def __bool__(self):
    return bool(self._attribs)

    def __nonzero__(self):
    return bool(self._attribs)

    if __name__ == '__main__':
    cfg = Config()
    cfg.a = 1
    cfg.b.c = 2
    cfg.d.e.f.g.h = [1, 2, 3]
    print cfg.a
    print cfg.b.c
    print cfg.d.e.f.g.h

    del cfg.b.c
    print cfg.b.c

    try:
    del cfg.d.e._attribs
    except AttributeError, ex:
    print ex

    cfg.seal()
    try:
    cfg.k.l.z = []
    except AttributeError, ex:
    print ex

    Once the config is loaded, it will be passed down to other user-
    written scripts and it's important that these scripts don't
    accidentally change the config. So the idea is that I'll call cfg.seal
    () to prevent any further changes before passing it on to these other
    scripts. Beyond the general fiddliness of the code, I think the way
    seal() currently works is particularly pants.

    I considered using a simpler approach:

    def mkdd(): return defaultdict(mkdd)
    cfg = mkdd()
    execfile(filename, {'cfg': cfg}, {})

    But I quite like the way the '.' separators quite naturally (IMO)
    indicate a hierarchy of settings.

    Comments and suggestions welcome!

    Kind regards,

    Edd
    Edd, Apr 11, 2009
    #1
    1. Advertising

  2. Re: [rfc] An object that creates (nested) attributes automaticallyonassignment

    On Fri, 10 Apr 2009 19:04:38 -0700, Edd wrote:

    > Hi folks,
    >
    > I'd like to use Python itself as the configuration language for my
    > Python application. I'd like the user to be able to write something like
    > this in their config file(s):
    >
    > cfg.laser.on = True
    > cfg.laser.colour = 'blue'
    > cfg.discombobulated.vegetables = ['carrots', 'broccoli'] # ...
    >
    > To this end, I've created a class that appears to allow instance
    > variables to be created on the fly.


    Um, don't all classes allow that?

    >>> class MyClass(): pass

    ....
    >>> instance = MyClass()
    >>>


    Or do you mean instance *attributes*? Again, apart from built-in types
    and classes that use __slots__, all classes allow that.

    >>> instance.parrot = 'Norwegian Blue'
    >>>


    > In other words, I can to the
    > following to read a config file:
    >
    > cfg = Config()
    > execfile(filename, {'cfg', cfg}, {})



    That's okay so long as you trust the user not to put malicious, or buggy,
    code in your config file. Personally, I think config files should be more
    tolerant of errors than a programming language.


    > However, I think my implementation of the Config class is a little
    > crappy. I'd really appreciate the critical eye of a pro. Here's the
    > sauce:


    For starters, where is your documentation? No doc strings, not even any
    comments! No, I tell a lie... *one* obscure comment that doesn't really
    explain much.


    > class Config(object):
    > def __init__(self, sealed=False):
    > def seal():
    > for v in self._attribs.values():
    > if isinstance(v, self.__class__): v.seal()
    > del self.__dict__['seal']
    >
    > d = {'_attribs': {}, '_a2p': None}
    > if not sealed: d['seal'] = seal
    >
    > self.__dict__.update(d)


    I'm going to try to guess what the above does. When you initialise an
    instance, you can tell the instance to be "sealed" or unsealed. I'm not
    sure what the difference is, or why you would choose one over the other.
    Sealed instances seem to be exactly the same as unsealed instances,
    except they have a seal() method (actually a closure). The seal method,
    when called, recursively seals any embedded Config instances inside the
    current instance, then deletes itself.

    Arghhh!!! Self-modifying code!!! Unclean, unclean!!!

    I'm not sure why seal() is necessary -- it seems to me that if present,
    all it does is delete itself. So why not just leave it out altogether?

    You also have a rather complicated way of adding instance attributes.
    Instead of

    d = {'_attribs': {}, '_a2p': None}
    self.__dict__.update(d)

    why not just do the more obvious:

    self._attribs = {}
    self._a2p = None

    ?



    > def __getattr__(self, key):
    > if not key in self._attribs:
    > d = Config(sealed='seal' not in self.__dict__) def
    > add2parent():
    > self._attribs[key] = d
    > if self._a2p:
    > self._a2p()
    > self._a2p = None



    It looks like you are just re-inventing the normal attribute mechanism of
    Python. I'm not sure why you feel this is necessary. And it contains MORE
    self-modifying code! Yuck! Frankly I don't care enough to dig into your
    code to understand how it works in detail.



    > # if anything is assigned to an attribute of d, # make
    > sure that d is recorded as an attribute of this
    > Config
    > d._a2p = add2parent
    > return d
    > else:
    > return self._attribs[key]
    >
    > def __setattr__(self, key, value):
    > if key in self.__dict__:
    > self.__dict__[key] = value
    > else:
    > if not 'seal' in self.__dict__:
    > clsname = self.__class__.__name__
    > raise AttributeError("'%s' object attribute '%s'
    > is read-only (object is sealed)" % (clsname, key))
    > self.__dict__['_attribs'][key] = value if self._a2p:
    > self._a2p()
    > self._a2p = None


    Does "sealed" mean that the instance is read-only? If so, and if I'm
    reading this correctly, I think it is buggy. You allow modifications to
    attributes inside __dict__ *without* checking to see if the instance is
    read-only. Then you get the test backwards: surely the existence, not the
    absence, of a 'seal' attribute should mean it is sealed?



    > def __delattr__(self, key):
    > if key in self.__dict__:
    > clsname = self.__class__.__name__
    > raise AttributeError("can't delete '%s' object
    > attribute '%s' as it is used for book-keeping!" % (clsname, key))
    > else:
    > if key in self._attribs:
    > del self._attribs[key]


    Nothing much to say here, except that you're doing more work re-inventing
    the wheel, storing attributes inside _attribs instead of using the
    general attribute mechanism. Seems unnecessary to me, but perhaps I don't
    understand your use-case.


    > Once the config is loaded, it will be passed down to other user- written
    > scripts and it's important that these scripts don't accidentally change
    > the config. So the idea is that I'll call cfg.seal () to prevent any
    > further changes before passing it on to these other scripts.


    Or you could pass a *copy* of the config, and let them change it to their
    heart's content, it won't matter.

    Or you could say "we're all adults here", simply document that any
    changes will have consequences, and let user scripts change the config.
    And why not?


    > Beyond the
    > general fiddliness of the code, I think the way seal() currently works
    > is particularly pants.


    Is "pants" slang for "fragile, hard to understand and difficult to debug"?



    --
    Steven
    Steven D'Aprano, Apr 11, 2009
    #2
    1. Advertising

  3. Edd

    John Machin Guest

    Re: An object that creates (nested) attributes automatically onassignment

    On Apr 11, 1:22 pm, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:
    > Is "pants" slang for "fragile, hard to understand and difficult to debug"?


    pommy slang for "sucks intensely, like the Deathstar's tractor
    beam" ... I think we agree with him.
    John Machin, Apr 11, 2009
    #3
  4. Edd

    Edd Guest

    Re: An object that creates (nested) attributes automatically onassignment

    Hi Steven,

    Thank you for your response!

    On Apr 11, 4:22 am, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:
    > On Fri, 10 Apr 2009 19:04:38 -0700, Edd wrote:
    > > Hi folks,

    >
    > > I'd like to use Python itself as the configuration language for my
    > > Python application. I'd like the user to be able to write something like
    > > this in their config file(s):

    >
    > >    cfg.laser.on = True
    > >    cfg.laser.colour = 'blue'
    > >    cfg.discombobulated.vegetables = ['carrots', 'broccoli'] # ...

    >
    > > To this end, I've created a class that appears to allow instance
    > > variables to be created on the fly.

    >
    > Um, don't all classes allow that?
    >
    > >>> class MyClass(): pass

    > ...
    > >>> instance = MyClass()

    >
    > Or do you mean instance *attributes*? Again, apart from built-in types
    > and classes that use __slots__, all classes allow that.
    >
    > >>> instance.parrot = 'Norwegian Blue'


    Yes I probably mean instance attributes. Forgive me, I am not
    particularly sure of the terminology. But your MyClass example, won't
    quite do what I want, as I'd like to be able to define instance
    attributes on top of instance attributes by assignment:

    >>> class MyClass(): pass

    ....
    >>> instance = MyClass()
    >>> instance.lasers.armed = True

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    AttributeError: MyClass instance has no attribute 'laser'
    >>>


    > > In other words, I can to the
    > > following to read a config file:

    >
    > >     cfg = Config()
    > >     execfile(filename, {'cfg', cfg}, {})

    >
    > That's okay so long as you trust the user not to put malicious, or buggy,
    > code in your config file. Personally, I think config files should be more
    > tolerant of errors than a programming language.


    That's certainly a valid remark, but this will be a tool for
    programmers. I am hoping that the user will make use of the power in
    moderation. Often, it really will be useful to allow functions to be
    defined in the config files, for example.

    > > However, I think my implementation of the Config class is a little
    > > crappy. I'd really appreciate the critical eye of a pro. Here's the
    > > sauce:

    >
    > For starters, where is your documentation? No doc strings, not even any
    > comments! No, I tell a lie... *one* obscure comment that doesn't really
    > explain much.


    Yes, you're quite right. I was about to add some doc strings, but I
    didn't think the implementation was good enough. That's somewhat
    backwards, though, right?! Especially considering I'm asking for
    improvements. Anyway, I had hoped that the example usage at the end
    would show what the purpose of the class is.

    >
    > >     class Config(object):
    > >         def __init__(self, sealed=False):
    > >             def seal():
    > >                 for v in self._attribs.values():
    > >                     if isinstance(v, self.__class__): v.seal()
    > >                 del self.__dict__['seal']

    >
    > >             d =  {'_attribs': {}, '_a2p': None}
    > >             if not sealed: d['seal'] = seal

    >
    > >             self.__dict__.update(d)

    >
    > I'm going to try to guess what the above does. When you initialise an
    > instance, you can tell the instance to be "sealed" or unsealed. I'm not
    > sure what the difference is, or why you would choose one over the other.
    > Sealed instances seem to be exactly the same as unsealed instances,
    > except they have a seal() method (actually a closure). The seal method,
    > when called, recursively seals any embedded Config instances inside the
    > current instance, then deletes itself.
    >
    > Arghhh!!! Self-modifying code!!! Unclean, unclean!!!


    Quite!

    > I'm not sure why seal() is necessary -- it seems to me that if present,
    > all it does is delete itself. So why not just leave it out altogether?


    As I said in the original post, such Config objects will be made
    available to other kinds of user-written script and it's important
    that the Config not change between the execution of one script and the
    next. The seal() mechanism was an attempt to help the user from
    *accidentally* doing this and then having to try to diagnose the
    problem and understand how changing the config might have broken the
    invariants of the software. I guess a big "DON'T CHANGE THE CONFIG IN
    YOUR SCRIPTS" message in the manual, might be sufficient, though :)

    > You also have a rather complicated way of adding instance attributes.
    > Instead of
    >
    > d =  {'_attribs': {}, '_a2p': None}
    > self.__dict__.update(d)
    >
    > why not just do the more obvious:
    >
    > self._attribs = {}
    > self._a2p = None


    Because that would go through __setattr__(), which does something else
    (which is the whole point of the class). At least, that was my
    understanding, which certainly could be at fault.

    This might be nicer I guess:

    self.__dict__['_attribs'] = {}
    self.__dict__['_a2p'] = None

    There used to be more instance attributes than just two so it was
    easier to put them in a dict and use update. I agree that it's rather
    obfuscated, though.

    [Edd's horrendous code snipped]

    > It looks like you are just re-inventing the normal attribute mechanism of
    > Python. I'm not sure why you feel this is necessary. And it contains MORE
    > self-modifying code! Yuck! Frankly I don't care enough to dig into your
    > code to understand how it works in detail.


    Ok. But is there a quick-and-easy way of creating an object, cfg, such
    that I can write:

    cfg.hovercraft.full.of = 'eels'

    without knowing in advance that the user will want a .hovercraft
    instance attribute, or a .full attribute inside that, or a .of inside
    that, or ... ?

    > >         def __setattr__(self, key, value):
    > >             if key in self.__dict__:
    > >                 self.__dict__[key] = value
    > >             else:
    > >                 if not 'seal' in self.__dict__:
    > >                     clsname = self.__class__.__name__
    > >                     raise AttributeError("'%s' object attribute '%s'
    > > is read-only (object is sealed)" % (clsname, key))
    > >                 self.__dict__['_attribs'][key] = value if self._a2p:
    > >                     self._a2p()
    > >                     self._a2p = None

    >
    > Does "sealed" mean that the instance is read-only? If so, and if I'm
    > reading this correctly, I think it is buggy. You allow modifications to
    > attributes inside __dict__ *without* checking to see if the instance is
    > read-only. Then you get the test backwards: surely the existence, not the
    > absence, of a 'seal' attribute should mean it is sealed?


    The absence of the seal() 'method' means that it has already been
    called i.e. the object has already been sealed. I agree it's somewhat
    fishy, which is why I'm asking for suggestions for improvements.
    Perhaps I should just forget the seal() idea.

    > > Once the config is loaded, it will be passed down to other user- written
    > > scripts and it's important that these scripts don't accidentally change
    > > the config. So the idea is that I'll call cfg.seal () to prevent any
    > > further changes before passing it on to these other scripts.

    >
    > Or you could pass a *copy* of the config, and let them change it to their
    > heart's content, it won't matter.


    I think that's probably the best thing. I was worried about some parts
    not being deep-copyable, but I think I'm happy to put that concern
    aside. Besides, right now, even though you can't add/delete attributes
    in a Config, you can still change existing ones:

    cfg = Config()
    cfg.a.b.c = [1, 2]
    cfg.seal()
    cfg.a.b.c[1] = 3 # pffff

    Yes, you've helped convince me that it's just a bad idea.

    > Or you could say "we're all adults here", simply document that any
    > changes will have consequences, and let user scripts change the config.
    > And why not?


    Even adults make the occasional tiny mistake which has confusing
    consequences in a larger system. It was an attempt to help prevent
    this. I probably worry too much about that.

    > > Beyond the
    > > general fiddliness of the code, I think the way seal() currently works
    > > is particularly pants.

    >
    > Is "pants" slang for "fragile, hard to understand and difficult to debug"?


    Yes! Rest assured that I am under no illusion that what I have written
    is good!

    Steven, I greatly appreciate your taking the time to understand the
    aforementioned horrors. Assuming that the seal() stuff is no longer a
    requirement, is there a cleaner way of creating an object where
    (nested) instance attributes can be defined by simple assignment?

    Perhaps it would have been better if I had left out my awful attempt
    altogether. It seems like it only made my question more confusing than
    it needed to be :( If it would help, I'd be happy to add some doc
    strings and tests but I was getting ready to throw this code away when
    a cleaner 5-line alternative was presented!

    Kind regards,

    Edd
    Edd, Apr 11, 2009
    #4
  5. Re: An object that creates (nested) attributes automatically onassignment

    On Sat, 11 Apr 2009 03:01:48 -0700, Edd wrote:

    > Yes I probably mean instance attributes. Forgive me, I am not
    > particularly sure of the terminology. But your MyClass example, won't
    > quite do what I want, as I'd like to be able to define instance
    > attributes on top of instance attributes by assignment:
    >
    >>>> class MyClass(): pass

    > ...
    >>>> instance = MyClass()
    >>>> instance.lasers.armed = True

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in <module>
    > AttributeError: MyClass instance has no attribute 'laser'


    Ah, now it is more clear.

    Okay, let's try this:


    >>> class C(object):

    .... def __getattr__(self, name):
    .... # Only called if self.name doesn't exist.
    .... inst = self.__class__()
    .... setattr(self, name, inst)
    .... return inst
    ....
    >>> c = C()
    >>> c.x.y.z = 45
    >>> c.__dict__

    {'x': <__main__.C object at 0xb7c3b78c>}
    >>> c.x.__dict__

    {'y': <__main__.C object at 0xb7c3b7ec>}
    >>> c.x.y.z

    45



    --
    Steven
    Steven D'Aprano, Apr 11, 2009
    #5
  6. Edd

    Edd Guest

    Re: An object that creates (nested) attributes automatically onassignment

    On Apr 11, 12:54 pm, Steven D'Aprano <st...@REMOVE-THIS-
    cybersource.com.au> wrote:

    > Ah, now it is more clear.
    >
    > Okay, let's try this:
    >
    > >>> class C(object):

    >
    > ...     def __getattr__(self, name):
    > ...             # Only called if self.name doesn't exist.
    > ...             inst = self.__class__()
    > ...             setattr(self, name, inst)
    > ...             return inst


    Ha! Perfect! I knew it should be simpler. Thanks very much!

    Kind regards,

    Edd
    Edd, Apr 11, 2009
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Amelyan
    Replies:
    2
    Views:
    427
    Amelyan
    May 25, 2005
  2. request@no_spam.com
    Replies:
    5
    Views:
    413
  3. =?Utf-8?B?am9uZWZlcg==?=

    Dropdown List in nested MasterPage creates an error

    =?Utf-8?B?am9uZWZlcg==?=, May 16, 2007, in forum: ASP .Net
    Replies:
    1
    Views:
    343
  4. crybaby
    Replies:
    1
    Views:
    297
    Peter Otten
    Sep 23, 2007
  5. Ivan Shmakov
    Replies:
    3
    Views:
    1,125
    Kari Hurtta
    Feb 13, 2012
Loading...

Share This Page