a ConfigParser wtf moment

Discussion in 'Python' started by Eric S. Johansson, Jan 14, 2005.

  1. I'm not sure if this is a real problem or if I have been staring at code
    too long. given this code

    #!/usr/bin/python

    from ConfigParser import *

    configuration_file = "test.conf"
    substitution = {"xyzzy":"maze"}
    configuration = SafeConfigParser()
    configuration.readfp(file(configuration_file))


    list = configuration.items("core")
    print list

    list = configuration.items("core",0, substitution)
    print list

    ---------------

    and this configuration file

    -----------

    [core]
    xyzzy=little bird
    dwarf = greasy smoke %(xyzzy)s plugh

    ------------
    why do I get the following results?

    -------------

    trial backup # python test.py
    [('dwarf', 'greasy smoke little bird plugh'), ('xyzzy', 'little bird')]
    [('dwarf', 'greasy smoke maze plugh'), ('xyzzy', 'maze')]
    trial backup #

    -------------

    if you're having a hard time seeing it, before the substitution, xyzzy
    is set to the value in the configuration file, afterwards, it is set to
    the value of the substitution in the code. It seems to me that
    substitutions should not affect any configuration file symbols of the
    same name.

    anyway to fix this problem without python diving?

    ---eric
     
    Eric S. Johansson, Jan 14, 2005
    #1
    1. Advertising

  2. Eric S. Johansson

    Guest

    Sort of hard to explain, but if you put another:

    list = configuration.items("core")
    print list

    at the end of the script, you will find that the original config hasn't
    been changed.
    It is a quirk of how the items() method is implemented using 'yield'
    that means that
    you see what you do.

    In particular to use 'yield' it it necessary to create a temporary
    dictionary which
    contains the key/value pairs from that section of the config and then
    overlay it
    with the user supplied vars. Ie., the items() code has:

    .. d = self._defaults.copy()
    .. try:
    .. d.update(self._sections[section])
    .. except KeyError:
    .. if section != DEFAULTSECT:
    .. raise NoSectionError(section)
    .. # Update with the entry specific variables
    .. if vars:
    .. d.update(vars)

    See the last line, that will replace the value of 'xyzzy' with that
    passed in
    as argument to items().

    To avoid this, you need to write something like:

    .. list = []
    .. for key in configuration.options("core"):
    .. list.append((key,configuration.get("core",substitution))
    .. print list

    This cause me problems for a different reason, ie., that user vars keys
    appear in what items() returns. I avoid using items() for this reason.
     
    , Jan 14, 2005
    #2
    1. Advertising

  3. wrote:
    > To avoid this, you need to write something like:
    >
    > . list = []
    > . for key in configuration.options("core"):
    > . list.append((key,configuration.get("core",substitution))
    > . print list
    >
    > This cause me problems for a different reason, ie., that user vars keys
    > appear in what items() returns. I avoid using items() for this reason.


    it turns out, I originally discovered this problem through the get with
    substitutions. It's too late to muck with it now but if you are really
    interested I will generate a test case showing the failure or else prove
    I was hallucinating.

    My current workaround is to look for a %( in every value returned and if
    so do a string substitution.

    value = base.get(section,item, 1)
    if value.find("%(") != -1:
    value = value% self.interpolation_symbols

    yes, it is as ugly as a roadkill toad but it gets the job done. I've
    spent away too much time on this problem for this particular project. I
    think we all know the feeling.

    ---eric
     
    Eric S. Johansson, Jan 14, 2005
    #3
  4. Eric S. Johansson

    Guest

    True, wasn't thinking. This will affect get() as well. My problem was a
    slightly different problem.

    In your case you would have got what you wanted if get()/items()
    instead of being implemented as:

    .. try:
    .. value = d[option]
    .. except KeyError:
    .. raise NoOptionError(option, section)

    Was implemented as:

    .. try:
    .. value = self._sections[section][option]
    .. except KeyError:
    .. raise NoOptionError(option, section)

    That is, get the raw value from the original dictionary for the
    section. That way you avoid picking up a value from the user
    supplied vars. It does also avoid picking a key which has come
    through from the default section, but that could easily be
    accomodated if that was perceived to be expected behaviour
    by having the except clause fall through to looking in the
    default section. Similarly if seen that getting it from vars is
    okay as well, fall back onto that as file step. All depends on
    what the perceived overridding priority is.

    At the moment vars overrides section which overrides default
    and the document sort of does say that that is what one should
    expect for vars at least:

    .. Additional substitutions may be provided using the
    .. `vars' argument, which must be a dictionary whose contents overrides
    .. any pre-existing defaults.

    Probably not what I would have expected either. I would have expected
    vars to be available for substitution but not for option selection in
    the
    first place.
    It is too late on a Friday, so I may be hallucinating now. :)
     
    , Jan 14, 2005
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. NWx
    Replies:
    2
    Views:
    2,305
  2. a twilight zone moment

    , Mar 3, 2005, in forum: ASP .Net
    Replies:
    5
    Views:
    318
    Matt Berther
    Mar 4, 2005
  3. Roedy Green

    A bafflement moment on enum

    Roedy Green, Dec 22, 2005, in forum: Java
    Replies:
    5
    Views:
    332
    Chris Smith
    Dec 23, 2005
  4. Macroman
    Replies:
    1
    Views:
    431
    Macroman
    Feb 18, 2004
  5. Replies:
    3
    Views:
    441
Loading...

Share This Page