Persuading ConfigParser to give me the section elements in the sameorder as the file

[geoffbache]

Hi all,

I recently needed to parse a file that was perfect for ConfigParser
apart from one thing: the elements in the sections, although
definitions, could in some cases clash with each other and therefore
it was important to be able to retrieve them in the same order as they
appeared in the file.

Unfortunately ConfigParser uses ordinary dictionaries for the section
elements and they are therefore returned in an arbitrary order.

The only solution I found was to copy ConfigParser.py and replace all
the dictionaries with "sequential dictionaries"
which are exactly like dictionaries except that elements are returned
in the order they were inserted. (see
http://home.arcor.de/wolfgang.grafen/Python/Modules/seqdict/Seqdict.html)

I wonder if there was a better way? For example, is there any hook
that could modify what is created by the statement

x = {}

I tried setting

__builtins__.dict = ndict.seqdict

But that didn't seem to have any effect on the above statement.

As a secondary question, I find sequential dictionaries to be an
essential part of programming in Python and I use them all the time. I
wondered a bit if there were any plans or proposals to include them as
part of the Python library?

Regards,
Geoff Bache

 

[Matimus]

Hi all,

I recently needed to parse a file that was perfect for ConfigParser
apart from one thing: the elements in the sections, although
definitions, could in some cases clash with each other and therefore
it was important to be able to retrieve them in the same order as they
appeared in the file.

Unfortunately ConfigParser uses ordinary dictionaries for the section
elements and they are therefore returned in an arbitrary order.

The only solution I found was to copy ConfigParser.py and replace all
the dictionaries with "sequential dictionaries"
which are exactly like dictionaries except that elements are returned
in the order they were inserted. (seehttp://home.arcor.de/wolfgang.grafen/Python/Modules/seqdict/Seqdict.html)

I wonder if there was a better way? For example, is there any hook
that could modify what is created by the statement

x = {}

I tried setting

__builtins__.dict = ndict.seqdict

But that didn't seem to have any effect on the above statement.

As a secondary question, I find sequential dictionaries to be an
essential part of programming in Python and I use them all the time. I
wondered a bit if there were any plans or proposals to include them as
part of the Python library?

Regards,
Geoff Bache

Have a look at this: http://www.python.org/dev/peps/pep-0372/

Looking at the config parser module, it looks like there are only a
couple of places where {} is used. I would create a mixin class to
replace the offending methods. That should work because it looks like
you only have to replace "__init__" and "add_section". So...


class OrderedConfigParserMixin:
def __init__(self, defaults=None):
self._sections = ndict.seqdict()
self._defaults = ndict.seqdict()
if defaults:
for key, value in defaults.items():
self._defaults[self.optionxform(key)] = value

def add_section(self, section):
"""Create a new section in the configuration.

Raise DuplicateSectionError if a section by the specified name
already exists.
"""
if section in self._sections:
raise DuplicateSectionError(section)
self._sections[section] = ndict.seqdict()

# Then you can use this to create your own ordered config parsers.
Note that
# multiple inheritance in python uses a breadth first search. If you
want
# the methods on your mixin to get called instead of the methods on
the
# original class you must include the mixin first.

from ConfigParser import RawConfigParser, ConfigParser,
SafeConfigParser

class OrderedRawConfigParser(OrderedConfigParserMixin,
RawConfigParser):
pass

class OrderedConfigParser(OrderedConfigParserMixin, ConfigParser):
pass

class OrderedSafeConfigParser(OrderedConfigParserMixin,
SafeConfigParser):
pass


I don't know if this is the _best_ approach, but it is certainly much
preferred over monkey patching the built-ins module. Note that I
haven't tested any of the above code.


Matt

 

[geoffbache]

Hi Matt,

Have a look at this:http://www.python.org/dev/peps/pep-0372/

Thanks, that was very useful. Good to know these things are being
considered.

Looking at the config parser module, it looks like there are only a
couple of places where {} is used. I would create a mixin class to
replace the offending methods. That should work because it looks like
you only have to replace "__init__" and "add_section". So...

class OrderedConfigParserMixin:
    def __init__(self, defaults=None):
        self._sections = ndict.seqdict()
        self._defaults = ndict.seqdict()
        if defaults:
            for key, value in defaults.items():
                self._defaults[self.optionxform(key)] = value

    def add_section(self, section):
        """Create a new section in the configuration.

        Raise DuplicateSectionError if a section by the specified name
        already exists.
        """
        if section in self._sections:
            raise DuplicateSectionError(section)
        self._sections[section] = ndict.seqdict()

# Then you can use this to create your own ordered config parsers.
Note that
# multiple inheritance in python uses a breadth first search. If you
want
# the methods on your mixin to get called instead of the methods on
the
# original class you must include the mixin first.

from ConfigParser import RawConfigParser, ConfigParser,
SafeConfigParser

class OrderedRawConfigParser(OrderedConfigParserMixin,
RawConfigParser):
    pass

class OrderedConfigParser(OrderedConfigParserMixin, ConfigParser):
    pass

class OrderedSafeConfigParser(OrderedConfigParserMixin,
SafeConfigParser):
    pass

I don't know if this is the _best_ approach, but it is certainly much
preferred over monkey patching the built-ins module. Note that I
haven't tested any of the above code.

Yes, I tried this first. But actually you missed the main place where
dictionaries are created, which is the monster method _read, the line
being

cursect = {'__name__': sectname}

I thought by the time I'd copied that whole method just to edit that
line I may as well just copy the whole file and forget the
inheritance

btw, the PEP you pointed me at indicated ConfigParser will take a
dict_type argument for exactly this purpose in Python 2.6, so I look
forward to when I can use that instead...

Regards,
Geoff