Dealing with config files what's the options

[Tom Willis]

How are the expert pythoneers dealing with config files?

Is there anything similair to .net's config files or java's .properties?

A quick search on google didn't return anything that looked useful,
and I almost would expect to see some module that would be for dealing
with config information.

I can think of at least one way to do it, but I'm sure there are
shortcomings I can't see yet, and I'd rather use something someone
smarter than me has written.

I see in the logging module that there's stuff to handle configs but
seems kind of odd to have to import logging to get your config
information

Any ideas?

have I used the word config enough in this message?

 

[Fuzzyman]

Hello Tom,

How are the expert pythoneers dealing with config files?

Is there anything similair to .net's config files or java's ..properties?

I'm not familiar with those config file formats - but ConfigObj
certainly makes handling config files easy. It uses the ini type layout
- which you're not so fond of, although it aloows lists for values as
well.

from configobj import ConfigObj
config = ConfigObj(filename)
value1 = config['section 1']['value 1']

See http://www.voidspace.org.uk/python/configobj.html

I'm interested in suggestions as to ways to take it forward. I've just
added unicode support (still experimental - wait for the next release)
and an interface for validation. Adding nested sections using
indentation will probably be the next major feature..... (as well as
preserving user formatting when writing back files)

Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml

 

[Jorgen Grahn]

How are the expert pythoneers dealing with config files? ....
Any ideas?

How about writing them in Python?

I have no URL handy, but it would surprise me if there wasn't a lot written
about different techniques for doing this.

/Jorgen

 

[Dave Brueck]

How about writing them in Python?

Depending on who will be editing the config files, this can be a great approach.

At the simplest level, a config.py file like this is so easy to use:

# Net settings
timeoutSec = 5.3
maxConnections = 3

# Other stuff
foo = 'bar'

This type of a format is easy to use for just about anybody who has ever had to
use config files before. What's nice is that the code to use it is
straightforward too:

import config
conn = config.maxConnections
....

A few times I've tried to use accessor functions to ensure that the values are
present or valid or whatever, but I stopped doing that because in practice it's
just not needed (again, for users who are familiar with the concept of config
files).

A slightly more elaborate approach gives you full structure:

class Net:
maxConnections = 12

class System:
class Logging:
root = '/var/logs'

This prevents individual setting names from getting unwieldy, and the code that
uses it can be pretty readable too:

logRoot = config.System.Logging.root

or, if there are lots of retrievals to do:

Logging = config.System.Logging
logRoot = Logging.root
.... etc ...

Using classes asks a little bit more of the users (they can break it a little
more easily), but again, in practice it really hasn't been a problem at all.

-Dave

 

[Tom Willis]

Depending on who will be editing the config files, this can be a great approach.

At the simplest level, a config.py file like this is so easy to use:

# Net settings
timeoutSec = 5.3
maxConnections = 3

# Other stuff
foo = 'bar'

This type of a format is easy to use for just about anybody who has ever had to
use config files before. What's nice is that the code to use it is
straightforward too:

import config
conn = config.maxConnections
...

A few times I've tried to use accessor functions to ensure that the values are
present or valid or whatever, but I stopped doing that because in practice it's
just not needed (again, for users who are familiar with the concept of config
files).

A slightly more elaborate approach gives you full structure:

class Net:
maxConnections = 12

class System:
class Logging:
root = '/var/logs'

This prevents individual setting names from getting unwieldy, and the code that
uses it can be pretty readable too:

logRoot = config.System.Logging.root

or, if there are lots of retrievals to do:

Logging = config.System.Logging
logRoot = Logging.root
... etc ...

Using classes asks a little bit more of the users (they can break it a little
more easily), but again, in practice it really hasn't been a problem at all.

-Dave

I actually thought of this, and I was kind of on the fence due to the
intended audience.

I don't think it's too much to ask that they are comfy with the
concept of variables. I mean, if it was a shell script they'd be at
the top of the file anyway.

Then again I'm some what hesitant to help them make the connection
that I'm giving them the ability to indirectly edit the code. Kind of
like opening pandoras box. Once the figure out they can open any file
(*.py) with notepad, there will be utter anarchy and I'll get the call
at 4am that somethings wrong with the production data.

 

[Dave Brueck]

Depending on who will be editing the config files, this can be a great approach.

[snip]
I actually thought of this, and I was kind of on the fence due to the
intended audience.

I don't think it's too much to ask that they are comfy with the
concept of variables. I mean, if it was a shell script they'd be at
the top of the file anyway.

Then again I'm some what hesitant to help them make the connection
that I'm giving them the ability to indirectly edit the code. Kind of
like opening pandoras box. Once the figure out they can open any file
(*.py) with notepad, there will be utter anarchy and I'll get the call
at 4am that somethings wrong with the production data.

If you're giving them an executable (i.e. py2exe'd), then you can just exclude
config.py from the exe.

Either way, if you're not so worried about malicious breakage but ignorant
breakage, then you could always name your config file something like
'options.cfg' and then:

import new, sys

# Do this at app startup
sys.modules['config'] = new.module('config')
exec file('options.cfg').read() in config.__dict__

# All other modules do this
import config
conn = config.maxConnections
.... etc ...

-Dave

 

[Jorgen Grahn]

....
I actually thought of this, and I was kind of on the fence due to the
intended audience.

I don't think it's too much to ask that they are comfy with the
concept of variables. I mean, if it was a shell script they'd be at
the top of the file anyway.

Then again I'm some what hesitant to help them make the connection
that I'm giving them the ability to indirectly edit the code. Kind of
like opening pandoras box. Once the figure out they can open any file
(*.py) with notepad, there will be utter anarchy and I'll get the call
at 4am that somethings wrong with the production data.

There's two issues there.

One is about hiding knowledge from the users. I'm against that; this isn't
the middle ages. If they shoot themselves in the foot, that is /their/
problem, not yours. You support /your/ code, not whatever they hack together.
And with 99.999% certainty, they won't touch it.

The other is about the config file messing up the environment for your main
program. Seems hard to do by mistake -- you'd have to write endless loops or
import modules and modify them, or someting. I seem to recall that you can
evaluate a piece of code in a separate environment/sandbox -- maybe that's
the way to go?

(Personally, I try to avoid designs which need config files. If I /did/ need
one I'd steal the design from some well-known Unix program, because that's
my primary target, and I'd prefer not to lock myself into Python -- I might
want to rewrite the thing in C, or Perl, or ...)

/Jorgen