Efficient configuration storage

S

sebsauvage

Hello.

In one of my programs ( http://sebsauvage.net/python/webgobbler/ ),
I have a global dictionnary containing the whole program configuration.
Sample follows:
CONFIG = { "network.http.useproxy" : True,
"network.http.proxy.address": "proxy.free.fr",
"network.http.proxy.port" : 3128
[etc.]
}

I'd like to be able to save/load to/from a file.

I do not want to use pickle because I want configuration to be human readable.

ConfigParser could do the trick, but the biggest trouble is that it does
not retain type (boolean, integer, string...).

Would I have to store everything as text and cast it everywhere
it's used (and try/except each cast of course) ?
'Looks ugly and inefficient to me.

Or have a configuration class which knows the type of each parameter
and casts appropriately from the configuration file ?


Is there a better way of doing this ?
 
P

Phil Frost

One possibility is to use pprint to serialize the dictionary, and then
read it again with eval(). This is more or less human readable, but has
security implications. It could be made somewhat more readable by making
a class who's repr() would return something like "key: <repr(value)>",
and provided a method for evaluating this expression. You might be able
to use the rfc822 module. However, eval() would still be used.

You might be able to use a save flavor of eval, such as presented here:
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/286134>.

Otherwise, implementing a class that knows what type a value should be,
then using the right constructor, str(), int(), etc, might be the best
solution.
 
A

Alexis Roda

sebsauvage said:
>
ConfigParser could do the trick, but the biggest trouble is that it does
not retain type (boolean, integer, string...).

Would I have to store everything as text and cast it everywhere
it's used (and try/except each cast of course) ?
'Looks ugly and inefficient to me.

ConfigParser has methods get_bool, get_int (can't remember the exact
name) ...
Or have a configuration class which knows the type of each parameter
and casts appropriately from the configuration file ?

Well, here the cast may fail, so you should catch exception/errors too.


HTH
--
////
(@ @)
----------------------------oOO----(_)----OOo--------------------------
<> Ojo por ojo y el mundo acabara ciego
/\ Alexis Roda - Universitat Rovira i Virgili - Reus, Tarragona (Spain)
-----------------------------------------------------------------------
 
I

Istvan Albert

sebsauvage wrote:

Would I have to store everything as text and cast it everywhere
it's used (and try/except each cast of course) ?
'Looks ugly and inefficient to me.

Are you sure that casting types onto configuration parameters
will make webgobbler inefficient?

And you don't have to catch the exceptions all the time,
just once overall. Usually configuration files don't change that
often and .ini file are very simple so it is easy to spot
the cause of error. So it is not even ugly anymore.

IMHO enforcing these types at load time is a good idea.
Having to track down an error caused inside the program by
a wrong type listed in a configuration file is not fun.

Istvan.
 
L

Larry Bates

ConfigParser is VERY efficient. I have a program that reads
and parses thousands of configuration parameters in sub-second
times.

The methods are there to get strings, boolean (supports 0/1,
Y/N, Yes/No, etc.), integer, float.

I wrap the calls in try: blocks so I can catch bad config
parameters early in the process.

section="email"
option="emailtoaddress"

try: emailtoaddress=INI.get(section, option)
except:
#
# This is not fatal just set to None
#
emailtoaddress=None


section="init"
option="numberofcopies"
try: numberofcopies=INI.getint(section, option)
except: numberofcopies=1

option="debug"
try: _debug=INI.getboolean(section, option)
except: _debug=0

You get the idea.

Larry Bates
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top