lightweight human-readable config?

M

Maxim Khesin

I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

can you guys suggest some format for this? thanks,
max
 
J

John Roth

Maxim Khesin said:
I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

can you guys suggest some format for this? thanks,
max

A text file containing lines formatted as:

key: value

for line in inFile.readlines():
key, value = line.split(":")
configdict[key] = value

I usually like to add a statement that filters out lines
that start with a "#" character.

Elaborate to taste.

John Roth
 
N

Noen

John Roth wrote:

I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

can you guys suggest some format for this? thanks,
max


A text file containing lines formatted as:

key: value

for line in inFile.readlines():
key, value = line.split(":")
configdict[key] = value

I usually like to add a statement that filters out lines
that start with a "#" character.

Elaborate to taste.

John Roth
You could try out
This format is already avaiable as a python module following the .ini
format (RFC 822).
The format is like this
[SECTION]
varname1: var
varname2: var
varname3: var
varname4: var
....
Compiled using the -OO, its size is.
30.10.2003 00:33 13 483 ConfigParser.pyo

Enjoy -- Noen
 
H

Heather Coppersmith

I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)
can you guys suggest some format for this? thanks,

Will the standard library ConfigParser work? Point your web
browser here:

http://www.python.org/doc/current/lib/module-ConfigParser.html

Regards,
Heather
 
V

Ville Vainio

Maxim Khesin said:
I want to have a config file with my python proggie, satisfying the
following requirements:

I usually use plain old python modules for configuration. If you don't
want to use 'import' and a name ending with .py, just exec the module
in some namaspace. You can do "default value" behaviour by execing
some default value module in the same namespace (often you might want
to parse /etc/fooconfig before ~/.fooconfig). Then you just harvest
the namespace for the config data.
 
A

Anthony Briggs

I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

It's probably reinventing the wheel, but here's one I wrote a while
ago. It might serve as a starting point, if nothing else. The actual
code's quite short - it's all those pesky comments and unit tests
that take up all the room ;)

It's also possible to have .py files imported (I forget how) from a
py2exe program, so you can also go down the python formatted config
file route.

Anthony


--

#!/usr/local/bin/python

# A stab at writing a config script
# We'll assume that # and ; are comment characters
# unless they're in quotes

# to run the unit tests, simply execute this script:
# config.py -v

import sys, os, string
import unittest

class config:
""" A class to store configuration values in a text file, and
read them out.

To use this class, simply declare it:

myconfig = config("config.txt")

it will automatically read in all of the values from config.txt,
and myconfig will then behave roughly like a dictionary:

myconfig['name'] # corresponds to name=... in config.txt

There are also the following functions:

myconfig.keys() # a list of all of the attributes
myconfig.delete('wibble') # delete attribute wibble
myconfig.readconfig() # read in the config file again
myconfig.writeconfig([filename]) # write the config file
# filename is optional
"""

def __init__(self, configfile):
"""Set up the class, and read in all of the values."""
self._info={}
self._file=configfile
if os.access(self._file, os.F_OK) == 1:
# we can read in the config file
self.readconfig()


# A couple of simple functions to make using the class easier
# ie. you can just say configclassinstance['blah']
def __getitem__(self, name):
return self._info[name]

def __setitem__(self, name, value):
self._info[name] = value

def keys(self):
"""return all of the config values."""
return self._info.keys()

def delete(self, item):
"""Delete a key from the config directory."""
if item not in self._info.keys():
raise IndexError, 'Item '+str(item)+' not found in the config.'
else:
del self._info[item]


def readconfig(self):
"""Read in the config file, line by line."""
# First, clear out the old config:
self._info = {}

# Now, open it and read in each line, calling _readconfigline
to parse it
file = open(self._file, 'r')
for line in file.readlines():
self._readconfigline(line)
file.close()

return

def _readconfigline(self,theline):
"""Given a line, parse it and put it into the self.info dictionary."""

theline = string.strip(theline)
if theline=='' or theline[0]=='#' or theline[0]==';':
return # this line is empty or a comment

attribute, value = string.split(theline, '=', 1)

attribute = string.strip(attribute)
value = string.strip(value)

if value[0] == '=':
raise ValueError, "Too many equal signs in "+theline

# First we check for quotes, just to be sure that
# there aren't any nested comment chars in them
value = string.strip(value)
if '"' in value:
if value[0]=='"' and string.count(value, '"') == 2:
value, ignored = string.split(value[1:], '"', 1)
else: # they're playing funny buggers, and gave us
# something like attr=blah"blah"
raise ValueError, "Quotes misplaced in "+theline

else: # no quotes - cut sick!
if '#' in value:
value, ignored = string.split(value, '#', 1)
value = string.strip(value)
if ';' in value:
value, ignored = string.split(value, ';', 1)
value = string.strip(value)

self._info[attribute] = value


def writeconfig(self, filename=""):
"""Write out all of the values in the config to a file."""

if filename=="":
filename = self._file

if os.access(self._file, os.F_OK) == 1:
# We might want to raise an error at this point...
raise IOError, "File "+self._file+" already exists!"

file = open(filename, 'w')
for item in self._info.keys():
file.write(item+'="'+self._info[item]+'"\n')
file.close()



class testItems(unittest.TestCase):
"""Test that config works correctly."""

configtest = config("testconfig.txt")

goodconfiglines = [ # line, attribute, value
['womble=blah', 'womble', 'blah'],
['womble2="blah2"', 'womble2', 'blah2'],

['blah2="wibble" #comment', 'blah2', 'wibble'],
['blah3="wibble2" ;comment', 'blah3', 'wibble2'],
['blah6="####" # A comment', 'blah6', '####'],
['blah7=";;;;" ; A comment', 'blah7', ';;;;'],
['blah10=quotes a go go! ; # # ; ; ##; ;#',
'blah10', 'quotes a go go!'],

['blah5 = "womble5"', 'blah5', 'womble5'],
['blah9=womble72 ; comment!', 'blah9', 'womble72'],
['blah11=" spaces should be preserved! "', 'blah11',
" spaces should be preserved! "],

["blah8=This won't break!", 'blah8', "This
won't break!"],

]

badconfiglines = [ # these config lines should throw a ValueError
'blah11="broken"quotes"',
'blah12="more"broken"quotes"',
'blah13=broken"quotes?"',
'strangequals=="blah"',
'strangeequals2="wibble"="blah"',
]

def testAddLines(self):
"""Test that lines can be added to the config object."""
# First, clear out self.testconfig:
for item in self.configtest.keys():
self.configtest.delete(item)

for item in self.goodconfiglines:
self.configtest._readconfigline(item[0])
self.assertEqual(item[2], self.configtest[item[1]])

def testDelete(self):
"""Test that delete works."""
# First, clear out self.testconfig:
for item in self.configtest.keys():
self.configtest.delete(item)

# Now add and delete
for item in self.goodconfiglines:
self.configtest[item[1]]=item[2]
self.failUnless(self.configtest[item[1]]==item[2])

self.configtest.delete(item[1])
self.failIf(item[2] in self.configtest.keys())

def testConfigKeys(self):
"""Test that items added appear in config.keys()."""
# First, clear out self.testconfig:
for item in self.configtest.keys():
self.configtest.delete(item)
self.failUnless(self.configtest.keys()==[])

# Now add and delete
for item in self.goodconfiglines:
self.configtest[item[1]]=item[2]
self.failUnless(item[1] in self.configtest.keys())


def testConfigLines(self):
"""Test that all of the types of config lines can be read
from a file."""
for item in self.goodconfiglines:
#write the line to a config file
file = open("testconfig.txt", "w")
file.write(item[0])
file.close()

# Now check that it can be read from the file
self.configtest.readconfig()
self.assertEqual(item[2], self.configtest[item[1]])

def testSanityCheck(self):
"""For each item in goodconfiglines, test that it can be
written to a file
and read back unchanged."""
# First, clear out self.testconfig:
for item in self.configtest.keys():
self.configtest.delete(item)

# Now test!
for item in self.goodconfiglines:
self.configtest[item[1]]=item[2]
os.unlink("testconfig.txt")
self.configtest.writeconfig("")
self.configtest.readconfig()
self.assertEqual(item[2],self.configtest[item[1]])

def testBadConfigLines(self):
"""Test that dodgy config lines will throw an exception."""
for item in self.badconfiglines:
self.assertRaises(ValueError,
self.configtest._readconfigline, theline=item)



if __name__ == "__main__":
unittest.main()
 
A

anton muhin

Maxim said:
I want to have a config file with my python proggie, satisfying the
following requirements:
1) support key->(value, default)
2) simple and intuitive to read and edit
3) easyly readable into a python datastructure (a dictionary?)
4) not requiring any heavy libraries needed (I am distributing my
proggie as a py2exe executable and do not want to bloat the size)

can you guys suggest some format for this? thanks,
max
I like YAML (www.yaml.org), although the only Python implementation I'm
aware of is a bit buggy.

hth,
anton.
 
N

Nick Vargish

John Roth said:
for line in inFile.readlines():
key, value = line.split(":")
configdict[key] = value

for line in inFile.readlines():
key, value = line.split(':', 1) # only split once, values could
# have ':' in them
key = key.rstrip() # remove trailing spaces from key
value = value.lstrip() # remove leading spaces from value
configdict[key] = value

You could reduce the last three lines to:

configdict[key.rstrip()] = value.lstrip()

Nick
 
S

Skip Montanaro

Maxim> Thanks for the pointer. It is not clear how this supports
Maxim> defaults

Take a look at the SpamBayes Options.py file.

Skip
 
M

Maxim Khesin

Nick said:
for line in inFile.readlines():
key, value = line.split(":")
configdict[key] = value


for line in inFile.readlines():
key, value = line.split(':', 1) # only split once, values could

Just one comment: using ':' for separator is not a great idea if Windows
paths are come of the potential values.
m
 
J

John Roth

Maxim Khesin said:
Nick said:
for line in inFile.readlines():
key, value = line.split(":")
configdict[key] = value


for line in inFile.readlines():
key, value = line.split(':', 1) # only split once, values could

Just one comment: using ':' for separator is not a great idea if Windows
paths are come of the potential values.

That's why the "1" as the second parameter. It makes sure that
split doesn't do more than one split.

John Roth
 
C

Corey Coughlin

I actually have a whole set of programs that use a format like this,
it's actually parsed on spaces and keywords, so you just say:

keyword1 value1 value2 value3

keyword2 value1

keyword3
value1
value2

and so on. It's pretty free format, and relatively easy to parse, all
you need to set up is a list of keywords, and in return from these
functions you get a dictionary with the value lists for each keyword.
You can also add comments by adding '#' signs, and the rest of the
line is a comment. Here are the functions:

def ParseCommandFile(commfn, commkeys):
commwords = ReadCommandFile(commfn)
return ParseList(commwords, commkeys)

def ReadCommandFile(commfn):
if not (commfn and os.path.exists(commfn):
raise OSError, "Can't find this path: %s" % fname
commf = open(commfn,'r')
commtext = ''
for line in commf:
if '#' in line:
line = line[:string.find(line,'#')]
commtext = commtext + ' ' + line
commf.close()
commwords = string.split(commtext)
return commwords

def ParseList(commlist, commkeys):
commvals = {}
for ck in commkeys:
commvals[ck] = []
currkey = ''
currval = []
foundkeys = commkeys[:]
for cword in commlist:
if currkey:
if cword in foundkeys:
commvals[currkey] = currval
currkey = cword
foundkeys.remove(currkey)
currval = []
else:
currval.append(cword)
else:
if cword in foundkeys:
currkey = cword
foundkeys.remove(currkey)
if currkey:
commvals[currkey] = currval
return commvals

There it is, maybe not the most efficient code, but it lets you write
a command file parsed on spaces, so there's no need to worry about
extra syntax characters, which keeps it nice and clean.
 
L

Larry Bates

You can have defaults in configparser .INI file if you like.

import ConfigParser
ini_filename='program.ini'
defaults={'key1':'value1','key2':'value2')
#
# This will initialize the ConfigParser with dictionary
# of key,value defaults.
INI=ConfigParser.ConfigParser(defaults)
#
# Make sure .INI file exists
#
if not os.path.exists:
print "Unable to locate .INI filename=%s" % ini_filename
sys.exit(2)
#
# Now read the .INI file's contents
#
INI.read(ini_filename)

section='INIT'
option='debug'
try: debug=INI.getboolean(section, option)
except: debug=0

option='trace'
try: trace=INI.getboolean(section, option)
except: trace=0

section='RUNTIME'
option='first_invoice'
try: first_invoice=INI.get(section, option)
except:
print ".INI file missing first_invoice key"
sys.exit(2)

option='last_invoice'
try: last_invoice=INI.get(section, option)
except:
print ".INI file missing last_invoice key"
sys.exit(2)

..
.. remainder of program
..



..INI files have following format

[SECTION1]
KEY1=
KEY2=
KEY3=

[SECTION2]
KEY1=
KEY2=
KEY3=

You can have as many sections as you like and keynames
are unique under a section. Normally I do something like:

[INIT]
debug=1
trace=1
logfile='program.log'

[RUNTIME]
first_invoice=12345
last_invoice=99999

This way I can turn on/off debug and trace modes and
have the program pick up RUNTIME parameters for processing.

Hope information helps.
-Larry
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,755
Messages
2,569,537
Members
45,020
Latest member
GenesisGai

Latest Threads

Top