lightweight human-readable config?

Discussion in 'Python' started by Maxim Khesin, Oct 29, 2003.

  1. Maxim Khesin

    Maxim Khesin Guest

    I want to have a config file with my python proggie, satisfying the
    following requirements:
    1) support key->(value, default)
    2) simple and intuitive to read and edit
    3) easyly readable into a python datastructure (a dictionary?)
    4) not requiring any heavy libraries needed (I am distributing my
    proggie as a py2exe executable and do not want to bloat the size)

    can you guys suggest some format for this? thanks,
    max
     
    Maxim Khesin, Oct 29, 2003
    #1
    1. Advertising

  2. Maxim Khesin

    John Roth Guest

    "Maxim Khesin" <> wrote in message
    news:p4Xnb.17367$...
    > I want to have a config file with my python proggie, satisfying the
    > following requirements:
    > 1) support key->(value, default)
    > 2) simple and intuitive to read and edit
    > 3) easyly readable into a python datastructure (a dictionary?)
    > 4) not requiring any heavy libraries needed (I am distributing my
    > proggie as a py2exe executable and do not want to bloat the size)
    >
    > can you guys suggest some format for this? thanks,
    > max


    A text file containing lines formatted as:

    key: value

    for line in inFile.readlines():
    key, value = line.split(":")
    configdict[key] = value

    I usually like to add a statement that filters out lines
    that start with a "#" character.

    Elaborate to taste.

    John Roth


    >
     
    John Roth, Oct 29, 2003
    #2
    1. Advertising

  3. Maxim Khesin

    Noen Guest

    John Roth wrote:


    > "Maxim Khesin" <> wrote in message
    > news:p4Xnb.17367$...
    >
    >>I want to have a config file with my python proggie, satisfying the
    >>following requirements:
    >>1) support key->(value, default)
    >>2) simple and intuitive to read and edit
    >>3) easyly readable into a python datastructure (a dictionary?)
    >>4) not requiring any heavy libraries needed (I am distributing my
    >>proggie as a py2exe executable and do not want to bloat the size)
    >>
    >>can you guys suggest some format for this? thanks,
    >>max

    >
    >
    > A text file containing lines formatted as:
    >
    > key: value
    >
    > for line in inFile.readlines():
    > key, value = line.split(":")
    > configdict[key] = value
    >
    > I usually like to add a statement that filters out lines
    > that start with a "#" character.
    >
    > Elaborate to taste.
    >
    > John Roth

    You could try out
    This format is already avaiable as a python module following the .ini
    format (RFC 822).
    The format is like this
    [SECTION]
    varname1: var
    varname2: var
    varname3: var
    varname4: var
    ....
    Compiled using the -OO, its size is.
    30.10.2003 00:33 13 483 ConfigParser.pyo

    Enjoy -- Noen
    >
    >
    >
    >
    >
     
    Noen, Oct 29, 2003
    #3
  4. On Wed, 29 Oct 2003 22:22:13 GMT,
    Maxim Khesin <> wrote:

    > I want to have a config file with my python proggie, satisfying the
    > following requirements:
    > 1) support key->(value, default)
    > 2) simple and intuitive to read and edit
    > 3) easyly readable into a python datastructure (a dictionary?)
    > 4) not requiring any heavy libraries needed (I am distributing my
    > proggie as a py2exe executable and do not want to bloat the size)


    > can you guys suggest some format for this? thanks,


    Will the standard library ConfigParser work? Point your web
    browser here:

    http://www.python.org/doc/current/lib/module-ConfigParser.html

    Regards,
    Heather

    --
    Heather Coppersmith
    That's not right; that's not even wrong. -- Wolfgang Pauli
     
    Heather Coppersmith, Oct 29, 2003
    #4
  5. Maxim Khesin

    Ville Vainio Guest

    Maxim Khesin <> writes:

    > I want to have a config file with my python proggie, satisfying the
    > following requirements:


    I usually use plain old python modules for configuration. If you don't
    want to use 'import' and a name ending with .py, just exec the module
    in some namaspace. You can do "default value" behaviour by execing
    some default value module in the same namespace (often you might want
    to parse /etc/fooconfig before ~/.fooconfig). Then you just harvest
    the namespace for the config data.

    --
    Ville Vainio http://www.students.tut.fi/~vainio24
     
    Ville Vainio, Oct 30, 2003
    #5
  6. At 10:22 PM +0000 29/10/03, Maxim Khesin wrote:
    >I want to have a config file with my python proggie, satisfying the
    >following requirements:
    >1) support key->(value, default)
    >2) simple and intuitive to read and edit
    >3) easyly readable into a python datastructure (a dictionary?)
    >4) not requiring any heavy libraries needed (I am distributing my
    >proggie as a py2exe executable and do not want to bloat the size)


    It's probably reinventing the wheel, but here's one I wrote a while
    ago. It might serve as a starting point, if nothing else. The actual
    code's quite short - it's all those pesky comments and unit tests
    that take up all the room ;)

    It's also possible to have .py files imported (I forget how) from a
    py2exe program, so you can also go down the python formatted config
    file route.

    Anthony


    --

    #!/usr/local/bin/python

    # A stab at writing a config script
    # We'll assume that # and ; are comment characters
    # unless they're in quotes

    # to run the unit tests, simply execute this script:
    # config.py -v

    import sys, os, string
    import unittest

    class config:
    """ A class to store configuration values in a text file, and
    read them out.

    To use this class, simply declare it:

    myconfig = config("config.txt")

    it will automatically read in all of the values from config.txt,
    and myconfig will then behave roughly like a dictionary:

    myconfig['name'] # corresponds to name=... in config.txt

    There are also the following functions:

    myconfig.keys() # a list of all of the attributes
    myconfig.delete('wibble') # delete attribute wibble
    myconfig.readconfig() # read in the config file again
    myconfig.writeconfig([filename]) # write the config file
    # filename is optional
    """

    def __init__(self, configfile):
    """Set up the class, and read in all of the values."""
    self._info={}
    self._file=configfile
    if os.access(self._file, os.F_OK) == 1:
    # we can read in the config file
    self.readconfig()


    # A couple of simple functions to make using the class easier
    # ie. you can just say configclassinstance['blah']
    def __getitem__(self, name):
    return self._info[name]

    def __setitem__(self, name, value):
    self._info[name] = value

    def keys(self):
    """return all of the config values."""
    return self._info.keys()

    def delete(self, item):
    """Delete a key from the config directory."""
    if item not in self._info.keys():
    raise IndexError, 'Item '+str(item)+' not found in the config.'
    else:
    del self._info[item]


    def readconfig(self):
    """Read in the config file, line by line."""
    # First, clear out the old config:
    self._info = {}

    # Now, open it and read in each line, calling _readconfigline
    to parse it
    file = open(self._file, 'r')
    for line in file.readlines():
    self._readconfigline(line)
    file.close()

    return

    def _readconfigline(self,theline):
    """Given a line, parse it and put it into the self.info dictionary."""

    theline = string.strip(theline)
    if theline=='' or theline[0]=='#' or theline[0]==';':
    return # this line is empty or a comment

    attribute, value = string.split(theline, '=', 1)

    attribute = string.strip(attribute)
    value = string.strip(value)

    if value[0] == '=':
    raise ValueError, "Too many equal signs in "+theline

    # First we check for quotes, just to be sure that
    # there aren't any nested comment chars in them
    value = string.strip(value)
    if '"' in value:
    if value[0]=='"' and string.count(value, '"') == 2:
    value, ignored = string.split(value[1:], '"', 1)
    else: # they're playing funny buggers, and gave us
    # something like attr=blah"blah"
    raise ValueError, "Quotes misplaced in "+theline

    else: # no quotes - cut sick!
    if '#' in value:
    value, ignored = string.split(value, '#', 1)
    value = string.strip(value)
    if ';' in value:
    value, ignored = string.split(value, ';', 1)
    value = string.strip(value)

    self._info[attribute] = value


    def writeconfig(self, filename=""):
    """Write out all of the values in the config to a file."""

    if filename=="":
    filename = self._file

    if os.access(self._file, os.F_OK) == 1:
    # We might want to raise an error at this point...
    raise IOError, "File "+self._file+" already exists!"

    file = open(filename, 'w')
    for item in self._info.keys():
    file.write(item+'="'+self._info[item]+'"\n')
    file.close()



    class testItems(unittest.TestCase):
    """Test that config works correctly."""

    configtest = config("testconfig.txt")

    goodconfiglines = [ # line, attribute, value
    ['womble=blah', 'womble', 'blah'],
    ['womble2="blah2"', 'womble2', 'blah2'],

    ['blah2="wibble" #comment', 'blah2', 'wibble'],
    ['blah3="wibble2" ;comment', 'blah3', 'wibble2'],
    ['blah6="####" # A comment', 'blah6', '####'],
    ['blah7=";;;;" ; A comment', 'blah7', ';;;;'],
    ['blah10=quotes a go go! ; # # ; ; ##; ;#',
    'blah10', 'quotes a go go!'],

    ['blah5 = "womble5"', 'blah5', 'womble5'],
    ['blah9=womble72 ; comment!', 'blah9', 'womble72'],
    ['blah11=" spaces should be preserved! "', 'blah11',
    " spaces should be preserved! "],

    ["blah8=This won't break!", 'blah8', "This
    won't break!"],

    ]

    badconfiglines = [ # these config lines should throw a ValueError
    'blah11="broken"quotes"',
    'blah12="more"broken"quotes"',
    'blah13=broken"quotes?"',
    'strangequals=="blah"',
    'strangeequals2="wibble"="blah"',
    ]

    def testAddLines(self):
    """Test that lines can be added to the config object."""
    # First, clear out self.testconfig:
    for item in self.configtest.keys():
    self.configtest.delete(item)

    for item in self.goodconfiglines:
    self.configtest._readconfigline(item[0])
    self.assertEqual(item[2], self.configtest[item[1]])

    def testDelete(self):
    """Test that delete works."""
    # First, clear out self.testconfig:
    for item in self.configtest.keys():
    self.configtest.delete(item)

    # Now add and delete
    for item in self.goodconfiglines:
    self.configtest[item[1]]=item[2]
    self.failUnless(self.configtest[item[1]]==item[2])

    self.configtest.delete(item[1])
    self.failIf(item[2] in self.configtest.keys())

    def testConfigKeys(self):
    """Test that items added appear in config.keys()."""
    # First, clear out self.testconfig:
    for item in self.configtest.keys():
    self.configtest.delete(item)
    self.failUnless(self.configtest.keys()==[])

    # Now add and delete
    for item in self.goodconfiglines:
    self.configtest[item[1]]=item[2]
    self.failUnless(item[1] in self.configtest.keys())


    def testConfigLines(self):
    """Test that all of the types of config lines can be read
    from a file."""
    for item in self.goodconfiglines:
    #write the line to a config file
    file = open("testconfig.txt", "w")
    file.write(item[0])
    file.close()

    # Now check that it can be read from the file
    self.configtest.readconfig()
    self.assertEqual(item[2], self.configtest[item[1]])

    def testSanityCheck(self):
    """For each item in goodconfiglines, test that it can be
    written to a file
    and read back unchanged."""
    # First, clear out self.testconfig:
    for item in self.configtest.keys():
    self.configtest.delete(item)

    # Now test!
    for item in self.goodconfiglines:
    self.configtest[item[1]]=item[2]
    os.unlink("testconfig.txt")
    self.configtest.writeconfig("")
    self.configtest.readconfig()
    self.assertEqual(item[2],self.configtest[item[1]])

    def testBadConfigLines(self):
    """Test that dodgy config lines will throw an exception."""
    for item in self.badconfiglines:
    self.assertRaises(ValueError,
    self.configtest._readconfigline, theline=item)



    if __name__ == "__main__":
    unittest.main()

    --
    ----------------------------------------------------
    HyPEraCtiVE? HeY, WhO aRE YoU cALliNg HypERaCtIve?!

    ----------------------------------------------------
     
    Anthony Briggs, Oct 30, 2003
    #6
  7. Maxim Khesin

    anton muhin Guest

    Maxim Khesin wrote:
    > I want to have a config file with my python proggie, satisfying the
    > following requirements:
    > 1) support key->(value, default)
    > 2) simple and intuitive to read and edit
    > 3) easyly readable into a python datastructure (a dictionary?)
    > 4) not requiring any heavy libraries needed (I am distributing my
    > proggie as a py2exe executable and do not want to bloat the size)
    >
    > can you guys suggest some format for this? thanks,
    > max
    >

    I like YAML (www.yaml.org), although the only Python implementation I'm
    aware of is a bit buggy.

    hth,
    anton.
     
    anton muhin, Oct 30, 2003
    #7
  8. Maxim Khesin

    Nick Vargish Guest

    "John Roth" <> writes:

    > for line in inFile.readlines():
    > key, value = line.split(":")
    > configdict[key] = value


    for line in inFile.readlines():
    key, value = line.split(':', 1) # only split once, values could
    # have ':' in them
    key = key.rstrip() # remove trailing spaces from key
    value = value.lstrip() # remove leading spaces from value
    configdict[key] = value

    You could reduce the last three lines to:

    configdict[key.rstrip()] = value.lstrip()

    Nick

    --
    # sigmask || 0.2 || 20030107 || public domain || feed this to a python
    print reduce(lambda x,y:x+chr(ord(y)-1),' Ojdl!Wbshjti!=obwAcboefstobudi/psh?')
     
    Nick Vargish, Oct 30, 2003
    #8
  9. Maxim Khesin

    Maxim Khesin Guest

    > Will the standard library ConfigParser work? Point your web
    > browser here:
    >
    > http://www.python.org/doc/current/lib/module-ConfigParser.html
    >
    > Regards,
    > Heather


    Thanks for the pointer. It is not clear how this supports defaults
    (without understanding windows .ini (I cannot find official docs for
    that) of RFC 822 (I found too much documentation for this :) )
    m
     
    Maxim Khesin, Oct 30, 2003
    #9
  10. Skip Montanaro, Oct 30, 2003
    #10
  11. Maxim Khesin

    Maxim Khesin Guest

    Nick Vargish wrote:
    > "John Roth" <> writes:
    >
    >
    >>for line in inFile.readlines():
    >> key, value = line.split(":")
    >> configdict[key] = value

    >
    >
    > for line in inFile.readlines():
    > key, value = line.split(':', 1) # only split once, values could


    Just one comment: using ':' for separator is not a great idea if Windows
    paths are come of the potential values.
    m
     
    Maxim Khesin, Oct 31, 2003
    #11
  12. Maxim Khesin

    John Roth Guest

    "Maxim Khesin" <> wrote in message
    news:1qvob.46881$...
    >
    > Nick Vargish wrote:
    > > "John Roth" <> writes:
    > >
    > >
    > >>for line in inFile.readlines():
    > >> key, value = line.split(":")
    > >> configdict[key] = value

    > >
    > >
    > > for line in inFile.readlines():
    > > key, value = line.split(':', 1) # only split once, values could

    >
    > Just one comment: using ':' for separator is not a great idea if Windows
    > paths are come of the potential values.


    That's why the "1" as the second parameter. It makes sure that
    split doesn't do more than one split.

    John Roth
    > m
    >
     
    John Roth, Oct 31, 2003
    #12
  13. I actually have a whole set of programs that use a format like this,
    it's actually parsed on spaces and keywords, so you just say:

    keyword1 value1 value2 value3

    keyword2 value1

    keyword3
    value1
    value2

    and so on. It's pretty free format, and relatively easy to parse, all
    you need to set up is a list of keywords, and in return from these
    functions you get a dictionary with the value lists for each keyword.
    You can also add comments by adding '#' signs, and the rest of the
    line is a comment. Here are the functions:

    def ParseCommandFile(commfn, commkeys):
    commwords = ReadCommandFile(commfn)
    return ParseList(commwords, commkeys)

    def ReadCommandFile(commfn):
    if not (commfn and os.path.exists(commfn):
    raise OSError, "Can't find this path: %s" % fname
    commf = open(commfn,'r')
    commtext = ''
    for line in commf:
    if '#' in line:
    line = line[:string.find(line,'#')]
    commtext = commtext + ' ' + line
    commf.close()
    commwords = string.split(commtext)
    return commwords

    def ParseList(commlist, commkeys):
    commvals = {}
    for ck in commkeys:
    commvals[ck] = []
    currkey = ''
    currval = []
    foundkeys = commkeys[:]
    for cword in commlist:
    if currkey:
    if cword in foundkeys:
    commvals[currkey] = currval
    currkey = cword
    foundkeys.remove(currkey)
    currval = []
    else:
    currval.append(cword)
    else:
    if cword in foundkeys:
    currkey = cword
    foundkeys.remove(currkey)
    if currkey:
    commvals[currkey] = currval
    return commvals

    There it is, maybe not the most efficient code, but it lets you write
    a command file parsed on spaces, so there's no need to worry about
    extra syntax characters, which keeps it nice and clean.

    Maxim Khesin <> wrote in message news:<p4Xnb.17367$>...
    > I want to have a config file with my python proggie, satisfying the
    > following requirements:
    > 1) support key->(value, default)
    > 2) simple and intuitive to read and edit
    > 3) easyly readable into a python datastructure (a dictionary?)
    > 4) not requiring any heavy libraries needed (I am distributing my
    > proggie as a py2exe executable and do not want to bloat the size)
    >
    > can you guys suggest some format for this? thanks,
    > max
     
    Corey Coughlin, Oct 31, 2003
    #13
  14. Maxim Khesin

    Larry Bates Guest

    You can have defaults in configparser .INI file if you like.

    import ConfigParser
    ini_filename='program.ini'
    defaults={'key1':'value1','key2':'value2')
    #
    # This will initialize the ConfigParser with dictionary
    # of key,value defaults.
    INI=ConfigParser.ConfigParser(defaults)
    #
    # Make sure .INI file exists
    #
    if not os.path.exists:
    print "Unable to locate .INI filename=%s" % ini_filename
    sys.exit(2)
    #
    # Now read the .INI file's contents
    #
    INI.read(ini_filename)

    section='INIT'
    option='debug'
    try: debug=INI.getboolean(section, option)
    except: debug=0

    option='trace'
    try: trace=INI.getboolean(section, option)
    except: trace=0

    section='RUNTIME'
    option='first_invoice'
    try: first_invoice=INI.get(section, option)
    except:
    print ".INI file missing first_invoice key"
    sys.exit(2)

    option='last_invoice'
    try: last_invoice=INI.get(section, option)
    except:
    print ".INI file missing last_invoice key"
    sys.exit(2)

    ..
    .. remainder of program
    ..



    ..INI files have following format

    [SECTION1]
    KEY1=
    KEY2=
    KEY3=

    [SECTION2]
    KEY1=
    KEY2=
    KEY3=

    You can have as many sections as you like and keynames
    are unique under a section. Normally I do something like:

    [INIT]
    debug=1
    trace=1
    logfile='program.log'

    [RUNTIME]
    first_invoice=12345
    last_invoice=99999

    This way I can turn on/off debug and trace modes and
    have the program pick up RUNTIME parameters for processing.

    Hope information helps.
    -Larry






    "Maxim Khesin" <> wrote in message
    news:U1fob.23140$...
    > > Will the standard library ConfigParser work? Point your web
    > > browser here:
    > >
    > > http://www.python.org/doc/current/lib/module-ConfigParser.html
    > >
    > > Regards,
    > > Heather

    >
    > Thanks for the pointer. It is not clear how this supports defaults
    > (without understanding windows .ini (I cannot find official docs for
    > that) of RFC 822 (I found too much documentation for this :) )
    > m
    >
     
    Larry Bates, Oct 31, 2003
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Konstantin Shemyak

    Human-readable storage for types (with no pointers)?

    Konstantin Shemyak, Sep 9, 2003, in forum: C Programming
    Replies:
    4
    Views:
    390
    Ben Pfaff
    Sep 12, 2003
  2. Jonah Bossewitch

    human readable IPTC field names

    Jonah Bossewitch, Jan 12, 2005, in forum: Python
    Replies:
    0
    Views:
    417
    Jonah Bossewitch
    Jan 12, 2005
  3. Alex Willmer

    Human readable number formatting

    Alex Willmer, Sep 28, 2005, in forum: Python
    Replies:
    9
    Views:
    653
  4. SpamMePlease PleasePlease

    Translating pysnmp oids to human readable strings

    SpamMePlease PleasePlease, Mar 5, 2009, in forum: Python
    Replies:
    15
    Views:
    3,400
    Shantanu Joshi
    Mar 9, 2009
  5. Ulf Meinhardt
    Replies:
    0
    Views:
    514
    Ulf Meinhardt
    Sep 13, 2009
Loading...

Share This Page