Comparing Dictionaries

[Kenneth Love]

Hello,

I am new to Python, but not programming. I would like to start my
Python career by developing programs according to the "best practices"
of the industry. Right now, that appears to be unit tests, patterns,
and source code control.

So, I am trying to write a unit test for some code that reads a Windows
style INI file and stores the section, key, and values in a dictionary
(recipe originally found on ASPN, but heavily modified for my needs).

------------------------------
class LoadIni(unittest.TestCase):
knownDefault = {
"database$dbms": "mysql",
"database$name": "thelarch",
"database$user": "nobody",
"database$password": "special",
"database$host": "127.0.0.1"
}

def testNoIniFilename(self):
"""A default configuration list should be returned."""
result = ini.load()
# These next two print statement show the same key/value pairs,
# but in a different order.
#print knownDefault
#print result
self.assertEqual(result, self.knownDefault) # does not work
-------------------------------

In this case, I need to compare the returned dictionary (result) with
the known default dictionary (knownDefault). This needs to be a
comparison of the dictionary key/value pairs. If all of the key/value
pairs are an exact match and no extra key/value pairs are in either
dictionary then I consider the two dictionaries to be equivalent.

In other words, I consider these two dictionaries to be equivalent:

{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'mickey' }
{ 'mouse' : 'mickey', 'dog' : 'bone', 'cat' : 'fever' }

while these two are not:

{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'mickey' }
{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'michael' }

Any suggestions on how to compare these via some assert statement?

Am I even on the right track with the INI file storing results in a
dictionary? When I found the code, it just "felt right".

NOTE: The code works, I can load the INI file, programmatically change
a few settings, save the results to a new INI file, and manually compare
the resulting files. I get the expected results. It is the automation
of this testing process that appears to be the stumbling block.

adTHANKSvance,
Kenneth Love


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Kenneth Love | Oklahoma Tax Commission
DP Programmer/Analyst | Information Technology
(405) 522 - 5864 | http://www.tax.ok.gov/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

[Ben Finney]

In other words, I consider these two dictionaries to be equivalent:

{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'mickey' }
{ 'mouse' : 'mickey', 'dog' : 'bone', 'cat' : 'fever' }

while these two are not:

{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'mickey' }
{ 'dog' : 'bone', 'cat' : 'fever', 'mouse' : 'michael' }

Any suggestions on how to compare these via some assert statement?

Dictionaries already know how to compare themselves to each other.
False

 

[Ali]

Hello,

I am new to Python, but not programming. I would like to start my
Python career by developing programs according to the "best practices"
of the industry. Right now, that appears to be unit tests, patterns,
and source code control.

I am not sure about "best", but one of my "favourite practices" is to
not re-write code that other people have written, and have released
under licenses that allow you to do anything with it.

So, I am trying to write a unit test for some code that reads a Windows
style INI file and stores the section, key, and values in a dictionary

So, in the standard library: http://docs.python.org/lib/module-ConfigParser.html
And if you want a more involved approach: http://www.voidspace.org.uk/python/configobj.html

Ali

 

[Kenneth Love]

I am not sure about "best", but one of my "favourite practices" is to
not re-write code that other people have written, and have released
under licenses that allow you to do anything with it.


So, in the standard library:
http://docs.python.org/lib/module-ConfigParser.html
And if you want a more involved approach:
http://www.voidspace.org.uk/python/configobj.html

Ali

The published recipe (based on ConfigParser) did not handle my INI
files. I have periods in both the section names and the key names.
The INI files contents were developed according to an internally
defined process that other non-Python programs rely on. The published
recipe *did not work* with this scenario.

Believe me, not modifying 3rd-party code is something I prefer and
already practice. The code has to work first.

In case anyone is curious, here is a link to the original:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/65334

I combined the load/save routines, slogged through the unfamiliar
syntax, changed the section/key separator to '$', and started
writing unit tests.

Sincerely,
Kenneth Love


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Kenneth Love | Oklahoma Tax Commission
DP Programmer/Analyst | Information Technology
(405) 522 - 5864 | http://www.tax.ok.gov/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

[Kenneth Love]

Dictionaries already know how to compare themselves to each other.

'mickey', 'dog': 'bone', 'cat': 'fever'}
True
'michael', 'dog': 'bone', 'cat': 'fever'}
False

After verifying this myself, I double checked my code. It turns out,
the changes I made to get rid of the "import string" from the original
recipe were the problem. Leaving the parenthesis off lower() and
strip() in the following code causes all sorts of problems:

config["%s$%s" % (name, key.lower())] = cp.get(sec, key).strip()

That should teach me not to change working code at the same time I am
writing unit tests. Even so, I realize it won't be the last time I
do something so silly. Yes, I know about TDD's "write the test first",
but I'm not comfortable with the philosophy of these new fangled unit
tests yet.

Thanks for the help!

Sincerely,
Kenneth Love


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Kenneth Love | Oklahoma Tax Commission
DP Programmer/Analyst | Information Technology
(405) 522 - 5864 | http://www.tax.ok.gov/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

[Steven D'Aprano]

The published recipe (based on ConfigParser) did not handle my INI
files. I have periods in both the section names and the key names.
The INI files contents were developed according to an internally
defined process that other non-Python programs rely on. The published
recipe *did not work* with this scenario.

I think you have made a mistake. ConfigParser as published certainly DOES
accept periods in section and option names. It just *works*.

Here's my INI file:

[SECTION.FRED]
option.wilma = 45



And here's how I read it:

import ConfigParser
D = ConfigParser.ConfigParser()
D.read('Desktop/data.ini') ['Desktop/data.ini']
D.sections() ['SECTION.FRED']
D.options('SECTION.FRED') ['option.wilma']
D.getint('SECTION.FRED', 'option.wilma')

45


Even if the existing ConfigParser doesn't do what you want, the right way
to fix the problem is not to recreate the entire module from scratch, but
to subclass it:


import ConfigParser

class MyConfigParser(ConfigParser.ConfigParser):
# Warning: untested. I haven't even looked at the source code
# of the original.
def getint(self, section, option):
value = super(MyConfigParser, self).getint(section, option)
return value + 1


(Although, it has to be said... if somebody can subclass ConfigParser so
that writing a modified file doesn't remove comments, I'll be mightily
impressed.)

 

[Paddy]

Hi Kenneth, being new to Python i wondered if you at least considered
Doctests as part of your testing solution.
Other languages don't have Doctest.

- Paddy.

 

[Martin P. Hellwig]

Kenneth Love wrote:

That should teach me not to change working code at the same time I am
writing unit tests. Even so, I realize it won't be the last time I
do something so silly. Yes, I know about TDD's "write the test first",
but I'm not comfortable with the philosophy of these new fangled unit
tests yet.

<cut>
I've been using python for about 3 years now, it was the first
programming language I learned and I haven't done any other since (well
that is if you dismiss sql and shell scripting on various platforms).

Though in my daily job I have some coding tasks, I do not see my self as
a programmer since in my perspective a programmer is someone who can
write good code regardless of the language. My code isn't any good, it
is barely functional (so I really needed unit-test way of coding, for my
own protection and not even because it gives more maintenance security
for the project).

But the funny thing that I have seen in the development scene is that
writing tests first and code later is a lot easier when you have a
technical specification to base it on. A technical specification is of
course based on a functional design. A functional design is written on
the base of the assignment and the scope definition.

The scope and design can change in the course of the project but good
practice is usually to do changes in another release. And that is what
software change management and release management is for.

Still there are tons of reasons why you shouldn't follow blindly the
best practice, since best practice is founded on theory molded around
the most occurring reality. And the only thing that I am absolutely sure
of is that reality differs enough from person to person and situation to
situation, that forcing the same practice to every person/situation is a
guarantee for wasting time and energy.

IMHO there are two extreme basic types of programmers, on the left are
the ones that have a problem them self, solve it while happily hacking
along the way, leaving the design part for a later stage, on the right
are people that know beforehand on an almost bit level what a program is
supposed to do, already have a design and aren't really that interested
in the problem that their code should solve in the first place.

Me I am a bit schizophrenic, I move in between of them, on a hourly
base. So for me the reality is that I have a certain 'problem' I write
some code with the built-in IDLE mostly following functional coding
practice (since I still have to figure out how some stuff works in
python).

Then when I have a an idea what I am doing, I create a new project tree
in subversion and write my technical specification (still having IDLE
open to test some of the wild ideas that pop-up along the way).

After that I open Eclipse/PyDEV and start working on the unit-test and
follow from there, when the code is finished I check it in and branch it
as 0.0.1. Sleep over it and probably refactor it later on because I
didn't like some variables names and want to add some comments in the
code or move out some code to other classes, etc. etc.

The only problem is that I wish I was a bit more stable, because several
times it has happend that I thought that the problem is a one-liner, so
skipped the whole part after the figure-out-the-problem-code and used
that what I have already written. Sometimes that works out well, but if
it doesn't it always bites me hard.

But all these are just experiences from my side, I have no idea whether
I am right or not, but for me it seems to work, so don't take it as
advice (I am not really the person to take advice from) but as a little
'story' along the way

 

[Ben Finney]

But the funny thing that I have seen in the development scene is
that writing tests first and code later is a lot easier when you
have a technical specification to base it on. A technical
specification is of course based on a functional design. A
functional design is written on the base of the assignment and the
scope definition.

This is, to my mind, one of the greatest advantages of test-driven
development: if you're not clear on how the code will be used, you
can't write the test, and (by the discipline of TDD) you can't write
the code either.

Therefore, you're forced to confront the fuzziness of your design
*before* it causes you to write meaningless code – but only to the
extent necessary to write the code at hand. If understanding the code
you're about to write requires some more extensive design thinking,
that's all to the good; but if you have enough understanding to write
a test for the small area you're on, you don't need to do a huge
amount of design. The stark confrontation of needing to write the unit
test up front shows you the difference, at exactly the time it's most
useful.

There are many who call TDD "test-driven design" for this same reason.

 

[Kenneth Love]

Hi Kenneth, being new to Python i wondered if you at least considered
Doctests as part of your testing solution.
Other languages don't have Doctest.

- Paddy.

Until I read your post, I had never even heard of Doctest. I will look
into it.

Here is the list of online sources I've found:

http://www.ibiblio.org/obp/thinkCSpy/index.html
http://www.diveintopython.org/toc/index.html
http://aspn.activestate.com/ASPN/Cookbook/Python


Here are the O'Reilly books I purchased from Barnes & Noble:

Python in a Nutshell (2nd ed.)
Python Cookbook (2nd ed.)
Programming Python (3rd ed.)

I am a slow reader. So, if Doctests are mentioned in any of the above,
I haven't encountered it yet.

Specifically, my information on unit testing comes from reading about
half of this online chapter:

http://www.diveintopython.org/unit_testing/index.html

I will search on Google for more info on Doctest.

Thanks!

Sincerely,
Kenneth Love


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Kenneth Love | Oklahoma Tax Commission
DP Programmer/Analyst | Information Technology
(405) 522 - 5864 | http://www.tax.ok.gov/
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

[Paddy]

Until I read your post, I had never even heard of Doctest. I will look
into it.

Here is the list of online sources I've found:

http://www.ibiblio.org/obp/thinkCSp...p://aspn.activestate.com/ASPN/Cookbook/Python

Here are the O'Reilly books I purchased from Barnes & Noble:

Python in a Nutshell (2nd ed.)
Python Cookbook (2nd ed.)
Programming Python (3rd ed.)

I am a slow reader. So, if Doctests are mentioned in any of the above,
I haven't encountered it yet.

Specifically, my information on unit testing comes from reading about
half of this online chapter:

http://www.diveintopython.org/unit_testing/index.html

I will search on Google for more info on Doctest.

Thanks!

Sincerely,
Kenneth Love

Add to your list:
http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-2-doctest.html
http://en.wikipedia.org/wiki/Doctest

Have fun.

- Paddy.

 

[Paul Rubin]

I will search on Google for more info on Doctest.

Doctest is recent. Try:

http://python.org/doc/lib/module-doctest.html

Diveintopython should probably be updated to use doctest instead
of unittest. unittest is Java-descended and doesn't fit into Python
as well as doctest does.

 

[Alex Martelli]

Python in a Nutshell (2nd ed.) ...
I am a slow reader. So, if Doctests are mentioned in any of the above,
I haven't encountered it yet.

Yep, I cover doctest in the chapter on testing, debugging, profiling and
optimizing.


Alex