classes vs dicts

[David MacQuigg]

Greetings,

I am pretty new to Python and like it very much, but there is one
thing I can't figure out and I couldn't really find anything in the
docs that addresses this.

Say I want to write an address book program, what is the best way to
define a person (and the like): create a class (as I would do in Java)
or use a dictionary?
I guess using dictionaries is fastest and easiest, but is this
recommended?

My problem is similar, with the additional requirement that I need a
convenient way to access data deep within a hierarchy of parameters.
I chose classes over dictionaries because the syntax to access items
in a deeply nested dictionary is awkward.

dict[window1][plot2][xaxis][label][font][size] = 12

vs

statefiles.window1.plot2.xaxis.label.font.size = 12

The problem is I can't easily save the whole hierarchy to disk.
Pickle doesn't work with classes. Also, I worry about the overhead of
classes when I am just needing a simple container. A typical
statefile will have 1000 parameters in 300 classes nested up to ten
levels deep. As a structure of nested classes this takes about 74KB
on disk. Importing the file creates a .pyc file that is 157KB !! It
does seem to import quickly, however, so speed may not be a problem.

Seems like Python could use a "container" structure which would be
like a class, but without the overhead and with the ability to
"pickle" the whole structure.

# ../CDP/Statefiles/bignest.py -- testfile with 960 parameters.

c960:
c480a:
c120a:
c30a:
c10a:
analyses:
transient:
start = '10u'
stop = '20u'
method = 'euler'
readic = 'final50u'
# . . . . . . . . . . . . . .
wavescan:
window1:
xaxis:
label:
text = "Frequency (MHz)"
position = -0.5
font:
size = 12
family = 'arial'
color = 'blue'
bold = True
# . . . . . . . . . . . . . .
c10b:
analyses:
transient:
start = '10u'
stop = '20u'
methd = 'euler'
readic = 'final50u'

-- Dave

 

[Dave Brueck]

My problem is similar, with the additional requirement that I need a
convenient way to access data deep within a hierarchy of parameters.
I chose classes over dictionaries because the syntax to access items
in a deeply nested dictionary is awkward.

dict[window1][plot2][xaxis][label][font][size] = 12

vs

statefiles.window1.plot2.xaxis.label.font.size = 12

The problem is I can't easily save the whole hierarchy to disk.
Pickle doesn't work with classes. Also, I worry about the overhead of
classes when I am just needing a simple container. A typical
statefile will have 1000 parameters in 300 classes nested up to ten
levels deep. As a structure of nested classes this takes about 74KB
on disk. Importing the file creates a .pyc file that is 157KB !! It
does seem to import quickly, however, so speed may not be a problem.

Seems like Python could use a "container" structure which would be
like a class, but without the overhead and with the ability to
"pickle" the whole structure.

Forgive me if I don't understand the problem, but why can't you just do:

class Bag:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)

Then if your data is like:

settings
network
ip = '10.20.30.40'
port = 1234
logging
level = 3

you would construct it by hand like:

settings = Bag(network=Bag(ip='10.20.30.40', port=1234), logging=Bag(level=3))

and you can access members like:

print settings.network.ip
or
settings.network.useSSL = 1
etc.

and you can easily pickle _instances_ so persistence isn't really a problem:

from cPickle import dumps, loads
q = dumps(settings)
newSettings = loads(q)

Unless I'm misunderstanding the problem, the only remaining issue is the
overhead. If that really is a problem (most likely not, but maybe if you're
transferring it over a slow network connection or something), then simply use
the built-in gzip or bz2 modules.

-Dave

 

[Terry Reedy]

My problem is similar, with the additional requirement that I need a
convenient way to access data deep within a hierarchy of parameters.
I chose classes over dictionaries because the syntax to access items
in a deeply nested dictionary is awkward.

dict[window1][plot2][xaxis][label][font][size] = 12

vs

statefiles.window1.plot2.xaxis.label.font.size = 12

The problem is I can't easily save the whole hierarchy to disk.
Pickle doesn't work with classes. Also, I worry about the overhead of
classes when I am just needing a simple container. A typical
statefile will have 1000 parameters in 300 classes nested up to ten
levels deep. As a structure of nested classes this takes about 74KB
on disk. Importing the file creates a .pyc file that is 157KB !! It
does seem to import quickly, however, so speed may not be a problem.

I believe you could write a class whose instances would correspond to
(wrap) a dictionary with possible subdicts. It would have a getattribute
method that turned the attribute name into a dict key and either return a
new instance or a 'raw' object depending on whether the corresponding value
was a nested dict or something else. You would start with one instance
representing the top level dict and work down from there. I'll leave
implementation to you.

I also believe you could instead do this as a subclass of dict itself. I
might have even seen such somewhere.

Terry J. Reedy

 

[David MacQuigg]

Excellent! Someday, I hope to be smart enough I can see such a simple
solution right away.

-- Dave MacQuigg

My problem is similar, with the additional requirement that I need a
convenient way to access data deep within a hierarchy of parameters.
I chose classes over dictionaries because the syntax to access items
in a deeply nested dictionary is awkward.

dict[window1][plot2][xaxis][label][font][size] = 12

vs

statefiles.window1.plot2.xaxis.label.font.size = 12

The problem is I can't easily save the whole hierarchy to disk.
Pickle doesn't work with classes. Also, I worry about the overhead of
classes when I am just needing a simple container. A typical
statefile will have 1000 parameters in 300 classes nested up to ten
levels deep. As a structure of nested classes this takes about 74KB
on disk. Importing the file creates a .pyc file that is 157KB !! It
does seem to import quickly, however, so speed may not be a problem.

Seems like Python could use a "container" structure which would be
like a class, but without the overhead and with the ability to
"pickle" the whole structure.

Forgive me if I don't understand the problem, but why can't you just do:

class Bag:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)

Then if your data is like:

settings
network
ip = '10.20.30.40'
port = 1234
logging
level = 3

you would construct it by hand like:

settings = Bag(network=Bag(ip='10.20.30.40', port=1234), logging=Bag(level=3))

and you can access members like:

print settings.network.ip
or
settings.network.useSSL = 1
etc.

and you can easily pickle _instances_ so persistence isn't really a problem:

from cPickle import dumps, loads
q = dumps(settings)
newSettings = loads(q)

Unless I'm misunderstanding the problem, the only remaining issue is the
overhead. If that really is a problem (most likely not, but maybe if you're
transferring it over a slow network connection or something), then simply use
the built-in gzip or bz2 modules.

-Dave

 

[David MacQuigg]

My problem is similar, with the additional requirement that I need a
convenient way to access data deep within a hierarchy of parameters.
I chose classes over dictionaries because the syntax to access items
in a deeply nested dictionary is awkward.

dict[window1][plot2][xaxis][label][font][size] = 12

vs

statefiles.window1.plot2.xaxis.label.font.size = 12

The problem is I can't easily save the whole hierarchy to disk.
Pickle doesn't work with classes. Also, I worry about the overhead of
classes when I am just needing a simple container. A typical
statefile will have 1000 parameters in 300 classes nested up to ten
levels deep. As a structure of nested classes this takes about 74KB
on disk. Importing the file creates a .pyc file that is 157KB !! It
does seem to import quickly, however, so speed may not be a problem.

I believe you could write a class whose instances would correspond to
(wrap) a dictionary with possible subdicts. It would have a getattribute
method that turned the attribute name into a dict key and either return a
new instance or a 'raw' object depending on whether the corresponding value
was a nested dict or something else. You would start with one instance
representing the top level dict and work down from there. I'll leave
implementation to you.

I also believe you could instead do this as a subclass of dict itself. I
might have even seen such somewhere.

I wrote a module, based on Dave Brueck's example, adding load and save
methods to his basic Bag class. The class definition is copied below.
The complete module, including unit tests is at
http://ece.arizona.edu/~edatools/Python/Statefiles/bag01.py

The work remaining is to make the output "statefile" an even neater,
more compact format. This will allow a user to scan through thousands
of parameters to find the ones he needs, then cut-and-paste a section
of the parameter file into a script, and have the copied parameters
load into his program.

-- Dave

# Organize statefiles as nested "bags" of parameters.
# Load and save in a neat format for external editing.

TAB = 3; SP = ' '*(TAB-1); LINE = 0; LMAX = 15
TABMARKS = (('#'+SP)+('.'+SP)*4)*3 + '\n'

class Bag:
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
self.Indent = ''

def load(infile):
strng = infile.read()
exec( 'bag = Bag(\n' + strng + ')' )
return bag
load = staticmethod(load)

def save(self, outfile):
global LINE
for k,v in self.__dict__.items():
if k == 'Indent': continue
Idt = self.Indent
if isinstance( v, Bag ):
if LINE > LMAX:
outfile.write( TABMARKS ); LINE = 0
outfile.write( Idt + k + '=Bag(\n' )
LINE += 1
v.Indent = Idt + ' '*TAB
v.save(outfile)
outfile.write( Idt + '),\n' )
else:
if type(v) is not str:
v = str(v)
else:
v = "'%s'" % v
outfile.write( Idt + k + ' = ' + v + ',\n' )
LINE += 1