acting on items passed to a method via a dictiomary

D

Donnal Walter

The following method is defined in one of my classes:

def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items['something']
# now do something with value
if 'another' in items:
value = items['another']
# do something else with value
if 'spam' in items:
value = items['spam']
# do yet another thing with value
if 'eggs' in items:
value = items['eggs']
# and so on

The purpose is to set up the object based on values contained in a
dictionary passed to the setup() method. The dictionary may be empty, or
it may contain a variable number of items, and it may contain items not
looked for by the setup() method. Moreover, this method may be
overridden in subclasses. The approach described above works ok, so I
suppose I should leave well enough alone, but I have this nagging
feeling that Python has a way to do this more simply or more elegantly.
Any suggestion?

Donnal Walter
Arkansas Children's Hospital
 
D

Diez B. Roggisch

Donnal said:
The following method is defined in one of my classes:

def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items['something']
# now do something with value
if 'another' in items:
value = items['another']
# do something else with value
if 'spam' in items:
value = items['spam']
# do yet another thing with value
if 'eggs' in items:
value = items['eggs']
# and so on

Any suggestion?

First of all, don't use {} as default value for items. Many have stepped
into this trap: While perfectly legal, it will be evaluated only once, when
the method setup is found the first time. So then an instance of a dict is
created. But now if subsequent calls to setup alter that dict items points
to, they all share the same dict!! This small example illustrates that
behaviour:

def foo(key, bar={}):
bar[key] = key
print bar

for i in xrange(3):
foo(i)

yields:

{1:1}
{1:1, 2:2}

where you surely have expected {2:2} in the second case. So the idiom for
this commonly is this:

def foo(key, bar=None):
if bar is None:
bar = {} # remember: bar is a local name, bound to a value - so next
time, bar will still be none if foo is called w/o bar as parameter
bar[key] = key

Now to your question: This works:

for k in ['something', 'another', 'spam', 'eggs']:
if items.has_key(k):
value = items[k]
break
 
V

Ville Vainio

Donnal> if 'something' in items:
Donnal> value = items['something']
Donnal> # now do something with value
Donnal> if 'another' in items:
Donnal> value = items['another']
Donnal> # do something else with value

Donnal> overridden in subclasses. The approach described above
Donnal> works ok, so I suppose I should leave well enough alone,
Donnal> but I have this nagging feeling that Python has a way to
Donnal> do this more simply or more elegantly. Any suggestion?

The code is quite ok - you might avoid the repetition of 'something'
by using

a.get(k[, x])

# a[k] if k in a, else x

If you feel 'clever' and don't care about performance (no, I didn't
benchmark this):

-----
def dictget(d, key):
if key in d:
yield d[key]
return

d = { 'spam' : 1, 'eggs' : 67 }

for value in dictget(d,'spam'):
print value

for value in dictget(d,'larch'):
print value + 999 # never printed


for value in dictget(d,'eggs'):
print value + 2
 
J

Jeremy Bowers

The following method is defined in one of my classes:

def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items['something']
# now do something with value
....

It mostly depends on the contents of the "do something with value" that
you have peppered throughout. I would pull out the various "do somethings"
into methods, but other then that, if you need the full flexibility of a
dict that can contain anything and you need to do things with arbitrary
combinations, there is little else to do.

After pulling things out into methods, I'd actually shift the "if"
statement into the method and do something like the following:

def setup(self, items = None):
if items is None:
items = {}

self.items = items

for method in ('something', 'another', 'spam', 'eggs'):
getattr(self, method)()

which will make it more flexible in the long term.

A few other things leap to mind (might want to separate the self.item
setting from the loop, so children can override the loop without messing
up items in calls to the super class; if you don't know what I mean
because I'm being somewhat vague here, don't worry :) ), but that's about
all you can do without getting silly.
 
V

Ville Vainio

Ville> def dictget(d, key):
Ville> if key in d:
Ville> yield d[key]
Ville> return

After putting down the crack pipe, I realized you can also do

def dictget(d,key):
if key in d:
return [d[key]]
return []

which is slightly less clever/cute. Too lazy to compare performance
right now.
 
M

Michael Loritsch

Jeremy Bowers said:
The following method is defined in one of my classes:

def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items['something']
# now do something with value
...

It mostly depends on the contents of the "do something with value" that
you have peppered throughout. I would pull out the various "do somethings"
into methods, but other then that, if you need the full flexibility of a
dict that can contain anything and you need to do things with arbitrary
combinations, there is little else to do.

After pulling things out into methods, I'd actually shift the "if"
statement into the method and do something like the following:

def setup(self, items = None):
if items is None:
items = {}

self.items = items

for method in ('something', 'another', 'spam', 'eggs'):
getattr(self, method)()

which will make it more flexible in the long term.

A few other things leap to mind (might want to separate the self.item
setting from the loop, so children can override the loop without messing
up items in calls to the super class; if you don't know what I mean
because I'm being somewhat vague here, don't worry :) ), but that's about
all you can do without getting silly.

I agree with Jeremy here that everything depends on what "do something
with value" means.

After looking over this thread though, it appears to me that having
the 'setup()' method perform a dispatch operation using a dictionary
defined at class scope would be an easy, and extensible choice.

This way, one would not have to override the 'setup' method in derived
classes, rather one would only have to replace the class level
dispatch map, which in the examples below is named 'setupMap'.

Consider the following code, which would take a list or tuple to the
setup function.

class ListSetup:

def setup(self, items = None):
if items is None:
items = ()
#Note, here a list is being used instead of a dictionary
for item in items:
if self.setupMap.has_key(item): #Ignore extra items
self.setupMap[item](self)

def setupfunction1(self):
print "Called ListSetup.setupfunction1"

def setupfunction2(self):
print "Called ListSetup.setupfunction2"

#This is the key to the whole thing -> a dictionary used for
#dispatching the correct calls
setupMap = {'something':setupfunction1, 'another':setupfunction2}


The key is in the class scope dictionary 'setupMap'. If a key is
provided in the setup call, each of the appropropiate member functions
will be called.

On the other hand, if the 'values' stored in the dictionary passed in
are also important, we can incorporate that into our solution, by
passing them to our setupfunctionX methods as arguments.

Here is a similar solution, using an items dictionary:

class DictSetup:

def setup(self, items = None):
if items is None:
items = {}
#Note, here we are expecting a dictionary
for key in items.keys():
if self.setupMap.has_key(key): #Ignore extra items
self.setupMap[key](self, items[key])

def setupfunction1(self, value):
print "Called DictSetup.setupfunction1 with value", value

def setupfunction2(self, value):
print "Called DictSetup.setupfunction2 with value", value

#Again, this is the key to the whole thing -> a dictionary used
#for dispatching the correct calls
setupMap = {'something':setupfunction1, 'another':setupfunction2}

I hope this helps!

Michael Loritsch
 
A

Alex Martelli

Diez B. Roggisch said:
...
First of all, don't use {} as default value for items. Many have stepped
into this trap: While perfectly legal, it will be evaluated only once, when
the method setup is found the first time. So then an instance of a dict is
created. But now if subsequent calls to setup alter that dict items points
to, they all share the same dict!! This small example illustrates that

However, there are no problems whatsoever with the issue you remark on,
as long as the method never alters the 'items' object. As long as only
nonmutating methods get called on 'items', i.e., 'items' is practically
treated as "read-only", Donnal Walter's approach is just fine. The
issue you remark on does deserve to be made, but it's also important to
understand when it does matter and when it doesn't.


Alex
 
A

Alex Martelli

Donnal Walter said:
The following method is defined in one of my classes:

def setup(self, items={}):
"""perform setup based on a dictionary of items"""
if 'something' in items:
value = items['something']
# now do something with value
if 'another' in items:
value = items['another']
# do something else with value
if 'spam' in items:
value = items['spam']
# do yet another thing with value
if 'eggs' in items:
value = items['eggs']
# and so on

The various 'do something', 'do something else', etc, pieces of code,
need to be factored out into methods or functions reachable on the basis
of the related names, 'something', 'another', etc. Depending on exacly
what the various 'somethings' are that you want to do, you might be
happier with placing them in other methods of this class with names
constructed in a systematic way from the related names (this makes it
feasible for subclasses to override some part of them) or local
functions in a dictionary (if you're keen to stop subclasses from
overriding only some part of this processing).

Normally one does want to allow subclasses more flexibility rather than
less (extended variants of the "Template Method" design pattern are very
abundant in the Python standard library, exactly because of this), so,
you might want to do something like...:

class Base(object):

crucial_names = 'something another spam eggs and so on'.split()

def method_name(self, crucial_name): return 'handle_'+crucial_name

def handle_something(self, value): print 'something is', value
def handle_eggs(self, value): print 'over easy for', value
# other handle_... deliberately omitted, default to no-op

def setup(self, items={}):
for crucial_name in self.crucial_names:
if crucial_name in items:
metname = self.method_name(crucial_name)
method = getattr(self, metname, None)
if method: method(items[crucial_name])

This lets a subclass alter the list of "crucial names" (via the highly
Pythonic concept of "data override"), as well as the processing to be
performed for each "crucial name", and even the way a method name is
constructed from the "crucial name". Of course there are many design
decisions here that you may want to tweak or reverse, depending on your
exact need -- e.g., it may be that in your case a missing method should
not be a plain no-op but a serious error warranting an exception.

Do notice a little detail: it's typically important that the method_name
method 'augment' the crucial_name argument in such a way that the
resulting string cannot possibly be a Python keyword. For example, if
you had 'def method_name(self, crucial_name): return crucial_name' the
handling of crucial name 'and' might be bothersome, since it's a Python
keyword; if you used, e.g., "return 'a'+crucial_name", a subclass would
have similar problems if it wanted to have 'nd' as a crucial name
(unless it overrode method_name and reimplemented all the existing
handling methods, probably not worth the bother)... it would be feasible
to work around such problems, but it's better to design method_name so
as to avoid the problems arising in the first place!-)


Alex
 
D

Diez B. Roggisch

However, there are no problems whatsoever with the issue you remark on,
as long as the method never alters the 'items' object. As long as only
nonmutating methods get called on 'items', i.e., 'items' is practically
treated as "read-only", Donnal Walter's approach is just fine. The
issue you remark on does deserve to be made, but it's also important to
understand when it does matter and when it doesn't.

You are right of course, but the high frequency of postings regarding
"strange default value behaviour" made me want to make this point before
Donnal steps into that pitfall. And while the example at hand didn't alter
the contents of items, I'd nevertheless settled for items=None and used
something like this:

def foo(items=None):
if not items is None:
....
 
B

Bengt Richter

You are right of course, but the high frequency of postings regarding
"strange default value behaviour" made me want to make this point before
Donnal steps into that pitfall. And while the example at hand didn't alter
the contents of items, I'd nevertheless settled for items=None and used
something like this:

def foo(items=None):
if not items is None:
....

I like to use the idiom

def foo(items=None):
if items is None: items = {} # or whatever non-shared mutable
...

Regards,
Bengt Richter
 
D

Donnal Walter

Michael said:
>
> I agree with Jeremy here that everything depends on what "do
> something with value" means.

Thanks again for the additional helpful replies. I am still learning how
to ask (good) questions, and in my original post I was guilty of
abstracting away too many important details.

First of all, I am/was aware of the potential problem associated with
using the default "items={}" in the signature of the setup() method. In
fact, my own code used:

def setup(self, items):

where items is generated by **kwds somewhere else. Without thinking,
though, I wrote "items = {}" in my posting merely to indicate that items
might be an empty dictionary in some cases, which several rightly
pointed out could lead to "strange default value behavior".

More importantly, I left out a couple of crucial features of the setup()
method itself. I did so deliberately because I thought it would add
confusion to what I hoped was a simple question, but now I see that this
is probably critical information. The first thing I left out is that
setup() is what I call a 'univeral' method. When it is called from the
__init__() method for the instance, setup() is bound to the instance,
but when it is called from the metaclass __init__() method, it is bound
to the class itself. This way, the specification values in *items* can
be passed as class attributes (via dict in the metaclass __init__) or as
keywords collected by the instance __init__(). Moreover, I "decorate"
this method/function as being "universal" from within the metaclass
__init__() rather than in the body of the class statement. At first I
thought this was pretty cool, but the only way I could figure out to
call a super setup() from a subclass (since this is a universal method)
is to supply a munged name "_klass.__setup__" for subclasses to call.
This is done in the metaclass as well. It is a hack, but it works.

With regard to factoring out the various 'do something' pieces of
code, I agree that this appears to be the most appropriate way to define
the setup process, and for example it would eliminate the need for
munging the name for subclasses. Indeed I started out this way. But here
is why I ended up putting all the code in one method. First, some of the
items interact (such as logical ORing several different flags). This
could be done in separate methods by using an appropriate instance
attribute, but it is easier to use a local variable to hold the
intermediate result. Second, the order in which some of the setup
functions are performed is significant. Dispatching from a list would
preserve an order, of course, but in this case, it reads better to
define the various functionalities one after another in the body of a
single method. And finally, having separate setup methods for each item
in the specification means that the metaclass would need to decorate
each one of these methods as being "universal". Better here than in the
body of the subclasses (IMHO), but this still adds more complexity than
I had hoped for.

From the standpoint of the API itself, it would be nice if each
subclass could simply define a set of methods "setup_xxxx" where "xxxx"
is the name of a specification to look for in "items". If I can figure
out how to avoid the need for these specs to be handled in a particular
order, it would make this approach much easier. If I end up keeping all
the code in a single method (for each subclass), I will likely adopt
Ville's suggestion (defining dictget(d, key)) to avoid repeating each
key name.

Thanks,
Donnal Walter
Arkansas Children's Hospital






#-----------------------------------------------------------------------------
# Revised: 2004-10-18
# Copyright: (c) 2003-2004 Donnal C. Walter (http://mindwrapper.org)
# License: BSD License (http://www.mindwrapper.org/license)
#-----------------------------------------------------------------------------

class universal(object):
"""universal method: binds to either a class or an instance

Thanks to Thomas Heller on comp.lang.python
http://tinyurl.com/51qd or http://tinyurl.com/51qm
"""
def __init__(self, func):
self.__func = func

def __get__(self, inst, type=None):
if inst is None:
# bind to the class
return self.__BoundMethod(self.__func, type)
else:
# bind to the instance
return self.__BoundMethod(self.__func, inst)

class __BoundMethod:
# Helper class.
def __init__(self, func, first):
self.__func = func
self.__first = first

def __call__(self, *args, **kwds):
return self.__func(self.__first, *args, **kwds)


class Item:

class __metaclass__(type):
def __init__(cls, name, bases, dict):
type.__init__(cls, name, bases, dict)
n = 'setup' # declarations handler
if n in dict: # if class def has this name
fnc = dict.pop(n) # retrieve the function
ufnc = universal(fnc) # make into universal method
setattr(cls, n, ufnc) # for class or instance
n = '_' + cls.__name__ + '__' + n # and local name
setattr(cls, n, ufnc) # for subclasses to call
cls.setup(dict) # then execute for this class

_args = []

def __init__(self, parent=None, name='', *refs, **kwds):
self._parent = parent # node is in a parent
self._name = name # node is named
if refs:
self._refs = refs # cache refs if present
# overrides class refs
# setup can override
self.setup(kwds) # handle specifications

def setup(self, kwds):
if 'refs' in kwds:
self._refs = aslist(kwds.pop('refs'))
return kwds
 
D

David M. Cooke

Diez B. Roggisch said:
You are right of course, but the high frequency of postings regarding
"strange default value behaviour" made me want to make this point before
Donnal steps into that pitfall. And while the example at hand didn't alter
the contents of items, I'd nevertheless settled for items=None and used
something like this:

def foo(items=None):
if not items is None:

Or the more readable (IMO):

def foo(items=None):
if items is not None:
...

They're equivalent, but a reader could see 'not items is None' as
'(not items) is None)' [although it's not].
 
D

Donnal Walter

David said:
Diez B. Roggisch said:
def foo(items=None):
if not items is None:


Or the more readable (IMO):

def foo(items=None):
if items is not None:
...

They're equivalent, but a reader could see 'not items is None' as
'(not items) is None)' [although it's not].

As I mentioned in a previous post this morning, the default items = {}
is now a somewhat irrelevant side-issue, but your comment here will help
me in a dozen other locations. For some reason it had never occurred to
me that "items is not None" would be equivalent to "not items is none"
because I made a wrong assumption about the nature of the *is* operator.
But a quick shell session shows that you are quite right.

Thank you.

Donnal Walter
Arkansas Children's Hospital
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,754
Messages
2,569,527
Members
44,998
Latest member
MarissaEub

Latest Threads

Top