search multiple dictionaries efficiently?

[Livin]

I'm a noobie so please be easy on me. I have searched a ton and did not find
anything I could understand.

I'm using py2.3

I've been using Try/Except but this gets long with multiple dictionaries.

I found some code on web pages and such but cannot get them to work. Any
help is appreciated as I need to search multiple dictionaries for keys.



here's the code I found but cannot get to work...

dict_set = (self.dictHSdevices, self.dictHSevents, self.dictHSbtnDevices)
<-- /creates set of dictionaries/

/code to search the set/-->

val = [ d for d in dict_set if d[value] in dict_set ]
OR
for key, value in dict.iteritems(): pass

 

[Paul McGuire]

I'm a noobie so please be easy on me. I have searched a ton and did not find
anything I could understand.

I'm using py2.3

I've been using Try/Except but this gets long with multiple dictionaries.

I found some code on web pages and such but cannot get them to work. Any
help is appreciated as I need to search multiple dictionaries for keys.



here's the code I found but cannot get to work...

dict_set = (self.dictHSdevices, self.dictHSevents, self.dictHSbtnDevices)
<-- /creates set of dictionaries/

/code to search the set/-->

val = [ d for d in dict_set if d[value] in dict_set ]
OR
for key, value in dict.iteritems(): pass

dict_set = [ {1:'a'}, {2:'b',3:'c'} ]
def lookup(value):

.... for d in dict_set:
.... try:
.... return d[value]
.... except KeyError:
.... pass
.... raise KeyError(value)
....Traceback (most recent call last):
File "<stdin>", line 1, in ?

 

[George Sakkis]

I'm a noobie so please be easy on me. I have searched a ton and did not find
anything I could understand.

I'm using py2.3

I've been using Try/Except but this gets long with multiple dictionaries.

I found some code on web pages and such but cannot get them to work. Any
help is appreciated as I need to search multiple dictionaries for keys.



here's the code I found but cannot get to work...

dict_set = (self.dictHSdevices, self.dictHSevents, self.dictHSbtnDevices)
<-- /creates set of dictionaries/

/code to search the set/-->

val = [ d for d in dict_set if d[value] in dict_set ]
OR
for key, value in dict.iteritems(): pass

If I understood correctly what you want to do, here's a one-liner that
does it:

def lookup(value, *dicts):
return [d.get(value) for d in dicts if value in d]
[2, 4]

It's not the *most* efficient way because value is looked up twice if
it is contained in the dictionary; if you absolutely need it to be as
efficient as possible and can't figure it out on your own, ask again
and someone will help you out.

George

 

[gene tani]

I'm a noobie so please be easy on me. I have searched a ton and did not find
anything I could understand.

I'm using py2.3

I've been using Try/Except but this gets long with multiple dictionaries.

I found some code on web pages and such but cannot get them to work. Any
help is appreciated as I need to search multiple dictionaries for keys.



here's the code I found but cannot get to work...

dict_set = (self.dictHSdevices, self.dictHSevents, self.dictHSbtnDevices)
<-- /creates set of dictionaries/

/code to search the set/-->

val = [ d for d in dict_set if d[value] in dict_set ]
OR
for key, value in dict.iteritems(): pass

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/305268
http://mail.python.org/pipermail/python-list/2005-August/294519.html

(I think these may be a little different from what you're asking above,
since they search dicts sequentially and return 1st hit)

 

[Duncan Booth]

It's not the *most* efficient way because value is looked up twice if
it is contained in the dictionary; if you absolutely need it to be as
efficient as possible and can't figure it out on your own, ask again
and someone will help you out.

How do you *know* it is not the *most* efficient way? Have you tried timing
different ways of approaching this problem and found that looking up the
value twice is slow?

I've tried timing dictionary lookups in the past, and the three obvious
solutions roughly come out as follows:

try/except is fastest when the value is in the dictionary, but it is a
*lot* slower if the exception gets thrown. If missing values are a very
rare occurrence this might be a good way to do it, but usually the code
doesn't read as well so its best to avoid. [0.26/4.11]

Test with the 'in' operator and then retrieving the value is the fastest
solution when the value isn't in the dictionary (it only does a single
lookup then), and is still fast when it is. [0.36/0.2]

Using the get method of the dictionary with a default value to be retrieved
if the key is not present is slower than using the 'in' operator in all
cases (it does beat try/except when an exception is thrown) [0.49/0.54]

The numbers above are the times produced in each case for a key present/key
missing using a simple test with timeit.py.

Part of the reason, BTW, that calling d.get(key,default) is slow is that is
also requires two dictionary lookups: one to find the get method and then
another to access the key in the dictionary, plus it has other overheads (a
method call) which test&get avoids.

These figures could of course be invalidated if the actual use is too far
from the simple string lookup I tried. For example if the key has a slow
hash function saving the second lookup would be worthwhile.

 

[Ben Sizer]

Test with the 'in' operator and then retrieving the value is the fastest
solution when the value isn't in the dictionary (it only does a single
lookup then), and is still fast when it is. [0.36/0.2]

Using the get method of the dictionary with a default value to be retrieved
if the key is not present is slower than using the 'in' operator in all
cases (it does beat try/except when an exception is thrown) [0.49/0.54]

Assuming those statistics are replicatable, it's quite unfortunate that
the obvious and concise way to do things works out more slowly than the
approach that you'd expect to take twice as long. Thankfully there
doesn't seem to be too many of these problems in Python.