Reply to thread

Message: [QUOTE="John Machin, post: 3610716"] Let's assume this need is real for the moment. Put this loop in your code after the creation of foo and bar and before you "combine" them: for key in bar: assert key in foo Does it cause an exception? If so, either: you have a bug in the creation of foo or bar (or both!), or: the certainty you had in making your opening statement "I am certain that all keys in bar belong to foo as well" was not well- founded. If however it is correct that all keys in bar are also to be found in foo, then the following snippets of code are equivalent for your purpose of adding bar frequencies into foo: (1) iterating over foo: for key in foo: foo[key] += bar.get(key, 0) (2) iterating over bar: for key in bar: foo[key] += bar[key] I (again) challenge you to say *why* you feel that the "iterating over bar" solution will not work. Let's start with "So I made foo dictionary that creates all POTENTIAL training bigrams with a smoothed frequency of 1". Let me guess that you have a set W of all words ever used/usable in the language of the texts that you are considering ... let N = len(W). So the number of potential bigrams is N**2. Hmmm, how large is N, and have you actually run the foo-building code yet? Now, assuming foo does fit in memory etc, you get to the stage where you have a test message containing a bigram b = (word1, word2). Its smoothed frequency will be foo[b]. If b is in bar, this should be equal to bar[b] + 1. Otherwise it will be 1. So: (1) foo[b] == bar.get(b, 0) + 1 (2) foo is redundant. If you want to check that b is "legal", use (word1 in W and word2 in W). Please attempt to refute the specific points above, rather than writing another essay :-) Cheers, John[/b][/b][/b] [/QUOTE]

Verification