dict comprehension question.

[Quint Rankid]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

 

[Roy Smith]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1

Why are you trying to do this mind-blowing thing? Other than as an
entry in an obfuscated code contest, what is this for?

Anyway, I don't think this is possible with a dict comprehension.
Entries in the dict depend on entries previously put into the dict. I
don't see any way a dict comprehension can deal with this.

 

[Mitya Sirenef]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

Will this do?:

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
{x: w.count(x) for x in w}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}


- mitya

 

[Mitya Sirenef]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

Will this do?:

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
{x: w.count(x) for x in w}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}


- mitya

I should probably add that this might be inefficient for large lists as
it repeats count for each item. If you need it for large lists, profile
against the 'for loop' version and decide if performance is good enough
for you, for small lists it's a nice and compact solution.

In a more general case, you can't refer to the list/dict/etc
comprehension as it's being constructed, that's just not a design goal
of comprehensions.

-m

 

[Joel Goldstick]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

Will this do?:

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
{x: w.count(x) for x in w}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}


- mitya

I should probably add that this might be inefficient for large lists as
it repeats count for each item. If you need it for large lists, profile
against the 'for loop' version and decide if performance is good enough
for you, for small lists it's a nice and compact solution.

In a more general case, you can't refer to the list/dict/etc
comprehension as it's being constructed, that's just not a design goal
of comprehensions.

Would this help:

w = [1,2,3,1,2,4,4,5,6,1]
s = set(w)
s set([1, 2, 3, 4, 5, 6])
{x:w.count(x) for x in s} {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

-m

 

[Peter Otten]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

The name a3 will not be rebound until after the right side is evaluate. To
spell it with a loop:

a3 = {}
_internal = {} # You have no access to this var.
# Even if you can beat a particular
# Python implementation -- you shouldn't

for x in w:
_internal[x] = a3.get(x, 0) + 1

a3 = _internal

That should make it clear that x will be looked up in the "old" empty a3
dict.

The closest you can get to a self-updating dict is probably

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
a1 = {}
a1.update((x, a1.get(x, 0)+1) for x in w)
a1

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

but that it works doesn't mean it is a good idea.
If your Python version supports it the obvious choice is

from collections import Counter
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
Counter(w)

Counter({1: 3, 2: 2, 4: 2, 3: 1, 5: 1, 6: 1})

 

[MRAB]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

You can't do it with a comprehension.

The best way is probably with the 'Counter' class:

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
from collections import Counter
Counter(w)

Counter({1: 3, 2: 2, 4: 2, 3: 1, 5: 1, 6: 1})

If you want the result be a dict, then just the result to 'dict':
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

 

[Mitya Sirenef]

On Sat, Dec 29, 2012 at 3:09 PM, Mitya Sirenef <[email protected]

Newbie question. I've googled a little and haven't found the answer.

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

I've tried a few things
eg
a1 = {x:self.get(x,0)+1 for x in w}
results in error messages.

And
a2 = {x:a2.get(x,0)+1 for x in w}
also results in error messages.

Trying to set a variable to a dict before doing the comprehension
a3 = {}
a3 = {x:a3.get(x,0)+1 for x in w}
gets this result, which isn't what I wanted.
{1: 1, 2: 1, 3: 1, 4: 1, 5: 1, 6: 1}

I'm not sure that it's possible to do this, and if not, perhaps the
most obvious question is what instance does the get method bind to?

TIA

Will this do?:

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
{x: w.count(x) for x in w}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}


- mitya

I should probably add that this might be inefficient for large
lists as
it repeats count for each item. If you need it for large lists,
profile
against the 'for loop' version and decide if performance is good
enough
for you, for small lists it's a nice and compact solution.

In a more general case, you can't refer to the list/dict/etc
comprehension as it's being constructed, that's just not a design goal
of comprehensions.


Would this help:

w = [1,2,3,1,2,4,4,5,6,1]
s = set(w)
s set([1, 2, 3, 4, 5, 6])
{x:w.count(x) for x in s} {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Indeed, this is much better -- I didn't think of it..

 

[Terry Reedy]

w = [1,2,3,1,2,4,4,5,6,1]
s = set(w)
s set([1, 2, 3, 4, 5, 6])
{x:w.count(x) for x in s}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Indeed, this is much better -- I didn't think of it..[/QUOTE]

It still turns an O(n) problem into an O(k*n) problem, where k is the
number of distinct items.

 

[Terry Reedy]

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Let me paraphrase this: "I have nice, clear, straightforward,
*comprehensible* code that I want to turn into an incomprehensible mess
with a 'comprehension." That is the ironic allure of comprehensions.

Comprehensions do not allow for interactions between the source items.
Mitya and Joel worked around this with solutions that do redundant
calculation and multiply the time order.

Reductions do allow for interactions. Doing everything as a reduction
was the fad before comprehensions came along ;-)

from functools import reduce
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
def update(dic, n):
"Mutate and return dic (contrary to usual Python policy)"
dic[n] = dic.get(n, 0) + 1
return dic
counts = reduce(update, w, {})
print(counts == {1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1})

# prints True

The above is how to rewrite your code in a functional language that does
not have statements and explicit iteration. In Python, I would only
bother to wrap the body of the loop in a function if I needed the same
body in multiple places.

Comprehensions are filtered mappings and that both filter and map can be
written as reduction, so reduction included comprehension. It is more
powerful because it can also do sequential interaction. Indeed, I would
say that it should only be used when there is sequential interaction.

 

[Tim Chase]

w = [1,2,3,1,2,4,4,5,6,1]
s = set(w)
s set([1, 2, 3, 4, 5, 6])
{x:w.count(x) for x in s}

{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Indeed, this is much better -- I didn't think of it..

Except that you're still overwhelmed by iterating over every element
in "w" for every distinct element. So you've gone from O(N**2) to
O(k*N).

The cleanest way to write it (IMHO) is MRAB's

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
from collections import Counter
results = dict(Counter(w))

which should gather all the statistics in one single pass across "w"
making it O(N), and it's Pythonically readable.

-tkc

 

[Joel Goldstick]

w = [1,2,3,1,2,4,4,5,6,1]
s = set(w)
s
set([1, 2, 3, 4, 5, 6])
{x:w.count(x) for x in s}
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Indeed, this is much better -- I didn't think of it..

Except that you're still overwhelmed by iterating over every element in
"w" for every distinct element. So you've gone from O(N**2) to O(k*N).

The cleanest way to write it (IMHO) is MRAB's

w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
from collections import Counter
results = dict(Counter(w))

which should gather all the statistics in one single pass across "w"
making it O(N), and it's Pythonically readable.

-tkc

I like this too. I haven't learned about collections module yet. Thanks

for the pointer

 

[Steven D'Aprano]

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Let me paraphrase this: "I have nice, clear, straightforward,
*comprehensible* code that I want to turn into an incomprehensible mess
with a 'comprehension." That is the ironic allure of comprehensions.

But... but... one liner! ONE LINNNNNNEEEERRRRR!!!! Won't somebody think
of the lines I'll save!!!!

*wink*


In case it's not obvious, I'm 100% agreeing with Terry here. List comps
and dict comps are wonderful things, but they can't do everything, and
very often even if they can do something they shouldn't because it makes
the code inefficient or unreadable.

There's nothing wrong with a two or three liner.

 

[88888 Dihedral]

On 12/29/2012 2:48 PM, Quint Rankid wrote:

Given a list like:
w = [1, 2, 3, 1, 2, 4, 4, 5, 6, 1]
I would like to be able to do the following as a dict comprehension.
a = {}
for x in w:
a[x] = a.get(x,0) + 1
results in a having the value:
{1: 3, 2: 2, 3: 1, 4: 2, 5: 1, 6: 1}

Let me paraphrase this: "I have nice, clear, straightforward,

*comprehensible* code that I want to turn into an incomprehensible mess

with a 'comprehension." That is the ironic allure of comprehensions.

But... but... one liner! ONE LINNNNNNEEEERRRRR!!!! Won't somebody think

of the lines I'll save!!!!



*wink*





In case it's not obvious, I'm 100% agreeing with Terry here. List comps

and dict comps are wonderful things, but they can't do everything, and

very often even if they can do something they shouldn't because it makes

the code inefficient or unreadable.



There's nothing wrong with a two or three liner.

This is useful for not being choked in sorting a list
by the notorious quick-sort.