Filtering a Python list to uniques

[kellygreer1]

What is the best way to filter a Python list to its unique members?
I tried some method using Set but got some "unhashable" error.

lstone = [ 1, 2, 3, 3, 4, 5, 5, 6 ]
# how do i reduce this to
lsttwo = [ 1, 2, 3, 4, 5, 6 ]

Is there a page on this in the Python in a Nutshell or the Python
Cookbook?
Did I miss something?

Kelly Greer
(e-mail address removed)
change nospam to yahoo

 

[bearophileHUGS]

Kelly Greer:

What is the best way to filter a Python list to its unique members?

If Python is "batteries included", then an industrial-strength
unique() seems one of the most requested 'batteries' that's not
included I feel that it's coming in Python 2.6/3.x. In the
meantime:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502263
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/438599

Bye,
bearophile

 

[Mensanator]

What is the best way to filter a Python list to its unique members?
I tried some method using Set but got some "unhashable" error.

lstone = [ 1, 2, 3, 3, 4, 5, 5, 6 ]
# how do i reduce this to
lsttwo = [ 1, 2, 3, 4, 5, 6 ]

Is there a page on this in the Python in a Nutshell or the Python
Cookbook?
Did I miss something?

I don't know, the set() soution worked for me.

lstone = [1,2,3,3,4,5,5,6]
setone = set(lstone)
lsttwo = list(setone)
lstone [1, 2, 3, 3, 4, 5, 5, 6]
setone set([1, 2, 3, 4, 5, 6])
lsttwo

[1, 2, 3, 4, 5, 6]

 

[Raymond Hettinger]

What is the best way to filter a Python list to its unique members?
I tried some method using Set but got some "unhashable" error.

lstone = [ 1, 2, 3, 3, 4, 5, 5, 6 ]
# how do i reduce this to
lsttwo = [ 1, 2, 3, 4, 5, 6 ]

If the elements are hashable, try this:
lsttwo = sorted(set(lstone))
If not hashable, try:
lsttwo = [k for k,v in itertools.groupby(sorted(lstone))]


Raymond

 

[hellt]

What is the best way to filter a Python list to its unique members?
I tried some method using Set but got some "unhashable" error.

lstone = [ 1, 2, 3, 3, 4, 5, 5, 6 ]
# how do i reduce this to
lsttwo = [ 1, 2, 3, 4, 5, 6 ]

Is there a page on this in the Python in a Nutshell or the Python
Cookbook?
Did I miss something?

Kelly Greer
(e-mail address removed)
change nospam to yahoo

or just look this thread for a fastest solution
http://groups.google.com/group/comp...d91ccea?lnk=gst&q=duplicates#af8961f1ed91ccea

 

[kellygreer1]

What is the best way to filter a Python list to its unique members?
I tried some method using Set but got some "unhashable" error.

lstone = [ 1, 2, 3, 3, 4, 5, 5, 6 ]
# how do i reduce this to
lsttwo = [ 1, 2, 3, 4, 5, 6 ]

Is there a page on this in the Python in a Nutshell or the Python
Cookbook?
Did I miss something?

Kelly Greer
(e-mail address removed)
change nospam to yahoo

or just look this thread for a fastest solutionhttp://groups.google.com/group/comp.lang.python/browse_frm/thread/709...

How come the Set() thing seems to work for some people and I get the
'unhashable' error?

How do you test for 'membership' on a dictionary?

# where tmp is the non-unique list
# dct is a dictionary where each unique key will be tied to a count
(the value)
# for testing I was setting the count to 0
for v in tmp:
if not v in dct: dct[v] = 0

# I get unhashable error here.
# Even if I write it.

for v in tmp:
if not v in dct.keys(): dct[v] = 0

What am I missing?

Thanks,
Kelly

 

[Jerry Hill]

How come the Set() thing seems to work for some people and I get the
'unhashable' error?

How do you test for 'membership' on a dictionary?

# where tmp is the non-unique list
# dct is a dictionary where each unique key will be tied to a count
(the value)
# for testing I was setting the count to 0
for v in tmp:
if not v in dct: dct[v] = 0

# I get unhashable error here.
# Even if I write it.

for v in tmp:
if not v in dct.keys(): dct[v] = 0

What am I missing?

Some of the elements of tmp are unhashable. Unhashable items can't be
the keys of a dictionary or members of a set. I don't think you've
said anywhere in the thread what these items are, you just started out
with an example of a list of integers. Do you believe the elements in
tmp are integers? If so, try the following -

for v in tmp:
print type(v), repr(v), hash(v)

and let us know what it spits out.

 

[Gabriel Genellina]

How come the Set() thing seems to work for some people and I get the
'unhashable' error?

How do you test for 'membership' on a dictionary?

# where tmp is the non-unique list
# dct is a dictionary where each unique key will be tied to a count
(the value)
# for testing I was setting the count to 0
for v in tmp:
if not v in dct: dct[v] = 0

# I get unhashable error here.
# Even if I write it.

for v in tmp:
if not v in dct.keys(): dct[v] = 0

What am I missing?

Mutable objects can't be used as keys in a dict nor be members of a set
(at least if the mutable part is used to implement the == comparison).
Technically, they should not implement __hash__ so hash(x) fails on them -
they are "unhashable".

Objects that can be used as keys in a dictionary or be members of a set
include:
instances of all numeric types, strings, tuples, frozensets; all of them
are immutable and hashable. Instances of user-defined classes that don't
implement __eq__ nor __cmp__ nor __hash__ are hashable and can be used
too. Classes that implement such special methods must ensure that (x==y)
=> (hash(x)==hash(y)) and in that case can be used too; else, they should
not implement __hash__ at all.

On the other hand, mutable containers can't be keys in a dict, nor be
members of a set; including instances of lists, sets, dicts, all of them
are mutable and unhashable objects.

Read the Cookbook recipes that someone posted before; one of them is
generic enough to handle all cases, trying the fastest approaches first.

If you think that all your objects should be hashable, try to find which
one isn't.