# Is there a unique method in python to unique a list?

Discussion in 'Python' started by Token Type, Sep 9, 2012.

1. ### Token TypeGuest

Is there a unique method in python to unique a list? thanks
Token Type, Sep 9, 2012

2. ### Chris AngelicoGuest

On Sun, Sep 9, 2012 at 3:43 PM, Token Type <> wrote:
> Is there a unique method in python to unique a list? thanks

I don't believe there's a method for that, but if you don't care about
order, try turning your list into a set and then back into a list.

ChrisA
Chris Angelico, Sep 9, 2012

3. ### Chris AngelicoGuest

On Sun, Sep 9, 2012 at 4:29 PM, John H. Li <> wrote:
> However, if I don't put list(set(lemma_list)) to a variable name, it works
> much faster.

Try backdenting that statement. You're currently doing it at every
iteration of the loop - that's why it's so much slower.

But you'll probably find it better to work with the set directly,
instead of uniquifying a list as a separate operation.

ChrisA
Chris Angelico, Sep 9, 2012
4. ### Token TypeGuest

> Try backdenting that statement. You're currently doing it at every
>
> iteration of the loop - that's why it's so much slower.

Thanks. I works now.

>>> def average_polysemy(pos):

synset_list = list(wn.all_synsets(pos))
sense_number = 0
lemma_list = []
for synset in synset_list:
lemma_list.extend(synset.lemma_names)
for lemma in list(set(lemma_list)):
sense_number_new = len(wn.synsets(lemma, pos))
sense_number = sense_number + sense_number_new
return sense_number/len(set(lemma_list))

>>> average_polysemy('n')

1

> But you'll probably find it better to work with the set directly,
>
> instead of uniquifying a list as a separate operation.

Yes, the following second methods still runs faster if I don't give a separate variable name to list(set(lemma_list)). Why will this happen?

>>> def average_polysemy(pos):

synset_list = list(wn.all_synsets(pos))
sense_number = 0
lemma_list = []
for synset in synset_list:
lemma_list.extend(synset.lemma_names)
for lemma in list(set(lemma_list)):
sense_number_new = len(wn.synsets(lemma, pos))
sense_number = sense_number + sense_number_new
return sense_number/len(set(lemma_list))

>>> average_polysemy('n')

1
Token Type, Sep 9, 2012
5. ### Token TypeGuest

> Try backdenting that statement. You're currently doing it at every
>
> iteration of the loop - that's why it's so much slower.

Thanks. I works now.

>>> def average_polysemy(pos):

synset_list = list(wn.all_synsets(pos))
sense_number = 0
lemma_list = []
for synset in synset_list:
lemma_list.extend(synset.lemma_names)
for lemma in list(set(lemma_list)):
sense_number_new = len(wn.synsets(lemma, pos))
sense_number = sense_number + sense_number_new
return sense_number/len(set(lemma_list))

>>> average_polysemy('n')

1

> But you'll probably find it better to work with the set directly,
>
> instead of uniquifying a list as a separate operation.

Yes, the following second methods still runs faster if I don't give a separate variable name to list(set(lemma_list)). Why will this happen?

>>> def average_polysemy(pos):

synset_list = list(wn.all_synsets(pos))
sense_number = 0
lemma_list = []
for synset in synset_list:
lemma_list.extend(synset.lemma_names)
for lemma in list(set(lemma_list)):
sense_number_new = len(wn.synsets(lemma, pos))
sense_number = sense_number + sense_number_new
return sense_number/len(set(lemma_list))

>>> average_polysemy('n')

1
Token Type, Sep 9, 2012
6. ### Serhiy StorchakaGuest

On 09.09.12 08:47, Donald Stufft wrote:
> If you don't need to retain order you can just use a set,

Only if elements are hashable.
Serhiy Storchaka, Sep 9, 2012
7. ### Paul RubinGuest

Token Type <> writes:
>>>> def average_polysemy(pos):

> synset_list = list(wn.all_synsets(pos))
> sense_number = 0
> lemma_list = []
> for synset in synset_list:
> lemma_list.extend(synset.lemma_names)
> for lemma in list(set(lemma_list)):
> sense_number_new = len(wn.synsets(lemma, pos))
> sense_number = sense_number + sense_number_new
> return sense_number/len(set(lemma_list))

I think you mean (untested):

synsets = wn.all_synsets(pos)
sense_number = 0
lemma_set = set()
for synset in synsets:
for lemma in lemma_set:
sense_number += len(wn.synsets(lemma,pos))
return sense_number / len(lemma_set)
Paul Rubin, Sep 9, 2012
8. ### Paul RubinGuest

Paul Rubin <> writes:
> I think you mean (untested):
>
> synsets = wn.all_synsets(pos)
> sense_number = 0
> lemma_set = set()
> for synset in synsets:
> for lemma in lemma_set:
> sense_number += len(wn.synsets(lemma,pos))
> return sense_number / len(lemma_set)

Or even:

lemma_set = set(synset for synset in wn.all_synsets(pos))
sense_number = sum(len(wn.synsets(lemma, pos)) for lemma in lemma_set)
return sense_number / len(lemma_set)
Paul Rubin, Sep 9, 2012
9. ### Token TypeGuest

Thanks. I try to use set() suggested by you. However, not successful. Please see:
>>> synsets = list(wn.all_synsets('n'))
>>> synsets[:5]

[Synset('entity.n.01'), Synset('physical_entity.n.01'), Synset('abstraction.n.06'), Synset('thing.n.12'), Synset('object.n.01')]
>>> lemma_set = set()
>>> for synset in synsets:

Traceback (most recent call last):
File "<pyshell#43>", line 2, in <module>
TypeError: unhashable type: 'list'
>>> for synset in synsets:

Traceback (most recent call last):
File "<pyshell#45>", line 2, in <module>
TypeError: unhashable type: 'set'
Token Type, Sep 9, 2012
10. ### Chris AngelicoGuest

On Sun, Sep 9, 2012 at 11:44 PM, Token Type <> wrote:

That tries to add the whole list as a single object, which doesn't
work because lists can't go into sets. There are two solutions,
depending on what you want to do.

1) If you want each addition to remain discrete, make a tuple instead:

2) If you want to add the elements of that list individually into the
set, use update:
lemma_set.update(synset.lemma_names)

I'm thinking you probably want option 2 here.

ChrisA
Chris Angelico, Sep 9, 2012