Oddity using sorted with key

[Josh English]

I am running into a strange behavior using the sorted function in Python 2.7. The key parameter is not behaving as the docs say it does:

Here is a snippet of code, simplified from my full program:

#begin code
class Thing(object):
def __init__(self, name):
self.name = name

def __repr__(self):
return "Thing %s" % self.name


stuff = [Thing('a'), Thing('C'), Thing('b'), Thing('2')]
more_stuff = [Thing('d'), Thing('f')]

all_the_stuff = stuff + more_stuff

print list(sorted(all_the_stuff, key=lambda x: x.name.lower))
print list(sorted(all_the_stuff, key=lambda x: x.name.lower()))

# END

The output is:

[Thing d, Thing f, Thing 2, Thing a, Thing b, Thing C]
[Thing 2, Thing a, Thing b, Thing C, Thing d, Thing f]

The second call to sorted works as expected. Just using the method doesn't sort properly.

Any ideas why I'm seeing two different results? Especially as the correct form is giving me the wrong results?

Josh English

 

[Chris Kaynor]

print list(sorted(all_the_stuff, key=lambda x: x.name.lower))

In this case, the key being sorted on is the function object x.name.lower,
not the result of the call.

It might make more sense if you break the lambda out into a separate def
statement, like this:
def myKeyFunc(x):
return x.name.lower # Return the function that produces the lower-cased
name.
print list(sorted(all_the_stuff, key=myKeyFunc))

print list(sorted(all_the_stuff, key=lambda x: x.name.lower()))

In this case, you are calling x.name.lower, and the key being used is the
result of that call.

And heres what this one looks like broken down:
def myKeyFunc(x):
return x.name.lower() # Return the lower-cased name.
print list(sorted(all_the_stuff, key=myKeyFunc))

Chris

 

[emile]

I am running into a strange behavior using the sorted function in Python 2.7. The key parameter is not behaving as the docs say it does:

Here is a snippet of code, simplified from my full program:

#begin code
class Thing(object):
def __init__(self, name):
self.name = name

def __repr__(self):
return "Thing %s" % self.name


stuff = [Thing('a'), Thing('C'), Thing('b'), Thing('2')]
more_stuff = [Thing('d'), Thing('f')]

all_the_stuff = stuff + more_stuff

print list(sorted(all_the_stuff, key=lambda x: x.name.lower))

in this case everything sorts by the same value of the bound method of
name.lower. You could have used "".lower, which has the same value as
x.name.lower, and gotten the same results.

print list(sorted(all_the_stuff, key=lambda x: x.name.lower()))

in this case you're sorting by the lower case value of x.name, which is
what you want.


Emile

# END

The output is:

[Thing d, Thing f, Thing 2, Thing a, Thing b, Thing C]
[Thing 2, Thing a, Thing b, Thing C, Thing d, Thing f]

The second call to sorted works as expected. Just using the method doesn't sort properly.

Any ideas why I'm seeing two different results? Especially as the correct form is giving me the wrong results?

Josh English

 

[John Gordon]

print list(sorted(all_the_stuff, key=lambda x: x.name.lower))
print list(sorted(all_the_stuff, key=lambda x: x.name.lower()))

# END

The output is:

[Thing d, Thing f, Thing 2, Thing a, Thing b, Thing C]
[Thing 2, Thing a, Thing b, Thing C, Thing d, Thing f]

Any ideas why I'm seeing two different results? Especially as the correct
form is giving me the wrong results?

Why do you say that 'key=lambda x: x.name.lower' is the correct form? That
returns the str.lower() function object, which is a silly thing to sort
on. Surely you want to sort on the *result* of that function, which is
what your second print does.

 

[Peter Otten]

I am running into a strange behavior using the sorted function in Python

print list(sorted(all_the_stuff, key=lambda x: x.name.lower))
print list(sorted(all_the_stuff, key=lambda x: x.name.lower()))

Let's simplify your example some more:

items = ["B", "2", "a"]
sorted(items, key=lambda x: x.lower())

['2', 'a', 'B']

That certainly works. Let's have a look at the keys used for the sorting:
.... print item, "-->", (lambda x: x.lower())(item)
....
B --> b
2 --> 2
a --> a

Now on to your first example. What are the keys in this case?
.... print item, "-->", (lambda x: x.lower)(item)
....
B --> <built-in method lower of str object at 0x7fd7ae45d260>
2 --> <built-in method lower of str object at 0x7fd7ae3e40f8>
a --> <built-in method lower of str object at 0x7fd7ae43d648>

Just a bunch of bound methods. These compare by id(), basically the memory
address of the bound method object, not something that makes a lot of sense
as a sort key.

2.7. The key parameter is not behaving as the docs say it does:

You may be misled by the mention of key=str.lower

str.lower is an unbound method, and if you pass it to sorted() together with
a list of strings, it results in a call

str.lower(item) # item is a string

which is equivalent to

item.lower() # item is a string

So while it doesn't work for a list of Thing instances, if you have bare
strings you can do
['2', 'a', 'B']

 

[Josh English]

Why do you say that 'key=lambda x: x.name.lower' is the correct form? That

returns the str.lower() function object, which is a silly thing to sort

on. Surely you want to sort on the *result* of that function, which is

what your second print does.

From http://docs.python.org/2/library/functions.html:

key specifies a function of one argument that is used to extract a comparison key from each list element: key=str.lower. The default value is None (compare the elements directly).

And from https://wiki.python.org/moin/HowTo/Sorting/:

The value of the key parameter should be a function that takes a single argument and returns a key to use for sorting purposes. This technique is fast because the key function is called exactly once for each input record.

....

Of course, now that I re-read that, I was mistaking the lambda function with the definition of the lambda function. Stupid me.

 

[Josh English]

A comprehensive and educational answer, Peter. Thank you.

Josh