sum() requires number, not simply __add__

[Buck Golemon]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

We can fix this in a backward-compatible way, I believe.

Demonstration:
I'd expect these two error messages to be identical, but they are
not.
TypeError: unsupported operand type(s) for +: 'C' and 'C'

 

[Buck Golemon]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

We can fix this in a backward-compatible way, I believe.

Demonstration:
    I'd expect these two error messages to be identical, but they are
not.

     >>> class C(object): pass
     >>> c = C()
     >>> sum((c,c))
    TypeError: unsupported operand type(s) for +: 'int' and 'C'
    >>> c + c
    TypeError: unsupported operand type(s) for +: 'C' and 'C'

Proposal:

def sum(values,
base=0):
values =
iter(values)

try:
result = values.next()
except StopIteration:
return base

for value in values:
result += value
return result

 

[Arnaud Delobelle]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

It can. You need to pass a second argument which will be the start
value. Try help(sum) for details.

 

[Chris Rebert]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

The time machine strikes again! sum() already can. You just need to
specify an appropriate initial value (the empty list in this example)
for the accumulator :

Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

sum([[1,2],[3,4]], [])

[1, 2, 3, 4]

Cheers,
Chris

 

[Buck Golemon]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

The time machine strikes again! sum() already can. You just need to
specify an appropriate initial value (the empty list in this example)
for the accumulator :

Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.>>>sum([[1,2],[3,4]], [])

[1, 2, 3, 4]

Cheers,
Chris
--http://rebertia.com

Thanks. I did not know that!

My proposal is still *slightly* superior in two ways:

1) It reduces the number of __add__ operations by one
2) The second argument isn't strictly necessary, if you don't mind
that the 'null sum' will produce zero.

def sum(values, base=0):
values = iter(values)

try:
result = values.next()
except StopIteration:
return base

for value in values:
result += value

return result

 

[Arnaud Delobelle]

def sum(values,
base=0):
     values =
iter(values)

     try:
         result = values.next()
     except StopIteration:
         return base

     for value in values:
         result += value
     return result

This is definitely not backward compatible. To get something that has
a better chance of working with existing code, try this (untested):

_sentinel = object()

def sum(iterable, start=_sentinel):
if start is _sentinel:
iterable = iter(iterable)
try:
start = iterable.next()
except StopIteration:
return 0
for x in iterable:
start += x
return start

del _sentinel

 

[Stefan Behnel]

Chris Rebert, 23.02.2012 22:32:

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

The time machine strikes again! sum() already can. You just need to
specify an appropriate initial value (the empty list in this example)
for the accumulator :

Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

sum([[1,2],[3,4]], [])

[1, 2, 3, 4]

I know that you just meant this as an example, but it's worth mentioning in
this context that it's not exactly efficient to "sum up" lists this way
because there is a lot of copying involved. Each adding of two lists
creates a third one and copies all elements into it. So it eats a lot of
time and space.

Stefan

 

[Chris Angelico]

_sentinel = object()

def sum(iterable, start=_sentinel):
   if start is _sentinel:

del _sentinel

Somewhat off-topic: Doesn't the if statement there do a lookup for a
global, which would mean that 'del _sentinel' will cause it to fail?
Or have I missed something here?

ChrisA

 

[Ian Kelly]

My proposal is still *slightly* superior in two ways:

1) It reduces the number of __add__ operations by one
2) The second argument isn't strictly necessary, if you don't mind
that the 'null sum' will produce zero.

It produces the wrong result, though:

sum([3,4], base=12)

7

If I'm starting with 12 and summing 3 and 4, I expect to get 19.

Ideally the second argument should be ignored only if it isn't passed
in at all, and I don't know off-hand why the built-in sum doesn't do
this. We really don't need to replace it, though. If you want a
different sum behavior, just write your own.

def sum(iterable, *args):
return reduce(operator.add, iterable, *args)

sum([3,4]) 7
sum([3,4], 12) 19
sum(['hello', 'world'])

'helloworld'

Cheers,
Ian

 

[Arnaud Delobelle]

Somewhat off-topic: Doesn't the if statement there do a lookup for a
global, which would mean that 'del _sentinel' will cause it to fail?
Or have I missed something here?

Yes, you're right Change the signature to

def sum(iterable, start=_sentinel, _sentinel=_sentinel):

This is not pretty...

 

[Ian Kelly]

Somewhat off-topic: Doesn't the if statement there do a lookup for a
global, which would mean that 'del _sentinel' will cause it to fail?
Or have I missed something here?

I believe you're correct. If you really want to delete the _sentinel
reference though, you could do:

def sum(iterable, start=object()):
if start is sum.func_defaults[0]:
...

Cheers,
Ian

 

[Chris Angelico]

def sum(iterable, start=_sentinel, _sentinel=_sentinel):

Is this a reason for Python to introduce a new syntax, such as:

def foo(blah, optional=del):
if optional is del: print("No argument was provided")

Basically, 'del' is treated like a unique non-providable object, only
possible in an argument list and only if the argument was omitted. No
more proliferation of individual sentinels... what do you think?

(I picked "del" because it's an existing keyword. Fairly arbitrary
choice though.)

Chris Angelico

 

[Steven D'Aprano]

Somewhat off-topic: Doesn't the if statement there do a lookup for a
global, which would mean that 'del _sentinel' will cause it to fail? Or
have I missed something here?

Yes, deleting _sentinel will cause the custom sum to fail, and yes, you
have missed something.

If the caller wants to mess with your library and break it, they have
many, many ways to do so apart from deleting your private variables.


del _sentinel
_sentinel = "something else"
sum.__defaults__ = (42,) # mess with the function defaults
sum.__code__ = (lambda a, b=None: 100).__code__ # and with func internals
sum = None # change your custom sum to something else
del sum # or just delete it completely
len = 42 # shadow a built-in
import builtins; del builtins.range # really screw with you


If your application stops working after you carelessly mess with
components your application relies on, the right answer is usually:

"Don't do that then."

Python doesn't try to prevent people from shooting themselves in the foot.


Monkey-patching-by-actual-monkeys-for-fun-and-profit-ly y'rs,

 

[Chris Angelico]

Yes, deleting _sentinel will cause the custom sum to fail, and yes, you
have missed something.

If the caller wants to mess with your library and break it, they have
many, many ways to do so apart from deleting your private variables.

I was looking at the module breaking itself, though, not even waiting
for the caller to do it.

ChrisA

 

[Peter Otten]

I feel like the design of sum() is inconsistent with other language
features of python. Often python doesn't require a specific type, only
that the type implement certain methods.

Given a class that implements __add__ why should sum() not be able to
operate on that class?

We can fix this in a backward-compatible way, I believe.

Demonstration:
I'd expect these two error messages to be identical, but they are
not.

TypeError: unsupported operand type(s) for +: 'C' and 'C'

You could explicitly provide a null object:
.... def __add__(self, other):
.... return other
........ def __init__(self, v):
.... self.v = v
.... def __add__(self, other):
.... return A("%s+%s" % (self, other))
.... def __str__(self):
.... return self.v
.... def __repr__(self):
.... return "A(%r)" % self. v
....Traceback (most recent call last):
A('a+b+c')

 

[Antoon Pardon]

If your application stops working after you carelessly mess with
components your application relies on, the right answer is usually:

"Don't do that then."

Python doesn't try to prevent people from shooting themselves in the foot.

Yes it does! A simple example is None as a keyword to prevent
assignments to it.

 

[Roy Smith]

Python doesn't try to prevent people from shooting themselves in the foot.

Yes it does! A simple example is None as a keyword to prevent
assignments to it.[/QUOTE]

Hmmm. Just playing around with some bizarre things to do with None, and
discovered this:

doesn't give an error, but also doesn't assign the module to the symbol
'None'. Weird.

 

[Terry Reedy]

Hmmm. Just playing around with some bizarre things to do with None, and
discovered this:


doesn't give an error, but also doesn't assign the module to the symbol
'None'. Weird.

In 3.2SyntaxError: invalid syntax