subtle error slows code by 10x (builtin sum()) - replace builtin sumwithout using import?

B

bdb112

First a trap for new players, then a question to developers

Code accelerated by numpy can be slowed down by a large factor is you
neglect to import numpy.sum .

from timeit import Timer
frag = 'x=sum(linspace(0,1,1000))'
Timer(frag ,setup='from numpy import linspace').timeit(1000)
# 0.6 sec
Timer(frag, setup='from numpy import sum, linspace').timeit(1000) #
difference is I import numpy.sum
# 0.04 sec 15x faster!

This is obvious of course - but it is very easy to forget to import
numpy.sum and pay the price in execution.

Question:
Can I replace the builtin sum function globally for test purposes so
that my large set of codes uses the replacement?

The replacement would simply issue warnings.warn() if it detected an
ndarray argument, then call the original sum
I could then find the offending code and use the appropriate import to
get numpy.sum
 
A

Albert Hopkins

Question:
Can I replace the builtin sum function globally for test purposes so
that my large set of codes uses the replacement?

The replacement would simply issue warnings.warn() if it detected an
ndarray argument, then call the original sum
I could then find the offending code and use the appropriate import to
get numpy.sum

You shouldn't do this, but you could use the __builtins__ module

e.g.
 
C

Chris Torek

First a trap for new players, then a question to developers

Code accelerated by numpy can be slowed down by a large factor is you
neglect to import numpy.sum .

from timeit import Timer
frag = 'x=sum(linspace(0,1,1000))'
Timer(frag ,setup='from numpy import linspace').timeit(1000)
# 0.6 sec
Timer(frag, setup='from numpy import sum, linspace').timeit(1000) #
difference is I import numpy.sum
# 0.04 sec 15x faster!

This is obvious of course - but it is very easy to forget to import
numpy.sum and pay the price in execution.

Question:
Can I replace the builtin sum function globally for test purposes so
that my large set of codes uses the replacement?
The replacement would simply issue warnings.warn() if it detected an
ndarray argument, then call the original sum
I could then find the offending code and use the appropriate import to
get numpy.sum


Sure, just execute code along these lines before running any of
the tests:

import __builtin__
import warnings

_sys_sum = sum # grab it before we change __builtin__.sum

def hacked_sum(sequence, start=0):
if isinstance(sequence, whatever):
warnings.warn('your warning here')
return _sys_sum(sequence, start)

__builtin__.sum = hacked_sum

(You might want to grab a stack trace too, using the traceback
module.) You said "without using import" but all you have to
do is arrange for python to import this module before running
any of your own code, e.g., with $PYTHONHOME and a modified
site file.
 

Members online

Forum statistics

Threads
473,733
Messages
2,569,440
Members
44,832
Latest member
GlennSmall

Latest Threads

Top