overload builtin operator

S

Shaun

Hi,

I'm trying to overload the divide operator in python for basic arithmetic.
eg. 10/2 ... no classes involved.

I am attempting to redefine operator.__div__ as follows:

# my divide function
def safediv(a,b):
return ...

# reassign buildin __div__
import operator
operator.__div__ = safediv

The operator.__dict__ seems to be updated OK but the '/' operator still
calls buildin __div__

Does anyone know if this is possible and if I'm going along the correct
path with my attempts above?
Is it possible to do this using a C extention?

Regards,
Shaun.
 
R

Robert Kern

Shaun said:
Hi,

I'm trying to overload the divide operator in python for basic arithmetic.
eg. 10/2 ... no classes involved.

I am attempting to redefine operator.__div__ as follows:

# my divide function
def safediv(a,b):
return ...

# reassign buildin __div__
import operator
operator.__div__ = safediv

The operator.__dict__ seems to be updated OK but the '/' operator still
calls buildin __div__

Actually,

int1 / int2

is, I believe, equivalent to

int.__div__(int1, int2)

operator.__div__ does not get called.
Does anyone know if this is possible and if I'm going along the correct
path with my attempts above?
Is it possible to do this using a C extention?

No, you can't replace the methods on builtin types.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
R

Reinhold Birkenfeld

Shaun said:
Hi,

I'm trying to overload the divide operator in python for basic arithmetic.
eg. 10/2 ... no classes involved.

I am attempting to redefine operator.__div__ as follows:

# my divide function
def safediv(a,b):
return ...

# reassign buildin __div__
import operator
operator.__div__ = safediv

The operator.__dict__ seems to be updated OK but the '/' operator still
calls buildin __div__

It won't work that way. You cannot globally modify the behaviour of an operator,
but you can customize how an operator works for your type.

Consider:

class safeint(int):
def __div__(self, other):
return safediv(self, other)

safeint(10)/2

Reinhold
 
B

Bengt Richter

It won't work that way. You cannot globally modify the behaviour of an operator,
but you can customize how an operator works for your type.

Consider:

class safeint(int):
def __div__(self, other):
return safediv(self, other)

safeint(10)/2
You are right that you cannot globally modify the behaviour of an operator in the
sense the OP seems to be envisioning, but with some trouble I think it would be possible
to interfere with the translation of '<numerator-term>/<denominator-term>' to become
'safediv(<numerator-term>, <denominator-term>)' by transforming the AST during a custom
import process, such that wherever a Div node is found, a CallFunc node is substituted. E.g.,

for a node like

Div((Name('numerator'), Name('denominator')))

substitute another node like

CallFunc(Name('safediv'), [Name('numerator'), Name('denominator')], None, None)

where the Name('numerator') and Name('denominator') in the latter are actually
the Div node's .left and .right, so as to operate on the same args (which can
be abitrary expression-representing subtrees).

Of course, you probably also want to replace

AugAssign(Name('a'), '/=', Name('b'))

with

Assign([AssName('a', 'OP_ASSIGN')], CallFunc(Name('safediv'), [Name('a'), Name('b')], None, None))

Unfortunately, the AST tree does not seem to be designed to be edited easily wrt
_replacing_ nodes found by walking via flattened lists vs. just _using_ them in order.

I think I can create special walker that can do replacements of nodes found anywhere, and
calling node-type-specific or default supplied callbacks to supply replacements for original nodes
passed to them, but this will require a number of specialized visitors etc., so I will have to get back
to this later. But at least the OP and you nudged me into thinking about AST rewriting again ;-)
(also affects my @@sourcedeco ideas ;-)

Regards,
Bengt Richter
 
S

Shaun

Thanks for your replies, obviously this isn't a simple thing to do so
I'll take a different tack.

The exact problem I am trying to solve here is to avoid the
ZeroDivisionError in division.
I have c++ code which delegates to python to calculate expressions on
table cells. The values of the table cell are arbitary numbers and the
expressions to be calculated are fairly simple python arithmetic and
math functions.

The problem being that some users want an expression like '(100/x)+ 3'
where x=0 to return 3. So that dividing a number by zero results in 0.

Apart from parsing the expression string myself and checking for divide
by zero I can't find another way to solve the problem.

Hopefully someone out there has some ideas.
Thanks,
Shaun.
 
R

Robert Kern

Shaun said:
Thanks for your replies, obviously this isn't a simple thing to do so
I'll take a different tack.

The exact problem I am trying to solve here is to avoid the
ZeroDivisionError in division.
I have c++ code which delegates to python to calculate expressions on
table cells. The values of the table cell are arbitary numbers and the
expressions to be calculated are fairly simple python arithmetic and
math functions.

The problem being that some users want an expression like '(100/x)+ 3'
where x=0 to return 3. So that dividing a number by zero results in 0.

You have silly users. There's only one place I can think of where
division of a finite, nonzero number by zero can sensibly result in zero
(underdetermined least squares via SVD). I'd be curious as to why they
want that behavior.
Apart from parsing the expression string myself and checking for divide
by zero I can't find another way to solve the problem.

It might be easier to parse the expression and wrap all of the numbers
by a subclass of float that does division like you want.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
B

Bengt Richter

It won't work that way. You cannot globally modify the behaviour of an operator,
but you can customize how an operator works for your type.

Consider:

class safeint(int):
def __div__(self, other):
return safediv(self, other)

safeint(10)/2
You are right that you cannot globally modify the behaviour of an operator in the
sense the OP seems to be envisioning, but with some trouble I think it would be possible
to interfere with the translation of '<numerator-term>/<denominator-term>' to become
'safediv(<numerator-term>, <denominator-term>)' by transforming the AST during a custom
import process, such that wherever a Div node is found, a CallFunc node is substituted. E.g.,

for a node like

Div((Name('numerator'), Name('denominator')))

substitute another node like

CallFunc(Name('safediv'), [Name('numerator'), Name('denominator')], None, None)

where the Name('numerator') and Name('denominator') in the latter are actually
the Div node's .left and .right, so as to operate on the same args (which can
be abitrary expression-representing subtrees).

Of course, you probably also want to replace

AugAssign(Name('a'), '/=', Name('b'))

with

Assign([AssName('a', 'OP_ASSIGN')], CallFunc(Name('safediv'), [Name('a'), Name('b')], None, None))

Unfortunately, the AST tree does not seem to be designed to be edited easily wrt
_replacing_ nodes found by walking via flattened lists vs. just _using_ them in order.

I think I can create special walker that can do replacements of nodes found anywhere, and
calling node-type-specific or default supplied callbacks to supply replacements for original nodes
passed to them, but this will require a number of specialized visitor methods etc., so I will
have to get back to this later. But at least the OP and you nudged me into thinking about
AST rewriting again ;-) (also affects my @@sourcedeco ideas ;-)

.... this didn't go out due to news server problem, so I'll add some clips from what I did re ast editing:

----< testdiv.py >-------------------------------------------------------------------------
def test():
print 1.0/2.0
print 12/3
a=12; b=3
print a/b
print 2**a/(b+1)
try:
print 'print 1/0 =>'
print 1/0
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'print a/(b*(a-12)) =>'
print a/(b*(a-12))
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'def foo(x=1/(b-3)): return x =>'
def foo(x=1/(b-3)): return x
print 'print foo() =>'
print foo()
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'b /= (a-12) =>'
b /= (a-12)
print 'b after b/=(a-12):', b
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)

if __name__ == '__main__': test()
-------------------------------------------------------------------------------------------
Outputfrom run without conversion of / :

0.5
4
4
1024
print 1/0 =>
ZeroDivisionError: integer division or modulo by zero
print a/(b*(a-12)) =>
ZeroDivisionError: integer division or modulo by zero
def foo(x=1/(b-3)): return x =>
ZeroDivisionError: integer division or modulo by zero
b /= (a-12) =>
ZeroDivisionError: integer division or modulo by zero


----< import_safediv.py >------------------------------------------------------------------
# import_safediv.py
from importast import importast
from compiler.ast import Div, CallFunc, Name
from compiler.ast import AugAssign, Assign, AssName

def safediv(num, den):
if den==0: result = 0*num
else: result = num/den
print 'safediv(%r, %r) => %r'%(num, den, result)
return result

def div2safediv(divnode):
"""replace Div nodes with CallFunc nodes calling safediv with same args"""
return CallFunc(Name('safediv'),[divnode.left, divnode.right], None, None, divnode.lineno)

def divaugass2safediv(auganode):
if auganode.op != '/=': return auganode
assname =auganode.node.name
return Assign([AssName(assname, 'OP_ASSIGN')],
CallFunc(Name('safediv'),[Name(assname), auganode.expr], None, None, auganode.lineno))

callbacks = {Div:div2safediv, AugAssign:divaugass2safediv}
def import_safediv(modname, verbose=False):
return importast(modname, callbacks, verbose)

if __name__ == '__main__':
import sys
modname, args = sys.argv[1], sys.argv[2:]
verbose = (args[0:] and args[0]=='-verbose') or False
modsafe = import_safediv(modname, verbose)
modsafe.safediv = safediv
if hasattr(modsafe, 'test'): modsafe.test()
-------------------------------------------------------------------------------------------

Result from running the above and specifying testdiv as the module
to import and enforce safediv on, and run test() on:

safediv(1.0, 2.0) => 0.5
0.5
safediv(12, 3) => 4
4
safediv(12, 3) => 4
4
safediv(4096, 4) => 1024
1024
print 1/0 =>
safediv(1, 0) => 0
0
print a/(b*(a-12)) =>
safediv(12, 0) => 0
0
def foo(x=1/(b-3)): return x =>
safediv(1, 0) => 0
print foo() =>
0
b /= (a-12) =>
safediv(3, 0) => 0
b after b/=(a-12): 0

Note that the exceptions went away ;-)

It was much messier than it should have been, and the code is not very efficient,
but at least it's a kind of proof of concept ;-)

Regards,
Bengt Richter
 
B

Bengt Richter

On Sun, 28 Aug 2005 04:09:10 GMT, (e-mail address removed) (Bengt Richter) wrote:

[... a response to the OP's apparent desire to "overload the divide operator" with a call
to his safediv function ...]

The part that rewrote the the AugAssign could only work for plain name Augassign targets, so
I introduced a helper function to generate a suitable assignment target node for attributes,
subscripts, and slices as well. So the test (still very alpha) looks like the following now:



----< testdiv.py >-------------------------------------------------------------------------
def test():
print 1.0/2.0
print 12/3
a=12; b=3
print a/b
print 2**a/(b+1)
try:
print 'print 1/0 =>'
print 1/0
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'print a/(b*(a-12)) =>'
print a/(b*(a-12))
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'def foo(x=1/(b-3)): return x =>'
def foo(x=1/(b-3)): return x
print 'print foo() =>'
print foo()
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
try:
print 'b /= (a-12) =>'
b /= (a-12)
print 'b after b/=(a-12):', b
except Exception, e: print '%s: %s' %(e.__class__.__name__, e)
print 's=[15]; s[0] /= 3; print s =>'
s=[15]; s[0] /= 3; print s
class T(list):
def __getitem__(self, i): print i; return 42
def __setitem__(self, i, v): print i, v
def __div__(self, other): print self, other; return 4242
t = T()
print 't.x=15; t.x /= 3; print t.x =>'
t.x=15; t.x /= 3; print t.x
print 't=T([15, 27]); t /= 3; print t =>'
t=T([15, 27]); t /= 3; print t

def foo(x, y):
"""for dis.dis to show code"""
z = x/y
x /= y
x.a /= y
x[z] /= y
x[:] /= y

if __name__ == '__main__': test()
------------------------------------------------------------------------------------------
Outputfrom run without conversion of / :


----< import_safediv.py >------------------------------------------------------------------
# import_safediv.py
from arborist import import_editing_ast, get_augassign_tgt
from compiler.ast import Div, CallFunc, Name, AugAssign, Assign

def safediv(num, den):
if den==0: result = 0*num
else: result = num/den
print 'safediv(%r, %r) => %r'%(num, den, result)
return result

def div2safediv(divnode):
"""replace Div nodes with CallFunc nodes calling safediv with same args"""
return CallFunc(Name('safediv'),[divnode.left, divnode.right], None, None, divnode.lineno)

def divaugass2safediv(auganode):
if auganode.op != '/=': return auganode
return Assign([get_augassign_tgt(auganode)],
CallFunc(Name('safediv'),[auganode.node, auganode.expr], None, None, auganode.lineno))

callbacks = {Div:div2safediv, AugAssign:divaugass2safediv}
def import_safediv(modname, verbose=False):
modsafe = import_editing_ast(modname, callbacks, verbose)
modsafe.safediv = safediv
return modsafe

if __name__ == '__main__':
modsafe = import_safediv('testdiv', verbose=True)
modsafe.test()
-------------------------------------------------------------------------------------------

Result from using the above interactively:

[14:21] C:\pywk\ut\ast>py24
Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright", "credits" or "license" for more information.

First import as usual and run test() 0.5
4
4
1024
print 1/0 =>
ZeroDivisionError: integer division or modulo by zero
print a/(b*(a-12)) =>
ZeroDivisionError: integer division or modulo by zero
def foo(x=1/(b-3)): return x =>
ZeroDivisionError: integer division or modulo by zero
b /= (a-12) =>
ZeroDivisionError: integer division or modulo by zero
s=[15]; s[0] /= 3; print s =>
[5]
t.x=15; t.x /= 3; print t.x =>
5
t=T([15, 27]); t /= 3; print t =>
[15, 27] 3
4242

Now import the import_safediv importer, to import with
conversion of ast to effect translation of divides to safediv calls:

Import and runs test() much as before:
safediv(1.0, 2.0) => 0.5
0.5
safediv(12, 3) => 4
4
safediv(12, 3) => 4
4
safediv(4096, 4) => 1024
1024
print 1/0 =>
safediv(1, 0) => 0
0
print a/(b*(a-12)) =>
safediv(12, 0) => 0
0
def foo(x=1/(b-3)): return x =>
safediv(1, 0) => 0
print foo() =>
0
b /= (a-12) =>
safediv(3, 0) => 0
b after b/=(a-12): 0
s=[15]; s[0] /= 3; print s =>
safediv(15, 3) => 5
[5]
t.x=15; t.x /= 3; print t.x =>
safediv(15, 3) => 5
5
t=T([15, 27]); t /= 3; print t =>
[15, 27] 3
safediv([15, 27], 3) => 4242
4242

Look at the code generated by normal import first
40 0 LOAD_FAST 0 (x)
3 LOAD_FAST 1 (y)
6 BINARY_DIVIDE
7 STORE_FAST 2 (z)

41 10 LOAD_FAST 0 (x)
13 LOAD_FAST 1 (y)
16 INPLACE_DIVIDE
17 STORE_FAST 0 (x)

42 20 LOAD_FAST 0 (x)
23 DUP_TOP
24 LOAD_ATTR 3 (a)
27 LOAD_FAST 1 (y)
30 INPLACE_DIVIDE
31 ROT_TWO
32 STORE_ATTR 3 (a)

43 35 LOAD_FAST 0 (x)
38 LOAD_FAST 2 (z)
41 DUP_TOPX 2
44 BINARY_SUBSCR
45 LOAD_FAST 1 (y)
48 INPLACE_DIVIDE
49 ROT_THREE
50 STORE_SUBSCR

44 51 LOAD_FAST 0 (x)
54 DUP_TOP
55 SLICE+0
56 LOAD_FAST 1 (y)
59 INPLACE_DIVIDE
60 ROT_TWO
61 STORE_SLICE+0
62 LOAD_CONST 1 (None)
65 RETURN_VALUE

and then corresponding import_safediv (with same line numbers):
40 0 LOAD_GLOBAL 0 (safediv)
3 LOAD_FAST 0 (x)
6 LOAD_FAST 1 (y)
9 CALL_FUNCTION 2
12 STORE_FAST 3 (z)

41 15 LOAD_GLOBAL 0 (safediv)
18 LOAD_FAST 0 (x)
21 LOAD_FAST 1 (y)
24 CALL_FUNCTION 2
27 STORE_FAST 0 (x)

42 30 LOAD_GLOBAL 0 (safediv)
33 LOAD_FAST 0 (x)
36 LOAD_ATTR 4 (a)
39 LOAD_FAST 1 (y)
42 CALL_FUNCTION 2
45 LOAD_FAST 0 (x)
48 STORE_ATTR 4 (a)

43 51 LOAD_GLOBAL 0 (safediv)
54 LOAD_FAST 0 (x)
57 LOAD_FAST 3 (z)
60 BINARY_SUBSCR
61 LOAD_FAST 1 (y)
64 CALL_FUNCTION 2
67 LOAD_FAST 0 (x)
70 LOAD_FAST 3 (z)
73 STORE_SUBSCR

44 74 LOAD_GLOBAL 0 (safediv)
77 LOAD_FAST 0 (x)
80 SLICE+0
81 LOAD_FAST 1 (y)
84 CALL_FUNCTION 2
87 LOAD_FAST 0 (x)
90 STORE_SLICE+0
91 LOAD_CONST 1 (None)
94 RETURN_VALUE


I guess this shows that code munging by AST rewriting is somewhat feasible,
but you can probably expect to have to dig into dark corners ;-)

I need to think through a few things I'm doing by intuition before I go much further...

Regards,
Bengt Richter
 
T

Terry Reedy

I suspect that PyPy, when further alone, will make it easier to do things
like develop a customized interpreter that has alternate definitions for
builtin operators. So maybe the OP should ask again in a year or two.

TJR
 
R

Robert Kern

Iain said:
Robert Kern wrote:

You mean you don't? Damn. Can I have some of yours?

No, you may not. Mine! All mine!

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
B

Bengt Richter

I suspect that PyPy, when further alone, will make it easier to do things
like develop a customized interpreter that has alternate definitions for
builtin operators. So maybe the OP should ask again in a year or two.
OTOH, did you notice that I provided (although I didn't post the underlying
walker framework) a way (that works now) of importing a module with
the / operator effectively converted to a safediv call, as the OP apparently wanted?
;-)
BTW, I posted an update that handles non-simple-name-target /= augassigns as well,
but though it shows on google groups, I can't see it yet. Still some news server
problem, apparently. Or maybe it's just slogging to catch up after previous problem...

Regards,
Bengt Richter
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,787
Messages
2,569,629
Members
45,331
Latest member
ElaneLyttl

Latest Threads

Top