decimal by default

Discussion in 'Python' started by Daniel, Jun 29, 2006.

  1. Daniel

    Daniel Guest

    I'm writing an application that (among other things) evaluates
    mathematical expressions. The user enters strings containing literals
    and names that later get evaluated using the Python interpreter. Here's
    a short (very simplified) example:

    >>> from decimal import Decimal
    >>> names = dict(a=Decimal('3.625'), b=Decimal(2))
    >>> expr = '(a + 2.625) / b' # expression entered by end-user
    >>> eval(expr, names)

    Traceback (most recent call last):
    ...
    TypeError: You can interact Decimal only with int, long or Decimal data
    types.

    I understand why I got the error, so there's no need to explain that.
    It is a requirement that the 'names' dict contains Decimal values. And
    of course it's unacceptable to expect my users to enter Decimal('...')
    every time they enter a non-integer number. My initial solutioin is to
    use a regular expression to wrap each float value with Decimal('...')
    before the expression is evaluated. But I don't like that solution for
    two reasons:

    1. It seems error prone and inelegant. Paraphrase: if you've got a
    problem and you think "Ahh, I'll use regular expressions..." now you've
    got two problems.

    2. Error reporting is not as intuitive (I'm using the Python
    interpreter and therefore my users see Python exceptions when their
    expressions don't evaluate). After the expressions have been shot up
    with all the extra Decimal junk to make them evaluate correctly they
    are not nearly as recognizable (or easy to read) and the user is likely
    to think "but I didn't even write that expression...where is that
    Decimal('...') stuff coming from?"

    Ideally I'd like to have a way to tell the interpreter to use Decimal
    by default instead of float (but only in the eval() calls). I
    understand the performance implications and they are of no concern. I'm
    also willing to define a single global Decimal context for the
    expressions (not sure if that matters or not). Is there a way to do
    what I want without rolling my own parser and/or interpreter? Is there
    some other alternative that would solve my problem?

    Thanks,
    ~ Daniel
     
    Daniel, Jun 29, 2006
    #1
    1. Advertising

  2. Daniel <> wrote:
    ...
    > Ideally I'd like to have a way to tell the interpreter to use Decimal
    > by default instead of float (but only in the eval() calls). I
    > understand the performance implications and they are of no concern. I'm
    > also willing to define a single global Decimal context for the
    > expressions (not sure if that matters or not). Is there a way to do
    > what I want without rolling my own parser and/or interpreter? Is there
    > some other alternative that would solve my problem?


    What about:

    c = compile(thestring, thestring, '<eval>')

    cc = new.code( ...all args from c's attributes, except the 5th
    one, constants, which should instead be:
    decimalize(c.co_consts)...)

    i.e.

    cc = new.code(c.co_argcount, c.co_nlocals, c.co_stacksize, c.co_flags,
    c.co_code, decimalize(c.co_consts), c.co_names,
    c.co_varnames, c.co_filename, c.co_name,
    c.co_firstlineno, c.co_lnotab)

    where

    def decimalize(tuple_of_consts):
    return tuple( maydec(c) for c in tuple_of_consts )

    and

    def maydec(c):
    if isinstance(c, float): c = decimal.Decimal(str(c))
    return c


    Yeah, the new.code call IS very verbose, just because it needs to
    tediously and boilerplatedly repeat every attribute as an argument, but
    you can easily wrap the boilerplate in an auxiliary function and forget
    about it. (If you often want to change one or two tiny thing in a code
    object you probably already have a suitable auxiliary function around).

    Now, you can eval(cc) [[I suggest you also explicitly pass dictionaries
    for locals and globals, but, hey, that's just a good idea with eval in
    general!-)]] and, ta-da!


    Alex
     
    Alex Martelli, Jun 29, 2006
    #2
    1. Advertising

  3. Daniel

    Daniel Guest

    Alex Martelli wrote:
    > What about:
    >
    > c = compile(thestring, thestring, '<eval>')
    >
    > cc = new.code( ...all args from c's attributes, except the 5th
    > one, constants, which should instead be:
    > decimalize(c.co_consts)...)


    Wow, what an elegant solution! I had no hope that it would be this
    simple. I always wondered what compile() was useful for and now I know
    at least one thing. I'll try it out tomorrow. Thanks a lot Alex!

    ~ Daniel
     
    Daniel, Jun 30, 2006
    #3
  4. Daniel <> wrote:

    > Alex Martelli wrote:
    > > What about:
    > >
    > > c = compile(thestring, thestring, '<eval>')
    > >
    > > cc = new.code( ...all args from c's attributes, except the 5th
    > > one, constants, which should instead be:
    > > decimalize(c.co_consts)...)

    >
    > Wow, what an elegant solution! I had no hope that it would be this


    Heh, funny, I was originally parsing your response as ironic, because
    _I_ don't think of this as elegant -- too boilerplatey (as the expansion
    I showed right after dispays!)... took me a sec to see you really mean
    it!-)

    > simple. I always wondered what compile() was useful for and now I know
    > at least one thing. I'll try it out tomorrow. Thanks a lot Alex!


    You're welcome! And, compile is also useful for many other things, such
    as any situation where you may need to run eval multiple times on the
    same string of source code (typically on multiple distinct
    dicts/namespaces): compile the source once, then in the loop eval the
    code object (bytecode) rather than the source -- that saves time.

    Also, you may do introspection on the code object -- for example, I show
    in the Nutshell's 2nd ed how this lets you perform a "safe eval" -- a
    way to let the user specify any Python "literal" without risking a
    malicious user running arbitrary code (essentially, you refuse to eval
    the code object if its co_names isn't empty -- or, you might let said
    co_names possibly contain just a few names you deem "safe", such as,
    say, 'sin', 'cos', 'tan', which you can get into your namespace from the
    math module). Such introspection on names may also allow some further
    optimization, particularly in the repeated-execution case, if there are
    "well-known names" that you're able to compute "just in time" (nowadays
    you can also use a special mapping to "only compute at need" the values
    for the names that are actually needed).

    Beyond which, we get into the realm of byecode hacks...!-)


    Alex
     
    Alex Martelli, Jun 30, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ven
    Replies:
    3
    Views:
    1,365
  2. Gilbert Fine
    Replies:
    8
    Views:
    932
    Zentrader
    Aug 1, 2007
  3. Vitaliy
    Replies:
    1
    Views:
    502
    Peter Otten
    May 29, 2008
  4. valpa
    Replies:
    11
    Views:
    1,609
    Steven D'Aprano
    Mar 24, 2009
  5. Replies:
    0
    Views:
    323
Loading...

Share This Page