help needed on decimal formatting issue

R

Robin Becker

I'm not really very used to the decimal module so I'm asking here if any one can
help me with a problem in a well known third party web framework

The code in question is

def format_number(value, max_digits, decimal_places):
"""
Formats a number into a string with the requisite number of digits and
decimal places.
"""
if isinstance(value, decimal.Decimal):
context = decimal.getcontext().copy()
context.prec = max_digits
return u'%s' % str(
value.quantize(decimal.Decimal(".1") ** decimal_places,
context=context))
else:
return u"%.*f" % (decimal_places, value)


we have set up decimal fields with max_digits=7 decimal_places=2 and
max_digits=10, decimal_places=4. We are getting issues with quantize failing
with the message 'quantize result has too many digits for current context'.

Anyhow, I believe that we need to adjust the above code so that the precision is
actually set to

context.prec = max_digits+decimal_places

but I'm not certain that is right. Presumably the author thought that the fields
could have large values of both max_digits and decimal_places. Initially I
thought precision must be to do with decimal_places only, but changing
context.prec = decimal_places

did not fix the issue. Can someone enlighten me? The failing case for the
original code was

format_number(Decimal('914146.80'),7,2)

with context.prec = decimal_places we failed differently with

format_number(Decimal('42.7571'),10,4)

so I adopted context.prec = max_digits+decimal_places and that seems to work. If
this is indeed a proper fix I will send a bug report to the well known jazz
based framework.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,048
Latest member
verona

Latest Threads

Top