don't NaN & infinities hide FP errors

[kartik]

Since Python floats are implemented using C doubles, assuming IEEE 754
compliance, certain operations should result in NaN or infinity. Don't
such special values hide errors? Would we want our variables to be in
such states? I feel it would be better if a floating point division by
zero (or other actions that produce NaNs or infinities) throws an
exception, just like an integer divide by zero.

Am I right?

 

[Peter Maas]

Since Python floats are implemented using C doubles, assuming IEEE 754
compliance, certain operations should result in NaN or infinity. Don't
such special values hide errors? Would we want our variables to be in
such states? I feel it would be better if a floating point division by
zero (or other actions that produce NaNs or infinities) throws an
exception, just like an integer divide by zero.

Am I right?

Just try it. 1./0. raises a ZeroDivisionError, math.log(0) raises a
math range error, and math.log(-1) yields nan. If you want math.log(-1)
raise an error, too, you must write your own log() that tests for nan
and raises a NaN error or write a trace function that looks for nan
returned by math.log() (see sys.settrace()).

 

[Gandalf]

Just try it. 1./0. raises a ZeroDivisionError, math.log(0) raises a
math range error, and math.log(-1) yields nan.

This is not true. On UNIX systems, math.log(-1) will be NaN.
On a Windows system, math.log(-1) will raise this:

ValueError: math domain error

math.log(0) will raise the same on both:

OverflowError: math range error

I do not have a clue about MACs and other OSes.

I wonder why it is this way? Python under Windows and UNIX is different,
even when the architecture is the same. (i386). Is it because of the OS?
Isn't it possible to implement floating point arithmetic with assembler
code,
and use that instead of the built-in floating point arithmetic of the C
compiler
under Windows? Clearly, the hardware is capable of using NaN and Inf;
I believe that Python should work the same way on any system, if there is a
way to do that.

Laci 2.0

 

[Tim Peters]

[Peter Maas]

Just try it. 1./0. raises a ZeroDivisionError,

That much is true -- Python forces this.

math.log(0) raises a math range error, and math.log(-1) yields nan.

The rest varies across platforms, as explained in this note at the end
of Python's math module docs:

"""
The math module consists mostly of thin wrappers around the platform C
math library functions. Behavior in exceptional cases is loosely
specified by the C standards, and Python inherits much of its
math-function error-reporting behavior from the platform C
implementation. As a result, the specific exceptions raised in error
cases (and even whether some arguments are considered to be
exceptional at all) are not defined in any useful cross-platform or
cross-release way. For example, whether math.log(0) returns -Inf or
raises ValueError or OverflowError isn't defined, and in cases where
math.log(0) raises OverflowError, math.log(0L) may raise ValueError
instead.
"""

 

[Gandalf]

The rest varies across platforms, as explained in this note at the end
of Python's math module docs:

"""
The math module consists mostly of thin wrappers around the platform C
math library functions. Behavior in exceptional cases is loosely
specified by the C standards, and Python inherits much of its
math-function error-reporting behavior from the platform C
implementation. As a result, the specific exceptions raised in error
cases (and even whether some arguments are considered to be
exceptional at all) are not defined in any useful cross-platform or
cross-release way. For example, whether math.log(0) returns -Inf or
raises ValueError or OverflowError isn't defined, and in cases where
math.log(0) raises OverflowError, math.log(0L) may raise ValueError
instead.
"""

I still feel that it would be possible to use some assembly code and
achieve the same behaviour on all platforms.
It is really not difficult to calculate log on a 387. It can be more
difficult on a RISC but not impossible.
Why the implementation stuck on the C standards? Okay, maybe it is not
possible or too difficult to use asm code
on some platforms. But at least Python should try to do the same thing
when it is possible. It could be an 'option'
to compile low level code for the most popular operating systems. What
do you think?

Laci 2.0

 

[Jeff Epler]

All you need to do is to contribute under the Python license, and
promise to maintain, a clearly high-quality, portable FP library and
associated test suite that will work on current and future platforms
(across instruction sets, CPU architectures, operating systems, and C
compilers), not produce a measurable slowdown in Python code compared to
platform doubles, and will never be worse (less accurate) than simply
using the platform's own maths.

Oh, and it would be bes if your code already has a proven track
record before you contribute it to Python.

Banging out a few inline assembler blocks for Pentium/gcc or
Pentium/MSVC++ is just not enough.

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFBm4LtJd01MZaTXX0RAgtBAKCU1FOsMQeP67723IgJGKpA3SWXkgCfWxm2
XBKUd1G8zuYgHwatn0KAsTg=
=CXiP
-----END PGP SIGNATURE-----

 

[John Roth]

I still feel that it would be possible to use some assembly code and
achieve the same behaviour on all platforms.

As Jeff points out, the core developers would really like this.
However, they don't have the time to do it, so if you want to
do it, they would be grateful - as long as you follow a few
simple rules.

As I understand it, it's not quite as bad as Jeff says: the
problem is nasty enough that if you simply handle several
major platforms _consistently_, then the minor platforms
can still behave inconsistently. Then someone else can do
any of them that she wants to.

I think making Windows (both native and Cygwin) and
all the Unices (including Linux and Mac OS 10) behave
consistently would be a good start. You might also want
to look at the new floating decimal package in release 2.4
for ideas about how to handle exceptions and so forth.

There are guidelines for both Python and C code in
the Peps, and becoming familiar with the source is something
you want to do.

Make sure you've got a thorough test suite: if something
breaks this will save a maintainer a lot of time figuring out
what.

Otherwise, have at it. Lots of people will bless you.

John Roth

 

[Tim Peters]

[Gandalf]

I still feel that it would be possible to use some assembly code and
achieve the same behaviour on all platforms.
It is really not difficult to calculate log on a 387. It can be more
difficult on a RISC but not impossible.
Why the implementation stuck on the C standards? Okay, maybe it is
not possible or too difficult to use asm code on some platforms. But at
least Python should try to do the same thing when it is possible. It
could be an 'option' to compile low level code for the most popular
operating systems. What do you think?

I think you're not burdened by actual experience writing
production-quality math libraries <0.3 wink>. I've done that for a
living at times, and writing a production-quality math library, even
at the shallow level C89 provides, and even for a single platform, is
a whale of a lot of work. OTOH, writing low-quality math libraries is
easy.

Python defers to C here because platform C vendors are generally in a much
better position to provide production-quality platform-specific math
libraries than the Python project will ever be.

BTW, "same behavior on all platforms" is a non-goal for me when it
comes to floating point. There's an enormous range of
speed-vs-accuracy tradeoffs that can be made here, and legitimately
so. By not trying to force a specific choice here on Python's users,
a Python user that *really* cares about this can get the tradeoff they
like by obtaining (probably buying) and installing a libm replacement
for their platform C libm. Python will automatically use that too
then.

 

[Grant Edwards]

No.

Assuming you're not using the royal we, yes "we" do.

What if I don't _want_ an exception? What if NaN or Infinity
is _the_correct_answer_? For what I do, most of the time, NaN
or Infinity is exactly what I want.

You may think, while dealing with your problems and programs, that NaN is
an error.

I may want them, because NaN, Inf, and exceptions are not the same things.

I certainly want them. I'd be right annoyed if somebody
decided all of a sudden than NaNs are errors. NaNs are
tremendously useful things, and making them errors would be
like deciding that negative integers are errors since you
obviously can have "minus seven" apples sitting on the table in
front of you.

Let's say you've got a bunch of process control modules. Each
of them takes a set of input values and produces a set of
outputs.

Now let's say one of the inputs fails (no valid value is
available). You can just shut down the whole refinery, you've
got to try to keep going as best you can. The easiest way to
do that is to feed a NaN in on the invalid input, and let it
propogate through the network of modules. The outputs that
don't depend on the invalid input remain valid. The outputs
that do depend on the invalid input are NaNs.

At the output end of things you don't have to do all sorts of
logic to figure out which outputs are valid and which ones
aren't -- all you have to do is decide what to do when each
output is a NaN.

 

[David M. Cooke]

This is not true. On UNIX systems, math.log(-1) will be NaN.

Not true; all the linux boxes and a Tru64 system I've just tried raise
ValueError. However, the three versions of Python I have on my Mac OS X
system give nan.

 

[Jeremy Bowers]

Since Python floats are implemented using C doubles, assuming IEEE 754
compliance, certain operations should result in NaN or infinity. Don't
such special values hide errors? Would we want our variables to be in
such states? I feel it would be better if a floating point division by
zero (or other actions that produce NaNs or infinities) throws an
exception, just like an integer divide by zero.

Am I right?

You may think, while dealing with your problems and programs, that NaN is
an error.

I may want them, because NaN, Inf, and exceptions are not the same things.

If you're working with floats, you're *already* in a domain where
*everything* might hide an error. Along with everybody else's reply
(including Tim Peter's which should be considered "definitive"), I'd add
that trying to create a float-based system that *doesn't* "hide errors" is
basically a contradiction in terms. All you can do is make a few choices
about which errors you want.

 

[kartik]

On UNIX systems, math.log(-1) will be NaN.

I don't think so. On Linux, I get a ValueError with the message "math domain error".

 

[kartik]

Let's say you've got a bunch of process control modules. Each
of them takes a set of input values and produces a set of
outputs.

Now let's say one of the inputs fails (no valid value is
available). You can just shut down the whole refinery, you've
got to try to keep going as best you can. The easiest way to
do that is to feed a NaN in on the invalid input, and let it
propogate through the network of modules. The outputs that
don't depend on the invalid input remain valid. The outputs
that do depend on the invalid input are NaNs.

At the output end of things you don't have to do all sorts of
logic to figure out which outputs are valid and which ones
aren't -- all you have to do is decide what to do when each
output is a NaN.

I understand. But I wasn't saying NaN should be removed, only that
arithmetic operations shouldn't produce it; you will still be free to
explicitly set a variable to NaN if you don't have valid input.

Besides, Python seems to raise exceptions instead of generating NaNs
or infinities, at least on my machine (Linux, GCC 3.3.2, Python
2.3.3). Doesn't that support my view?

 

[Grant Edwards]

I understand. But I wasn't saying NaN should be removed, only
that arithmetic operations shouldn't produce it; you will
still be free to explicitly set a variable to NaN if you don't
have valid input.

I don't think you do understand. The whole _point_ of my post
was that NaNs propagate. They're useless if arithmetic
operations don't produce NaNs.

Besides, Python seems to raise exceptions instead of
generating NaNs or infinities, at least on my machine (Linux,
GCC 3.3.2, Python 2.3.3).

I don't see any exceptions for operations involving NaNs:

$ python
Python 2.3.3 (#2, Feb 17 2004, 11:45:40)
[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for moreinformation.
Python does generate exceptions for divide by zero. I would
find it far more useful for it to produce an infinity.

Doesn't that support my view?

I don't think so.

 

[Dan Bishop]

I understand. But I wasn't saying NaN should be removed, only that
arithmetic operations shouldn't produce it; you will still be free to
explicitly set a variable to NaN if you don't have valid input.

Besides, Python seems to raise exceptions instead of generating NaNs
or infinities, at least on my machine (Linux, GCC 3.3.2, Python
2.3.3). Doesn't that support my view?

Python's inconsistent about OverflowErrors:

Python 2.3.4 (#2, Nov 13 2004, 18:58:41)
[GCC 3.3.5 (Debian 1:3.3.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in ?
OverflowError: (34, 'Numerical result out of range')

Python 2.1.1 (#1, Aug 25 2001, 04:19:08)
[GCC 3.0.1] on sunos5
Type "copyright", "credits" or "license" for more information.Infinity

 

[Bengt Richter]

I understand. But I wasn't saying NaN should be removed, only that
arithmetic operations shouldn't produce it; you will still be free to
explicitly set a variable to NaN if you don't have valid input.

Besides, Python seems to raise exceptions instead of generating NaNs
or infinities, at least on my machine (Linux, GCC 3.3.2, Python
2.3.3). Doesn't that support my view?

Python's inconsistent about OverflowErrors:

Python 2.3.4 (#2, Nov 13 2004, 18:58:41)
[GCC 3.3.5 (Debian 1:3.3.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
File "<stdin>", line 1, in ?
OverflowError: (34, 'Numerical result out of range')

Python 2.1.1 (#1, Aug 25 2001, 04:19:08)
[GCC 3.0.1] on sunos5
Type "copyright", "credits" or "license" for more information.Infinity

I would guess that 1e100 is passed to a c library so the inconsistency
comes from c libraries. I'm not sure what would be the best policy. Silent
promotion to infinity is kind of squirrely in a context where a long
can easily represent the number exactly and finitely, unless you view
floating point as a special context of its own, in which case I might
like an optional exception hook that by default creates infinities and NaNs
but that I could use if I wanted to raise whatever or trace stuff etc.

But in the larger picture of numbers, overflow is a representation failure.
With decimal available, ISTM we could as well promote floats to decimal as
ints to longs, to get a unified policy across numerical representations.

BTW, my 2.4b1 returns a strange representation of infinity, that has no
round trip capability:

Python 2.4b1 (#56, Nov 3 2004, 01:47:27)
[GCC 3.2.3 (mingw special 20030504-1)] on win32
Type "help", "copyright", "credits" or "license" for more information. 1.0

???
For calculated-by-python floats it seems fairly consistent, if limited:
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to float

.... But not too large for decimal (or rational based on longs).

Regards,
Bengt Richter

 

[Jive]

I kind of like the various flavors of NaN. I have on occasion written
numerical code that exploits them. But seeing as how Python aspires to some
level of cross-platform portability, it really should throw exceptions
rather than return NaN's, Inf's, and the lot. I am surprised that it
doesn't always do that.

jdadson at yahoo dott com

[Peter Maas]

Just try it. 1./0. raises a ZeroDivisionError,

That much is true -- Python forces this.

math.log(0) raises a math range error, and math.log(-1) yields nan.

The rest varies across platforms, as explained in this note at the end
of Python's math module docs:

"""
The math module consists mostly of thin wrappers around the platform C
math library functions. Behavior in exceptional cases is loosely
specified by the C standards, and Python inherits much of its
math-function error-reporting behavior from the platform C
implementation. As a result, the specific exceptions raised in error
cases (and even whether some arguments are considered to be
exceptional at all) are not defined in any useful cross-platform or
cross-release way. For example, whether math.log(0) returns -Inf or
raises ValueError or OverflowError isn't defined, and in cases where
math.log(0) raises OverflowError, math.log(0L) may raise ValueError
instead.
"""

 

[Tim Peters]

[Jive]

I kind of like the various flavors of NaN. I have on occasion written
numerical code that exploits them. But seeing as how Python aspires
to some level of cross-platform portability, it really should throw
exceptions rather than return NaN's, Inf's, and the lot. I am surprised
that it doesn't always do that.

Python is written to the C89 standard. That standard says nothing
about NaNs, infs, signed zeroes, or subnormals, and says little about
exceptional numeric conditions. C99 goes on to promise almost nothing
about exceptional conditions in libm functions; it says a lot about
754 fp semantics, but leaves 754 conformance optional. As a result,
there's no consistency across vendor C compilers or runtimes wrt these
gimmicks, neither in basic arithmetic nor in libm behavior.

If someone wants to contribute portable C89 code to get at all that
stuff in all cases, then (a) fine; (b) they wouldn't be the first to
try it; and (c) they would be the first not to give up <wink>.

The implementation of the new decimal module does arithmetic entirely
via Python-coded emulation, so all its behavior is under our control.
As a result, it fully conforms, on all platforms, to the proposed IBM
standard for decimal arithmetic, including the IEEE 854 subset wrt
exceptional semantics, values (infs, NaNs, etc), status flags, and
trap-enable flags. Also as a result, it may run about 100x slower
than your binary fp HW.