How to print floating point in scientific format?

[sdhyok]

I want to change the DEFAULT behavior of python
to print out all floating points in scientific format?
For instance,

x=0.01
print x

1.000000E-2 #Like print "%E"%x

How can I do it?

Daehyok Shin

 

[Ben Finney]

I want to change the DEFAULT behavior of python

Then you can modify the Python source to do whatever you want. This is
a bad idea for your current stated purpose, though.

Changing the default behaviour of a native Python type will only be
local to machines that have this altered Python; if you write code that
depends on that, it will break unexpectedly on other machines.

to print out all floating points in scientific format?
For instance,

1.000000E-2 #Like print "%E"%x

How can I do it?

Since changing the way native flot object operate is a bad idea, two
ways seem reasonable.

One is to always use output formatting, since it will make your code
clearer.
1.000000E-02

The other way is to subclass the 'float' class to overload its __str__
method. This gives you objects that default to what you want, without
breaking Python.
... """ A floating-point number class
... that defaults to scientific representation
... """
... def __init__( self, *args, **kwargs ):
... float.__init__( self, args, kwargs )
... return
... def __str__( self ):
... return "%E" % self
... ... x = SciFloat( 0.01 )
... print x
...
1.000000E-02

 

[sdhyok]

I am disappointed that there is no elegant solution for this problem
in python.
As you may notice, my primary usage of python is for scientific
programming.
For the purpose, it is critical for users to flexibly modify the
printing format
of floating points depending on their ranges and precisions.

Think about this case.

import Numeric as N
x = N.arange(10.)
print x

[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]

print "%E"%x

TypeError: float argument required

Even the format string option in print command doesn't work for numpy
array.
This limitation can be a serious handicap in the application of python
for scientific programming.
I like to listen to all of you who are interested in this issue.

Shin, Daehyok

 

[Cousin Stanley]

The conversion to e format in the print statement
seems to work OK if I iterate over the items in the array ....

import Numeric as N

x = N.arange( 10. )

print x [ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]

for this_item in x :

.... print '%e' % this_item
....
0.000000e+000
1.000000e+000
2.000000e+000
3.000000e+000
4.000000e+000
5.000000e+000
6.000000e+000
7.000000e+000
8.000000e+000
9.000000e+000

 

[Alex Martelli]

I want to change the DEFAULT behavior of python
to print out all floating points in scientific format?
For instance,

1.000000E-2 #Like print "%E"%x

How can I do it?

You need to download Python's sources, modify them, and build a modified
Python interpreter and libraries that impose the behavior you want rather
than what the normal Python interpreter and libraries do.

Specifically, look at function format_float, line 233 of file
Objects/floatobject.c in the current 2.3 maintenance branch for example.
Currently it formats the float with "%.*g" [variable precision passed
in as an argument to format_float]; if you want to use a different C level
format string, that string is the one you need to change.

Of course, that's likely to break some of the tests in Python's unit-tests
suite, so you'll probably want to modify those, too. And then, you get to
maintain your "slightly divergent" Python yourself forevermore. I do not
think there is much likelihood that a patch in this regard would be
accepted in the Python core, even if you made it flexible enough to keep
the current behavior by default and change it only upon specific request
(e.g., use a variable string for the format, and let the Python coder
modify the letter in it, only, keeping "%.*g" as the default but letting
the 'g' be changed) -- such "big global" settings, which would let one
idiosyncratic library module modify Python behavior enough to break other
innocent modules, are looked at with disfavour, for reasons that should
be obvious (each and every such 'big global' _damages_ Python's suitability
for writing very large, multi-authors applications -- that suitability is
currently very high, and _extremely_ convincing arguments would need to
be brought to bear in order to convince Guido to deliberately lower it).


Alex

 

[Andrew Dalke]

sdhyok:

As you may notice, my primary usage of python is for scientific
programming.
For the purpose, it is critical for users to flexibly modify the
printing format
of floating points depending on their ranges and precisions.

Python gives complete control over you can represent numbers.
It just doesn't change the default representation. Users *can*
flexibly modify the printing format.

I can't see why you would want to change it globally. That
just sounds wrong. Eg, I want my app to print the resolution
in Angstroms in the form "%3.2f" while I want the mass values
in "%3.1f". A global setting is a very clumsy way to change that.

This limitation can be a serious handicap in the application of python
for scientific programming.
I like to listen to all of you who are interested in this issue.

I do scientific programming. The lack of specialized formatting
hasn't bothered me at all.

Andrew
(e-mail address removed)

 

[sdhyok]

Cousin, you are right.
But, do you want to write for loops
whenever you print arrays?
Maybe not if you are a hard-core scientific programmer.

The conversion to e format in the print statement
seems to work OK if I iterate over the items in the array ....

import Numeric as N

x = N.arange( 10. )

print x [ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]

for this_item in x :

... print '%e' % this_item
...
0.000000e+000
1.000000e+000
2.000000e+000
3.000000e+000
4.000000e+000
5.000000e+000
6.000000e+000
7.000000e+000
8.000000e+000
9.000000e+000

 

[sdhyok]

As you recommend, I won't modify the default behavior of python.
But, still I need a better treatment of python on arrays.

Daehyok Shin

I want to change the DEFAULT behavior of python
to print out all floating points in scientific format?
For instance,

1.000000E-2 #Like print "%E"%x

How can I do it?

You need to download Python's sources, modify them, and build a modified
Python interpreter and libraries that impose the behavior you want rather
than what the normal Python interpreter and libraries do.

Specifically, look at function format_float, line 233 of file
Objects/floatobject.c in the current 2.3 maintenance branch for example.
Currently it formats the float with "%.*g" [variable precision passed
in as an argument to format_float]; if you want to use a different C level
format string, that string is the one you need to change.

Of course, that's likely to break some of the tests in Python's unit-tests
suite, so you'll probably want to modify those, too. And then, you get to
maintain your "slightly divergent" Python yourself forevermore. I do not
think there is much likelihood that a patch in this regard would be
accepted in the Python core, even if you made it flexible enough to keep
the current behavior by default and change it only upon specific request
(e.g., use a variable string for the format, and let the Python coder
modify the letter in it, only, keeping "%.*g" as the default but letting
the 'g' be changed) -- such "big global" settings, which would let one
idiosyncratic library module modify Python behavior enough to break other
innocent modules, are looked at with disfavour, for reasons that should
be obvious (each and every such 'big global' _damages_ Python's suitability
for writing very large, multi-authors applications -- that suitability is
currently very high, and _extremely_ convincing arguments would need to
be brought to bear in order to convince Guido to deliberately lower it).


Alex

 

[Carl Banks]

As you recommend, I won't modify the default behavior of python.
But, still I need a better treatment of python on arrays.

Continually repeating your question isn't going to make a better
treatment magically appear.

Just use the loop method someone suggested.

If you don't like writing a for loop every time, put it in a function.

 

[Cousin Stanley]

| Cousin, you are right.
| But, do you want to write for loops
| whenever you print arrays?
|
| Maybe not if you are a hard-core scientific programmer.

Cousin sdhyok ....

I believe that the nature of science and the numerical tools
that it employs for problem solving is highly context dependent
upon the particular problem at hand and that it might be difficult
to find a one-size-fits-all solution for array printing since
the nature of the arrays that are employed and their content
can vary greatly from context to context ....

The dimensonality, homogeniety, and population density
of the arrays employed must be considered as design parameters
in each problem space and treated accordingly ....

However, one can define specialized array functions,
print functions or otherwise, for specific problems
and use those without having to code the loops each time ....

I do have an extensive background in computational chemistry ....

Some would call that discipline hard-core science ....

Others would call it hand-waving in hopeless depths ....

 

[Bengt Richter]

I want to change the DEFAULT behavior of python
to print out all floating points in scientific format?
For instance,

1.000000E-2 #Like print "%E"%x

How can I do it?

You need to download Python's sources, modify them, and build a modified
Python interpreter and libraries that impose the behavior you want rather
than what the normal Python interpreter and libraries do.

Specifically, look at function format_float, line 233 of file
Objects/floatobject.c in the current 2.3 maintenance branch for example.
Currently it formats the float with "%.*g" [variable precision passed
in as an argument to format_float]; if you want to use a different C level
format string, that string is the one you need to change.

Of course, that's likely to break some of the tests in Python's unit-tests
suite, so you'll probably want to modify those, too. And then, you get to
maintain your "slightly divergent" Python yourself forevermore. I do not
think there is much likelihood that a patch in this regard would be
accepted in the Python core, even if you made it flexible enough to keep
the current behavior by default and change it only upon specific request
(e.g., use a variable string for the format, and let the Python coder
modify the letter in it, only, keeping "%.*g" as the default but letting
the 'g' be changed) -- such "big global" settings, which would let one
idiosyncratic library module modify Python behavior enough to break other
innocent modules, are looked at with disfavour, for reasons that should
be obvious (each and every such 'big global' _damages_ Python's suitability
for writing very large, multi-authors applications -- that suitability is
currently very high, and _extremely_ convincing arguments would need to
be brought to bear in order to convince Guido to deliberately lower it).

OTOH, why not a sys.printhook ? There is a kind of precedent in
... if isinstance(obj, (int, long, float)): return sys.__displayhook__('%e'%float(obj))
... else: return sys.__displayhook__(obj)
...(1, 2L, 3.0)

I didn't bother with recursive list and tuple mods. That would be up to the printhook user.
ISTM a printhook would be easy to implement and not disruptive overall, whatever silliness
it might inspire in particular individuals ;-)

Actually, I think there could even be some good use for it, since you could do things with
the object stream that you couldn't do by intercepting strings in sys.stdout.write.

Regards,
Bengt Richter

 

[Alex Martelli]

As you recommend, I won't modify the default behavior of python.
But, still I need a better treatment of python on arrays.

Arrays are supplied by extension modules, most popularly Numeric.

If you're doing scientific computations in Python and use arrays but
not Numeric (or its slated-to-be-replacement numarray, of which I
don't know much yet; or the many things layered atop of and onside
of Numeric, such as scipy), switch. You'll be happy you did.

I've never heard a scientific programmer complain too badly about
how Numeric treats arrays, whether they were coming from Fortran
(the typical case), other higher-level languages, or general purpose
languages not really suited for scientific computation (such as C).

You seem to be focused on readable display of arrays (quite an
unusual focus for scientific programming). So, worst case, it's trivial
to write a general purpose function that takes an array of ANY size
and rank and emits it in the way you prefer -- easier than in any
other language commonly used for scientific programming.

So, you don't need anything from Python -- just write that blessed
function and scientifically program to your heart's content.


Alex

 

[David M. Cooke]

You seem to be focused on readable display of arrays (quite an
unusual focus for scientific programming). So, worst case, it's trivial
to write a general purpose function that takes an array of ANY size
and rank and emits it in the way you prefer -- easier than in any
other language commonly used for scientific programming.

With Numeric at least, you can set globally the functions used for
str() and repr() on arrays, giving exactly this, without requiring a
separate function for representations. Have a look at Numeric.py,
multiarray.set_string_function(), and ArrayPrinter.py from the Numeric
distribution.

The simple way of controlling output with Numeric is to use the
variables sys.output_line_width, sys.float_output_precision and
sys.float_output_supress_small (if true, replace in the output numbers that
are much smaller than the rest with zero, which is very nice when most of
your numbers are 1.0, and you've got some annoying 1.8342e-17 that you want
to ignore). These are added by Numeric to the sys module.

With numarray, you could subclass the array object, write your own
__str__ and __repr__ methods, and use that instead.

 

[Ben Finney]

Writing my own function is definitely one choice.
But, my point is that if some of us are really serious about
scientific/engineering programming, we must have a common function or
command to print out whole array elements easily with any format we
want.

We eagerly await your patches for improving the Numeric code in whatever
way you want.

 

[sdhyok]

Writing my own function is definitely one choice.
But, my point is that if some of us are really serious
about scientific/engineering programming,
we must have a common function or command
to print out whole array elements easily with any format we want.

It can be in Numeric (or numarray) package like,

import Numeric as N
N.print(array, format="%.2E")


Daehyok Shin

 

[sdhyok]

Thanks, Alex.

snip

If you're doing scientific computations in Python and use arrays but
not Numeric (or its slated-to-be-replacement numarray, of which I
don't know much yet; or the many things layered atop of and onside
of Numeric, such as scipy), switch. You'll be happy you did.

I refer all sequences, but, as you say, particularly numpy arrays.

I've never heard a scientific programmer complain too badly about
how Numeric treats arrays, whether they were coming from Fortran
(the typical case), other higher-level languages, or general purpose
languages not really suited for scientific computation (such as C).

Right, we can use scientific binary formats, like NetCDF, HDF.
But, we often want to read the data in our naked eyes
and exchange them with spreadsheet.

You seem to be focused on readable display of arrays (quite an
unusual focus for scientific programming). So, worst case, it's trivial
to write a general purpose function that takes an array of ANY size
and rank and emits it in the way you prefer -- easier than in any
other language commonly used for scientific programming.

So, you don't need anything from Python -- just write that blessed
function and scientifically program to your heart's content.

Trivial to create my own function.
But, surprisingly no common function or command for the work.
Maybe, one solution is

import Numeric as N
N.print(array, format="%.2E")


Daehyok Shin

 

[Alex Martelli]

sdhyok wrote:
...

Right, we can use scientific binary formats, like NetCDF, HDF.
But, we often want to read the data in our naked eyes
and exchange them with spreadsheet.

For "exchange with spreadsheets" see the csv module (new in 2.3), e.g.:

import csv
import sys
csv.writer(sys.stdout).writerow([ 100.+i for i in range(3,8) ]) 103.0,104.0,105.0,106.0,107.0

It doesn't, of course, force scientific format (it uses the general
format) -- spreadsheets don't need that. However, it does have the
concept of a "Dialect" (here, I've let the dialect default to 'excel'),
a bundle of formatting parameters that tweak the output into the best
form for a given spreadsheet; that doesn't provide the specific bit
of formatting control you want, but it gives what's clearly the right
"hook" on which to hang any kind of formatting control. If you need
to pursue "exchange with spreadsheet" and you have a use-case for
needing scientific format in such a setting, then proposing a patch
to module csv [having the dialect optionally record the preferred
way to format floating point numbers] might stand a good chance.

Right now, _csv.c [line 1126 in the sources) just transforms the
fields it's emitting with PyObject_Str(field) -- equivalent to the
Python call str(field), just as the print statement does. But just
as it nows checks dialect->quoting before coming to that, it might
well check some other field of the '*dialect* structure in order
to influence output formatting selectively -- *IF* good use cases
could be found and verified where a spreadsheet's behavior can be
made better by special formatting of numbers fed to it.

Trivial to create my own function.

We agree.

But, surprisingly no common function or command for the work.

Actually, Numeric.array2string IS exactly such a "common function
or command". It's been written 7 years ago by Konrad Hinsen and
happily used ever since by thousands of happy users of Numeric --
how much more common than this can you get?

However, among its very numerous formatting parameters, it does
NOT have one to let you force exponential format as opposed to
fixedpoint format -- it always decides that tidbit (i.e. whether
an 'e' or 'f' is ultimately used in the format string) by itself.

Clearly, no Numeric user has had substantial need for this kind
of forced output formatting over these many years -- which I do
not find at all surprising. Still, array2string is a Python
coded function -- see its sources in Numeric/ArrayPrinter.py --
so that experimenting with changes to it is trivial, and maybe
the Numeric maintainers will look with favour upon a patch,
should you decide to offer one. It might be as simple as adding
yet one more optional parameter to the array2string function,
say "forcedformat" defaulting to None, passing it on to the
underlying internal _floatFormat function it calls to determine
the format string, and having _floatFormat just use it, if not
None, instead of determining the format itself. I would
suggest you make a copy of ArrayPrinter.py, tweak it to your
own satisfaction, then send the resulting patch to the Numeric
maintainers proposing inclusion in Numeric's next release.

Maybe, one solution is

import Numeric as N
N.print(array, format="%.2E")

In my opinion, the solution I just sketched for you, i.e.:

print N.array2string(array, precision=2, forcedformat="E")

stands a much better chance than the addition of a new function
that does direct printing (rather than the more useful and
flexibile formatting-to-string) and merges precision and
formatcharacter into one argument (note that array2string
does already accept a precision argument) -- even without
considering the fact that 'print' is a keyword and thus not
usable as a function-name.


Alex