Using Numeric 24.0b2 with Scientific.IO.NetCDF

[bandw]

I am having a problem using Numeric-24.0b2 in conjunction with
the NetCDF module from ScientificPython (version 2.4.9).
This problem does not surface using Numeric-23.8. The problem
arises in using the "min" function on a NetCDF floating array.
In 23.8, the "min" function returns a floating scalar, while in
24.0b2 it returns an *array* of length "1". Below I list a
simple NetCDF file and a Python script that illustrate the
problem. When I run the script using 23.8, I get the result:

1.0 <type 'float'>

whereas using 24.0b2 I get:

1.0 <type 'array'>

This creates a backward incompatibility that breaks several of
my codes.

NetCDF file simple.cdl (used to create simple.nc with "ncgen")
--------------------------------------------------------------

netcdf simple {
dimensions:
num = 3 ;
variables:
float temp(num) ;
data:

temp = 1, 2, 3 ;
}


Python script
-------------

import Numeric
from Scientific.IO.NetCDF import NetCDFFile

cdf_file1 = NetCDFFile("simple.nc","r")
temp = cdf_file1.variables["temp"][:]

print min(temp), type(min(temp))

 

[Robert Kern]

I am having a problem using Numeric-24.0b2 in conjunction with
the NetCDF module from ScientificPython (version 2.4.9).
This problem does not surface using Numeric-23.8. The problem
arises in using the "min" function on a NetCDF floating array.
In 23.8, the "min" function returns a floating scalar, while in
24.0b2 it returns an *array* of length "1". Below I list a
simple NetCDF file and a Python script that illustrate the
problem. When I run the script using 23.8, I get the result:

1.0 <type 'float'>

whereas using 24.0b2 I get:

1.0 <type 'array'>

This creates a backward incompatibility that breaks several of
my codes.

Call float(temp) if you really need a Python float. The change was
intentional such that A would always be an array regardless of the
shape of A. This greatly simplifies certain types of code although the
change does have its transition costs for some specific pieces of older
code like yours.

BTW, you don't want to use the builtin min(). That iterates over the
array as if it were a Python list. Use minimum.reduce().

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[bandw]

Robert,

Thanks for your reply. However, I am still having problems. Sometimes
I get a scalar return
and sometimes I get an array. For example, using the netCDF file:

netcdf simple {
dimensions:
num = 3 ;
variables:
float temp0(num) ;
int temp1(num) ;
data:

temp0 = 1., 2., 3. ;
temp1 = 1, 2, 3 ;
}

and running:

#
import Numeric
print Numeric.__version__
from Scientific.IO.NetCDF import NetCDFFile

cdf_file1 = NetCDFFile("simple.nc","r")

var1 = cdf_file1.variables["temp0"][:]
var2 = cdf_file1.variables["temp1"][:]
min1 = reduce(Numeric.minimum,var1)
min2 = reduce(Numeric.minimum,var2)

print "Types of var1, min(var1), min1:",type(var1), type(min(var1)),
type(min1)
print "Types of var2, min(var2), min2:",type(var2), type(min(var2)),
type(min2)

I get:

24.0b2
Types of var1, min(var1), min1: <type 'array'> <type 'array'> <type
'array'>
Types of var2, min(var2), min2: <type 'array'> <type 'int'> <type
'int'>

Even something like:

import Numeric
a = Numeric.array([1.,2.])
print type(a),type(min(a))

<type 'array'> <type 'float'>

does not produce an array.

Any comments woud be appreciated.

Fred Clare

 

[Robert Kern]

Robert,

Thanks for your reply. However, I am still having problems. Sometimes
I get a scalar return
and sometimes I get an array. For example, using the netCDF file:

netcdf simple {
dimensions:
num = 3 ;
variables:
float temp0(num) ;
int temp1(num) ;
data:

temp0 = 1., 2., 3. ;
temp1 = 1, 2, 3 ;
}

and running:

#
import Numeric
print Numeric.__version__
from Scientific.IO.NetCDF import NetCDFFile

cdf_file1 = NetCDFFile("simple.nc","r")

var1 = cdf_file1.variables["temp0"][:]
var2 = cdf_file1.variables["temp1"][:]
min1 = reduce(Numeric.minimum,var1)
min2 = reduce(Numeric.minimum,var2)

print "Types of var1, min(var1), min1:",type(var1), type(min(var1)),
type(min1)
print "Types of var2, min(var2), min2:",type(var2), type(min(var2)),
type(min2)

I get:

24.0b2
Types of var1, min(var1), min1: <type 'array'> <type 'array'> <type
'array'>
Types of var2, min(var2), min2: <type 'array'> <type 'int'> <type
'int'>

Even something like:

import Numeric
a = Numeric.array([1.,2.])
print type(a),type(min(a))

<type 'array'> <type 'float'>

does not produce an array.

Hmm, odd. Anyways, follow my advice: use minimum.reduce() and wrap
results in float() or array() if you really need floats or rank-0 arrays.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[bandw]

Thanks again. I will take your advice. My concern is in not knowing
where in all
my python code I am assuming a scalar return in certain circumstances.
But I
guess I can take care of the errors as they come up.

Fred

 

[bandw]

I am having more problems with 24.0b2. Consider the NetCDF file:

netcdf very_simple {
dimensions:
num = 2 ;
variables:
float T(num) ;
T:mv = 5.0f ;
data:
T = 1., 2. ;
}

and the python script:

import Numeric
from Scientific.IO.NetCDF import NetCDFFile
file = NetCDFFile("simple.nc","r")
T = file.variables["T"]

a = T.mv
print "T.mv = ", a
print "type(T.mv) = ", type(a)
print "len(T.mv) = ", len(a)
print "T.mv[0] = ", a[0]
print "len(T.mv[0]) = ", len(a[0])
print "type(T.mv[0]) = ", type(a[0])

which produces the output:

T.mv = [ 5.]
type(T.mv) = <type 'array'>
len(T.mv) = 1
T.mv[0] = 5.0
len(T.mv[0]) = 1
type(T.mv[0]) = <type 'array'>

I can see no reason why T.mv[0] should be typed as an array.