netCDF4 variable manipulation

[Sheldon]

Hi,

I'm trying to read a netCDF4 variable from a file (no problem) and
then scale it before writing over the original variable in the file.

I'm using python 2.7 and the latest netCDF4 module. It's not that I
keep getting an error message but I want to do this without using for
loops and all the manuals seems to be skipping this task as if it is
never done. I'm new to python and coming over from matlab. Does anyone
know how to modify the netCDF4 variable in python and then write it
back without creating a new variable, or using for loops?

Appreciate the help,
/S

 

[Dave Angel]

Hi,

I'm trying to read a netCDF4 variable from a file (no problem) and
then scale it before writing over the original variable in the file.

I'm using python 2.7 and the latest netCDF4 module. It's not that I
keep getting an error message but I want to do this without using for
loops and all the manuals seems to be skipping this task as if it is
never done. I'm new to python and coming over from matlab. Does anyone
know how to modify the netCDF4 variable in python and then write it
back without creating a new variable, or using for loops?

Appreciate the help,
/S

Are you using netcdf version 4, or is that name just a coincidence? If
you're using a 3rd party library, you really should say so, and post a
link, for those curious.

in any case, if you are, I can't help you. This is apparently some
special kind of variable similar to a numpy array.


On the other hand, you may just be asking how to update some bytes in a
file, in place. if you read the file in binary mode, then you can use
seek() to go back the appropriate distance, and simply write it.

 

[Steven D'Aprano]

Hi,

I'm trying to read a netCDF4 variable from a file (no problem) and then
scale it before writing over the original variable in the file.

I'm using python 2.7 and the latest netCDF4 module. It's not that I keep
getting an error message but I want to do this without using for loops
and all the manuals seems to be skipping this task as if it is never
done. I'm new to python and coming over from matlab. Does anyone know
how to modify the netCDF4 variable in python and then write it back
without creating a new variable, or using for loops?

There is no such thing as "the netCDF4 variable" in Python -- it is not a
built-in part of the language, and therefore there is no built-in feature
for manipulating it.

You are going to have to be more specific about what you want than just
"how do I modify a variable with for loops?".

My wild guess is that you have some sort of config file which includes a
field called "netCDF4" and you want to update it in place. We can't tell
you how to do this without knowing what the config file is -- is it an
INI file, XML, JSON, YAML, Unix-style rc file, a binary pickle, or
something else?

 

[Sheldon]

There is no such thing as "the netCDF4 variable" in Python -- it is not a
built-in part of the language, and therefore there is no built-in feature
for manipulating it.

You are going to have to be more specific about what you want than just
"how do I modify a variable with for loops?".

My wild guess is that you have some sort of config file which includes a
field called "netCDF4" and you want to update it in place. We can't tell
you how to do this without knowing what the config file is -- is it an
INI file, XML, JSON, YAML, Unix-style rc file, a binary pickle, or
something else?

Hi,

My apologies. I didn't realize this module was not popular. I'm using
the python module "netCDF4"
that I installed using PIP. This module gives me the ability to read
and write netcdf4 files.
The module is an interface to the netcdf4.1.3 and HDF51.8.8 libraries.
What I'm trying to do is open an existing netcdf file, extract an
variable, modify the variable, and then
write it again to the file - effectively replacing the old variable.

rgrp = netCDF4.Dataset('ODIN_NWP_2001_01_01_00.NC','r+')
pv = rgrp.groups['Data_3D'].variables['PV']

print pv
<type 'netCDF4.Variable'>
float32 PV(u'level', u'lat', u'lon')
longname: Potential Vorticity
_FillValue: -999.0
units: K m**2 kg**-1 s**-1
code: 60
scale_factor: 1.0
add_offset: 0.0
path = /Data_3D
unlimited dimensions = ()
current size = (60, 181, 360)

As you can see, pv is described here as a netCDF4.Variable. I'm trying
to modify this pv and then write it back to the file
but the manual is vague on this part. I tried this:

pv = pv*2
---------------------------------------------------------------------------
TypeError Traceback (most recent call
last)
----> 1 pv = pv*2
TypeError: unsupported operand type(s) for *: 'netCDF4.Variable' and
'int'

I'm unsure how this is suppose to work.

/S

 

[Chris Rebert]

My apologies. I didn't realize this module was not popular. I'm using
the python module "netCDF4"
that I installed using PIP. This module gives me the ability to read
and write netcdf4 files.
The module is an interface to the netcdf4.1.3 and HDF51.8.8 libraries.
What I'm trying to do is open an existing netcdf file, extract an
variable, modify the variable, and then
write it again to the file - effectively replacing the old variable.

rgrp = netCDF4.Dataset('ODIN_NWP_2001_01_01_00.NC','r+')
pv = rgrp.groups['Data_3D'].variables['PV']

print pv
<type 'netCDF4.Variable'>
float32 PV(u'level', u'lat', u'lon')
   longname: Potential Vorticity
   _FillValue: -999.0
   units: K m**2 kg**-1 s**-1
   code: 60
   scale_factor: 1.0
   add_offset: 0.0
path = /Data_3D
unlimited dimensions = ()
current size = (60, 181, 360)

As you can see, pv is described here as a netCDF4.Variable. I'm trying
to modify this pv and then write it back to the file
but the manual is vague on this part. I tried this:

pv = pv*2
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call
last)
----> 1 pv = pv*2
TypeError: unsupported operand type(s) for *: 'netCDF4.Variable' and
'int'

I'm unsure how this is suppose to work.

The API docs suggest that Variables don't support any arithmetic
operations, just slicing and attribute manipulation. They seem to
interoperate with numpy arrays though, so presumably you could avoid
explicit looping by loading the Variable's contents into a numpy
array, doing your calculations on that, and then assigning the
resulting numpy array back into a slice of the Variable.

Cheers,
Chris