How to manipulate/compare/examine netcdf files using cdo or nco?

Author: S. Luyssaert and J. Ryder
Last revision: 2020/02/28, B. Guenet

nco

nco is a very powerful tool to print data, modify variable names, concatenate, etc on netcdf file. A detailled documentation can be found here ​http://nco.sourceforge.net/ We will just summarize here some of the most useful nco commands often used in the orchidee context.

• ncdump

This command print all the informations stored in your netcdf file. Some prefix can extract only the information you need.

• To look at header information:

ncdump -h myJob_20060101_20061231_1M_sechiba_history.nc

• To extract one variable (here for 'fluxlat', the latent heat flux)

ncdump -v fluxlat myJob_20060101_20061231_1M_sechiba_history.nc 

• To extract one variable and show them in a tabulated way (here for 'fluxlat', the latent heat flux)

ncdump -v fluxlat -f fortran -n fluxlat myJob_20060101_20061231_1M_sechiba_history.nc 

• ncrename

This command change the name of a given variable/dimension depending on the prefix you are using

• To rename a variable within the file (here for 'fluxlat', the latent heat flux)

  ncrename -v fluxlat,latent_heat_flux myJob_20060101_20061231_1M_sechiba_history.nc 
  

• To rename a variable a creating a new file (here for 'fluxlat', the latent heat flux)

  ncrename -v fluxlat,latent_heat_flux myJob_20060101_20061231_1M_sechiba_history.nc mynewfile.nc
  

• To rename a dimension within the file (here for 'lon', the longitude)

  ncrename -d lon,longitude myJob_20060101_20061231_1M_sechiba_history.nc 
  

• To rename a variable a creating a new file (here for 'lon', the longitude)

  ncrename -d lon,longitude myJob_20060101_20061231_1M_sechiba_history.nc mynewfile.nc
  

• ncks

If you would like to remove a certain time series from a netcdf file (for instance, in this example, all data from 'temp_atmos_pres' in the file jamescdf.nc for which the height dimension is 20) with the output written to jamestest.nc

ncks -H -d height,20 -d temp_atmos_pres,: jamescdf.nc jamestest.nc

• ncap2

The most powerful and quickly evolving tool from the nco set of functions

search what you are looking on this growing list of answers ​forum nco

A simple example showing different capacities (creation of a variable, use of a mask, call to an attribute, count the total and the size of the field (we can restrict this operation to a dimension as shown on the second line)

 ncap2 -O -s 'missing_flag[$y,$x]=0;where(t2m_daily!=t2m_daily@missing_value) missing_flag=1;missing_count=float(missing_flag.total())/missing_flag.size();print(missing_count)' stomate_restart.nc  ~/foo.nc
 ncap2 -O -s 'missing_flag[$y,$x]=0;where(t2m_daily!=t2m_daily@missing_value) missing_flag=1;missing_count=float(missing_flag.total($x))/missing_flag.size($x);print(missing_count)' stomate_restart.nc  ~/foo.nc

• ncatted

Sometimes, one just need to add the attribute coordinates (linking to the variables defining the longitude and latitude positions) to enable the recognition of the grid that is used. In that cases the ncatted command might be useful

   ncatted -a coordinates,my_var,o,c,"nav_lon nav_lat" sst_data.nc

or to correct a wrong spelling of an attribute

   ncatted -a units,latitude,m,c,"degrees_north" force2002.nc

cdo

An alternative to nco is cdo. They are basically doing similar things but dependending on what you want to do the command line might be easier with one or another. Details on cdo are here ​https://code.mpimet.mpg.de/projects/cdo cdo is useful for :

• changing of time axis
• regridding (many grid are already implemented and it is an even more versatile tool)
• complex operations...

Below a list of classical command:

• print list of available operators
  

     cdo -v 
  

• print the version you are using (if very old, think of downloading the source and install a new version, as it is a tool that is evolving quickly)

     cdo -V
  

• more info on the file

     cdo sinfov my_file.nc
  

• grid description

     cdo griddes my_file.nc 
  

• print some information about the command remapcon

     cdo -h remapcon
  

• an example of a conservative regridding to an other resolution

     cdo remapcon,my_grid my_climatology.nc my_regridded_climatology.nc 
  

my_grid could be a file like :

   gridtype  = lonlat
   gridsize  = 220
   xname     = longitude
   xunits    = degrees_east
   yname     = latitude
   yunits    = degrees_north
   xsize     = 20
   ysize     = 11
   xfirst    = -18.75
   xinc      = 3.75
   yfirst    = 20

Operator chaining

It is one of the main features of CDO. Use it as But operators with a arbitrary list of input files cannot be combined with other operators

Simple combination:

   cdo sub -dayavg ifile2 -timavg ifile1 ofile

instead of :

  cdo timavg ifile1 tmp1

  cdo dayavg ifile2 tmp2

  cdo sub tmp2 tmp1 ofile

  rm tmp1 tmp2

CDO can also be used to regrid NetCDF files. It is very straight and fast. Contrary to Ferret I [SL] did not find how to increase the memory allocation for CDO. Thus for very large files Ferret seems to be the software of your choice.

The call has the following structure:

cdo -f nc -regriddingMethod,gridTemplate.nc input.nc output.nc

There are three regridding methods: remapbil (bilinear interpolation), remapbic (bicubic interpolation), remapnn (nearest neighbour), remapcon (conservative regridding). Use remapnn and remapcon to avoid weird values if your variable is heterogeneous.

For the template file with your target grid, it can be any netcdf with regular lat/lon. A trick here is to use "full paths" and not relative paths to the files in order to work.

Some examples:

cdo -f nc -remapbil,/gpfs/cru/rst08auu/code/target.0.5degree.gpp.grid.45N.nc esa.ecv.smo.0.25deg.grid.monthly.1979.2010.nc esa.ecv.smo.0.25deg.grid.monthly.1979.2010.cdo.remapped.0.5deg.remapbil.45N.nc
cdo -f nc -remapnn,/gpfs/cru/rst08auu/code/target.0.5degree.gpp.grid.45N.nc esa.ecv.smo.0.25deg.grid.monthly.1979.2010.nc esa.ecv.smo.0.25deg.grid.monthly.1979.2010.cdo.remapped.0.5deg.remapnn.45N.nc

ncview: a quick alternative

The quickest way of all to look at NetCDF files and to quickly plot some outputs is to use ncview:

ncview myJob_20060101_20061231_1M_sechiba_history.nc

Unfortunately the ncview application can be sometimes a little unstable.