Xios in ORCHIDEE

The interface for XIOS is implemented in ORCHIDEE/trunk rev ​rev 1788 for usage of ORCHIDEE offline.
This page is about the on going work with XIOS in ORCHIDEE. The work with XIOS is done directly in the trunk of ORCHIDEE. Here are some notes during implementation and testing phase. Note ! All tests for now have only been done with ORCHIDEE offline mode. The implementation in ORCHIDEE will need to change when adapting to run with LMDZ.

Validation

The implementation of XIOS interface in ORCHIDEE is uder validation work. See details here : Branches/Xios/Validation

How to install ORCHIDEE with XIOS for offline use : at CURIE

• Extract

  > svn co http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/trunk modipsl
  > cd modipsl/util
  
  Add extraction of XIOS by adding following line in mod.def : use XIOS revision 446, latest revision 452 of XIOS do not work with ORCHIDEE, at curie :
  #-C- ORCHIDEE_trunk  XIOS/trunk                               446     12 XIOS               modeles
  
  > ./model ORCHIDEE_trunk
  

• Adapt compiling with netcdf parallel

  # Change netcdf in modipsl/util/AA_make.gdef to following :
  
  #-Q- curie  NCDF_INC = /usr/local/netcdf-4.2_hdf5_parallel/include
  #-Q- curie  NCDF_LIB = -L/usr/local/netcdf-4.2_hdf5_parallel/lib -lnetcdff -lnetcdf -lstdc++
  
  cd ../modeles/ORCHIDEE/arch
  # Change netcdf in arch-X64_CURIE.path to following : 
  NETCDF_LIBDIR="/usr/local/netcdf-4.2_hdf5_parallel/lib -lnetcdff -lnetcdf -lstdc++"
  NETCDF_INCDIR=/usr/local/netcdf-4.2_hdf5_parallel/include
  
  # Regenerate makefiles
  cd ../../../util
  ./ins_make
  

• Compile using fcm

  > cd ../config/ORCHIDEE_OL
  > gmake with_xios
  

• Lance
  • The executable can now be used with XIOS or with IOIPSL or with both at the same time.
    • Activate XIOS by adding XIOS_ORCHIDEE_OK=y
    • Deactivate IOIPSL by setting WRITE_STEP=0 and STOMATE_HIST_DT=0 in run.def
  • This executable must be used with netcdf and hdf5 parallel, add in the job just before exexcution following lines :

    module unload netcdf hdf5
    module load netcdf/4.2_hdf5_parallel
    module load hdf5/1.8.9_parallel
    

  • The executable can be launched in
    • attached mode : this means that the server XIOS is not used. You only launches the orchidee_ol executable.
    • using the server xios : in this case, one or several cores are dedicated to the server and the reste to orchiee_ol.

• Lance using libIGCM
  • xios files can not have suffix _xios if IOIPSL output is activated as standard. Instead change to use .xios
  • if using multiple_file, add min_digits="4" on the same line in iodef.xml. This can be kept even for one_file :

    <file_definition type="one_file" par_access="collective" enabled=".TRUE." min_digits="4">
    

• Rebuild
  • If using file_definition type="multiple_file" then rebuild must be done. Use this rebuild : /ccc/cont003/home/dsm/p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild. For exemple :

    /ccc/cont003/home/dsm/p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -o sechiba_history_xios.nc sechiba_history_xios_*
    

  • If using file_definition type="one_file" then no rebuild is needed.

• Visulize netcdf output
  • ferret : compatible to all netcdf versions, IOIPSL and XIOS output
  • nco : all versions adapted for IOIPSL output. To vizualize XIOS output, you need to have loaded netcdf and hdf5 parallel.
  • cdo : all versions adapted for IOIPSL output. To vizualize XIOS output, you need to have loaded netcdf and hdf5 parallel and also a newer version of cdo.
  • Load following modules to use interactivly these tools :

    module unload netcdf hdf5 cdo
    module load netcdf/4.2_hdf5_parallel
    module load hdf5/1.8.9_parallel
    module load cdo/1.6.0
    

Note : Sometimes following error message appears when linking

ar: creating /ccc/work/cont003/dsm/p86ghatt/XIOS/Validation/modipsl/modeles/ORCHIDEE/tmp_src/lib__fcm__dim2_driver.a
mpif90 -o dim2_driver.exe /ccc/work/cont003/dsm/p86ghatt/XIOS/Validation/modipsl/modeles/ORCHIDEE/.config/obj/driver.o -L/ccc/work/cont003/dsm/p86ghatt/XIOS/Validation/modipsl/modeles/ORCHIDEE/lib -l__fcm__dim2_driver -i4 -r8 -auto -L/usr/local/Intel_compilers/c/composer_xe_2011_sp1.9.293/mkl/lib/intel64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lm -O3 -fp-model precise -L/usr/local/netcdf-4.2_hdf5_parallel/lib -lnetcdff -lnetcdf -lstdc++ -L/ccc/work/cont003/dsm/p86ghatt/XIOS/Validation/modipsl/modeles/ORCHIDEE/../../lib -L/ccc/work/cont003/dsm/p86ghatt/XIOS/Validation/modipsl/modeles/ORCHIDEE/../XIOS/lib -lxios -lioipsl -lnetcdf
/usr/local/netcdf-4.2_hdf5_parallel/lib/libnetcdf.so: undefined reference to `H5Pset_dxpl_mpio'
/usr/local/netcdf-4.2_hdf5_parallel/lib/libnetcdf.so: undefined reference to `H5Pset_fapl_mpio'
/usr/local/netcdf-4.2_hdf5_parallel/lib/libnetcdf.so: undefined reference to `H5Pset_fapl_mpiposix'
fcm_internal load failed (256)
gmake[1]: *** [dim2_driver.exe] Error 1

Solution, load modules as in XIOS :

module unload netcdf
module unload hdf5
module load netcdf/4.2_hdf5_parallel
module load hdf5/1.8.9_parallel

Implementation

In the definition phase using IOIPSL, in the call to histdef, an operator is declared on the field. Using XIOS, some of these operators will be set in the xml file (which contains the definition phase) and some will be applied to the variables in the call to xios_orchidee_send_field.

operation in histdef	terme in xios_orchidee_send_field	operation in xml
flux_op	one_day/dt	-
flux_scinsec	1/dt (*)	-
ave	-	average
once	-	once
tmaxcels	-	maximum
tmincels	-	minimum
sumscatter	-	accumulate

(*) 1/dt is replaced by 0.0005555556 to get the same results as with IOIPSL. This is also commited temporary in revision [1788]. It will soon be replaced by 1/dt_sechiba.