wiki:DevelopmentActivities/Branches/ORCHIDEE-MICT-IMBALANCE-P/Compile

Context Navigation

How to compile MICT
1. modipsl/libIGCM
2. Libraries
Modules
Guide
1. Obelix
Old guide (do not use)
1. IOIPSL
  1. After 8.4.0 (rev 4057)
  2. Before 8.4.0 (rev 4057)
2. ORCHIDEE-MICT
Compilation
1. Automatic
2. Manual
Debug
Notes
1. Supported machines
Errors

How to compile MICT

Be aware there other ways to compile Orchidee. All of them are valid. But the use of makeorchidee_fcm should make things easier:

It enables the compilation with multiple cores at the same time.
No need to modify any file to change the compilation mode (prod/dev/debug).
Automatically manages all dependencies between files.

modipsl/libIGCM

Despite of having all files inside the modipsl/libIGCM structure. This guide does not guarantee it will work with this framework. For this purpose, go to the appropiate guide.

Libraries

Since MICT V7.2, XIOS 2 is enabled on MICT. This is a new library to manage the output for history files. It is recommended to use over the old one ( IOIPSL). For several reason:

It is much faster (https://forge.ipsl.fr/orchidee/wiki/DevelopmentActivities/Branches/ORCHIDEE-MICT-IMBALANCE-P/SimulationTimes#TrunkvsMICTComparision18022016)
No need for output history rebuild
IOIPSL unbalance on Orchidee computation time. All processors except for the last one (always) need to wait. Because it deals with more data (due to sea gridcells).
IOISPL max number of processors is limited to the number of latitude pixels
- XIOS has no limitation

Modules

Required modules to load in your environment to make Orchidee and its dependencies -IOIPSL and XIOS- properly compile and run. Take into account it is always required to use the same modules during compilation as well as at a running time.

On Irene

module purge
module load intel/17.0.4.196
module load mpi/openmpi/2.0.2
module load flavor/hdf5/parallel
module load netcdf-fortran/4.4.4
module load mkl
module load hdf5/1.8.20
module load feature/bridge/heterogenous_mpmd
module load nco
module load cdo
module load ncview

On obelix

module load intel-fcomp/2018
module load openmpi/2.1.5
module load pnetcdf/1.5.0
module load hdf5/1.10.5
module load netcdf/4.7

Do NOT use the modules below.

python/* -> it crashes orchidee

Old configuration

module load intel-fcomp/2018
module load openmpi/2.1.5
module load pnetcdf/1.5.0
module load hdf5/1.8.9p
module load netcdf/4p

known issues

Since the introduction of the new module openmpi/2.1.5 a new bug raised in one of the libraries listed below for Orchidee-MICT only. It prevents the writing of the stomate_Cforcing_permafrost.nc under certain cases (e.g: 2nodes + 4 cores each). So it is not possible to use forcesoil anymore:

Affected modules:

netcdf/4p
hdf5/1.8.9p

On curie

module load intel/16.0.3.210
module load netcdf/4.3.3.1_hdf5_parallel
module load hdf5/1.8.9_parallel

Guide

This guide will help you install Orchidee-MICT in a faster way than the old guide. Follow the steps below:

Download Modipsl

svn co http://forge.ipsl.fr/igcmg/svn/modipsl/trunk modipsl

Get into your new download modipsl folder

cd modipsl

Modify util/AA_make.gdef (diff notation)

[ajornet@obelix3 util]$ svn diff
Index: AA_make.gdef
===================================================================
--- AA_make.gdef	(révision 4740)
+++ AA_make.gdef	(copie de travail)
@@ -379,8 +379,8 @@
 #-Q- lxiv8    C_O =
 #-Q- lxiv8    C_L = cc
 #-Q- lxiv8    #-
-#-Q- lxiv8    NCDF_INC = /usr/local/install/netcdf-4.3.2p/include 
-#-Q- lxiv8    NCDF_LIB = -L/usr/local/install/netcdf-4.3.2p/lib -lnetcdff -lnetcdf
+#-Q- lxiv8    NCDF_INC = $(shell nc-config --includedir)
+#-Q- lxiv8    NCDF_LIB = -L$(shell nc-config --libdir) -lnetcdff -lnetcdf
 #-Q- lxiv8    #-
 #-Q- occigen  #-
 #-Q- occigen  #- Global definitions for Curie at TGCC

Add the lines below into util/mod.def at the end of the file:

#---- Configurations for ORCHIDEE-MICT

#-H- ORCHIDEE_mict_latest  ORCHIDEE MICT offline configuration
#-H- ORCHIDEE_mict_latest  This is a working configuration using the latest revision of ORCHIDEE mict
#-M- ORCHIDEE_mict_latest  albert.jornet-puig@lsce.ipsl.fr
#-C- ORCHIDEE_mict_latest  branches/ORCHIDEE-MICT/IOIPSL            HEAD    14 IOIPSL             modeles
#-C- ORCHIDEE_mict_latest  trunk/libIGCM                            1487    10 libIGCM            .
#-C- ORCHIDEE_mict_latest  branches/ORCHIDEE-MICT/ORCHIDEE          HEAD    14 ORCHIDEE           modeles
#-C- ORCHIDEE_mict_latest  branches/ORCHIDEE-MICT/ORCHIDEE_OL       HEAD    14 ORCHIDEE_OL        config
#-C- ORCHIDEE_mict_latest  XIOS/branchs/xios-2.5                    1550    12 XIOS               modeles

#-H- ORCHIDEE_mict_tag891  ORCHIDEE MICT version 8.9.1 offline configuration
#-H- ORCHIDEE_mict_tag891  This is a working configuration using the version 8.9.1 of ORCHIDEE mict
#-M- ORCHIDEE_mict_tag891  albert.jornet-puig@lsce.ipsl.fr
#-C- ORCHIDEE_mict_tag891  branches/ORCHIDEE-MICT/IOIPSL                    HEAD    14 IOIPSL             modeles
#-C- ORCHIDEE_mict_tag891  trunk/libIGCM                                    1487    10 libIGCM            .
#-C- ORCHIDEE_mict_tag891  branches/ORCHIDEE-MICT/tags/ORCHIDEE_MICT_8.9.1  HEAD    14 ORCHIDEE           modeles
#-C- ORCHIDEE_mict_tag891  branches/ORCHIDEE-MICT/ORCHIDEE_OL               HEAD    14 ORCHIDEE_OL        config
#-C- ORCHIDEE_mict_tag891  XIOS/branchs/xios-2.5                            1550    12 XIOS               modeles

#-H- ORCHIDEE_mict_tag890  ORCHIDEE MICT version 8.9.0 offline configuration
#-H- ORCHIDEE_mict_tag890  This is a working configuration using the version 8.9.0 of ORCHIDEE mict
#-M- ORCHIDEE_mict_tag890  albert.jornet-puig@lsce.ipsl.fr
#-C- ORCHIDEE_mict_tag890  branches/ORCHIDEE-MICT/IOIPSL                    HEAD    14 IOIPSL             modeles
#-C- ORCHIDEE_mict_tag890  trunk/libIGCM                                    1487    10 libIGCM            .
#-C- ORCHIDEE_mict_tag890  branches/ORCHIDEE-MICT/tags/ORCHIDEE_MICT_8.9.0  HEAD    14 ORCHIDEE           modeles
#-C- ORCHIDEE_mict_tag890  branches/ORCHIDEE-MICT/ORCHIDEE_OL               HEAD    14 ORCHIDEE_OL        config
#-C- ORCHIDEE_mict_tag890  XIOS/branchs/xios-2.5                            1550    12 XIOS               modeles

Go to the "util" folder

cd util

Download the whole model by running the command (chose one):

Latest Orchidee MICT development:
```
./model ORCHIDEE_mict_latest
```

(preferred) Orchidee MICT version 8.9.1:
```
./model ORCHIDEE_mict_tag891
```

Inside the same folder -util/-, Install the Makefiles all across the model:

./ins_make

Obelix

This guide is meant to work for any module found in your environment in Obelix. For that reason, it requires some additional steps.

Go to XIOS

cd ../modeles/XIOS

Modify the file arch/arch-ifort_LSCE.path according to (diff notation):

[ajornet@obelix3 XIOS]$ svn diff
Index: arch/arch-ifort_LSCE.path
===================================================================
--- arch/arch-ifort_LSCE.path	(révision 1550)
+++ arch/arch-ifort_LSCE.path	(copie de travail)
@@ -1,6 +1,6 @@
-NETCDF_INCDIR="-I /usr/local/install/netcdf-4.3.2p/include"
-NETCDF_LIBDIR="-L/usr/local/install/netcdf-4.3.2p/lib"
-NETCDF_LIB="-lnetcdf -lnetcdff"
-HDF5_INCDIR="-I /usr/local/install/hdf5-1.8.9p/include"
-HDF5_LIBDIR="-L/usr/local/install/hdf5-1.8.9p/lib"
-HDF5_LIB="-lhdf5_hl -lhdf5 -lhdf5 -lz -lcurl"
+NETCDF_INCDIR=`nc-config --fflags`
+NETCDF_LIBDIR=`nc-config --flibs`
+NETCDF_LIB=""
+HDF5_INCDIR=""
+HDF5_LIBDIR=""
+HDF5_LIB=""

Go to Orchidee

cd ../ORCHIDEE

For Orchidee MICT 8.9.0 or below you have to apply this modification to arch/arch-ifort_LSCE.path :

Index: arch/arch-ifort_LSCE.path
===================================================================
--- arch/arch-ifort_LSCE.path	(révision 6217)
+++ arch/arch-ifort_LSCE.path	(révision 6218)
@@ -1,5 +1,7 @@
-NETCDF_LIBDIR="/usr/local/install/netcdf-4.3.2p/lib -lnetcdff -lnetcdf -L/usr/local/install/hdf5-1.8.9p/lib -lhdf5_hl -lhdf5 -lhdf5 -lz -lcurl -lstdc++"
-NETCDF_INCDIR=/usr/local/install/netcdf-4.3.2p/include
+NC_LIB=`nc-config --libs`
+NC_FLIB=`nc-config --flibs`
+NETCDF_LIBDIR="$NC_LIB $NC_FLIB -lstdc++"
+NETCDF_INCDIR=`nc-config --includedir`
 IOIPSL_INCDIR=${ORCHDIR}/../../lib
 IOIPSL_LIBDIR=${ORCHDIR}/../../lib
 XIOS_INCDIR=${ORCHDIR}/../XIOS/inc

From here you can move to the next section compilation.

Old guide (do not use)

This guide is now obsolete for obelix. It was meant to work with the default ORCHIDEE_trunk environment modules (netcdf/4p and hdf/1.8.9p). But due to a bug in one of the related libraries it is not possible to use it anymore. It prevents the use of forcesoil.

This guide assumes modipsl/ligIGCM is already download with the ORCHIDEE_trunk configuration:

https://forge.ipsl.fr/orchidee/wiki/Documentation/UserGuide/InstallingORCHIDEEBasic#ORCHIDEEtrunkversion

The following steps describes what is required to change to compile Orchidee-MICT.

IOIPSL

This library takes care of multiple Input/Output? tasks:

Read/write restart files
Read/write history files
Read configuration file (run.def)
Calendar options

After 8.4.0 (rev 4057)

Starting at this version (also included), MICT is using a modified IOIPSL. Restart subroutines to Read/Write? allows more than 3 dimensions. It allows an increase of performance for the restart files. For such purpose, follow the instructions below in order to install it properly:

1- Go to the modeles folder. All source code is located here.

cd YOURPATH/modipsl/modeles/

2- Remove the current IOIPSL. This one is the IOIPSL trunk version.

rm -rf IOIPSL

3- Download the modified IOISPL-MICT:

3.A- Starting at 8.6.0

svn co -r4850 svn://forge.ipsl.fr/orchidee/branches/ORCHIDEE-MICT/IOIPSL IOIPSL

3.B- Before 8.6.0

svn co -r4720 svn://forge.ipsl.fr/orchidee/branches/ORCHIDEE-MICT/IOIPSL IOIPSL

4- Go to the IOIPSL folder

cd IOIPSL/src

5- Install all makefiles required to compile this library:

../../../util/ins_make

6- (optional) Modify the line inside the Makefile in order to get better performance

For Irene:

F_O = -DCPP_PARA -O3 -xHost -align all -align array64byte $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise

For obelix:

F_O = -DCPP_PARA -O3 -align all $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise

For curie/ADA:

F_O = -DCPP_PARA -O3 -xAVX -align all -align array32byte $(F_D) $(F_P) -I$(MODDIR) -module $(MODDIR) -fp-model precise

Before 8.4.0 (rev 4057)

Go to IOIPSL source code folder

cd YOURPATH/modipsl/modeles/IOIPSL/src

Install all makefiles.

../../util/ins_make

Go to the IOIPSL folder

cd ../IOIPSL/src

Modify the files shown below:

Index: histcom.f90
===================================================================
--- histcom.f90	(revision 2705)
+++ histcom.f90	(working copy)
@@ -73,7 +73,7 @@
 !-
 ! Fixed parameter
 !-
-  INTEGER,PARAMETER :: nb_files_max=20,nb_var_max=600, &
+  INTEGER,PARAMETER :: nb_files_max=20,nb_var_max=1400, &
  &                     nb_hax_max=5,nb_zax_max=10,nbopp_max=10
   REAL,PARAMETER :: missing_val=nf90_fill_real
   INTEGER,PARAMETER,PUBLIC :: &
Index: restcom.f90
===================================================================
--- restcom.f90	(revision 2705)
+++ restcom.f90	(working copy)
@@ -39,7 +39,7 @@
 ! the out file.
 !-
   INTEGER,PARAMETER :: &
- &  max_var=500, max_file=50, max_dim=NF90_MAX_VAR_DIMS
+ &  max_var=1000, max_file=50, max_dim=NF90_MAX_VAR_DIMS
 !-
   CHARACTER(LEN=9),SAVE :: calend_str='unknown'
 !-

Orchidee-MICT is using lots of variables. So it is necessary to tell IOIPSL increase the maximum allowed.

ORCHIDEE-MICT

In order to compile Orchidee MICT:

Go to the main model folder:

cd YOURPATH/modipsl/modeles/

Remove Orchidee trunk

rm -rf ORCHIDEE

Download MICT from svn (under development)

svn co svn://forge.ipsl.fr/orchidee/branches/ORCHIDEE-MICT/ORCHIDEE ORCHIDEE

Compilation

Once modipsl is properly set up for Orchidee-MICT, you can compile the model and its dependencies following the instructions below. For that purpose, you can choose two methods:

Automatic: the script will do all the steps for you
Manual: the same as the script but by hand

Automatic

Go to configuration folder for Orchidee offline:

cd YOURPATH/modipsl/config/ORCHIDEE_OL

Modify Makefile to include "-otherexec", from:

...
 # driver : Only compiling ORCHIDEE without linking to XIOS (with fcm method)
driver :
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm        -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver ; )

# driver_xios : Only compiling ORCHIDEE and linking to XIOS 1.0 (with fcm method)
#               Note: For linking with XIOS 2.0 change -xios into -xios2
driver_xios :
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver ; )

to:

...
 # driver : Only compiling ORCHIDEE without linking to XIOS (with fcm method)
driver :
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm        -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver -otherexec ; )

# driver_xios : Only compiling ORCHIDEE and linking to XIOS 1.0 (with fcm method)
#               Note: For linking with XIOS 2.0 change -xios into -xios2
driver_xios :
        (cd ../../modeles/ORCHIDEE ; ./makeorchidee_fcm -xios2 -j 8 -parallel mpi -prod -arch ${FCM_ARCH} -driver -otherexec ; )

Compile the whole model (IOIPSL, XIOS and Orchidee):

gmake clean && gmake

Manual

Orchidee can also be manually compiled.

IOIPSL

Go to IOIPSL folder

cd modipsl/modeles/IOIPSL/src

Compile IOIPSL

gmake

XIOS

XIOS is a new Input/Output? library aimed to replace IOIPSL at some point. Currently, it only replaces the write of history files. This means IOIPSL is still required and must be always compiled.

Go to XIOS folder:

cd modipsl/modeles/XIOS

Compile XIOS 2

./make_xios --prod --arch SUPPORTED_MACHINE --full --j 8

Orchidee-MICT

Move to Orchidee-MICT

cd modipsl/modeles/ORCHIDEE

Compile Orchidee

./makeorchidee_fcm  -parallel mpi -arch SUPPORTED_MACHINE -driver -prod -full -xios2 -j 8 -otherexec

Replace -arch MACHINE with the machine you are working with. Find more information in makeorchidee_fcm -h command.

After that, when you do a modification you can simply run the command line below. So you avoid the whole recompilation from the code:

./makeorchidee_fcm  -parallel mpi -arch SUPPORTED_MACHINE -driver -prod -j 8 -xios2 -otherexec

In some cases it might be necessary to recompile from zero.

Debug

If you need to debug use:

./makeorchidee_fcm  -parallel mpi -arch SUPPORTED_MACHINE (-xios2) -driver -debug -full -j 8 -otherexec

Notes

the use of gmake produces an error due to dependencies. For this reason, it is only required to compile orchidee libraries (ioipsl, ...).
This guide expects you have selected the proper module libraries in the HPC you are working with.
There is a new option for makeorchidee_fcm called otherexec. It creates forcesoil.exe

Supported machines

Refer to the following keys depending on the machine:

obelix: ifort_LSCE
Irene: X64_IRENE
Curie: X64_CURIE
ADA: X64_ADA

More machines are available. To discover them:

[ -arch XXX ] : name of the archicture file containg platform dependent compile options. 
                The files arch/arch-XXX.fcm and arch/arch-XXX.path must exist.

Errors

Please, report any error to albert.jornet-puig@…

Last modified 4 months ago Last modified on 2024-10-15T14:45:56+02:00

Download in other formats:

Plain Text