source: CPL/oasis3-mct/branches/OASIS3-MCT_5.0_branch/doc/UG3_Namcouple_generation.tex @ 6331

\newpage
\chapter{The configuration file {\it namcouple}}
\label{sec_namcouple}

The OASIS3-MCT configuration file {\it namcouple} contains, below
pre-defined keywords, all user-defined information necessary to
configure a particular coupled run.

The {\it namcouple} is a text file with the following characteristics:

\begin{itemize}
\item the keywords used to separate the information can appear in any
  order;
\item the number of blanks between two character strings is
  non-significant;
\item all lines beginning with \# are ignored and considered as
  comments;
\item blank lines are supported, but only since OASIS3-MCT\_4.0 version.
\end{itemize}

The first part of {\it namcouple } is devoted to configuration of
general parameters such as the total run time or the desired debug level.
The second part gathers specific
information on each coupling (or I/O) field, e.g. their coupling
period, the list of transformations or remapping to be performed
by OASIS3-MCT and associated configuring lines (described in more
details in chapter \ref{sec_transformations}).

In OASIS3-MCT, several {\it namcouple} inputs have been deprecated
but, for backwards compatibility, they are still allowed.  These
inputs will be noted in the following text using the notation
``UNUSED'' and not fully described. Information below these keywords
is obsolete; they will not be read and will not be used.

In the next sections, a {\it namcouple} example is given and
all configuring parameters are described. Additional lines
containing different parameters for each transformation
are described in section \ref{sec_transformations}. A realistic {\it
  namcouple} can be found in {\tt
  oasis3-mct/examples/tutorial/data\_oasis3/} directory.

\section{An example of a simple {\it namcouple}}
\label{subsec_examplenamcouple}

The following simple {\it namcouple} configures a run into which e.g. an
ocean, an atmosphere and an atmospheric chemistry components are
coupled. The ocean running on grid {\tt toce} provides only the SOSSTSST field to the atmosphere (grid {\tt atmo}),
which in return provides the field CONSFTOT to the ocean. One field
COSENHFL is exchanged from the atmosphere to the atmospheric
chemistry (also running on grid {\tt atmo}), and one field SOALBEDO is read from a file by the ocean.

\begin{verbatim}
########## First section #############################################
 $NFIELDS
    4
#
 $RUNTIME
    432000
#
 $NLOGPRT
   2     1    0
#
 $NCDFTYP
   cdf1
#
 $NUNITNO
   901     920
#
 $NMAPDEC
   decomp_wghtfile
#
 $NMATXRD
   ceg
#
 $NWGTOPT
   ignore_bad_index
#
 $SEQMODE
 $CHANNEL
 $JOBNAME
 $NBMODEL
 $INIDATE
 $MODINFO
 $CALTYPE
#
########## Second section #############################################
#
 $STRINGS
#
# Field 1
 SOSSTSST SISUTESU 1 86400  5  sstoc.nc  EXPOUT
 182  149  128  64  toce  atmo   LAG=+14400  SEQ=+1
 P 2 P 0
 LOCTRANS CHECKIN MAPPING  BLASNEW CHECKOUT
#
  AVERAGE
  INT=1
  map_toce_atmo_120315.nc src opt
  1.0  1
  CONSTANT     273.15
  INT=1
#
# Field 2
 CONSFTOT SOHEFLDO 6 86400  4   flxat.nc  EXPORTED
 atmo   toce  LAG=+14400  SEQ=+2
 P 0 P 2
 LOCTRANS  CHECKIN  SCRIPR CHECKOUT
#
  ACCUMUL
  INT=1
  BILINEAR LR SCALAR LATLON 1
  INT=1
#
# Field 3
 CONSFTOT  CONSFTOT 1  86400   1  flda3.nc  OUTPUT
 128  64  128  64 atmo   atmo 
 LOCTRANS
 AVERAGE
#
# Field 4
 SOALBEDO SOALBEDO  17  86400  0  SOALBEDO.nc  INPUT
\end{verbatim}

% section{An example of a simple {\it namcouple}}

\section{ First section of {\it namcouple} file}
\label{subsec_namcouplefirst}

The first section of {\it namcouple } uses some predefined keywords
prefixed by the \$ sign to locate the related information. The \$ sign
must be the first non-blank character on the line but can be in any column.
8 keywords are used by OASIS3-MCT and 5 of these are optional :

\begin{itemize}

\item {\tt \$NFIELDS}: On the line below this keyword, put a number equal (or greater) to the total
  number of field entries in the second part of the {\it
    namcouple}. If more than one field are described on the same line, this
  counts as only one entry.

%\item {\tt \$NBMODEL}: On the line below this keyword is the number of
%  models running in the given experiment followed by {\tt
%    CHARACTER$\star$6} variables giving their names, which must
%  correspond to the name announced by each model when calling {\tt
%    oasis\_init\_comp} (second argument, see section
%  \ref{subsubsec_Initialisation}).
%
%  Then the user may indicate on the same line the maximum Fortran unit
%  number used by the models. In the example, Fortran units above 55,
%  70, and 99 are free for respectively the ocean, atmosphere, and
%  atmospheric chemistry models. {\bf In all cases, OASIS3-MCT library
%    assumes, during the initialization phase, that units 1025 and 1026
%    are free and temporarily uses these units to read the {\it
%      namcouple} and to write corresponding log messages to file {\tt
%      nout.000000}.} After the initialization phase, OASIS3-MCT will
%  still suppose that units above 1024 are free, unless maximum unit
%  numbers are indicated here in the {\it namcouple}.
%  % If {\tt \$CHANNEL} is {\tt NONE}, {\tt \$NBMODEL} has to be 0 and
%  % there should be no model name and no unit number.

\item {\tt \$RUNTIME}: On the line below this keyword, put the total
  simulated time of the run, expressed in seconds (or any other time
  units as long as the same are used in all components and in the {\it
    namcouple}, see \ref{subsubsec_sendingreceiving}). Note that by convention the first coupling of a run occurs at the beginning of the run and all other couplings occur at a time strictly smaller than {\tt \$RUNTIME}.
  % If {\tt \$CHANNEL} is {\tt NONE}, {\tt \$RUNTIME} has to be the
  % number of time occurrences of the field to interpolate from the
  % restart file.

\item {\tt \$NLOGPRT}: The first, second and third numbers on the line below
  this keyword refer to (i) the debug verbosity, (ii) internal timing statistics
  and (iii) component load balancing analysis. The information is written by OASIS3-MCT
  for each component and (optionnally) for each process.
  
  The first number (that can be modified at runtime with the {\tt
    oasis\_set\_debug} routine, see section
  \ref{subsubsec_auxroutines}) may be:
  \begin{itemize}
  \item 0 : production mode. One file debug.root.xx is open by the master process of
    each component and one file debug\_notroot.xx is open for all the
    other processes of each component to write only error information.
  \item 1 : one file debug.root.xx is open by the master process of
    each component to write information equivalent to level 10 (see
    below) and also to write memory usage information;
    one file debug\_notroot.xx is open for all the other
    processes of each component to write error information.
  \item 2 : one file debug.yy.xxxxxx is open by each process of each
    component (with ``yy” being the component number and ``xxxxxx” the process number)
    to write normal production diagnostics and memory usage information
  \item 5 : as for 2 with in addition some initial debug info
  \item 10: as for 5 with in addition the routine calling tree
  \item 12: as for 10 with in addition some routine calling notes
  \item 15: as for 12 with even more debug diagnostics
  \item 20: as for 15 with in addition some extra runtime analysis
  \item 30: full debug information
  \end{itemize}
  The second number defines how time statistics are written out to
  file {\it comp\_name}.timers\_xxxx (with {\it comp\_name} being the component name, see section \ref{init_comp}); it can be:
  \begin{itemize}
  \item 0 : nothing is calculated or written.
  \item 1 : some time statistics are calculated and written in a
    single file by the processor 0 as well as the min and the max
    times over all the processors.
  \item 2 : some time statistics are calculated and each processor
    writes its own file ; processor 0 also writes the min and the max
    times over all the processors in its file.
  \item 3 : some time statistics are calculated and each processor
    writes its own file ; processor 0 also writes in its file the min
    and the max times over all processors and also writes in its file
    all the results for each processor.
  \end{itemize}
 For more information on the time statistics written out, see section
  \ref{timestat}.

The third number (new in OASIS3-MCT\_5.0) can be set to 1 to activate a load balancing diagnostic.
 An efficient use of the allocated computing resources in a coupled system requires the harmonisation of the components speed. This operation, called load balancing, is often neglected, either because of the apparent resource abundance and practical difficulties.
 To facilitates this work, OASIS3-MCT can output the full timeline of all coupling related events, for any of the allocated resources. This timeline is saved in one netCDF file per coupled component ({\tt timeline\_XXX\_component.nc}). It provides the comprehensive sequence of any operations related to the coupling (field exchange through MPI, field output on disk, field interpolation and mapping, field reading on disk, restart writing, initialisation and termination phase of the OASIS3-MCT setup) so that any simulation slow down in link with the use of the OASIS library can be identified.
 The analysis of the coupling field exchanges, amongst all the
 coupling events, allows not only to identify the resources waste of components which are recurrently waiting for their coupling fields but it also reveals other bottlenecks such as disk access, OS interruptions or model internal load imbalance. The full picture of these events makes possible an optimum load balancing, even for the most complex configurations.
  For a detailed information on load balancing analysis and timeline visualisation see respectively \cite{maisonnave20} and in \cite{piacentini20}.
  In addition to the timeline, computing information (time to solution, speed, cost) and a synthesis of the time spent on MPI routines for each coupled component can also help, in the simpler cases, to allocate resources in a balanced way ( see file {\tt load\_balancing\_info.txt} ).


\item {\tt \$NCDFTYP}: Optional (new in OASIS3-MCT\_5.0); on the line below
  this keyword is a character string that indicates the NetCDF file type
  for all (i.e. mapping, restart, output) new NetCDF files generated by OASIS3.
  The options are {\tt cdf1},
  {\tt cdf2}, and {\tt cdf5}. The mode {\tt cdf1} is also known as classic mode,  {\tt cdf2} as large file format or 64bit\_offset and supports larger
  files, {\tt cdf5} as 64bit\_data and supports both larger files
  and larger variables.  More information about these file types can
  be found in NetCDF documentation.  Because {\tt cdf5} may not be generally available in
  all NetCDF installations, use of this option requires that the C preprocessor
  directive {\tt CDF\_64BIT\_DATA} be used when compiling OASIS3-MCT.  If that
  preprocessor is not used, {\tt cdf5} is not a valid option.  
  The file format for any NetCDF file can be diagnosed by using ``ncdump -k filename''.
  \$NCDFTYP only affects new files created by OASIS3-MCT.  NetCDF will read and/or 
  overwrite existing files of any NetCDF file type, and the file type will remain
  unchanged in that case.

\item {\tt \$NUNITNO}: Optional (new in OASIS3-MCT\_4.0); on the line below this keyword are two integers
  that indicate the minimum and maximum unit numbers to be used for
  input and output files in the coupling layer.  The user should
  choose values that will NOT conflict or overlap with unit numbers in
  use in any of the component models. The defaults are 1024 for the minimum and 9999
  for the maximum unit number if not explicitly set by the user.

\item {\tt \$NMAPDEC}: Optional (new in OASIS3-MCT\_4.0); on the line below this keyword is a character string
  that indicates the mapping decomposition value to be used during local mapping.  The
  options are {\tt decomp\_1d} and {\tt decomp\_wghtfile}.  Option {\tt decomp\_1d} decomposes the grid in a simple
  one dimensional way while {\tt decomp\_wghtfile} decomposes the grid using the
  information in the remapping weight file to reduced mapping communication. Option {\tt decomp\_wghtfile}
  will take some extra time in initialization but it should result in faster mapping.
  The default is {\tt decomp\_1d} but it is recommended to test {\tt decomp\_wghtfile} to see if that
  option improves performance. More details can be found in \cite {craig18} and in \cite{valcke11}.

\item {\tt \$NMATXRD}: Optional (new in OASIS3-MCT\_4.0); on the line below this keyword is a character string
  that indicates the method used to read remapping weights.  There are two options, {\tt orig}
  and {\tt ceg}.  In both, the weights are read in chunks by the root process.  In the {\tt orig} option,
  the weights are then broadcasted to all processes and each process then saves the weights needed in
  order to be consistent with the mapping decomposition.  In the {\tt ceg} option, the root process
  reads the weights and then decides which process each weight should be assigned to.  A
  series of exchanges are then done and just the weights needed on
  each process are sent.  The {\tt orig} method sends much more data but is more parallel.  The {\tt ceg}
  method does most of the work on the root process but less data is communicated.  The default option
  is {\tt ceg}. More details can be found in \cite {craig18}.

\item {\tt \$NWGTOPT} : Optional (new in OASIS3-MCT\_4.0); on the line below this keyword is a character string
  that indicates how to handle bad remapping weights.  There are four options \newline
  {\tt abort\_on\_bad\_index}, {\tt ignore\_bad\_index}, {\tt ignore\_bad\_index\_silently}, and
 \newline  {\tt use\_bad\_index}.  Bad weights are defined as weights in the mapping file for which either
  the source or destination index are out of bounds relative to the number of grid cells
  in the grid; in that case, the weight is referencing a gridcell that does not physically
  exist.  Note that an index equal to zero will not be considered as a bad index if the associated weight
  is also zero. There are other situations where the value of the actual mapping weight is
  scientifically incorrect, but this is not easy to detect and is not dealt with in OASIS3-MCT.
  \begin{itemize}
  \item {\tt abort\_on\_bad\_index} will write error messages to the log files and abort if a bad weight
  index is detected. This is the default option.
  \item {\tt ignore\_bad\_index} will write an error message and then remove bad
  weights internally before continuing.
  \item {\tt ignore\_bad\_index\_silently} will remove bad weights and continue without writing an error
  message.
  \item {\tt use\_bad\_index} will attempt to keep bad weights in the interpolation computation,
  but this can result in memory corruption, silent dropping of weights, and incorrect results ; this is not recommended.
  \end{itemize}

  Note that the ability to check mapping files at runtime in OASIS3-MCT is limited.  It is always
  recommended that mapping files be analyzed offline before long production runs are carried out.
  Checks can be done to make sure the source and destination indices are valid, that weights values
  are reasonable (for instance, between 0 and 1, although this will depend on the mapping method),
  and that the sum of weights on the destination cells are reasonable (for instance, 1, in many cases).
  In addition, offline tests can be run with analytical functions to verify conservation, gradient
  preserving features and other characteristics associated with the particular mapping approach.

\item {\tt \$NNOREST}: Optional (new in OASIS3-MCT\_4.0); on the line below this keyword is a character
  string that can override the requirement that restart files must exist
  if they are needed.  If the character string value starts with T, t, .T,
  or .t (as in true), then OASIS3-MCT will initialise with zero any variable that normally requires
  a restart (for instance, variables with LAG $>$ 0) if the restart file does not exist. By default, missing
  restart files will cause the model to abort.  It is strongly recommended
  that this keyword NOT be used in production runs.  It exists to provide a
  quick shortcut for running technical tests.
  Note that if {\tt \$NNOREST} is true but the restart file nonetheless exists, it will be used.

\item {\tt Keywords \$SEQMODE, \$CHANNEL, \$JOBNAME, \$NBMODEL, \$INIDATE, \$MODINFO, \$CALTYPE} are not used anymore.

\end{itemize}

% {Description of {\it namcouple} first section}

\section{Second section of {\it namcouple} file }
\label{subsec_namcouplesecond}

The second part of the {\it namcouple}, starting after the keyword
{\tt \$STRINGS}, contains coupling information for each coupling (or
I/O) field.  Its format depends on the field status given by the last
entry on the field first line ({\tt EXPORTED}, {\tt IGNOUT} or {\tt
  INPUT} in the example above). The field may be (status {\tt AUXILARY} is now UNUSED) :

\begin{itemize}
\item {\tt EXPORTED}: exchanged between components and
  transformed by OASIS3-MCT
\item {\tt EXPOUT}: exchanged, transformed and also written to two
  debug NetCDF files, one before the sending action in the source
  component below the {\tt oasis\_put} call (after local transformations
  {\tt LOCTRANS} and {\tt BLASOLD} if present), and one after the
  receiving action in the target component below the {\tt oasis\_get} call
  (after all transformations). {\tt EXPOUT} should be used only when
  debugging the coupled model. The name of the debug NetCDF file
  (one per field) is automatically defined based on the field and
  component names.
\item {\tt IGNORED}: with OASIS3-MCT, this setting is equivalent to
  and converted to EXPORTED
\item {\tt IGNOUT}: with OASIS3-MCT, this setting is equivalent to and
  converted to EXPOUT
\item {\tt INPUT}: read in from the input file by the target component
  below the {\tt oasis\_get} call at
  appropriate times corresponding to the input period indicated by the
  user in the {\it namcouple}. See section \ref{subsec_inputdata} for
  the format of the input file.
\item {\tt OUTPUT}: written out to an output debug NetCDF file by the
  source component below the {\tt oasis\_put} call, after local
  transformations {\tt LOCTRANS} and {\tt BLASOLD}, at appropriate
  times corresponding to the output period indicated by the user in
  the {\it namcouple}.

\end{itemize}

\subsection{Second section of {\it namcouple} for {\tt EXPORTED} and
  {\tt EXPOUT} fields}
\label{subsubsec_secondEXPORTED}

The first 3 lines for fields with status {\tt EXPORTED} and {\tt
  EXPOUT} are as follows:
  \begin{verbatim}
   SOSSTSST SISUTESU 1 86400  5  sstoc.nc  EXPOUT
   182  149    128  64  toce  atmo   LAG=+14400 SEQ=+1
   P 2 P 0
\end{verbatim}
%\vspace{-0.2cm}
where the different entries are:
\begin{itemize}
\item Field first line:
  \begin{itemize}

  \item {\tt SOSSTSST} : symbolic name for the field in the source
    component (80 characters maximum). It has to match the argument {\tt name}
    of the corresponding field declaration in the source component; see
    {\tt oasis\_def\_var} in section \ref{subsubsec_Declaration}
  \item {\tt SISUTESU} : symbolic name for the field in the target
    component (80 characters maximum).  It has to match the argument {\tt
      name} of the corresponding field declaration in the target
    component; see {\tt oasis\_def\_var} in section
    \ref{subsubsec_Declaration}
  \item 1 : UNUSED but still required for parsing
  \item 86400 : coupling and/or I/O period for the field, in seconds
  \item 5 : number of transformations to be performed by OASIS3 on
    this field
  \item sstoc.nc : name of the coupling restart file for the field
    (32 characters maximum); mandatory even if no coupling restart file is
    effectively used (for more detail, see section
    \ref{subsec_restartdata})
  \item {\tt EXPOUT} : field status
  \end{itemize}
\item Field second line:
  \begin{itemize}
  \item 182 : number of points for the source grid first dimension
    (optional)
  \item 149 : number of points for the source grid second dimension
    (optional)\footnote{For 1D field, put {\tt 1} as the second dimension}
  \item 128 : number of points for the target grid first dimension
    (optional)
  \item 64 : number of points for the target grid second dimension
    (optional)$^{1}$

    These source and target grid dimensions are optional but note that
    in order to have 2D fields written as 2D arrays in the debug
    files, these dimensions must be provided in the {\it namcouple};
    otherwise, the fields will be written out as 1D arrays.

  \item toce : prefix of the source grid name in grid data files (see
    section \ref{subsec_griddata}) (80 characters maximum)
  \item atmo : prefix of the target grid name in grid data files (80 characters maximum)
  \item {\tt LAG=+14400}: optional lag index for the field (see section \ref{subsec_lag})
  \item {\tt SEQ=+1}: optional sequence index for the field (see
    section \ref{subsec_sec})
  \end{itemize}
\item Field third line
  \begin{itemize}
  \item P : source grid first dimension characteristic (`P':
    periodical; `R': regional).
  \item 2 : source grid first dimension number of overlapping grid
    points.
  \item P : target grid first dimension characteristic (`P':
    periodical; `R': regional).
  \item 0 : target grid first dimension number of overlapping grid
    points.
  \end{itemize}

\item The fourth line gives the list of transformations to be performed for
this field. In addition, there is one or more configuring lines
describing some parameters for each transformation. These additional
lines are described in more details in the chapter
\ref{sec_transformations}.

\end{itemize}

{\bf Support to couple multiple fields via a single communication}

With OASIS3-MCT, it is possible to couple mutiple fields via a
single communication. To activate this option, the user must list the
related fields on a single entry line (with a maximum of 5000 characters on one line) through a colon
delimited list in the {\it namcouple}, for example:

{\tt ATMTAUX:ATMTAUY:ATMHFLUX  TAUX:TAUY:HEATFLUX 1 3600 3 rstrt.nc EXPORTED}

All fields will then use the same
{\it namcouple} settings (source and target grids, transformations, etc.) for that entry. In the component model codes,
these fields are still apparently sent or received one at a
time through individual {\tt oasis\_put} and {\tt oasis\_get}. Inside OASIS3-MCT, the fields are stored and a single mapping
and send or receive instruction is executed for all fields. This is
useful in cases where multiple fields have the same coupling
transformations and to reduce communication costs by aggregating multiple
fields into a single communication.

This option does not put any constraint
on the order of the related {\tt oasis\_put} and {\tt oasis\_get} in the codes.

As they appear in one single entry line, these fields must share the same coupling restart file
but this restart file may contain other fields.

\subsection{Second section of {\it namcouple} for {\tt OUTPUT} fields}
\label{subsubsec_secondOUTPUT}
The first 3 lines for fields with status {\tt OUTPUT} are as follows:
  \begin{verbatim}
 CONSFTOT  CONSFTOT 1  86400   1  flda3.nc  OUTPUT
 128  64  128  64 atmo   atmo 
 LOCTRANS
\end{verbatim}
% \vspace{-0.2cm}
where the different entries are:
\begin{itemize}
  \item Field first line:
    \begin{itemize}
    \item the source symbolic name must be repeated twice on the field
      first line
    \item 1 : UNUSED but still required for parsing
    \item 86400 : output period for the field, in seconds
    \item 1 : number of transformations to be performed by OASIS3-MCT
      on this field
    \item flda3.nc : name of the coupling restart file for the field
      (32 characters maximum); needed only if  a {\tt LOCTRANS} transformation is present
    \item {\tt OUTPUT} : field status
    \end{itemize}
  \item Field second line:
    \begin{itemize}
    \item 128 : number of points for the source grid first dimension (optional)
    \item 64 : number of points for the source grid second dimension (optional)\footnote{For 1D field, put {\tt 1} as the second dimension}
    \item 128 : number of points for the target grid first dimension (optional)
    \item 64 : number of points for the target grid second dimension (optional)$^{1}$
    \item atmo : prefix of the source grid name in grid data files repeated twice (80 characters maximum)
    \end{itemize}
  \item The third line is {\tt LOCTRANS} if this transformation is chosen for
the field. Note that {\tt LOCTRANS} is the only transformation
supported for {\tt OUTPUT} fields.
\end{itemize}

\subsection{Second section of {\it namcouple} for {\tt INPUT} fields}
\label{subsubsec_secondINPUT}

The first and only line for fields with status {\tt INPUT} is:

  \begin{verbatim}
  SOALBEDO SOALBEDO  1  86400  0  SOALBEDO.nc  INPUT
  \end{verbatim}
\vspace{-0.5cm}
where the different entries are:
\begin{itemize}
\item {\tt SOALBEDO}: symbolic name for the field in the target component
  (80 characters maximum, repeated twice)
\item 1: UNUSED but still required for parsing (as for
  EXPORTED fields above)
\item 86400: input period in seconds
\item 0: number of transformations (always 0 for {\tt INPUT} fields)
\item {\tt SOALBEDO.nc}:  the input file name (32 characters maximum)
  (for more detail on its format, see section \ref{subsec_inputdata})
\item {\tt INPUT}: field status.
\end{itemize}