source: XIOS/dev/dev_trunk_omp/doc/XIOS_testcase_user_guide.tex @ 1742

% !TEX TS-program = pdflatex
% !TEX encoding = UTF-8 Unicode
% This is a simple template for a LaTeX document using the "article" class.
% See "book", "report", "letter" for other types of document.

\documentclass[11pt]{article} % use larger type; default would be 10pt

\usepackage[utf8]{inputenc} % set input encoding (not needed with XeLaTeX)

%%% Examples of Article customizations
% These packages are optional, depending whether you want the features they provide.
% See the LaTeX Companion or other references for full information.

%%% PAGE DIMENSIONS
\usepackage{geometry} % to change the page dimensions
\geometry{a4paper} % or letterpaper (US) or a5paper or....
% \geometry{margin=2in} % for example, change the margins to 2 inches all round
% \geometry{landscape} % set up the page for landscape
%   read geometry.pdf for detailed page layout information

\usepackage{graphicx} % support the \includegraphics command and options

% \usepackage[parfill]{parskip} % Activate to begin paragraphs with an empty line rather than an indent

%%% PACKAGES
\usepackage{booktabs} % for much better looking tables
\usepackage{array} % for better arrays (eg matrices) in maths
\usepackage{paralist} % very flexible & customizable lists (eg. enumerate/itemize, etc.)
\usepackage{verbatim} % adds environment for commenting out blocks of text & for better verbatim
\usepackage{subfig} % make it possible to include more than one captioned figure/table in a single float
% These packages are all incorporated in the memoir class to one degree or another...

%%% HEADERS & FOOTERS
\usepackage{fancyhdr} % This should be set AFTER setting up the page geometry
\pagestyle{fancy} % options: empty , plain , fancy
\renewcommand{\headrulewidth}{0pt} % customize the layout...
\lhead{}\chead{}\rhead{}
\lfoot{}\cfoot{\thepage}\rfoot{}

%%% SECTION TITLE APPEARANCE
\usepackage{sectsty}
\allsectionsfont{\sffamily\mdseries\upshape} % (See the fntguide.pdf for font help)
% (This matches ConTeXt defaults)

%%% ToC (table of contents) APPEARANCE
\usepackage[nottoc,notlof,notlot]{tocbibind} % Put the bibliography in the ToC
\usepackage[titles,subfigure]{tocloft} % Alter the style of the Table of Contents
\renewcommand{\cftsecfont}{\rmfamily\mdseries\upshape}
\renewcommand{\cftsecpagefont}{\rmfamily\mdseries\upshape} % No bold!

\usepackage{listings}

\usepackage{color}
\usepackage{hyperref}

\definecolor{gray}{rgb}{0.4,0.4,0.4}
\definecolor{darkblue}{rgb}{0.0,0.0,0.6}
\definecolor{cyan}{rgb}{0.0,0.6,0.6}

\lstset{
  basicstyle=\ttfamily,
  columns=fullflexible,
  showstringspaces=false,
  commentstyle=\color{gray}\upshape
}

\lstdefinelanguage{XML}
{
  morestring=[b]",
  moredelim=[s][\bfseries\color{Maroon}]{<}{\ },
  moredelim=[s][\bfseries\color{Maroon}]{</}{>},
  moredelim=[l][\bfseries\color{Maroon}]{/>},
  moredelim=[l][\bfseries\color{Maroon}]{>},
  morecomment=[s]{<?}{?>},
  morecomment=[s]{<!--}{-->},
  commentstyle=\color{DarkOliveGreen},
  stringstyle=\color{blue},
  identifierstyle=\color{red},
  keywordstyle=\color{brown},
  morekeywords={xmlns,version,type, grid, field_ref, grid_ref, id, 
  build_workflow_graph, build_workflow_graph_begin, build_workflow_graph_end}% list your attributes here
  }
\newcommand{\quotes}[1]{``#1''}
\usepackage[dvipsnames]{xcolor}
\newcommand{\visbutton}[1]{\colorbox[RGB]{92,184,92}{\color{white}{#1}}}
%%% END Article customizations

%%% The "real" document content comes below...

\title{User Guide for XIOS testcase}
\author{Yushan Wang}
%\date{} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed 
\makeatletter         
\def\@maketitle{
\raggedright
\begin{minipage}[l]{0.45\textwidth}
\flushleft\includegraphics[height = 30mm]{inputs/images/logo_IPSL}
\end{minipage}
\begin{minipage}[r]{0.45\textwidth}
\flushright\includegraphics[height = 30mm]{inputs/images/logo_LSCE}
\end{minipage}
\\[10ex]
\begin{center}
{\Huge \bfseries \sffamily \@title }\\[4ex] 
{\Large  \@author}\\[4ex] 
\@date\\[8ex]
\end{center}}
\makeatother


\begin{document}
\maketitle


\section{What is the XIOS testcase}

The XIOS generic testcase is a set of unit tests that test functionalities of 
XIOS. It is reserved for XIOS developers and it is designed to ensure that the 
XIOS does not introduce errors after each code modification. 

\section{Organization of the testcase}
The XIOS generic testcase is located in the \verb|GENERIC_TESTCASE| folder inside 
the XIOS directory. Inside the \verb|GENERIC_TESTCASE| folder, we have for the moment 
5 folders for testing specific XIOS functionalities. {\color{red}{Under construction}}
\begin{itemize}
    \item \verb|test_function|: test the temporal reduce functionalities (accumulate, 
    maximum, minimum, average, ...)
    \item \verb|test_scalar_algo|: test the scalar algorithms (extract, reduce, 
    transformation, ...)
    \item \verb|test_axis_algo|: test the axis algorithms (duplicate, extract, 
    interpolate, inverse, transformation, ...)
    \item \verb|test_domain_algo|: test the domain algorithms (connectivity, expand, extract, 
    interpolate, reorder, transformation, ...)
    \item \verb|test_grid_algo|: test the grid algorithms (transformation, ...) 
\end{itemize}

\section{How to configure XIOS testcase}

\subsection{Define the configuration variables}
At the moment, we provide 7 configuration variables with default values.
\begin{itemize}
  \item \verb|UsingServer2|: when set to \verb|true|, the two-level server mode is enabled. 
  Otherwise, the classic one-level server mode is used.
  \item \verb|RatioServer2|: the ratio of the second level servers. Significant only when 
  two-lever server mode enabled.
  \item \verb|NumberPoolsServer2|: number of the pools of the second level servers. 
  Significant only when two-lever server mode enabled.
  \item \verb|NumberClients|: number of client processes.
  \item \verb|NumberServers|: number of servers.
  \item \verb|Duration|: duration of the simulation.
  \item \verb|ATMdomain|: type of the domain used in the simulation.
\end{itemize}

The default values of all 7 configuration variables can be found and modified in 
the file \verb|default_param.py| located in the root folder \verb|GENERIC_TESTCASE|.

Besides of the default value, one can also set a temporal value for the configuration variables in the file \verb|user_params.def|.
In this file, each variable can have multiple values separated by a semicolon. For example, \verb|NumberServers=2,4,6|.
From such definition, the testcase will generate 3 different configurations and tests will be run for each configuration.


\subsection{Define the output file check list}
The validation of the testcase is ensured by the comparison between the newly generated output 
\verb|NetCDF| files and the pre-stored reference results. In some cases, one maybe interested in only one or several output files.
It will not be necessary to compare all output files for this will slow down the whole test execution.
To specify the output files to compare with the reference, one can modify the \verb|checkfile.def| file.
If nothing is set in this file, or \verb|all| is set, then all output files will be examined.

\section{How to launch XIOS testcase}
For the sake of simplicity, we suppose in the following context that the computing resource 
is Juliot-Curie from the TGCC computing center\footnote{\url{http://www-hpc.cea.fr/en/complexe/tgcc-JoliotCurie.htm}}.

Inside the \verb|GENERIC_TESTCASE| folder, we have a sob submission script \verb|job_irene.sh|. 
This script will execute all the unit tests in all the 5 test folders. We also prepared 
job scripts for each test folder. If one want to run unit tests for a specific folder, 
\textit{e.g.} \verb|test_domain_algo|, it is sufficient to launch the \verb|job_irene.sh| 
script inside the \verb|test_domain_algo| folder. 

\section{Output of XIOS testcase}
An example of the XIOS testcase output is as follows:
\begin{lstlisting}[frame=single]
-- Configuring done
-- Generating done
Test project /GENERIC_TESTCASE
      Start  1: test_domain_algo_config0
 1/16 Test  #1: test_domain_algo_config0 .........   Passed    3.51 sec
      Start  2: test_domain_algo_config1
 2/16 Test  #2: test_domain_algo_config1 .........   Passed    3.70 sec
      Start  3: test_domain_algo_config2
 3/16 Test  #3: test_domain_algo_config2 .........   Passed    2.68 sec
 ...
      Start 14: test_axis_algo_config1
14/16 Test #14: test_axis_algo_config1 ...........   Passed    2.31 sec
      Start 15: test_grid_algo_config0
15/16 Test #15: test_grid_algo_config0 ...........   Passed    2.23 sec
      Start 16: test_grid_algo_config1
16/16 Test #16: test_grid_algo_config1 ...........   Passed    2.24 sec

100% tests passed, 0 tests failed out of 16

Total Test time (real) =  39.63 sec
Built target report
\end{lstlisting}

This CTest style summary report shows the overall execution status of all unit tests. 
Besides of this console output, we also have \verb|report.html| which can be viewed
by any web browser. 

We also generate job submission script for each configuration in each tet folder. 
These configuration specified job scripts allow one to focus on one configuration of the given test.
\begin{verbatim}
  /GENERIC_TESTCASE/test_axis_algo/
  irene_job_config_UsingSrv2=true_NbServers=2_ATMdomain=lmdz.sh
\end{verbatim}

\section{References results of XIOS testcase}

If no reference results exist for a given configuration, the output \verb|NetCDF| files along with 
the setup files will be stored in the configuration folder of the test.
\begin{verbatim}
  GENERIC_TESTCASE/test_axis_algo/
        config_UsingSrv2=true_NbServers=2_ATMdomain=lmdz/
              tmp_reference/
              setup/
              xios_output/
        config_UsingSrv2=true_NbServers=4_ATMdomain=lmdz/
              tmp_reference/
              setup/
              xios_output/
  GENERIC_TESTCASE/test_domain_algo/
        ...
\end{verbatim}

These results are considered to be temporal. Once the results are confirmed, one can 
store these results elsewhere for later use. We provide the \verb|copy_to_reference.py|
script for storing the confirmed reference results in the \verb|$WORK| directory of the 
Juliot-Curie supercomputer. Similarly, we can retrieve the reference results from the 
\verb|$WORK| folder to the test folder by using the \verb|copy_from_reference.py| script.

\end{document}