Tools developed by IPSL and IPSL partners
Many software programs are developed at IPSL or at IPSL's partners (see list below).
Table of contents
1. modipsl
modipsl is a suite of scripts which allow you to :
- obtain a given model configuration from a server;
- compile a configuration on a specific machine.
More information on how to retrieve and use modipsl can be found here.
2. libIGCM
libIGCM is a running environment library which allow you to setup and run a simulation on a specific machine by launching computing and post-processing jobs.
3. XIOS
XIOS (XIOS stands for XML-IO-SERVER) is an input/output library which allows you to manage parallel I/Os in climate models in a flexible and efficient way. XIOS (version >= 2.5) is currently used in all of IPSL components. Here are XIOS main features :
- library dedicated to I/O management of climate code, developed at IPSL.
- XML configuration file.
- attached mode (library) or server mode (asynchronous transfer), multiple (sequential writing) or single (parallel writing) output file.
- NetCDF format.
3.1. Short presentation on how to manage outputs files using XIOS model in IPSL configurations
Each model of one configuration will be seen by XIOS as a specific context. For each context XIOS is waiting for 4 types of xml files :
- iodef.xml : Main configuration file for XIOS.
- context.xml : Axis, domain and grid information related to one context.
- field_def.xml : Definition of each variable (so called "field") sent to XIOS from the model. A variable to be written in output file must be either defined in this file or a combination of fields defined in this file.
- file_def.xml : Definition of all output files and their variables.
To write a new variable in output files there are 3 steps to follow :
- define it in field_def.xml file (use of unique id to identify it in the model, name of the variable to write in the output file, grid the variable is calculated on, unit of the variable, temporal operation to be performed on the field,... )
- for this specific variable, add the call of routine xios_send_field in the code. In many models this routine is encapsulated by model routines to manage all type and size of variables.
- add the variable in an existing output file (or create a new one) in file_def.xml specifying output frequency.
In field_def.xml :
<field id="varID" name="varNAME" long_name="xxx" unit="xxx" operation="average" grid_ref="xxx"/>
In model code :
Call xios_inca_send_field(“varID”, VarCode(:))
In file_def.xml :
<file id="fileID" name="fileNAME" output_freq="1mo" enabled=".TRUE.">
<field field_ref="varID" />
</file>
--> the model will write a file fileNAME.nc containing one monthly average variable named varNAME in the file, corresponding to the variable VarCode in the code.
More information can be found here.
4. IOIPSL and REBUILD
4.1. IOIPSL
IOIPSL is an I/O library for the IPSL models. This library is not being further developed (except to fix bugs). The XIOS library has replaced IOIPSL in the IPSL models for output handling. IOIPSL is still used in IPSL components for restart and parameters input actions.
4.2. Rebuild
Rebuild is a tool which allows you to concatenate model output files in NetCDF format. At the end of a simulation, files for each process are automatically (thanks to libIGCM) assembled to a global grid into one single output file. Rebuild is developed at IPSL and is a fortran code included in IOIPSL extended version.
For each machine a Rebuild executable is provided in the shared tools folder.
Thanks to XIOS output library (parallel writing functionality) rebuild is not used anymore in v6 configuration.
5. Tools to prepare a coupled configuration
Tools to creates interpolations between ocean and atmospheres grid, and initial state files for the coupler.
5.1. MOZAIC
MOZAIC is the legacy code used to build interpolation weights between ocean and atmosphere. It should be considerer as obsolete. It's a messy sequential code. Poor documentation available here.
5.2. MOSAIX
MOSAIX is the new code that creates the interpolation weights between ocean and atmosphere. It's a set a bash, python and fortran code that relies on XIOS for computing the weigths. It can be used in parallel. See Mosaix.
5.3. CPLRESTART
CPLRESTART is a small bash that builds (using nco utilities) the restart file for OASIS. It needs outputs from forced configurations (grid_T file from NEMO, histmth file from LMDZ).
5.4. Bug bilinear
In old coupled configurations, the wind stress is interpolated with the bilinear interpolation scheme of the Scrip library included in OASIS-MCT.
In IPSLCM6 using ORCA1, the bilinear interpolation of the wind stress is bugged. A small set of python routines has been developped to overcome the problem : see http://forge.ipsl.fr/igcmg/wiki/IPSLCM6/Coupling/BugBilinear.
New coupled configurations are not affected, as they use weights computed by MOSAIX (see above).
6. AddNoise
AddNoise allows you to apply an equi-distributed white noise which amplitude can be specified in a restart file in order to prepare ensemble experiments. See also : Doc/Setup/Ensemble
AddNoise also allows to restart a crashed model by adding a small perturbation in the restart. See Doc/FAQ for the procedure.
7. FAST/ATLAS
FAST is a library to facilitate post-processing and visualization with Ferret.
FAST is a collection of Ferret scripts and K-shell scripts. Its purpose is to hide a certain number of low-level and repetitive commands which can transform your Ferret scripts into long and unclear programs. With the scripts provided here, you will more easily control and set pages, viewports, texts, legends, colorbars, logos, ratios, zooms and also produce polygon maps and vectors maps.
Ferret is a very powerful application developed by the PMEL. We strongly recommend first reading the documentation and the tutorial they provide from their web site.
ATLAS repository is a collection of useful files (scripts, palette files, configuration files, etc.) to help produce plots to visualize and analyze output from the different components of the Earth System Model (ESM) from Institut Pierre Simon Laplace (IPSL), France. ATLAS is mainly base on FAST scripts.
Detailed information about Fast and Atlas can be found at fast and atlas.
8. WEB applications
web site to acccess to IPSL's Trusting and Intermonitoring software http://webservices.ipsl.fr
8.1. Trusting
The trusting is a software used by the IPSL model developers to check the reliability of their codes.
8.1.1. Standard trustings
Here you can find the overview page of trusting results, gathering information on regular trustings (possibly automatically launched), or on-the-fly trustings, for different versions of the IPSL climate model, and on different computing machines used at IPSL.
8.1.2. "Open trusting"
It is also possible for regular developers of the IPSL model to launch their own trustings. The results with the Open Trusting will not be displayed on the standard trusting result page. A summary of the trusting launched, detailling its experiment runs, will be found in the RUNDIR created by the Open Trusting script and an e-mail will be sent to the user.
Here is a simple how-to guide, for IPSLCM6.2 (as of November 2023).
8.2. Intermonitoring
information at webservices page : http://webservices.ipsl.jussieu.fr/interMonitoring
8.3. CPU Consumption Monitoring
Documentation is here
The cpu Consumption Monitoring is available for each each computing project either at IDRIS or TGCC on this page.
The graphs are updated daily.
8.4. QUOTA monitoring
Similarly from CPU monitoring, a QUOTA monitoring is available. It consist in daily log of the usage of the STORE/WORK/SCRATCH dir for a project. Both Inodes and Data volume are logged and used to provide timeseries.
The output curves are available at https://thredds-su.ipsl.fr/thredds/fileServer/igcmg/SUIVI_CONSO/VISUALISATION/index.html on the link "Stockage [...] TIMESERIES"
8.5. Data Management Tools
Documentation is here
A tool to helps run diagnostics on the usage of the storage facilities on IRENE is available (in development). It allows you to visualise all the children folders from a given path on IRENE and assess their DATA/INODES ratio.
Once the prolematic folder have been identified, you will be able either to tar them or to deleted the obsolete ones.
Diagnostic :
To access the diagnostic tool : https://outils.ipsl.fr/conso-calcul/app The page is accessible from IPSL network or via vpn. You need to provide you 'Fédé' credential. 'login [at] ipsl.fr' and your ipsl password. This is the same as to access documentations.ipsl.fr.
It consists in a diagnostic application to help the user assess which of their directories have the most data or takes the most inodes. Thanks to the tool, the user can more easily select what experiments they want to delete and download a path list to delete manually on the computing center.
The tool is still under development. A beta version is available on demand. Ask edupont at ipsl.fr for it.
Cleaning :
Once the folders with a too big inodes/data ratio have been identified. You should tar them to decrease their number of inodes. (More on this to come).
9. OASIS
OASIS is a tool developed at Cerfacs (Toulouse, France) which allows you to couple different climate model components. It allows the synchronized exchange and the interpolation of information between model components. OASIS is used in the IPSL Earth System model to couple NEMO (the ocean model) and LMDZ (the atmospheric model). The modipsl and libIGCM tools manage the OASIS setup and compilation. OASIS3 (sequential coupler) was used in IPSLCM5 coupled model. OASIS3-MCT (parallel coupler) is currently used in IPSLCM6 coupled model.
More information on OASIS is available here.
10. External Tools
Attachments (1)
- XIOS.png (80.5 KB) - added by acosce 4 years ago.
Download all attachments as: .zip
