wiki:Doc/ComputingCenters/ESPRImesocenter

1. Working on ESPRI mesocenter: Spirit and SpiritX

  1. 1. Working on ESPRI mesocenter: Spirit and SpiritX
    1. 1.1. General information
      1. 1.1.1. Documentation
      2. 1.1.2. The machines and file systems
      3. 1.1.3. Shared account
      4. 1.1.4. Individual account
      5. 1.1.5. How to define your environment
      6. 1.1.6. Settings for subversion (svn)
      7. 1.1.7. Adapt your environnement if you previously worked at ciclad or climserv
    2. 1.2. Tested configurations
    3. 1.3. Running using libIGCM at spirit and spiritx
    4. 1.4. Mixte MPI-OMP

For several years, all the IPSL infrastructure is federated as the ​ESPRI mesocenter.

In this infrastructure there are:

  • Spirit: a computing cluster located on the Jussieu campus (Paris, France), which hosts mostly modeling data (such as CORDEX, CMIP...)
  • SpiritX: a computing cluster located at Polytechnique (Palaisseau, France), which hosts mostly observation data

The both clusters have the same software and some file systems are cross-mounted, thus libIGCM is used in the same way on both clusters. The shared account, located at spirit, is used also at spiritx.

There are also the old computing clusters Ciclad(now replaced by Spirit) and ClimServ(now replaced by SpiritX). Read the previous documentation for the clusters Ciclad and ClimServ.

1.1. General information

1.1.1. Documentation

The full documentation for ESPRI mesocenter can be found here: ​https://documentations.ipsl.fr/spirit/ and ​https://mesocentre.ipsl.fr.

For questions related to the installation and usage of the cluster, but not related to libIGCM, you can send an email to : ​meso-support (at) ipsl.fr

1.1.2. The machines and file systems

The front-end machine can be accessed via spirit1.ipsl.fr or spiritx1.ipsl.fr. A login at the mesocenter always gives acces to both clusters.

Output files written by libIGCM are stored by default in /data/yourlogin/IGCM_OUT at spirit and in /homedata/yourlogin/IGCM_OUT at spiritx.

1.1.3. Shared account

The repository for shared files are found in /projsu/igcmg/IGCM.

Read more: Repository for shared files and shared tools

1.1.4. Individual account

To acces input files, you must belong to the igcmg users' group. Use following command to check to which groups you belong: 

id -a

1.1.5. How to define your environment

You do not need anything special to compile using the recent configurations (from v6.2 and more recent). This is due to the compilation script which loads the modules that are needed and libIGCM which loads the same modules. Older configurations have not been tested at spirit and spiritx.

At spirit and spiritx, it can be usefull to add ulimit -s unlimited if you run the modeles without libIGCM.

To receive the end-of-job messages returned by the job itself (e.g. end of simulation, error,...) you must specify your email address in the file ${HOME}/.forward.

Even if it is not necessary, if you want you can copy the starting files we propose for your personal environnement at spirit and spiritx.

Do the following at spirit, only first time you connect: 

cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc ~/.bashrc
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc_spirit ~/.bashrc_spirit

Do the following at spiritx, only first time you connect: 

cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc ~/.bashrc
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc_spiritx ~/.bashrc_spiritx

You can add what you want in the file .bashrc_spirit or .bashrc_spiritx. Note that the files .bashrc_spirit and .bashrc_spiritx are currently the same. But you might want to have a different environnement depending on the cluster, therfor the files are duplicated.

1.1.6. Settings for subversion (svn)

We recommand you to use the module subversion/1.13.0 for svn. This is loaded by default in the above recommanded .bashrc_spirit(x) files. If you receive following error messages when using svn, you probably didn't load this module and you're using the default svn, if so just add "module load subversion/1.13.0" in your .bashrc_spirit(x): 

Couldn't start kwalletd:  QDBusError("org.freedesktop.DBus.Error.Spawn.ChildSignaled", "Process org.kde.kwalletd5 received signal 6")
Couldn't start kwalletd:  QDBusError("org.freedesktop.DBus.Error.Spawn.ChildSignaled", "Process org.kde.kwalletd5 received signal 6")
Couldn't start kwalletd:  QDBusError("org.freedesktop.DBus.Error.Spawn.ChildSignaled", "Process org.kde.kwalletd5 received signal 6")
Couldn't start kwalletd:  QDBusError("org.freedesktop.DBus.Error.Spawn.ChildSignaled", "Process org.kde.kwalletd5 received signal 6")
Couldn't start kwalletd:  QDBusError("org.freedesktop.DBus.Error.Spawn.ChildSignaled", "Process org.kde.kwalletd5 received signal 6")

The installation by default doesn't allow to store passwords so you need to type your password each time. To avoid this, you can adapt your configuration by adding following in the file .subversion/config, section [auth]: 

vi ~/.subversion/config
[auth]
password-stores =
store-passwords = yes
store-auth-creds = yes
...

1.1.7. Adapt your environnement if you previously worked at ciclad or climserv

If you worked previously at ciclad or climserv, you already have your environnemental file .bashrc working for corresponding cluster. This .bashrc will not be suitable for the new clusters mainly for the versions of the modules which are not the same. We advice you to move this file into ~/.bashrc_ciclad or ~/.bashrc_climserv and to set a switch in .bashrc to source corresponding file depending on the cluster you connect to. You can find this switch based on the hostname in a bashrc file we propose

Do the following at spirit (or ciclad), only first time you connect: 

mv .bashrc .bashrc_ciclad
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc ~/.bashrc
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc_spirit ~/.bashrc_spirit

Do the following at spiritx (or ClimServ), only first time you connect: 

mv .bashrc .bashrc_climserv
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc ~/.bashrc
cp /ciclad-home/igcmg/MachineEnvironment/mesoipsl/bashrc_spiritx ~/.bashrc_spiritx

You'll keep your old environnement at ciclad and climserv and at spirit and spiritx, you can add what you want in the new file .bashrc_spirit(x).

1.2. Tested configurations

Currently the following forced configurations have been tested at spirit and spiritx with the ifort compiler:

  • NEMO forced mode
  • ORCHIDEE offline: ORCHIDEE_trunk, ORCHIDEE_4_1, ORCHIDEE_3_head and ORCHIDEE_2_2

LMDZOR_v6.2 and LMDZOR_v6.4 will be adapted soon.

1.3. Running using libIGCM at spirit and spiritx

libIGCM can be used at spirit and spiritx. The simulation progress, TS and SE are implemented. The options MONITORING, PACK and ATLAS are not implemented for Spirit and Spiritx.

1.4. Mixte MPI-OMP

This has not been tested.

Last modified 4 days ago Last modified on 05/02/24 15:12:45