Shared folders

In this section you will find information about the MICT shared folder and how to Contribute.

HPC's common folder is available at:

• Obelix (primary)

In the near future Curie and ADA will also have the files ( mirroring ).

Obelix

• /home/surface7/mict/ -> general files
• /home/surface7/mict/data -> data files (forcings, ...)

Long term

For files required to be saved for a longterm period use the tapes (~300TB) system.

More info: ​https://intranet.lsce.ipsl.fr/informatique/en/stage/index.php

What is archive:

The LSCE long term archive system allows users to store on a tape library old files that need to be kept.
Files stored on the system are never destroyed and can always be retrieved and brought-back on disks, as long as the tape is not destroyed.
A data tape is a passive device and its expected life-time is supposed to be many years (theory says 30 years...).
The LSCE tape library uses LTO5 tapes cartridges and the total space is around 300Tb.

BEWARE: As in all electronic systems, failure and errors can occur. There is no guarantee that the files can be retrieved...
Usage:

All files and directories under /home can be archived.

Commands available:

    stageput to save files and directories
    stagein to get back files and directories
    stagelist to list saved files
    stageq to list current operations (save/retry) in queue
    stageclear to remove the file's entry from the tape/file database.

All these have some important options. Read the man for more details.

The system does not automatically destroy files after saving them.

The archive system is available from all asterix and obelix computers (including the batch system).
To use it directly from a linux desktop (Fedora), run the following command to install the software:

    yum -y install stager-client-lsce 

Importante notes:

    No file can be removed from a tape.
    Confidentiality of archived files can't be guaranteed.

Good rules:

    For efficiency, it's better not to write small files on tapes. To save a big "tar" file is much better than saving a lot of small files.
    The minimum size for a file should be 1Gb.
    Give a good name to the file, to retrieve it easily.

Contributing

Files place in the MICT folder must be specific to this branch. Each developper is responsible to take care of its files. The basic structure (mict and mict/data) won't change. New folders can be created depending on our needs. Use common sense.

Versioning files

In case you created different versions of the same file. You can find here a suggestion to maintain them to the rest of the group

• lai_something_0.1_01022016.nc
• lai_something_0.2_01022017.nc
• lai_something_0.3_01022018.nc

• lai_something.nc -> link to lai_something_0.3_01022018.nc (the latest version of the file)

The ​symbolic link always point out to the latest version of the file. Users won't have to worry to change their simulations to refer to the latest. How to create a symbolic link in Linux:

ln -s lai_something.nc lai_something_0.3_01022018.nc