Working on TGCC

---

1. TGCC users' manual

​http://www-hpc.cea.fr/fr/complexe/tgcc.htm

2. TGCC's machines and file systems

3. How to install your environment on TGCC

• More information on the open-access website: ​http://www-hpc.cea.fr/fr/complexe/tgcc.htm
• Online access to the machines' users manual (you will need a TGCC login and password): ​https://www-tgcc.ccc.cea.fr/
• ​TGCC scripts environment
• The available TGCC's machine is currently curie (Bull Sandybridge).
• Note: the $HOME/.snapshot directory contains hourly, daily, and weekly backups of your $HOME files.

It is important to take the time to install a comfortable and efficient environment.

We suggest the user to use the p86ipsl login's environment (in bash) as an example (either copy or source the ~p86ipsl/.bashrc file). See the ~p86ipsl/.bashrc file

ryyy999@curie: cat ~/.bashrc
#------------------------------------------------------
# PLATFORM ENVIRONMENT
#------------------------------------------------------
source ~p86ipsl/.bashrc
ryyy999@curie: cp ~p86ipsl/.profile .

In this environement is specified:

• the path to the compiler tool fcm and to the rebuild tool which recombines output files from a parallel model:

  export PATH=~p86ipsl/fcm/bin:~p86ipsl/X64_CURIE/bin:$PATH
  

• the load of modules giving access to computing or post processing libraries and tools needed on our platform (done in ~p86ipsl/ .atlas_env_calcul_curie_ksh and ~p86ipsl/ .atlas_env_post_curie_ksh). The revision numbers may change, currently (21/01/2014) following modules are loaded for computing. Note the netcdf 3.6.3 library:

  module load ghostscript/9.04
  module load ferret/6.6.7
  module load netcdf/3.6.3
  module load gsl/1.14
  module load hdf5/1.8.8
  module load nco/4.0.5
  module load cdo/1.4.6
  module load netpbm/10.47.34
  module load imagemagick/6.7.4
  

The revision numbers may change, currently (21/01/2014) following modules are loaded for postprocessing:

module load ghostscript/9.04
module load ferret/6.6.7
module load netcdf/4.2
module load gsl/1.14
module load hdf5/1.8.8
module load nco/4.1.0
module load cdo/1.4.6
module load netpbm/10.47.34
module load imagemagick/6.7.4

4. Project and computing needs

• To find out the computing time used by the projects you are involved in (daily update):

  ryyy999@curie: ccc_myproject
  

• Specify in the header the project from which your job will use computing time:

  #MSUB -A genxxx
  

5. About file systems

5.1. Quotas

To check the available and used storage capacities of HOME, SCRATCH, CCCWORKDIR and CCCSTOREDIR:

ryyy999@curie: ccc_quota

On the curie machine this command will also return the space used by scratch (a specificity of the curie machine).

5.2. CCCWORKDIR

The $CCCWORKDIR directory corresponds to the $WORKDIR directory on curie. It is large but its content is not backed up.

5.3. CCCSTOREDIR

To manipulate the files in /ccc/store a few commands are useful:

# Demigrate a list of files on CCCSTOREDIR, see also "ccc_hsm -h"
ccc_hsm get $CCCSTOREDIR/FICHIER1 $CCCSTOREDIR/FICHIER2 ...

# Demigrate recursively the files from a CCCSTOREDIR directory, see also "ccc_hsm -h"
ccc_hsm get -r $CCCSTOREDIR/REPERTOIRE

# Find out the used space on CCCSTOREDIR
cd $CCCSTOREDIR ; find . -printf "%y %s %p \n" | awk '{ SUM+=$2 } END {print "SUM " SUM/1000000 " Mo " SUM/1000000000 " Go" }'

# or use --apparent-size with du :
du -sh --apparent-size

5.4. ccc_home command to know directory complete pathname

ccc_home could help you to find directory complete pathname for an other user or for you .

>ccc_home -h
ccc_home: Print the path of a user directory (default: home directory).
usage: ccc_home [ -H | -s | -t | -W | -A | -G | -a] [-u user]
                [-h, --help]

 -H, --home         :  (default) print the home directory path ($HOME)
 -s, -t, --scratch  :  print the scratch directory path   ($SCRATCHDIR)
 -W, --cccwork      :  print the CCC work directory path  ($CCCWORKDIR)
 -A, --cccstore     :  print the CCC store directory path ($CCCSTOREDIR)
 -G, --cccgenostore :  print the CCC genostore directory path ($CCCGENOSTOREDIR)
 -a, --all          :  print all paths
 -u user            :  show paths for the specified user instead of the current user
 -h, --help         :  display this help and exit

> ccc_home -A -u p86denv   
/ccc/store/cont003/dsm/p86denv

5.5. Storage spaces available from DODS

To store a file for the first time on dods, you must ask for dods write access by mail to the TGCC hotline access : hotline.tgcc@cea.fr.

6. End-of-job messages

To receive the end-of-job messages sent by the job itself: end of simulation, error,... you must specify your address in the $HOME/.forward file.

7. Simulation outputs

Final simulation outputs are stored in $CCCSTOREDIR/IGCM_OUT and on $CCCWORKDIR/IGCM_OUT regarding the ATLAS and MONITORING directories.

The dods servers on TGCC are available via: dods.extra.cea.fr/store (files such as Analyse/TS and Analyse/SE) and dods.extra.cea.fr/work for ATLAS and MONITORING.

8. About password

ccc_password_expiration helps you to know expiration date of your password. Currently password have to be changed one time per year.

 > ccc_password_expiration
Password for xxxxx@USERS-CCRT.CCC.CEA.FR: PPPPPPPPPP
Your password will expire in 70 days on Fri Nov 22 08:42:59 2013
 > ccc_password_expiration -h
Usage: ccc_password_expiration [username[@realm]]

9. The TGCC's machines

• Curie