source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_obelix.ksh @ 1624

Last change on this file since 1624 was 1624, checked in by jgipsl, 5 months ago

Added possibility to declare a restart file as optional which means that libIGCM want stop if the file is not found. Only a warning will be written.

For example, in the following case, if the routing_restart file is not available when starting the model or if it is not produced, libIGCM will not stop. But if the file sechiba_rest is not available, libIGCM will stop as usual.

[RestartFiles]
List=   (sechiba_rest_out.nc, sechiba_rest.nc, sechiba_rest_in.nc) ,\
        (routing_restart.nc, routing_restart.nc, routing_start.nc, OPTIONAL)
  • Read optional argument OPTIONAL on the line where the restart file is declared.
  • Each time when a restart file is not found for coping in or out, if the argument OPTIONAL is set, then just continue. If not set, the file is considered as MANDATORY and libIGCM will exist if it is missing as before.
  • on irene-amd and jean-zay, corrected a bug for the case in config.card DataProject?=DEFAULT


  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 41.5 KB
Line 
1#!/bin/ksh
2
3#**************************************************************
4# Author: Sebastien Denvil, Martial Mancip, Christian Laguerre
5# Contact: Martial.Mancip__at__ipsl.jussieu.fr
6# $Revision::                                          $ Revision of last commit
7# $Author::                                            $ Author of last commit
8# $Date::                                              $ Date of last commit
9# IPSL (2006)
10#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11#
12#**************************************************************
13
14#=========================================================
15# The documentation of this file can be automatically generated
16# if you use the prefix #D- for comments to be extracted.
17# Extract with command: cat lib* | grep "^#D-" | cut -c "4-"
18#=========================================================
19
20#D-#==================================================
21#D-LibIGCM_sys for obelix
22#D-#==================================================
23#D-
24#D- This ksh library if a layer under some usefull
25#D-environment variables and shell commands.
26#D-All those definitions depend on host particularities.
27#D-It manages a stack mechanism and test validity of operations.
28#D-All function described bellow must be prefixed by IGCM_sys.
29
30#====================================================
31# libIGCM_sys PARAMETERS
32#====================================================
33
34#====================================================
35# set DEBUG_sys to true to output calls of function
36typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38#====================================================
39# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40if ([ X${DRYRUN} != X ] && [ ${DRYRUN} = 4 ]) ; then
41  typeset -r DRYRUN=0
42  typeset -r DRYRUN_DEBUG=4
43else
44  typeset -r DRYRUN=${DRYRUN:=0}
45  typeset -r DRYRUN_DEBUG=0
46fi
47
48# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
49# -------------------------------------------------------------------------------------
50# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
51# |          |  Cp/Exe/param/files |            |         |                           |
52# |          |  Chmod Qsub         |            |         |                           |
53# -------------------------------------------------------------------------------------
54# |    0     |       yes           |    yes     |  yes    |      yes                  |
55# -------------------------------------------------------------------------------------
56# |    1     |       yes           |    yes     |  yes    |      no                   |
57# -------------------------------------------------------------------------------------
58# |    2     |       yes           |    yes     |  no     |      no                   |
59# -------------------------------------------------------------------------------------
60# |    3     |       yes           |    no      |  no     |      no                   |
61# -------------------------------------------------------------------------------------
62# |    4     |       yes           |    yes     |  creation of RUN_DIR and Job_debug  |
63# -------------------------------------------------------------------------------------
64
65
66#=====================================================
67# Global Variables :
68#=====================================================
69# Language : "fr" or "en"
70typeset -r MYLANG="fr"
71
72#=====================================================
73# Host user names project maxCpuTime
74# $hostname ou hostname
75typeset HOST=${HOST:=$( hostname )}
76# $username ou whoami
77typeset LOGIN=${LOGIN:=$( whoami )}
78# $hostname of the MASTER job
79typeset MASTER=${MASTER:=$( hostname )}
80# project name
81typeset PROJECT=NONE
82# jobWarningDelay in seconds
83typeset jobWarningDelay=${PBS_WALLTIME}
84
85#D-
86#D-#==================================================
87#D-Program used in libIGCM
88#D-#==================================================
89
90# Submit command
91typeset SUBMIT=${SUBMIT:=qsub}
92# rsync with path
93typeset -r RSYNC=/usr/bin/rsync
94# RSYNC_opt args to rsync
95typeset -r RSYNC_opt="-va"
96# ie storage filesystem
97typeset -r STOREHOST=${MASTER}
98
99#====================================================
100# Access to module command
101#====================================================
102. /usr/share/Modules/init/ksh
103
104#====================================================
105# Set environment tools (ferret, nco, cdo, rebuild, ...)
106#====================================================
107if [ X${TaskType} = Xcomputing ] ; then
108    IGCM_debug_Print 1 "Modules will be loaded later in IGCM_sys_activ_variables."
109else
110    . /home/orchideeshare/igcmg/MachineEnvironment/obelix/env_atlas_obelix
111fi
112
113[ ! X${TaskType} = Xchecking ] && IGCM_debug_Print 1 "List of loaded modules:"
114[ ! X${TaskType} = Xchecking ] && module list
115
116# Load python
117module load python/2.7.5
118
119#====================================================
120# Host specific DIRECTORIES
121#====================================================
122# For rebuild
123export PATH=${PATH}:${MODIPSL}/modeles/IOIPSL/bin:/home/orchideeshare/igcmg/IOIPSL_PLUS/modipsl.tagv2_2_4.28082019/bin/
124
125#====================================================
126#- MirrorlibIGCM for frontend
127typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
128
129#====================================================
130#- libIGCM_POST for frontend
131typeset -r libIGCM_POST=${libIGCM}
132
133#====================================================
134#- R_EXE   (==> BIN_DIR = ${MODIPSL}/bin )
135typeset -r R_EXE="${MODIPSL}/bin"
136
137#====================================================
138#- SUBMIT_DIR : submission dir
139if [ X${PBS_O_WORKDIR} != X ] ; then
140  typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
141else
142  typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PWD}}
143fi
144
145#====================================================
146#- IN
147typeset -r R_IN=${R_IN:=/home/orchideeshare/igcmg/IGCM}
148
149#====================================================
150#- RUN_DIR_PATH : Temporary working directory (=> TMP)
151if [ X${PBS_JOBID} != X ] ; then
152    typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/home/scratch01/$LOGIN/RUN_DIR/$PBS_O_LOGNAME.$PBS_JOBID}
153else
154    typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/home/scratch01/$LOGIN/RUN_DIR/tmp$$}
155fi
156
157
158#====================================================
159#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
160typeset -r OUTCOMMAND_PATH=/tmp
161
162#====================================================
163#- HOST_MPIRUN_COMMAND
164typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="time mpirun"}
165
166#====================================================
167#- Max number of arguments passed to nco operator or demigration command
168UNIX_MAX_LIMIT=360
169
170#====================================================
171#- set PackDefault to false on obelix
172PackDefault=false
173
174#====================================================
175#- Default number of MPI task for IPSL coupled model
176#- required for backward compatibility
177#-
178DEFAULT_NUM_PROC_OCE=5
179DEFAULT_NUM_PROC_CPL=1
180(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
181
182#D-#==================================================
183#D-function IGCM_sys_defineArchives
184#D-* Purpose:
185#D-* Define ARCHIVE : Dedicated to large files
186#D-* Define STORAGE : Dedicated to small/medium files
187#D-* Define R_OUT   : Output tree located on ARCHIVE
188#D-* Define R_FIG   : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
189#D-* Define R_BUF   : Output tree located on STORAGE hosting files waiting for rebuild or pack processes
190#D-* if SpaceName=TEST nothing special will hapen
191#D-* Examples:
192#D-
193function IGCM_sys_defineArchives {
194  IGCM_debug_PushStack "IGCM_sys_defineArchives"
195
196  if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
197    #====================================================
198    #- ARCHIVE (dedicated to large files)
199    ARCHIVE=${config_UserChoices_ARCHIVE}
200  else
201    #====================================================
202    #- ARCHIVE (dedicated to large files)
203    ARCHIVE=${ARCHIVE:=/home/scratch01/${LOGIN}}
204  fi
205
206  if [ ! X${config_UserChoices_STORAGE} = X ]; then
207    #====================================================
208    #- STORAGE (dedicated to small/medium files)
209    STORAGE=${config_UserChoices_STORAGE}
210  else
211    #====================================================
212    #- STORAGE (dedicated to small/medium files)
213    STORAGE=${ARCHIVE}
214  fi
215
216  # ON OBELIX NO SPECIAL CASE WHEN X${config_UserChoices_SpaceName} = XTEST
217
218  #====================================================
219  #- R_OUT
220  R_OUT=${ARCHIVE}/IGCM_OUT
221
222  #====================================================
223  #- R_FIG (hosting figures : monitoring and atlas, and/or small files)
224  R_FIG=${STORAGE}/IGCM_OUT
225
226  #====================================================
227  #- CMIP6 (hosting CMIP6 files produced by XIOS2 and configured by dr2xml)
228  CMIP6_BUF=${STORAGE}/IGCM_OUT
229
230  #====================================================
231  #- R_BUF (ONLY FOR double copy an scratch)
232  R_BUF=${STORAGE}/IGCM_OUT
233
234  IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
235  IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
236  IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
237
238  IGCM_debug_PopStack "IGCM_sys_defineArchives"
239}
240
241#D-#==================================================
242#D-function IGCM_sys_RshArchive
243#D-* Purpose: Archive rsh command
244#D-* Examples:
245#D-
246function IGCM_sys_RshArchive {
247  IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
248  /bin/ksh <<-EOF
249    ${@}
250EOF
251  status=$?
252  if [ ${status} -gt 0 ] ; then
253    IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
254    IGCM_debug_Exit "IGCM_sys_RshArchive"
255  fi
256  IGCM_debug_PopStack "IGCM_sys_RshArchive"
257}
258
259#D-#==================================================
260#D-function IGCM_sys_RshArchive_NoError
261#D-* Purpose: Archive rsh command, without error
262#D-*          used only in monitoring.job
263#D-* Examples:
264#D-
265function IGCM_sys_RshArchive_NoError {
266  IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
267  /bin/ksh <<-EOF
268    ${@} 2> /dev/null
269EOF
270  IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
271}
272
273#D-#==================================================
274#D-function IGCM_sys_MkdirArchive
275#D-* Purpose: Mkdir on Archive
276#D-* Examples:
277#D-
278function IGCM_sys_MkdirArchive {
279  IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
280  if ( $DEBUG_sys ) ; then
281    echo "IGCM_sys_MkdirArchive :" $@
282  fi
283  #- creation de repertoire sur le serveur fichier
284  if [ ! -d ${1} ]; then
285    \mkdir -p $1
286    status=$?
287
288    if [ ${status} -gt 0 ] ; then
289      IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
290      IGCM_debug_Exit "IGCM_sys_MkdirArchive"
291    fi
292  fi
293  IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
294}
295
296#D-#==================================================
297#D-function IGCM_sys_TestDirArchive
298#D-* Purpose: Test Directory that must exists on Archive
299#D-* Examples:
300#D-
301function IGCM_sys_TestDirArchive {
302  IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
303  if ( $DEBUG_sys ) ; then
304    echo "IGCM_sys_TestDirArchive :" $@
305  fi
306  typeset ExistFlag
307  ExistFlag=$( [ -d $1 ] && echo 0 || echo 1 )
308  IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
309  return ${ExistFlag}
310}
311
312#D-#==================================================
313#D-function IGCM_sys_IsFileArchived
314#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
315#D-* Examples:
316#D-
317function IGCM_sys_IsFileArchived {
318  IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
319  if ( $DEBUG_sys ) ; then
320    echo "IGCM_sys_IsFileArchived :" $@
321  fi
322  typeset IsArchivedFlag
323  # Never archived for this system
324  IsArchivedFlag=1
325  IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
326
327  return ${IsArchivedFlag}
328}
329
330#D-#==================================================
331#D-function IGCM_sys_TestFileArchive
332#D-* Purpose: Test file that must NOT EXISTS on Archive
333#D-* Examples:
334#D-
335function IGCM_sys_TestFileArchive {
336  IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
337  typeset ExistFlag
338  ExistFlag=$( [ -f $1 ] && echo 0 || echo 1 )
339  IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
340
341  return ${ExistFlag}
342}
343
344#D-#==================================================
345#D-function IGCM_sys_CountFileArchive
346#D-* Purpose: Count files on Archive filesystem
347#D-* Examples:
348#D-
349function IGCM_sys_CountFileArchive {
350  IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
351  ls ${@} 2>/dev/null | wc -l
352  if [ $? -gt 0 ] ; then
353    echo "IGCM_sys_CountFileArchive : erreur."
354  fi
355  IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
356}
357
358#D-#==================================================
359#D-function IGCM_sys_Tree
360#D-* Purpose: Tree directories with files on ${ARCHIVE}
361#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
362#D-
363function IGCM_sys_Tree {
364  IGCM_debug_PushStack "IGCM_sys_Tree" $@
365  if ( $DEBUG_sys ) ; then
366    echo "IGCM_sys_Tree :" $@
367  fi
368
369  \ls -lR ${@}
370
371  IGCM_debug_PopStack "IGCM_sys_Tree"
372}
373
374#D-#==================================================
375#D-function IGCM_sys_Qsub
376#D-* Purpose: Qsub new job
377#D-* Examples:
378#D-
379function IGCM_sys_Qsub {
380  IGCM_debug_PushStack "IGCM_sys_Qsub" $@
381  if ( $DEBUG_sys ) ; then
382    echo "IGCM_sys_Qsub :" $@
383  fi
384  typeset options status
385  options="-o ${SUBMIT_DIR}/${Script_Output}"
386  /usr/local/bin/qsub ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
387  status=$?
388
389  cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
390  if [ ${status} -gt 0 ] ; then
391    IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
392    IGCM_debug_Exit "IGCM_sys_Qsub"
393  else
394    JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
395  fi
396  IGCM_debug_PopStack "IGCM_sys_Qsub"
397}
398
399#D-#==================================================
400#D-function IGCM_sys_QsubPost
401#D-* Purpose: Qsub new job on scalaire
402#D-* Examples:
403#D-
404function IGCM_sys_QsubPost {
405  IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
406  if ( $DEBUG_sys ) ; then
407    echo "IGCM_sys_QsubPost :" $@
408  fi
409  typeset options status
410  options="-o ${POST_DIR}/${Script_Post_Output}.out -v ${listVarEnv}"
411  /usr/local/bin/qsub ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
412  status=$?
413
414  cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
415  if [ ${status} -gt 0 ] ; then
416    IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
417    IGCM_debug_Exit "IGCM_sys_QsubPost"
418  else
419    JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
420  fi
421  IGCM_debug_PopStack "IGCM_sys_QsubPost"
422}
423
424#D-*************************
425#D- File transfer functions
426#D-*************************
427#D-
428
429#D-#==================================================
430#D-function IGCM_sys_RmRunDir
431#D-* Purpose: rm tmpdir (dummy function most of the time batch
432#D-                      scheduler will do the job)
433#D-* Examples:
434#D-
435function IGCM_sys_RmRunDir {
436  IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
437  if ( $DEBUG_sys ) ; then
438    echo "IGCM_sys_RmRunDir :" $@
439  fi
440
441  typeset status
442
443  echo rm $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
444  \rm $@ >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
445  status=$?
446
447  if [ ${status} -gt 0 ] ; then
448    IGCM_debug_Print 1 "IGCM_sys_RmRunDir : rm error code is ${status}."
449    cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
450    IGCM_debug_Exit "IGCM_sys_RmRunDir"
451  else
452    \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
453  fi
454  IGCM_debug_PopStack "IGCM_sys_RmRunDir"
455}
456
457#D-#==================================================
458#D-function IGCM_sys_Put_Dir
459#D-* Purpose: Copy a complete directory on $(ARCHIVE)
460#D-* Examples:
461#D-
462function IGCM_sys_Put_Dir {
463  IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
464  if ( $DEBUG_sys ) ; then
465    echo "IGCM_sys_Put_Dir :" $@
466  fi
467  if [ $DRYRUN = 0 ]; then
468    if [ ! -d ${1} ] ; then
469      echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
470      IGCM_debug_PopStack "IGCM_sys_Put_Dir"
471      return
472    fi
473
474    typeset status
475
476    # Only if we use rsync
477    #IGCM_sys_TestDirArchive $( dirname $2 )
478    #
479    #USUAL WAY
480    \cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
481    status=$?
482
483    if [ ${status} -gt 0 ] ; then
484      IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
485      cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
486      IGCM_debug_Exit "IGCM_sys_Put_Dir"
487    else
488      \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
489    fi
490  fi
491  IGCM_debug_PopStack "IGCM_sys_Put_Dir"
492}
493
494#D-#==================================================
495#D-function IGCM_sys_Get_Dir
496#D-* Purpose: Copy a complete directory from ${ARCHIVE}
497#D-* Examples:
498#D-
499function IGCM_sys_Get_Dir {
500  IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
501  if ( $DEBUG_sys ) ; then
502    echo "IGCM_sys_Get_Dir :" $@
503  fi
504  if [ $DRYRUN = 0 ]; then
505    typeset NB_ESSAI DELAI status i
506    # number of tentative
507    NB_ESSAI=3
508    # time delay between tentative
509    DELAI=2
510
511    #
512    # USUAL WAY
513    i=0
514    while [ $i -lt $NB_ESSAI ] ; do
515      \cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
516      status=$?
517      if [ ${status} -gt 0 ] ; then
518        IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
519        IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
520        sleep $DELAI
521      else
522        break
523      fi
524      (( i = i + 1 ))
525    done
526
527    if [ ${status} -gt 0 ] ; then
528      IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
529      cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
530      IGCM_debug_Exit "IGCM_sys_Get_Dir"
531    else
532      \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
533    fi
534  fi
535  IGCM_debug_PopStack "IGCM_sys_Get_Dir"
536}
537
538#D-#==================================================
539#D-function IGCM_sys_Put_Rest
540#D-* Purpose: Put computied restarts on ${ARCHIVE}.
541#D-           File and target directory must exist.
542#D-* Examples:
543#D-
544function IGCM_sys_Put_Rest {
545  IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
546  if ( $DEBUG_sys ) ; then
547    echo "IGCM_sys_Put_Rest :" $@
548  fi
549  if [ $DRYRUN = 0 ]; then
550     
551    if [ ! -f ${1} ] ; then
552        # The file is missing     
553        if [ X${3} = X'OPTIONAL' ] ; then
554            echo "WARINING : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
555            IGCM_debug_Print 1 "IGCM_sys_Put_Rest: Optional restart file ${1} is missing, now continue for the next"
556        else
557            echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
558            IGCM_debug_Exit "IGCM_sys_Put_Rest"
559        fi
560
561    else
562      # The file exist
563
564    typeset status
565    #
566    # USUAL WAY
567    \cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
568    status=$?
569
570#       #RSYNC WITH NETWORK SSH CALL
571#       echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
572#       ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
573
574#       #RSYNC WITH NFS USE
575#       echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
576#       ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
577
578#       status=$?
579#       IGCM_sys_Rsync_out $status
580
581#       ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
582#       (( status=status+$? ))
583
584    if [ ${status} -gt 0 ] ; then
585      IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
586      [ -f ${1} ] && ls -l ${1}
587      [ -f ${2} ] && ls -l ${2}
588      [ -f ${2}/${1} ] && ls -l ${2}/${1}
589      cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
590      IGCM_debug_Exit "IGCM_sys_Put_Rest"
591    else
592
593      if [ X${JobType} = XRUN ] ; then
594        [ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
595        [ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
596      fi
597
598      \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
599    fi
600    fi
601  fi
602  IGCM_debug_PopStack "IGCM_sys_Put_Rest"
603}
604
605#D-#==================================================
606#D-function IGCM_sys_Put_Out
607#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
608#D-* Examples:
609#D-
610function IGCM_sys_Put_Out {
611  IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
612  if ( $DEBUG_sys ) ; then
613    echo "IGCM_sys_Put_Out :" $@
614  fi
615
616  typeset status
617
618  if [ $DRYRUN = 0 ]; then
619    if [ ! -f ${1} ] ; then
620      echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
621      IGCM_debug_PopStack "IGCM_sys_Put_Out"
622      return 1
623    fi
624    #
625    IGCM_sys_MkdirArchive $( dirname $2 )
626    #
627    if [ X${JobType} = XRUN ] ; then
628      if [ X${3} = X ] ; then
629        IGCM_sys_Chmod 444 ${1}
630      fi
631    fi
632    #
633
634    echo ${RSYNC} ${RSYNC_opt} $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
635    ${RSYNC} ${RSYNC_opt} $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
636    status=$?
637    IGCM_sys_Rsync_out $status
638
639    ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
640    (( status=status+$? ))
641
642    if [ ${status} -gt 0 ] ; then
643      IGCM_debug_Print 2 "IGCM_sys_Put_Out : rsync failed error code ${status}"
644      cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
645      IGCM_debug_Exit "IGCM_sys_Put_Out"
646    else
647      \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
648    fi
649  fi
650  IGCM_debug_PopStack "IGCM_sys_Put_Out"
651  return 0
652}
653
654#D-#==================================================
655#D-function IGCM_sys_Get
656#D-* Purpose: Get a file from ${ARCHIVE}
657#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
658#D-            IGCM_sys_Get /l Array_contain_myfiles /destpath/
659function IGCM_sys_Get {
660  IGCM_debug_PushStack "IGCM_sys_Get" $@
661
662  typeset DEST status dm_liste
663
664  if ( $DEBUG_sys ) ; then
665    echo "IGCM_sys_Get :" $@
666  fi
667  if [ $DRYRUN -le 2 ]; then
668    if [ X${1} = X'/l' ] ; then
669      eval set +A dm_liste \${${2}}
670    else
671      dm_liste=${1}
672    fi
673    eval DEST=\${${#}}
674
675    # test if the (first) file is present in the old computation :
676    IGCM_sys_TestFileArchive ${dm_liste[0]}
677    status=$?
678    if [ ${status} -gt 0 ] ; then
679      echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
680      IGCM_debug_Exit "IGCM_sys_Get"
681      return
682    fi
683
684    #USUAL WAY
685    \cp ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
686    status=$?
687
688    if [ ${status} -gt 0 ] ; then
689      IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status}"
690      cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
691      IGCM_debug_Exit "IGCM_sys_Get"
692    else
693      \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
694    fi
695  fi
696  IGCM_debug_PopStack "IGCM_sys_Get"
697}
698
699#D-#==================================================
700#D-function IGCM_sys_GetDate_Monitoring
701#D-* Purpose: get the last year for which the monitoring has been computed
702#D-* Examples:
703#D-
704function IGCM_sys_GetDate_Monitoring {
705  IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
706  if ( $DEBUG_sys ) ; then
707    echo "IGCM_sys_GetDate_Monitoring :" $@
708  fi
709
710  eval ${2}=$( cdo showyear ${1} 2> /dev/null | gawk '{ print $NF }' )
711
712  IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
713}
714
715#D-#==================================================
716#D-function IGCM_sys_Dods_Rm
717#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
718#D-* Examples:
719#D-
720function IGCM_sys_Dods_Rm {
721  if ( $DEBUG_sys ) ; then
722    echo "IGCM_sys_Dods_Rm :" $@
723  fi
724  return 0
725}
726
727#D-#==================================================
728#D-function IGCM_sys_Dods_Cp
729#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
730#D-* Examples:
731#D-
732function IGCM_sys_Dods_Cp {
733  if ( $DEBUG_sys ) ; then
734    echo "IGCM_sys_Dods_Cp :" $@
735  fi
736  return 0
737}
738
739#D-#==================================================
740#D-function IGCM_sys_Put_Dods
741#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
742#D-* Examples:
743#D-
744function IGCM_sys_Put_Dods {
745  IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
746  if ( $DEBUG_sys ) ; then
747    echo "IGCM_sys_Put_Dods :" $@
748  fi
749  IGCM_debug_PopStack "IGCM_sys_Put_Dods"
750}
751
752##############################################################
753# REBUILD OPERATOR
754
755#D-#==================================================
756#D-function IGCM_sys_sync
757#D-* Purpose: flush buffer on disk (dummy function on Obelix)
758#D-* Examples:
759#D-
760function IGCM_sys_sync {
761  IGCM_debug_PushStack "IGCM_sys_sync" $@
762  if ( $DEBUG_sys ) ; then
763    echo "IGCM_sys_sync :" $@
764    echo "Dummy call, let the system do that."
765  fi
766  IGCM_debug_PopStack "IGCM_sys_sync"
767}
768
769############################################################
770# Activate Running Environnment Variables
771
772#D-#==================================================
773#D-function IGCM_sys_activ_variables
774#D-* Purpose: set environement variables prior to execution
775#D-* Examples:
776#D-
777function IGCM_sys_activ_variables {
778  IGCM_debug_PushStack "IGCM_sys_activ_variables"
779  if ( $DEBUG_sys ) ; then
780    echo "IGCM_sys_activ_variables"
781  fi
782
783# --------------------------------------------------------------------
784#D- MODULE specifications
785# --------------------------------------------------------------------
786  # Source the file EnvFile containing all module load needed to run the model.
787  # EnvFile can be specified in config.card. If this is the case and if the file exists,
788  # it will be used. Otherwise a default file will be used.
789  if [ ! X${config_UserChoices_EnvFile} = X ] && [ -f ${config_UserChoices_EnvFile} ] ; then
790      # EnvFile is set in config.card and the file exists
791      IGCM_debug_Print 1 "EnvFile set in config.card will be used"
792      EnvFile=${config_UserChoices_EnvFile}
793  else
794      IGCM_debug_Print 1 "IGCM_sys_active_variables : Default modules will be used"
795      EnvFile=/home/orchideeshare/igcmg/MachineEnvironment/obelix/env_atlas_obelix
796  fi
797  IGCM_debug_Print 1 "IGCM_sys_active_variables : Following EnvFile will be used :" 
798  IGCM_debug_Print 1 ${EnvFile}
799  . ${EnvFile}
800
801  IGCM_debug_Print 1 "IGCM_sys_active_variables : Now loaded modules for obelix. "
802  module list
803
804# --------------------------------------------------------------------
805#D- MPI specifications
806# --------------------------------------------------------------------
807
808# --------------------------------------------------------------------
809#D- Other specifications
810# --------------------------------------------------------------------
811
812  ulimit -s unlimited
813
814  IGCM_debug_PopStack "IGCM_sys_activ_variables"
815}
816
817############################################################
818# Desactivate Running Environnment Variables
819
820#D-#==================================================
821#D-function IGCM_sys_desactiv_variables
822#D-* Purpose: unset environement variables after execution
823#D-* Examples:
824#D-
825function IGCM_sys_desactiv_variables {
826  IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
827  if ( $DEBUG_sys ) ; then
828    echo "IGCM_sys_desactiv_variables"
829  fi
830# --------------------------------------------------------------------
831#D- MPI specifications
832# --------------------------------------------------------------------
833
834# --------------------------------------------------------------------
835#D- Other specifications
836# --------------------------------------------------------------------
837
838  IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
839}
840
841############################################################
842# Update job headers to be used by the scheduler
843
844#D-#==================================================
845#D-function IGCM_sys_updateHeaders
846#D-* Purpose: Update job headers to be used by the scheduler
847#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
848#D-
849function IGCM_sys_updateHeaders {
850  IGCM_debug_PushStack "IGCM_sys_updateHeaders"
851  if ( $DEBUG_sys ) ; then
852    echo "IGCM_sys_updateHeaders"
853  fi
854  typeset file
855  file=$1
856
857  if [ ${executionType} -eq 1 ] ; then
858    # MPMD + MPI
859    sed -e "/::openMPthreads::/d"                  \
860        -e "s/::JobNumProcTot::/${coreNumber}/"    \
861      ${file} > ${file}.tmp
862
863  elif [ ${executionType} -eq 2 ] ; then
864    # MPMD + MPI + OMP
865    sed -e "s/::openMPthreads::/${openMPthreads}/" \
866        -e "s/::JobNumProcTot::/${coreNumber}/"    \
867      ${file} > ${file}.tmp
868
869  elif [ ${executionType} -eq 3 ] ; then
870    # SPMD + MPI/OMP
871    sed -e "s/::openMPthreads::/${openMPthreads}/" \
872        -e "s/::JobNumProcTot::/${mpiTasks}/"      \
873      ${file} > ${file}.tmp
874
875  elif [ ${executionType} -eq 4 ] ; then
876    # SPMD + MPI only
877    sed -e "s/::JobNumProcTot::/${mpiTasks}/"      \
878        -e "/::openMPthreads::/d"                  \
879      ${file} > ${file}.tmp
880
881  elif [ ${executionType} -eq 5 ] ; then
882    # SPMD + OMP only
883    sed -e "s/::openMPthreads::/${openMPthreads}/" \
884        -e "/::JobNumProcTot::/d"                  \
885      ${file} > ${file}.tmp
886
887  elif [ ${executionType} -eq 6 ] ; then
888    # SEQUENTIAL THEN
889    sed -e "s/::JobNumProcTot::/1/"                \
890        -e "/::openMPthreads::/d"                  \
891      ${file} > ${file}.tmp
892
893  fi
894
895  IGCM_sys_Mv ${file}.tmp ${file}
896
897  IGCM_debug_PopStack "IGCM_sys_updateHeaders"
898}
899
900############################################################
901# Build MPI/OMP scripts run file (dummy function)
902
903#D-#==================================================
904#D-function IGCM_sys_build_run_file
905#D-* Purpose: build run file (deprecated)
906#D-* Examples:
907#D-
908function IGCM_sys_build_run_file {
909  IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
910}
911
912############################################################
913# Build MPI/OMP scripts
914
915#D-#==================================================
916#D-function IGCM_sys_build_execution_scripts
917#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
918#D-* Examples:
919#D-
920function IGCM_sys_build_execution_scripts
921{
922  IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
923  if ( $DEBUG_sys ) ; then
924    echo "IGCM_sys_build_execution_scripts " $@
925  fi
926
927  EXECUTION=${HOST_MPIRUN_COMMAND}
928
929  # MPMD mode
930  if ( ${OK_PARA_MPMD} ) ; then
931
932    # Only MPI (MPMD)
933    if  ( ! ${OK_PARA_OMP} ) ; then
934
935      if [ -f run_file ] ; then
936        IGCM_sys_Rm -f run_file
937      fi
938      touch run_file
939
940      # Build run_file
941
942      # First loop on the components for the coupler ie oasis (only if oasis3)
943      # the coupler ie oasis3 must be the first one
944      for comp in ${config_ListOfComponents[*]} ; do
945
946        eval ExeNameIn=\${config_Executable_${comp}[0]}
947        eval ExeNameOut=\${config_Executable_${comp}[1]}
948
949        # for CPL component only
950        if [ "X${comp}" = "XCPL" ]  && [ "X${ExeNameOut}" != X\"\" ] ; then
951          eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
952          eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
953          echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut} " >> run_file
954        fi
955      done
956
957      # Then second loop on the components
958      for comp in ${config_ListOfComponents[*]} ; do
959
960        eval ExeNameIn=\${config_Executable_${comp}[0]}
961        eval ExeNameOut=\${config_Executable_${comp}[1]}
962
963        # Only if we really have an executable for the component and not the coupler ie oasis:
964        if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
965
966          eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
967          eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
968          echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
969        fi
970      done
971
972      EXECUTION="${HOST_MPIRUN_COMMAND} --app ./run_file"
973
974      IGCM_sys_Chmod u+x run_file
975      if ( $DEBUG_sys ) ; then
976        echo "run_file contains : "
977        cat run_file
978      fi
979 
980    # MPI-OpenMP (MPMD)
981    else
982
983      #  Hosts treatment
984      ${HOST_MPIRUN_COMMAND} hostname | sort | uniq > hosts.tmp
985
986      i=0
987      rm -f hosts
988      IGCM_debug_Print 1 "sys Obelix, Hosts available :"
989      for nodes in `cat hosts.tmp` ; do
990        host[$i]=$nodes
991        echo "${host[$i]} slots=1 max_slots=1" >> hosts
992        IGCM_debug_Print 1 ${host[$i]}
993        i=$((i+1))
994      done
995      rm -f hosts.tmp
996
997      listnodes=${host[*]}
998
999      EXECUTION="${HOST_MPIRUN_COMMAND} -hostfile hosts"
1000
1001      # Initialisation
1002      rank=0
1003      current_core=0
1004      core_per_node=6
1005      init_exec=n
1006
1007      # Loop on the components
1008      for comp in ${config_ListOfComponents[*]} ; do
1009
1010        eval ExeNameIn=\${config_Executable_${comp}[0]}
1011        eval ExeNameOut=\${config_Executable_${comp}[1]}
1012
1013        # Not possible if oasis has an executable (i.e old version of oasis3)
1014        if ( [ "X${ExeNameOut}" != X\"\" ] &&  [ "X${comp}" = "XCPL" ] ) ; then
1015          IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
1016          IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
1017          IGCM_debug_Verif_Exit
1018        fi
1019
1020        # Only if we really have an executable for the component :
1021        if [ "X${ExeNameOut}" != X\"\" ] ; then
1022
1023          eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
1024          eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1025
1026          echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1027          echo ""  >> script_${ExeNameOut}.ksh
1028          if [ ${comp_proc_omp_loc} -gt 1 ] ; then
1029
1030            # Check if the number of threads is correct
1031            case ${comp_proc_omp_loc} in
1032            2|4|6)
1033              IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
1034              ;;
1035            *)
1036              IGCM_debug_Exit "ERROR with OMP parameters !"
1037              IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
1038              IGCM_debug_Print 2 "Only 2,4,6 as number of OMP threads are possible "
1039              IGCM_debug_Verif_Exit
1040              ;;
1041            esac
1042            #echo "export KMP_STACKSIZE=3g"  >> script_${ExeNameOut}.ksh
1043            #echo "export KMP_LIBRARY=turnaround"  >> script_${ExeNameOut}.ksh
1044            #echo "export MKL_SERIAL=YES"  >> script_${ExeNameOut}.ksh
1045            echo "export OMP_STACKSIZE=200M" >> script_${ExeNameOut}.ksh
1046            echo "export OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1047          fi
1048
1049          #echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
1050          echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK )) " >> script_${ExeNameOut}.ksh
1051          echo "MYMPIRANK=\$(printf '%3.3d\n' \${MYMPIRANK})" >> script_${ExeNameOut}.ksh
1052          echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
1053          IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1054
1055          if [ ${init_exec} = y ] ; then
1056            EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
1057          else
1058            EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
1059            init_exec=y
1060          fi
1061
1062          # Build rankfile : method used to assign cores and nodes for the MPI process
1063          # Ex :
1064          #rank 0=curie5296 slot=0,1,2,3
1065          #rank 1=curie5296 slot=4,5,6,7
1066          # Example of final command :
1067          # mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
1068          # with script_lmdz.x.ksh :
1069          # #!/bin/ksh
1070          #export OMP_STACKSIZE=200M
1071          #export OMP_NUM_THREADS=4
1072          #./lmdz.x
1073
1074          for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
1075            (( index_host = current_core / core_per_node ))
1076            host_value=${host[${index_host}]}
1077            (( slot =  current_core % core_per_node ))
1078            virg=","
1079            string_final=""
1080            for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
1081              string=$index$virg
1082              string_final=$string_final$string
1083            done
1084            string_final=$( echo $string_final | sed "s/.$//" )
1085            echo "rank $rank=$host_value slot=$string_final" >> rankfile
1086            (( rank = rank + 1 ))
1087            (( current_core = current_core + comp_proc_omp_loc ))
1088          done
1089        fi
1090      done
1091    fi
1092
1093  # Only one executable (SPMD mode).
1094  else
1095
1096    for comp in ${config_ListOfComponents[*]} ; do
1097
1098      # Only if we really have an executable for the component :
1099      eval ExeNameOut=\${config_Executable_${comp}[1]}
1100      if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
1101
1102        echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
1103        echo ""  >> script_${ExeNameOut}.ksh
1104        if ( ${OK_PARA_OMP} ) ; then
1105          eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
1106          echo "export OMP_STACKSIZE=200M" >> script_${ExeNameOut}.ksh
1107          echo "export OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
1108        fi
1109
1110        if [ ${mpiTasks} -eq 1 ] && [ ${openMPthreads} -eq 1 ] ; then
1111          # Case with only one proc
1112          # mpirun should not be used at obelix for only one proc
1113          # Output text per process
1114          echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
1115          IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1116          EXECUTION="time ./script_${ExeNameOut}.ksh"
1117        elif  ( ${OK_PARA_MPI} ) ; then
1118          # Default : mpirun used if nb_proc gt 1
1119          # Output text per process
1120          echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${OMPI_COMM_WORLD_RANK} 2>out_${ExeNameOut}.err.\${OMPI_COMM_WORLD_RANK}"  >> script_${ExeNameOut}.ksh
1121          IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1122          EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
1123        elif [ ${executionType} -eq 6 ] ; then
1124          # Sequentiel case: this is the same as the first case.
1125          echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
1126          IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
1127          EXECUTION="time ./script_${ExeNameOut}.ksh"
1128        else
1129          # Strange case.
1130          IGCM_debug_Exit "ERROR with OK_PARA_MPI"
1131          IGCM_debug_Print 2 "sys Obelix : OK_PARA_MPI=False and mpiTask/openMPthreds different from 1"
1132          IGCM_debug_Verif_Exit
1133        fi
1134
1135        IGCM_debug_Print 1 "sys Obelix : script_${ExeNameOut}.ksh contains"
1136        cat script_${ExeNameOut}.ksh
1137
1138      fi
1139
1140    done
1141
1142  fi
1143
1144  IGCM_debug_Print 1 "sys Obelix : execution command is"
1145  IGCM_debug_Print 1 "$EXECUTION"
1146
1147  IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
1148}
1149
1150#D-#==================================================
1151#D-function IGCM_sys_check_path
1152#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
1153#D-* do not point to an important use directory. Stop immediately in that case.
1154#D-* Examples:
1155#D-
1156function IGCM_sys_check_path {
1157  IGCM_debug_PushStack "IGCM_sys_check_path"
1158  if ( $DEBUG_sys ) ; then
1159    echo "IGCM_sys_check_path"
1160  fi
1161
1162  if ( [ X${RUN_DIR_PATH} = X${HOME} ] || [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
1163    IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
1164    IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
1165    IGCM_debug_Exit "This will stop the job"
1166  fi
1167  IGCM_debug_PopStack "IGCM_sys_check_path"
1168}
1169
1170#D-#==================================================
1171#D-function IGCM_sys_check_quota. Dummy call here
1172#D-* Purpose: check user quota. Stop the simulation if quota above 90%
1173#D-* Examples:
1174#D-
1175function IGCM_sys_check_quota {
1176  IGCM_debug_PushStack "IGCM_sys_check_quota"
1177  if ( $DEBUG_sys ) ; then
1178    echo "IGCM_sys_check_quota"
1179  fi
1180
1181  # TO BE IMPLEMENTED
1182
1183  IGCM_debug_PopStack "IGCM_sys_check_quota"
1184}
1185
1186#D-#==================================================
1187#D-function IGCM_sys_projectAccounting
1188#D-* Purpose: store project accounting information in a file
1189#D-* Examples:
1190#D-
1191function IGCM_sys_projectAccounting {
1192  IGCM_debug_PushStack "IGCM_sys_projectAccounting"
1193  if ( $DEBUG_sys ) ; then
1194    echo "IGCM_sys_check_quota"
1195  fi
1196
1197  touch $1
1198
1199  IGCM_debug_PopStack "IGCM_sys_projectAccounting"
1200}
1201
1202#D-#==================================================
1203#D-function IGCM_sys_getJobSchedulerID
1204#D-* Purpose: Get the job ID during execution
1205#D-* Examples: IGCM_sys_getJobSchedulerID jobSchedulerID
1206#D-
1207function IGCM_sys_getJobSchedulerID {
1208  IGCM_debug_PushStack "IGCM_sys_getJobSchedulerID"
1209  if ( $DEBUG_sys ) ; then
1210    echo "IGCM_sys_getJobSchedulerID"
1211  fi
1212
1213  eval ${1}=$( echo ${PBS_JOBID} | awk -F. '{print $1}' )
1214
1215  IGCM_debug_PopStack "IGCM_sys_getJobSchedulerID"
1216}
1217
1218#D-#==================================================
1219#D-function IGCM_sys_GetJobID
1220#D-* Purpose: Get the job ID from the JobName
1221#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
1222#D-
1223function IGCM_sys_GetJobID {
1224  IGCM_debug_PushStack "IGCM_sys_GetJobID"
1225  if ( $DEBUG_sys ) ; then
1226    echo "IGCM_sys_GetJobID"
1227  fi
1228
1229  # With -f option, the full job name is given in the last column
1230  ID="$( qstat -u $2 | grep -w $1 | gawk '-F ' '{print $10}' )"
1231
1232  eval ${3}=${ID}
1233  IGCM_debug_PopStack "IGCM_sys_GetJobID"
1234}
1235
1236#D-#==================================================
1237#D-function IGCM_sys_CountJobInQueue
1238#D-* Purpose: Count number of users job
1239#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
1240#D-
1241function IGCM_sys_CountJobInQueue {
1242  IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
1243  if ( $DEBUG_sys ) ; then
1244    echo "IGCM_sys_CountJobInQueue"
1245  fi
1246  IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
1247}
1248
1249#D-#==================================================
1250#D-function IGCM_sys_atlas
1251#D-* Purpose: encapsulate atlas call so as to manage error code and curie specificity
1252#D-* Examples:
1253#D-
1254function IGCM_sys_atlas {
1255  IGCM_debug_PushStack "IGCM_sys_atlas" $@
1256  if ( $DEBUG_sys ) ; then
1257    echo "IGCM_sys_atlas :" $@
1258  fi
1259
1260  typeset status
1261
1262  \atlas $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
1263  status=$?
1264  if [ ${status} -gt 0 ] ; then
1265    echo "IGCM_sys_atlas : error code ${status}"
1266    cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1267    \rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
1268    IGCM_debug_PopStack "IGCM_sys_atlas"
1269    return 1
1270  else
1271    IGCM_debug_PopStack "IGCM_sys_atlas"
1272    return 0
1273  fi
1274
1275  IGCM_debug_PopStack "IGCM_sys_atlas"
1276}
1277
1278
1279#D-#==================================================
1280#D-function IGCM_sys_rebuild_nemo
1281#D-* Purpose: rebuild nemo parallel files with general rebuild
1282#D-* Examples:
1283#D-
1284
1285function IGCM_sys_rebuild_nemo {
1286  IGCM_debug_PushStack "IGCM_sys_rebuild_nemo" $@
1287  if ( $DEBUG_sys ) ; then
1288    echo "IGCM_sys_rebuild_nemo :" $@
1289  fi
1290
1291  nemo_generic_restart_file_name_out=${1}
1292  nemo_extension_out=${3}
1293  shift ; shift ; shift
1294
1295  IGCM_sys_rebuild -o ${nemo_generic_restart_file_name_out}.${nemo_extension_out} $@
1296
1297  IGCM_debug_PopStack "IGCM_sys_rebuild_nemo"
1298}
1299
Note: See TracBrowser for help on using the repository browser.