source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1529

Last change on this file since 1529 was 1528, checked in by aclsce, 4 years ago

Irene-amd : added 2 functionalities to increase computing performances :

  • increase this number of cores per MPI process (OpenMP thread) by using DEP keyword in config.card
  • use of dedicated nodes for XIOS servers by using DEDICATED keyword in config.card

Example of config.card :
ATM= (gcm.e, lmdz.x, 71MPI, 8OMP, 2DEP)
SRF= ("" ,"" )
SBG= ("" ,"" )
OCE= (opa, opa.xx, 360MPI, 2DEP)
ICE= ("" ,"" )
MBG= ("" ,"" )
CPL= ("", "" )
IOS= (xios_server.exe, xios.x, 12MPI,2DEP, DEDICATED)
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 56.1 KB
Line 
1#!/bin/ksh
2
3#**************************************************************
4# Author: Sebastien Denvil, Martial Mancip
5# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6# $Revision::                                          $ Revision of last commit
7# $Author::                                            $ Author of last commit
8# $Date::                                              $ Date of last commit
9# IPSL (2006)
10#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11#
12#**************************************************************
13
14#===================================
15function IGCM_config_CommonConfiguration
16{
17  IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19  # Debug Print :
20  [ ${Verbosity} -gt 0 ] && echo
21  IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23  # config.card path
24  configCardPath=$1
25
26  #==================================
27  typeset option auxprint
28
29  #==================================
30  # Read UserChoices section:
31  [ ${Verbosity} -gt 0 ] && echo
32  IGCM_debug_Print 1 "DefineArrayFromOption  : config_UserChoices"
33
34  IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35  for option in ${config_UserChoices[*]} ; do
36    IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37    eval auxprint=\${config_UserChoices_${option}}
38    IGCM_debug_Print 3 "${option} : ${auxprint}"
39  done
40
41  # Set variables with shorter names on some variables which 
42  # are used in the executable name in config.card
43  ResolAtm=${config_UserChoices_ResolAtm}
44  ResolOce=${config_UserChoices_ResolOce}
45  ConfChem=${config_UserChoices_ConfChem}
46  OptMode=${config_UserChoices_OptMode}
47
48  #==================================
49  # Read Ensemble section:
50  [ ${Verbosity} -gt 0 ] && echo
51  IGCM_debug_Print 1 "DefineArrayFromOption  : config_Ensemble"
52
53  IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
54  for option in ${config_Ensemble[*]} ; do
55    IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
56    eval auxprint=\${config_Ensemble_${option}}
57    IGCM_debug_Print 3 "${option} : ${auxprint}"
58  done
59
60  #==================================
61  # Read Post section:
62  [ ${Verbosity} -gt 0 ] && echo
63  IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
64
65  IGCM_card_DefineArrayFromSection ${configCardPath} Post
66  for option in ${config_Post[*]} ; do
67    IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
68    eval auxprint=\${config_Post_${option}}
69    IGCM_debug_Print 3 "${option} : ${auxprint}"
70  done
71  [ ${Verbosity} -gt 0 ] && echo
72
73  #==================================
74  # Define default value to keep compatibility with previous card: means before changes due to TGCC
75  # Apply some overrules to ensure proper usage of computing centres resources
76  #
77
78  # if RebuildFrequency is not define in config.card - we initilize it to NONE
79  if [ X${config_Post_RebuildFrequency} = X ] ; then
80      config_Post_RebuildFrequency=NONE
81  fi
82
83  if [ X${PackDefault} = Xtrue ] ; then
84    if [ X${config_UserChoices_SpaceName} = XTEST ]; then
85      # TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
86      IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
87      config_Post_PackFrequency=NONE
88    else
89      # Default to RebuildFrequency if nothing has been set up related to PackFrequency
90      [ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
91    fi
92  else
93    # If we DO NOT apply pack in this computing center
94    config_Post_PackFrequency=NONE
95  fi
96
97  #====================================================
98  # Define ARCHIVE : Dedicated to large files
99  # Define STORAGE : Dedicated to small/medium files
100  # Define R_OUT   : Output tree located on ARCHIVE
101  # Define R_BUF   : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
102  # Define R_FIG   : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
103  # Define R_TMP   : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
104  IGCM_sys_defineArchives
105
106  #====================================================
107  # R_SAVE : Job output directory
108  # R_BUFR : Job output buffered directory
109  # R_CMIP : Job output directory for CMIP6 standard files
110
111  if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
112    FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
113    if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
114      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
115      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
116      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
117      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
118      R_CMIP=${CMIP6_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
119    else
120      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
121      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
122      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
123      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
124      R_CMIP=${CMIP6_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
125    fi
126  else
127    if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
128      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
129      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
130      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
131      R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
132    else
133      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
134      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
135      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
136      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
137    fi
138  fi
139
140  #====================================================
141  # Define R_OUT_KSH : Storage place for job output
142  # Define R_OUT_EXE : Storage place for binary used during simulation
143  R_OUT_KSH=${R_SAVE}/Out
144  R_OUT_EXE=${R_SAVE}/Exe
145
146  #====================================================
147  # Define R_BUF_KSH : Buffer place for job output
148  # Define R_BUF_EXE : Buffer place for binary used during simulation
149  R_BUF_KSH=${R_BUFR}/Out
150  R_BUF_EXE=${R_BUFR}/Exe
151
152  #====================================================
153  # Define REBUILD_DIR : where we store files needing rebuild process
154  REBUILD_DIR=${R_BUFR}/REBUILD
155  if [ ! X${TaskType} = Xchecking ] ; then
156    IGCM_sys_MkdirWork ${REBUILD_DIR}
157  fi
158
159  #====================================================
160  # DodsCopy : apply default value if not defined
161  if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] || [ X${config_Post_DodsCopy} = X ] ) ; then
162    config_Post_DodsCopy=TRUE
163  fi
164
165  #====================================================
166  # AtlasIPSL : apply default value if not defined
167  if ( [ X${config_Post_AtlasIPSL} = X${NULL_STR} ] || [ X${config_Post_AtlasIPSL} = X ] ) ; then
168    config_Post_AtlasIPSL=FALSE
169  fi
170
171  #====================================================
172  # ParserXIOS : apply default value if not defined
173  if ( [ X${config_Post_ParserXIOS} = X${NULL_STR} ] || [ X${config_Post_ParserXIOS} = X ] ) ; then
174    config_Post_ParserXIOS=FALSE
175  fi
176
177  #====================================================
178  # MetricsPCMDI : apply default value if not defined
179  if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] || [ X${config_Post_MetricsPCMDI} = X ] ) ; then
180    config_Post_MetricsPCMDI=FALSE
181  fi
182
183  #====================================================
184  # IgnoreNonMonotonic : apply default value if not defined
185  if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] || [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
186    config_Post_IgnoreNonMonotonic=FALSE
187  fi
188
189  #====================================================
190  # LightRestartPack : apply default value if not defined
191  if ( [ X${config_Post_LightRestartPack} = X${NULL_STR} ] || [ X${config_Post_LightRestartPack} = X ] ) ; then
192    config_Post_LightRestartPack=FALSE
193  fi
194
195  #====================================================
196  # Define StackFileLocation : directory where we store stack files
197  # Define StackFileName : stack file containing call tree and instrumentation
198  # Stack file containing call tree will be stored there.
199  if ( $DEBUG_debug ) ; then
200    StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
201    [ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
202    if [ X${TaskType} = Xcomputing ]; then
203      StackFileName=computing.stack.$$
204    elif [ X${TaskType} = Xpost-processing ]; then
205      StackFileName=${Script_Post_Output}.stack.$$
206    elif [ X${TaskType} = Xchecking ]; then
207      StackFileName=checking.stack.$$
208    else
209      IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
210      IGCM_debug_Verif_Exit
211    fi
212
213    # This boolean will trigger the filling of the stack
214    # Only now we know where things should be ...
215    # We don't fill the stack when we perform checking task
216    if [ ! X${TaskType} = Xchecking ] ; then
217      ActivateStackFilling=true
218    fi
219  fi
220
221  IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
222}
223
224#===================================
225function IGCM_config_Initialize
226{
227  IGCM_debug_PushStack "IGCM_config_Initialize"
228
229  # Debug Print :
230  echo
231  IGCM_debug_Print 1 "IGCM_config_Initialize"
232
233  # Test modipsl tree existence.
234  IGCM_sys_TestDir ${MODIPSL}
235  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
236  IGCM_sys_TestDir ${libIGCM}
237  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
238  IGCM_sys_TestDir ${R_EXE}
239  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
240  IGCM_sys_TestDir ${SUBMIT_DIR}
241  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
242
243  if ( $DEBUG_debug ) ; then
244    echo "Keep trace of inital SUBMIT_DIR : "
245    ls -lta ${SUBMIT_DIR}
246  fi
247
248  #==================================
249  # Read ListOfComponents section:
250  echo
251  IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
252
253  IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
254  for comp in ${config_ListOfComponents[*]} ; do
255    IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
256  done
257  IGCM_debug_Print 3 ${config_ListOfComponents[*]}
258
259  #==================================
260  # Read Executable section:
261  IGCM_card_DefineArrayFromSection   ${SUBMIT_DIR}/config.card Executable
262
263  #==================================
264  # Read Restarts section:
265  # Restarts : Gerneral rule or local for each component.
266  echo
267  IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
268
269  IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
270  for option in ${config_Restarts[*]} ; do
271    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
272    eval auxprint=\${config_Restarts_${option}}
273    IGCM_debug_Print 3 "${option} : ${auxprint}"
274  done
275
276  #==================================
277  # Define Job Outputs Name
278  echo
279  IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
280  Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
281  IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
282  Exe_Output=out_execution
283  IGCM_debug_Print 3 "Exe_Output           = ${Exe_Output}"
284
285  #===================================================================#
286  # Prepare variables available for ${COMP}.card and ${COMP}.driver   #
287  #             But available to any son functions                    #
288  #===================================================================#
289
290  # Convert yyyy-mm-dd date to gregorian yyyymmdd
291  DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
292  DateEnd=$(   IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd}   )
293
294  # Period Length In Days between DateBegin and DateEnd
295  (( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} )  + 1 ))
296  if [ ${ExperienceLengthInDays} -lt 0 ] ; then
297    IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
298    IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
299    IGCM_debug_Verif_Exit
300  fi
301
302  # Day and Year of Initial State (Given in julian format)
303  InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
304  InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
305  InitYear_m1=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 - 1 ))
306
307  #================================================================#
308  #                  Test and Prepare directories                  #
309  #================================================================#
310
311  # ==> 4 kinds of input files :
312  #     1) R_INIT  : Initial State Files   (Etat0, carteveg)
313  #     2) R_BC    : Boundary Conditions   (Forcages, lai)
314  #     3) Parameters files (allready define through ${SUBMIT_DIR})
315  #     4) Restarts files   (allready define in IGCM_config_Initialize)
316
317  # Here we offer the possibility to redefine R_INIT, R_BC
318  # and PeriodNb through config.card
319  R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
320  echo
321  IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
322  IGCM_debug_Print 3 "R_IN=${R_IN}"
323  IGCM_debug_Print 3 "R_INIT=${R_INIT}"
324  R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
325  IGCM_debug_Print 3  "R_BC=${R_BC}"
326  PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
327  IGCM_debug_Print 3  "Loop in main Job with ${PeriodNb} period(s)"
328
329  # SD ADA SPECIFIC #
330  #      TO FIX     #
331  #IGCM_sys_TestDirArchive ${R_IN}
332  #[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
333
334  if ( ${FirstInitialize} ) ; then
335    IGCM_sys_MkdirArchive   ${R_SAVE}
336    [ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
337  else
338    IGCM_sys_TestDirArchive ${R_SAVE}
339    [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
340
341    if [ ! ${config_Post_PackFrequency} = NONE ] ; then
342      IGCM_sys_TestDir ${R_BUFR}
343      [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
344    fi
345
346    # Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
347    IGCM_config_StateCheck
348
349    # And EXIT if not OK
350    IGCM_debug_Verif_Exit
351  fi
352
353  #====================================================
354  # Experience type : DEB(ug), DEV(elopment), RUN
355  if [ X${JobType} != XRUN ] ; then
356    echo
357    echo "===================================================="
358    echo "libIGCM JOB is NOT in RUN type mode."
359    echo "!! OUTPUT files will NOT be PROTECTED !!"
360    echo "Be carefull : you can ERASE the result of this job !"
361
362    case ${JobType} in
363    DEB)
364      echo "DEBUG mode : activation of 'set -vx' mode."
365      echo "DEBUG mode : no protection for output files."
366      echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
367      ;;
368    DEV)
369      echo "DEVelopment mode : no protection for output files."
370      echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
371      ;;
372    esac
373
374    if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
375      if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
376        echo "------------"
377        echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
378        echo "          to PeriodLength : ${config_UserChoices_PeriodLength}"
379        echo "------------"
380        config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
381      fi
382    fi
383    echo "===================================================="
384    echo
385  fi
386
387  IGCM_debug_PopStack "IGCM_config_Initialize"
388}
389
390#===================================
391function IGCM_config_DaysInPeriodLength
392{
393  IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
394
395  typeset i
396
397  # Determine number of day(s) in PeriodLength :
398  case ${config_UserChoices_PeriodLength} in
399  *Y|*y)
400    PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[yY]//' )
401    echo
402    IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
403    PeriodLengthInDays=0
404    i=0
405    until [ $i -ge $PeriodLengthInYears ] ; do
406      (( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
407      (( i=i+1 ))
408    done
409    ;;
410  *M|*m)
411    PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[mM]//' )
412    echo
413    IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
414    PeriodLengthInDays=0
415    i=0
416    until [ $i -ge $PeriodLengthInMonths ] ; do
417      if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
418        (( PeriodLengthInDays  = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
419      else
420        (( PeriodLengthInDays  = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
421      fi
422      (( i=i+1 ))
423    done
424    ;;
425  *D|*d)
426    PeriodLengthInMonths=0
427    PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[dD]//' )
428    echo
429    IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
430  *)
431    IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in *Y, *M, *D."
432    IGCM_debug_Verif_Exit ;;
433  esac
434
435  IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
436}
437
438#===================================
439function IGCM_config_DateCoherency
440{
441  IGCM_debug_PushStack "IGCM_config_DateCoherency"
442
443  echo
444  IGCM_debug_Print 1 "IGCM_config_DateCoherency"
445  echo
446
447  typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
448
449  # check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
450  # DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
451  echo
452
453  case ${config_UserChoices_PeriodLength} in
454  *Y|*y)
455    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
456    ;;
457  *M|*m)
458    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
459    ;;
460  *D|*d)
461    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
462    ;;
463  esac
464  VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
465
466  if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
467    IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
468    IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
469    IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
470    IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
471    IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
472    IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
473
474    IGCM_debug_Exit "STOP here to avoid further issues."
475  fi
476
477  # PeriodDateBegin + PeriodLength = PeriodDateEnd
478  VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
479
480  IGCM_debug_PopStack "IGCM_config_DateCoherency"
481}
482
483#===================================
484function IGCM_config_StateCheck
485{
486  IGCM_debug_PushStack "IGCM_config_StateCheck"
487
488    #Test state of run in run.card
489    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
490
491    if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
492      echo
493      IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
494      IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
495      IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
496      IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
497      IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean lastest pack period."
498      IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
499      IGCM_debug_Exit
500    elif [ $( echo ${run_Configuration_PeriodState} | grep Fatal | wc -l ) -eq 1 ] ; then
501      echo
502      IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
503      IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
504      IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
505      IGCM_debug_Print 1 "https://hermes.ipsl.upmc.fr/static/simulation.detail.html?uid=${simuid}"
506      IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
507      IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
508      IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_PeriodLenght.job to clean latest failed period"
509      IGCM_debug_Print 1 "Or try running ${libIGCM}/clean_latestPackperiod.job to clean latest pack period."
510      IGCM_debug_Print 1 "Or use ${libIGCM}/purge_simulation.job to delete the full simulation."
511      IGCM_debug_Exit
512    fi
513
514  IGCM_debug_PopStack "IGCM_config_StateCheck"
515}
516
517#===================================
518function IGCM_config_Check
519{
520  IGCM_debug_PushStack "IGCM_config_Check"
521
522  # If one of the following modulo is not zero :
523  # we will issue an error then explain and exit in
524  # AA_job IGCM_debug_Verif_Exit call before binary submission
525
526  echo
527  IGCM_debug_Print 1 "IGCM_config_Check"
528  echo
529
530  typeset i
531
532  # Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
533  if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
534    AsynchronousRebuild=true
535    IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
536    echo
537    # modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
538    IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
539    IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
540    IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
541    IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
542    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
543    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
544    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
545    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
546  else
547    AsynchronousRebuild=false
548    IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
549    IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
550    echo
551    #modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
552    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
553    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
554    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
555    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency   config_UserChoices_PeriodLength
556  fi
557
558  # Check PackFrequency against other key frequencies
559  # Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
560  if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
561    Pack=true
562    #
563    IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
564    IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
565    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
566    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
567    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
568    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
569  else
570    Pack=false
571  fi
572
573  # modulo (TimeSeriesFrequency and all Chunck2D) must be zero
574  NbJob=${#CHUNCK2D_SIZE[@]}
575  i=0
576  until [ $i -ge $NbJob ]; do
577    value=${CHUNCK2D_SIZE[${i}]}
578    IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
579    IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
580    case ${value} in
581    *Y|*y) ;;
582    *)
583      IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year *Y|*y in comp.card"
584      IGCM_debug_Exit "This will stop the job" ;;
585    esac
586    (( i=i+1 ))
587  done
588
589  # modulo (TimeSeriesFrequency and all Chunck3D) must be zero
590  NbJob=${#CHUNCK3D_SIZE[@]}
591  i=0
592  until [ $i -ge $NbJob ]; do
593    value=${CHUNCK3D_SIZE[${i}]}
594    IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
595    IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
596    case ${value} in
597    *Y|*y) ;;
598    *)
599      IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year *Y|*y in comp.card"
600      IGCM_debug_Exit "This will stop the job" ;;
601    esac
602    (( i=i+1 ))
603  done
604
605  # check to be sure there is enough space on temporary filesystems to run
606  echo
607  IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
608  IGCM_sys_check_quota
609
610  # check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
611  echo
612  IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
613  IGCM_sys_check_path
614
615
616  IGCM_debug_PopStack "IGCM_config_Check"
617}
618
619#===================================
620function IGCM_config_ConfigureExecution
621{
622  IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
623
624  #echo
625  IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
626  #echo
627
628  typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
629  typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
630
631  # config.card path
632  configCardPath=$1
633
634  coreNumber=0
635  mpiTasks=0
636  openMPthreads=0
637  NbExec=0
638
639  OK_PARA_MPI=false
640  OK_PARA_OMP=false
641  OK_PARA_NOD=false
642  OK_PARA_MPMD=false
643
644  for comp in ${config_ListOfComponents[*]} ; do
645
646    # Manage component executable
647    IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
648
649    eval ExeNameIn=\${config_Executable_${comp}[0]}
650
651    # NO order in config.card for parallelized values !
652    # just use suffix : MPI , OMP and NOD (for number of NODes.)
653
654    # NOD is the number of NODes allocated
655    eval ${comp}_PROC_NOD=0
656
657    # MPI is the number of MPI processus per nodes
658    eval ${comp}_PROC_MPI=0
659
660    # OMP is the number of OpenMP threads per MPI processus
661    eval ${comp}_PROC_OMP=0
662
663    # DEP is the factor to increase the number of cores per process
664    eval ${comp}_PROC_DEP=0
665
666    # DEDICATED is the flag to activate the use of dedicated node for XIOS server only
667    eval ${comp}_OK_DEDICATED=0
668
669    # Only if we really have an executable for the component :
670    if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
671
672      IGCM_debug_Print 1 ${comp}
673
674      # Keep the first executable found and the first CompName
675      eval ExeNameFirst=${ExeNameIn}
676      CompNameFirst=${comp}
677
678      # Are we a second executable?
679      (( NbExec = NbExec + 1 ))
680
681      # set 1 MPI task, 1 OpenMP thread and 1 node as default
682      eval ${comp}_PROC_MPI=1
683      eval ${comp}_PROC_OMP=1
684      eval ${comp}_PROC_NOD=1
685      eval ${comp}_PROC_DEP=1
686
687      eval NbElts=\${#config_Executable_${comp}[@]}
688      if [ ${NbElts} -gt 2 ] ; then
689        #
690        # CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
691        #
692        i=2
693        while [ ${i} -lt ${NbElts} ] ; do
694          eval tempvar=\${config_Executable_${comp}[${i}]}
695          IGCM_debug_Print 2 ${tempvar}
696
697          if [ X${tempvar} = X ] ; then
698            IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
699            IGCM_debug_Exit "Check your config.card. Exit now"
700            IGCM_debug_Verif_Exit
701          fi
702
703          case ${tempvar} in
704          *[mM][pP][iI]*)
705            # Read MPI parameter for composante
706            eval ${comp}_PROC_MPI=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/MPI//" )
707            OK_PARA_MPI=true;;
708          *[oO][mM][pP]*)
709            # Read OMP parameter for composante
710            eval ${comp}_PROC_OMP=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/OMP//" )
711            ;;
712          *[nN][oO][dD]*)
713            # Read NOD (NumBer of Nodes) parameter for composante
714            eval ${comp}_PROC_NOD=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/NOD//" )
715            OK_PARA_NOD=true
716            OK_PARA_MPI=true
717            ;;
718            *[dD][eE][pP]*)
719            # Read DEP parameter for composante
720            eval ${comp}_PROC_DEP=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/DEP//" )
721            ;;
722            *[dD][eE][dD][iI][cC][aA][tT][eE][dD]*)
723            # Read DEDICATED parameter for composante
724            eval ${comp}_OK_DEDICATED=1
725          esac
726          (( i = i + 1 ))
727        done
728      else
729        #
730        # BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
731        #
732        IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
733        IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
734        IGCM_debug_Exit "Please modify ${configCardPath}"
735        exit
736      fi
737      if [ ${comp}_PROC_NOD -gt 1 ] ; then
738            if ( [ ${comp}_PROC_MPI -gt 1 ] && [ ${comp}_PROC_OMP -gt 1 ] ) ; then
739              IGCM_debug_Print 2 "Error using MPI/OMP/NOD parameters !!!"
740              IGCM_debug_Exit "It is not allowed to use MPI and OMP and NOD parameters for the same component"
741              IGCM_debug_Exit "Check your config.card. Exit now"
742              IGCM_debug_Verif_Exit
743            fi
744            if [ "${comp}" != "IOS" ] ; then
745              IGCM_debug_Print 2 "Error using NOD parameter !!!"
746              IGCM_debug_Exit "NOD parameter is only available for IOS component "
747              IGCM_debug_Exit "Check your config.card. Exit now"
748              IGCM_debug_Verif_Exit
749            fi
750      fi
751      if ( [ ${comp}_OK_DEDICATED -eq 1 ] && [ "${comp}" != "IOS" ] ) ; then
752            IGCM_debug_Print 2 "Error using DEDICATED parameter !!!"
753            IGCM_debug_Exit "DEDICATED parameter is only available for IOS component "
754            IGCM_debug_Exit "Check your config.card. Exit now"
755            IGCM_debug_Verif_Exit
756      fi
757
758      eval tempvarMPI=\${${comp}_PROC_MPI}
759      eval tempvarNOD=\${${comp}_PROC_NOD}
760      eval tempvarOMP=\${${comp}_PROC_OMP}
761      eval tempvarDEP=\${${comp}_PROC_DEP}
762
763      # set OMP mode if more than 1 OMP thread.
764      [ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
765
766      # Number of OMP threads
767      [ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
768
769      # SUM UP NUMBER OF CORES
770      if [ ${comp}_PROC_NOD -gt 1 ] ; then 
771          (( coreNumber = coreNumber + tempvarNOD * NB_CORE_PER_NODE ))
772      else
773          (( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP * tempvarDEP ))
774      fi
775      # SUM UP NUMBER OF MPI TASKS
776      (( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
777    fi
778  done
779
780  # MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
781  # WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
782  NUM_PROC_CPL=${CPL_PROC_MPI}
783  NUM_PROC_OCE=${OCE_PROC_MPI}
784  NUM_PROC_ATM=${ATM_PROC_MPI}
785
786  # set MPMD mode if more than 2 executable names.
787  [ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true 
788
789  # Define the execution type we are running in
790  if ( ${OK_PARA_MPMD} ) ; then
791    if ( ${OK_PARA_MPI} ) ; then
792      # MPMD always implies MPI
793      executionType=1
794    fi
795    if ( ${OK_PARA_OMP} ) ; then
796      # MPMD + MPI/OMP
797      executionType=2
798    fi
799  else
800    if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
801      # SPMD + MPI/OMP
802      executionType=3
803    elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
804      # SPMD + MPI only
805      executionType=4
806    elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
807      # SPMD + OMP only
808      executionType=5
809    elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
810      # SEQUENTIAL THEN
811      executionType=6
812      coreNumber=1
813    fi
814  fi
815
816  IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
817  IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
818  IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
819  IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
820
821  IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
822}
823
824#===================================
825function IGCM_config_PeriodStart
826{
827  IGCM_debug_PushStack "IGCM_config_PeriodStart"
828
829  echo
830  IGCM_debug_Print 1 "IGCM_config_PeriodStart"
831  echo
832
833  if ( ${FirstInitialize} ) ; then
834    #================================================#
835    #         Initialize date/period information     #
836    #================================================#
837
838    IGCM_date_GetYearMonthDay ${DateBegin} year month day
839    IGCM_config_DaysInPeriodLength
840
841    PeriodDateBegin=${DateBegin}
842    PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
843    CumulPeriod=1
844
845    #=================================================#
846    #              Write updated run.card             #
847    #=================================================#
848
849    #Correct run.card Configuration for this period
850    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
851    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
852    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
853    if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
854      IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
855    fi
856
857    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
858
859  else
860    #================================================#
861    #         The file run.card allready exist       #
862    #================================================#
863
864    # Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
865    IGCM_config_StateCheck
866    # And EXIT if not OK
867    IGCM_debug_Verif_Exit
868
869    #===================================#
870    #        Read updated run.card      #
871    #===================================#
872
873    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
874    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
875    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
876
877    PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
878    PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
879    CumulPeriod=${run_Configuration_CumulPeriod}
880
881    LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
882
883    if [ ${Period} = 1 ]; then
884      # save last Job output and current run.card
885      typeset Potential
886      IGCM_sys_Cd ${SUBMIT_DIR}
887      #
888      IGCM_debug_Print 2 "Save previous ksh job output"
889      for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
890        if [ X${Pack} = Xtrue ] ; then
891          ( IGCM_sys_TestFileBuffer  ${R_BUF_KSH}/${Potential} ) || IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
892        else
893          ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) || IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ NOMOVE
894          IGCM_debug_Print 1 "Save of previous ksh job output is not done for Pack=false"
895        fi
896      done
897
898      #
899      IGCM_debug_Print 2 "Save current run.card"
900      IGCM_card_CheckConflict run.card
901
902      if [ X${Pack} = Xtrue ] ; then
903        IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
904      else
905        IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card NOMOVE
906        IGCM_debug_Print 1 "Save of run.card is not done for Pack=false"
907      fi
908      #
909      IGCM_sys_Cd ${RUN_DIR}
910    else
911      unset FileToBeDeleted
912    fi
913
914    # Determine number of day(s) in PeriodLength
915    IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
916    IGCM_config_DaysInPeriodLength
917
918    # Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
919    IGCM_config_DateCoherency
920    # And EXIT if not OK
921    IGCM_debug_Verif_Exit
922
923    # Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
924    IGCM_config_StateCheck
925    # And EXIT if not OK
926    IGCM_debug_Verif_Exit
927
928    # We can say we are "Running" now.
929    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
930  fi
931
932  # BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
933  # Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
934  year_m1=$(( year - 1 ))
935  year_p1=$(( year + 1 ))
936  # Compute month_m1 (month minus 1M)
937  # Compute yyyymm_m1 (yyyymm minus 1M)
938  month_m1=$(( 10#${month} - 1 ))
939  if [ ${month_m1} = 0 ]; then
940    month_m1=12
941    yyyymm_m1=${year_m1}12
942  elif [ ${month_m1} -le 9 ]; then
943    month_m1=0${month_m1}
944    yyyymm_m1=${year}${month_m1}
945  else
946    yyyymm_m1=${year}${month_m1}
947  fi
948  # Compute month_p1 (month plus 1M)
949  # Compute yyyymm_p1 (yyyymm plus 1M)
950  month_p1=$(( 10#${month} + 1 ))
951  if [ ${month_p1} = 13 ]; then
952    month_p1=01
953    yyyymm_p1=${year_p1}01
954  elif [ ${month_p1} -le 9 ]; then
955    month_p1=0${month_p1}
956    yyyymm_p1=${year}${month_p1}
957  else
958    yyyymm_p1=${year}${month_p1}
959  fi
960
961  #===================================================================#
962  # Calculate CyclicYear to be used for looping over a given forcing  #
963  # period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
964  #===================================================================#
965
966  # To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
967  # CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
968  if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
969    CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
970    CyclicYear_p1=NOTDEFINED
971
972    # For current year
973    yeartmp=$year
974    diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
975    while [ $diffy -lt 0 ] ; do
976      yeartmp=$(( ${yeartmp} + ${CycleNb} ))
977      diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
978    done
979    CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
980
981    # For next coming year
982    yeartmp=$(( $year + 1 ))
983    diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
984    while [ $diffy -lt 0 ] ; do
985      yeartmp=$(( ${yeartmp} + ${CycleNb} ))
986      diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
987    done
988    CyclicYear_p1=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
989
990    IGCM_debug_Print 1 "CyclicYear   = ${CyclicYear}, CyclicYear_p1 = ${CyclicYear_p1}, current year=$year"
991  else
992    CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
993    CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
994    IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
995  fi
996
997  # END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
998
999  #===================================================================#
1000  # Prepare variables available for ${COMP}.card and ${COMP}.driver   #
1001  #             But available to any son functions                    #
1002  #===================================================================#
1003
1004  # Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
1005  (( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
1006
1007  # Debug Print :
1008  echo
1009  IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
1010  IGCM_debug_Print 1 "Year of simulation      : ${year}"
1011  IGCM_debug_Print 1 "Month of simulation     : ${month}"
1012  IGCM_debug_Print 1 "PeriodLengthInDays      : ${PeriodLengthInDays}"
1013  IGCM_debug_Print 1 "PeriodDateBegin         : ${PeriodDateBegin}"
1014  IGCM_debug_Print 1 "PeriodDateEnd           : ${PeriodDateEnd}"
1015  IGCM_debug_Print 1 "SimulationLengthInDays  : ${SimulationLengthInDays}"
1016  IGCM_debug_Print 1 "ExperienceLengthInDays  : ${ExperienceLengthInDays}"
1017
1018  #================================================================#
1019  #         Prepare variables available for comp_finalyze          #
1020  #================================================================#
1021
1022  # Period for save files
1023  DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
1024
1025  # Prefix for save files of this period
1026  PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
1027
1028  # List of files that will be deleted in RUN_DIR after run
1029  [ -f stack ] && FileToBeDeleted[0]="stack"
1030
1031  # Test if the same run as already been saved :
1032  if [ X${JobType} = XRUN ] ; then
1033    if [ ${DRYRUN} -le 0 ] ; then
1034      if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1035        IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently re-running an old job."
1036        IGCM_debug_Print 1 "Because of readonly permissions, you can't re-run a job when saved files"
1037        IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
1038        IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See different clean_*.job in ${libIGCM} directory."
1039        IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
1040        IGCM_debug_Verif_Exit
1041      fi
1042    fi
1043  else
1044    if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
1045      IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
1046    fi
1047  fi
1048
1049  #================================================================#
1050  #       Prepare variables available for binary execution         #
1051  #       Call function for creation of run script                 #
1052  #       Only done once per job                                   #
1053  #================================================================#
1054
1055  if [ ${Period} -eq 1 ]; then
1056    # Define the execution context (MPMD, SPMD, MPI/OMP ...)
1057    IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
1058    # Create the execution script for the current context
1059    IGCM_sys_build_execution_scripts
1060  fi
1061
1062  ExecutionFail=false
1063
1064  # Update the rabbitMQ queue
1065  IGCM_debug_BigBro_Update
1066
1067  IGCM_debug_PopStack "IGCM_config_PeriodStart"
1068}
1069
1070#===================================
1071function IGCM_config_SaveSourceModifications
1072{
1073  IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
1074
1075  typeset ExeOutDateMax listVarEnv
1076  ExeOutDateMax=$1
1077
1078  listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
1079  IGCM_sys_RshMaster "\
1080    . ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1081    export ExeOutDateMax=${ExeOutDateMax};\
1082    export R_OUT_EXE=${R_OUT_EXE};\
1083    export PREFIX=${PREFIX};\
1084    export SUBMIT_DIR=${SUBMIT_DIR};\
1085    export listVarEnv=${listVarEnv};\
1086    Script_Output=out_SaveSourceModifications;\
1087    IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1088
1089  IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1090}
1091
1092#===================================
1093function IGCM_config_PeriodEnd
1094{
1095  IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1096
1097  echo
1098  IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1099  echo
1100
1101  if [ ${DRYRUN} -le 1 ] ; then
1102
1103    IGCM_debug_Print 1 "Check components binary : size and creation date"
1104
1105    typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1106    typeset comp i
1107    typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1108
1109    #==================================#
1110    #        Get last Exe Size         #
1111    #==================================#
1112
1113    (( i=0 ))
1114    if ( ${FirstInitialize} ) ; then
1115      run_Log_LastExeSize=""
1116      for comp in ${config_ListOfComponents[*]} ; do
1117        run_Log_LastExeSize[$i]=0
1118        (( i=i+1 ))
1119      done
1120    else
1121      IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1122    fi
1123    #==================================#
1124    #         And Build ExeDate        #
1125    #==================================#
1126
1127    # ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1128    # Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1129    # ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1130    ExeDate=""
1131    NextExeSize="( "
1132    (( i=0 ))
1133    UpdateExe=false
1134    (( ExeSecDateMax = 0 ))
1135    for comp in ${config_ListOfComponents[*]} ; do
1136
1137      IGCM_debug_Print 3 ${comp}
1138
1139      eval ExeNameIn=\${config_Executable_${comp}[0]}
1140      eval ExeNameOut=\${config_Executable_${comp}[1]}
1141      # Only if we really have an executable for the component :
1142      if [ X${ExeNameIn} = X\"\" ] ; then
1143        # If there is no exe file for this component
1144        (( ExeSize=0 ))
1145      else
1146        eval ExeNameIn=${ExeNameIn}
1147        LS_bin=${R_EXE}/${ExeNameIn}
1148        IGCM_sys_FileSize ${LS_bin} ExeSize
1149
1150        set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1151        if [ X${ExeDate} = X ] ; then
1152          # First component exe date
1153          ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1154        else
1155          ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1156        fi
1157        ExeDate=${ExeDate}_${LS_comp[7]}
1158      fi
1159
1160      if [ ${i} -eq 0 ] ; then
1161        # First component
1162        NextExeSize="( "${ExeSize}
1163      else
1164        NextExeSize=${NextExeSize}", "${ExeSize}
1165      fi
1166      LastCompExeSize=${run_Log_LastExeSize[$i]}
1167      (( i=i+1 ))
1168
1169      if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1170        if ( ${FirstInitialize} ) ; then
1171          IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1172        else
1173          IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1174          IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1175          FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1176        fi
1177        IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1178        UpdateExe=true
1179
1180        # SD : switch off for now
1181        #IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1182        #if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1183        #  ExeSecDateMax=$ExeSecDate
1184        #fi
1185      fi
1186    done
1187
1188    # SD : switch off for now
1189    #if ( ${UpdateExe} ) ; then
1190    #  echo "Launch SaveSourceModifications."
1191    #  IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1192    #fi
1193
1194    NextExeSize=${NextExeSize}" )"
1195    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1196
1197    if [ ${DRYRUN} -le 1 ] ; then
1198      tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1199      ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1200      RET=$?
1201      if [ $RET -eq 0 ] ; then
1202        # ExeCpuLog variable contents 5 fields
1203        echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" |   \
1204          gawk '{printf("# %11d | %15s | %15s | %19s | %19s | %15.5f | %15.5f | %15.5f | %s\n", \
1205            $1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1206      fi
1207      FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1208    fi
1209
1210  fi
1211
1212  #==================================#
1213  #         Save Job output          #
1214  #==================================#
1215  if [ X${Pack} = Xtrue ] ; then
1216    IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1217  else
1218    IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1219  fi
1220  FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1221
1222  # All was right ? no ? then we stop.
1223  IGCM_debug_Verif_Exit
1224
1225  # If all was OK, we can delete all files not necessary for next Job
1226  echo
1227  IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1228
1229  if [ ${DRYRUN} -le 2 ] ; then
1230    for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1231  else
1232    echo ${FileToBeDeleted[@]}
1233  fi
1234
1235  # Send some accounting element to the user if CumulPeriod=3
1236  if [ ${CumulPeriod} -eq 3 ] ; then
1237    echo
1238    IGCM_debug_Print 1 "Send email containing some accounting information : "
1239
1240    RealCpuTime=$( echo ${ExeCpuLog} | gawk '{print $3}' )
1241
1242    consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" | bc )
1243
1244    consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" | bc )
1245
1246    recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" | bc )
1247
1248    IGCM_sys_SendMail Accounting
1249  fi
1250
1251  #=================================================#
1252  #         Modification of libIGCM behaviour       #
1253  #=================================================#
1254
1255  # To use this function, one must copy libIGCM.card from ${libIGCM} directory
1256  # and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1257  # variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1258  if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1259    echo
1260    echo "########################################################################"
1261    echo "!!!                 Modification of libIGCM behaviour                !!!"
1262    echo
1263
1264    IGCM_debug_Print 1 "DefineArrayFromOption  : libIGCM_UserChanges in libIGCM.card"
1265    IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1266    IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1267
1268    # Special treatments for libIGCM internals
1269    for option in ${libIGCM_UserChanges[*]} ; do
1270      IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1271
1272      echo "We will change : ${option}."
1273      eval echo "Previous value : " \${${option}}
1274      eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1275
1276      eval ${option}=\${libIGCM_UserChanges_${option}}
1277
1278      case ${option} in
1279      config_UserChoices_DateEnd)
1280        IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1281        DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1282
1283        # Period Length In Days between DateBegin and DateEnd
1284        (( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} )  + 1 ))
1285        if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1286          IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1287          IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1288          IGCM_debug_Verif_Exit
1289        fi
1290        ;;
1291      config_UserChoices_PeriodLength)
1292        IGCM_debug_Print 1  "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1293        ;;
1294      PeriodNb)
1295        IGCM_debug_Print 1  "Loop in main Job with ${PeriodNb} period(s)"
1296        ;;
1297      config_Post_RebuildFrequency)
1298        IGCM_debug_Print 1  "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1299        IGCM_post_Configure
1300        ;;
1301      config_Post_TimeSeriesFrequency)
1302        IGCM_debug_Print 1  "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1303        IGCM_post_Configure
1304        ;;
1305      config_Post_SeasonalFrequency)
1306        IGCM_debug_Print 1  "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1307        IGCM_post_Configure
1308        ;;
1309      esac
1310    done
1311
1312    echo
1313    echo "########################################################################"
1314    echo
1315  fi
1316
1317  #=================================================#
1318  #         Determine next computed period          #
1319  #=================================================#
1320
1321  PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1322  IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1323  year_m1=$(( year - 1 ))
1324  year_p1=$(( year + 1 ))
1325  IGCM_config_DaysInPeriodLength
1326  PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1327
1328  # Debug Print :
1329  echo
1330  IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1331  IGCM_debug_Print 1 "PeriodDateBegin       : ${PeriodDateBegin}"
1332  IGCM_debug_Print 1 "PeriodDateEnd         : ${PeriodDateEnd}"
1333  IGCM_debug_Print 1 "PeriodLengthInDays    : ${PeriodLengthInDays}"
1334
1335  PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1336  PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1337
1338  (( CumulPeriod = CumulPeriod + 1 ))
1339
1340  # Debug Print :
1341  echo
1342  IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1343  IGCM_debug_Print 3 "PeriodDateEnd Human   : ${PeriodDateEnd}"
1344  IGCM_debug_Print 3 "CumulPeriod           : ${CumulPeriod}"
1345
1346  #=================================================#
1347  #             Write updated run.card              #
1348  #=================================================#
1349
1350  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1351  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1352  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1353
1354  if ( ${FirstInitialize} ) ; then
1355    # It's no more the first time
1356    FirstInitialize=false
1357  fi
1358
1359  IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1360}
1361
1362#===================================
1363function IGCM_config_Finalize
1364{
1365  IGCM_debug_PushStack "IGCM_config_Finalize"
1366
1367  echo
1368  IGCM_debug_Print 1 "IGCM_config_Finalize"
1369  echo
1370
1371  # Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1372  IGCM_config_StateCheck
1373
1374  # And EXIT if not OK
1375  IGCM_debug_Verif_Exit
1376
1377  if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1378    #==========================#
1379    # End of entire simulation #
1380    #==========================#
1381    simulationIsOver=true
1382
1383    # Mail notification
1384    IGCM_sys_SendMail
1385    #
1386    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1387    #
1388    IGCM_debug_Print 1 "Normal End of computation."
1389
1390  else
1391    #=================#
1392    # Submit next job #
1393    #=================#
1394    simulationIsOver=false
1395
1396    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1397
1398    # Name of next Ksh Script output :
1399    Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1400
1401    IGCM_debug_Print 1 "Submit next job"
1402    # SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1403    IGCM_sys_Cd ${SUBMIT_DIR}
1404    # Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1405    ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null | wc -l )
1406    [ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? | head -$(( ${ScriptTot} - 5 )) )
1407    # Submit next job and come back
1408    IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1409    IGCM_sys_Cd -
1410  fi
1411
1412  # Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1413  # Only for production run (No clean up in DEV or DEB mode)
1414  # and command sent from .. directory.
1415  IGCM_sys_Cd ..
1416  [ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1417
1418  # Inform the rabbitMQ queue
1419  IGCM_debug_BigBro_Finalize
1420
1421  IGCM_debug_PopStack "IGCM_config_Finalize"
1422}
1423
1424#===================================
Note: See TracBrowser for help on using the repository browser.