source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1053

Last change on this file since 1053 was 1053, checked in by sdipsl, 10 years ago

bug fixes affecting only debug mode

  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 51.3 KB
RevLine 
[2]1#!/bin/ksh
2
3#**************************************************************
4# Author: Sebastien Denvil, Martial Mancip
[373]5# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6# $Revision::                                          $ Revision of last commit
7# $Author::                                            $ Author of last commit
8# $Date::                                              $ Date of last commit
[2]9# IPSL (2006)
10#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11#
12#**************************************************************
13
14#===================================
[726]15function IGCM_config_CommonConfiguration
16{
17  IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19  # Debug Print :
[769]20  [ ${Verbosity} -gt 0 ] && echo
[726]21  IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23  # config.card path
24  configPath=$1
25
26  #==================================
27  typeset option auxprint CompatibilityTag
28
29  #==================================
30  # Read libIGCM compatibility version in config.card
31  IGCM_card_DefineVariableFromOption ${configPath} Compatibility libIGCM
[939]32  CompatibilityTag=${config_Compatibility_libIGCM}
[726]33
34  if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
35    IGCM_debug_Exit "config.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
36  fi
37
38  #==================================
39  # Read UserChoices section:
[769]40  [ ${Verbosity} -gt 0 ] && echo
[726]41  IGCM_debug_Print 1 "DefineArrayFromOption  : config_UserChoices"
42
43  IGCM_card_DefineArrayFromSection ${configPath} UserChoices
44  for option in ${config_UserChoices[*]} ; do
45    IGCM_card_DefineVariableFromOption ${configPath} UserChoices ${option}
46    eval auxprint=\${config_UserChoices_${option}}
47    IGCM_debug_Print 3 "${option} : ${auxprint}"
48  done
49
50  #==================================
51  # Read Ensemble section:
[769]52  [ ${Verbosity} -gt 0 ] && echo
[726]53  IGCM_debug_Print 1 "DefineArrayFromOption  : config_Ensemble"
54
55  IGCM_card_DefineArrayFromSection ${configPath} Ensemble
56  for option in ${config_Ensemble[*]} ; do
57    IGCM_card_DefineVariableFromOption ${configPath} Ensemble ${option}
58    eval auxprint=\${config_Ensemble_${option}}
59    IGCM_debug_Print 3 "${option} : ${auxprint}"
60  done
61
62  #==================================
63  # Read Post section:
[769]64  [ ${Verbosity} -gt 0 ] && echo
[726]65  IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
66
67  IGCM_card_DefineArrayFromSection ${configPath} Post
68  for option in ${config_Post[*]} ; do
69    IGCM_card_DefineVariableFromOption ${configPath} Post ${option}
70    eval auxprint=\${config_Post_${option}}
71    IGCM_debug_Print 3 "${option} : ${auxprint}"
72  done
[769]73  [ ${Verbosity} -gt 0 ] && echo
[726]74
[728]75  #==================================
76  # Define default value to keep compatibility with previous card: means before changes due to TGCC
77  # Apply some overrules to ensure proper usage of computing centres resources
[804]78  #
[728]79  if [ X${PackDefault} = Xtrue ] ; then
80    if [ X${config_UserChoices_SpaceName} = XTEST ]; then
81      # TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
82      IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
83      IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE destination path directories"
84      config_Post_PackFrequency=NONE
85      IGCM_sys_ChangeArchive
86    else
87      # Default to RebuildFrequency if nothing has been set up related to PackFrequency
88      [ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
89    fi
90  else
91    # If we DO NOT apply pack in this computing center
92    config_Post_PackFrequency=NONE
93  fi
94
[726]95  #====================================================
96  # R_SAVE : Job output directory
97  # R_BUFR : Job output buffered directory
98
99  if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
100    FreeName=$( echo ${config_UserChoices_JobName} | sed 's/.*_//' )
101    if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
102      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
103      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
104      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
105      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
106    else
107      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
108      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
109      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
110      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
111    fi
112  else
113    if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
114      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
115      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
116      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
117      R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
118    else
119      R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
120      R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
121      R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
122      R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
123    fi
124  fi
125
[782]126  #====================================================
[1031]127  # Define REBUILD_DIR : where we store files needing rebuild process 
128  REBUILD_DIR=${R_BUFR}/REBUILD
129  if [ ! X${TaskType} = Xchecking ] ; then
130    IGCM_sys_MkdirWork ${REBUILD_DIR}
[782]131  fi
132
[903]133  #====================================================
134  # DodsCopy : apply default value if not defined
135  if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] || [ X${config_Post_DodsCopy} = X ] ) ; then
136    config_Post_DodsCopy=TRUE
137  fi
138
[726]139  IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
140}
141
142#===================================
[2]143function IGCM_config_Initialize
144{
[544]145  IGCM_debug_PushStack "IGCM_config_Initialize"
[2]146
[544]147  # Debug Print :
148  echo
149  IGCM_debug_Print 1 "IGCM_config_Initialize"
[2]150
[544]151  # Test modipsl tree existence.
152  IGCM_sys_TestDir ${MODIPSL}
153  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
154  IGCM_sys_TestDir ${libIGCM}
155  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
156  IGCM_sys_TestDir ${R_EXE}
157  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
158  IGCM_sys_TestDir ${SUBMIT_DIR}
159  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
[2]160
[544]161  if ( $DEBUG_debug ) ; then
162    echo "Keep trace of inital SUBMIT_DIR : "
163    ls -lta ${SUBMIT_DIR}
164  fi
[144]165
[544]166  #==================================
[619]167  # Read ListOfComponents section:
[726]168  echo
169  IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
170
[544]171  IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
172  for comp in ${config_ListOfComponents[*]} ; do
173    IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
174  done
175  IGCM_debug_Print 3 ${config_ListOfComponents[*]}
[2]176
[544]177  #==================================
[726]178  # Read Executable section:
[544]179  IGCM_card_DefineArrayFromSection   ${SUBMIT_DIR}/config.card Executable
[619]180
[544]181  #==================================
[726]182  # Read Restarts section:
183  # Restarts : Gerneral rule or local for each component.
184  echo
185  IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
186
187  IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
188  for option in ${config_Restarts[*]} ; do
189    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
190    eval auxprint=\${config_Restarts_${option}}
191    IGCM_debug_Print 3 "${option} : ${auxprint}"
192  done
193
194  #==================================
195  # Define Job Outputs Name
196  echo
[544]197  IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
[939]198  Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
[544]199  IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
[939]200  Exe_Output=out_execution
[544]201  IGCM_debug_Print 3 "Exe_Output           = ${Exe_Output}"
[2]202
[544]203  #===================================================================#
204  # Prepare variables available for ${COMP}.card and ${COMP}.driver   #
205  #             But available to any son functions                    #
206  #===================================================================#
[2]207
[544]208  # Convert yyyy-mm-dd date to gregorian yyyymmdd
209  DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
[619]210  DateEnd=$(   IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd}   )
[2]211
[544]212  # Period Length In Days between DateBegin and DateEnd
213  (( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} )  + 1 ))
214  if [ ${ExperienceLengthInDays} -lt 0 ] ; then
215    IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
216    IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
[619]217    IGCM_debug_Verif_Exit
[544]218  fi
[2]219
[544]220  # Day and Year of Initial State (Given in julian format)
221  InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
222  InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
[2]223
[544]224  #================================================================#
225  #                  Test and Prepare directories                  #
226  #================================================================#
[2]227
[544]228  # ==> 4 kinds of input files :
229  #     1) R_INIT  : Initial State Files   (Etat0, carteveg)
230  #     2) R_BC    : Boundary Conditions   (Forcages, lai)
231  #     3) Parameters files (allready define through ${SUBMIT_DIR})
232  #     4) Restarts files   (allready define in IGCM_config_Initialize)
[2]233
[544]234  # Here we offer the possibility to redefine R_INIT, R_BC
235  # and PeriodNb through config.card
236  R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
[731]237  echo
[544]238  IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
239  IGCM_debug_Print 3 "R_INIT=${R_INIT}"
240  R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
241  IGCM_debug_Print 3  "R_BC=${R_BC}"
242  PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
243  IGCM_debug_Print 3  "Loop in main Job with ${PeriodNb} period(s)"
[2]244
[544]245  # Test Archive input/output.
[726]246  echo
[544]247  IGCM_sys_TestDirArchive ${ARCHIVE}
248  [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
[2]249
[771]250  # SD ADA SPECIFIC #
251  #      TO FIX     #
252  #IGCM_sys_TestDirArchive ${R_IN}
253  #[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
254
[619]255  if ( ${FirstInitialize} ) ; then
[726]256    IGCM_sys_MkdirArchive   ${R_SAVE}
[805]257    [ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
[544]258  else
259    IGCM_sys_TestDirArchive ${R_SAVE}
260    [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
[2]261
[805]262    if [ ! ${config_Post_PackFrequency} = NONE ] ; then
263      IGCM_sys_TestDir ${R_BUFR}
264      [ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
265    fi
[545]266
[544]267    #Test state of run in run.card
268    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
[619]269
[544]270    if ( [ ${run_Configuration_PeriodState} != "Start" ] && [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
[619]271      echo
[544]272      IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
[985]273      IGCM_debug_Print 1 "Check post-processing jobs carefully by running ${libIGCM}/RunChecker.job -p ${SUBMIT_DIR}"
274      IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_month.job to rerun one period"
275      IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_year.job to rerun more."
[544]276      IGCM_debug_Exit
277      IGCM_debug_Verif_Exit
[2]278    fi
[544]279  fi
[2]280
[544]281  #====================================================
[619]282  # Experience type : DEB(ug), DEV(elopment), RUN
[544]283  if [ X${JobType} != XRUN ] ; then
284    echo
285    echo "===================================================="
286    echo "libIGCM JOB is NOT in RUN type mode."
287    echo "!! OUTPUT files will NOT be PROTECTED !!"
288    echo "Be carefull : you can ERASE the result of this job !"
[128]289
[544]290    case ${JobType} in
291    DEB)
292      echo "DEBUG mode : activation of 'set -vx' mode."
293      echo "DEBUG mode : no protection for output files."
294      echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
295      ;;
296    DEV)
297      echo "DEVelopment mode : no protection for output files."
298      echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
299      ;;
300    esac
[166]301
[544]302    if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
303      if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
[619]304        echo "------------"
305        echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
[786]306        echo "          to PeriodLength : ${config_UserChoices_PeriodLength}"
[619]307        echo "------------"
308        config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
[544]309      fi
[128]310    fi
[544]311    echo "===================================================="
312    echo
313  fi
[128]314
[544]315  #====================================================
316  #R_OUT_KSH : Storage place for job output
317  #R_OUT_EXE : Storage place for binary used during simulation
318  R_OUT_KSH=${R_SAVE}/Out
319  R_OUT_EXE=${R_SAVE}/Exe
[166]320
[544]321  #====================================================
322  #R_BUF_KSH : Buffer place for job output
323  #R_BUF_EXE : Buffer place for binary used during simulation
324  R_BUF_KSH=${R_BUFR}/Out
325  R_BUF_EXE=${R_BUFR}/Exe
326
327  IGCM_debug_PopStack "IGCM_config_Initialize"
[2]328}
329
330#===================================
[890]331function IGCM_config_DaysInPeriodLength
[118]332{
[890]333  IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
[118]334
[544]335  typeset i
[124]336
[544]337  # Determine number of day(s) in PeriodLength :
338  case ${config_UserChoices_PeriodLength} in
339  *Y|*y)
[619]340    PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[yY]//' )
[731]341    echo
[544]342    IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
343    PeriodLengthInDays=0
344    i=0
[619]345    until [ $i -ge $PeriodLengthInYears ] ; do
346      (( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
347      (( i=i+1 ))
348    done
[544]349    ;;
350  *M|*m)
[619]351    PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[mM]//' )
[731]352    echo
[544]353    IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
354    PeriodLengthInDays=0
355    i=0
[619]356    until [ $i -ge $PeriodLengthInMonths ] ; do
[829]357      if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
358        (( PeriodLengthInDays  = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
[619]359      else
[829]360        (( PeriodLengthInDays  = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
[619]361      fi
362      (( i=i+1 ))
363    done
[544]364    ;;
[619]365  *D|*d)
[544]366    PeriodLengthInMonths=0
[731]367    PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} | sed -e 's/[dD]//' )
368    echo
369    IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
[619]370  *)
[890]371    IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in *Y, *M, *D."
[544]372    IGCM_debug_Verif_Exit ;;
373  esac
[118]374
[890]375  IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
[380]376}
[118]377
[380]378#===================================
[890]379function IGCM_config_DateCoherency
[891]380{
[890]381  IGCM_debug_PushStack "IGCM_config_DateCoherency"
382
383  echo
384  IGCM_debug_Print 1 "IGCM_config_DateCoherency"
385  echo
386
387  typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
388
389  # check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
390  # DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
391  echo
[922]392
[890]393  case ${config_UserChoices_PeriodLength} in
394  *Y|*y)
395    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
396    ;;
397  *M|*m)
398    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
399    ;;
400  *D|*d)
401    Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
402    ;;
403  esac
[892]404  VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
[890]405
[892]406  if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
407    IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
408    IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
409    IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
410    IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
411    IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
412    IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
413
414    IGCM_debug_Exit "STOP here to avoid further issues."
415  fi
416
[890]417  # PeriodDateBegin + PeriodLength = PeriodDateEnd
418  VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
419
420  IGCM_debug_PopStack "IGCM_config_DateCoherency"
421}
422
423
424#===================================
[380]425function IGCM_config_Check
426{
[544]427  IGCM_debug_PushStack "IGCM_config_Check"
[380]428
[544]429  # If one of the following modulo is not zero :
430  # we will issue an error then explain and exit in
431  # AA_job IGCM_debug_Verif_Exit call before binary submission
[380]432
[544]433  echo
434  IGCM_debug_Print 1 "IGCM_config_Check"
435  echo
436
437  typeset i
438
[554]439  # Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
[545]440  if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
[544]441    AsynchronousRebuild=true
442    IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
[380]443    echo
[544]444    # modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
445    IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
446    IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
[554]447    IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
448    IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
[544]449    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
450    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
451    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
452    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
453  else
454    AsynchronousRebuild=false
455    IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
456    IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
[380]457    echo
[554]458    #modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
[544]459    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
460    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
461    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
462    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency   config_UserChoices_PeriodLength
463  fi
[380]464
[619]465  # Check PackFrequency against other key frequencies
[554]466  # Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
467  if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
468    Pack=true
469    #
470    IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
471    IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
472    IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
473    IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
474    IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
475    IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
476  else
477    Pack=false
478  fi
479
[890]480  # modulo (TimeSeriesFrequency and all Chunck2D) must be zero
[544]481  NbJob=${#CHUNCK2D_SIZE[@]}
482  i=0
483  until [ $i -ge $NbJob ]; do
484    value=${CHUNCK2D_SIZE[${i}]}
485    IGCM_debug_Print 1 "Check coherence between All Chunck2D frequency and TimeSeriesFrequency"
486    IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
[619]487    case ${value} in
488    *Y|*y) ;;
[786]489    *)
490      IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year *Y|*y in comp.card"
[544]491      IGCM_debug_Exit "This will stop the job" ;;
492    esac
493    (( i=i+1 ))
494  done
[387]495
[890]496  # modulo (TimeSeriesFrequency and all Chunck3D) must be zero
[544]497  NbJob=${#CHUNCK3D_SIZE[@]}
498  i=0
499  until [ $i -ge $NbJob ]; do
500    value=${CHUNCK3D_SIZE[${i}]}
501    IGCM_debug_Print 1 "Check coherence between All Chunck3D frequency and TimeSeriesFrequency"
502    IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
[619]503    case ${value} in
504    *Y|*y) ;;
[786]505    *)
506      IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year *Y|*y in comp.card"
[544]507      IGCM_debug_Exit "This will stop the job" ;;
508    esac
509    (( i=i+1 ))
510  done
[380]511
[890]512  # check to be sure there is enough space on temporary filesystems to run
[731]513  echo
[651]514  IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
515  IGCM_sys_check_quota
516
[933]517  # check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
518  echo
519  IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
520  IGCM_sys_check_path
521
522
[544]523  IGCM_debug_PopStack "IGCM_config_Check"
[118]524}
525
526#===================================
[2]527function IGCM_config_PeriodStart
528{
[544]529  IGCM_debug_PushStack "IGCM_config_PeriodStart"
[2]530
[544]531  echo
532  IGCM_debug_Print 1 "IGCM_config_PeriodStart"
533  echo
[2]534
[544]535  if ( ${FirstInitialize} ) ; then
536    #================================================#
537    #         Initialize date/period information     #
538    #================================================#
[2]539
[890]540    IGCM_date_GetYearMonthDay ${DateBegin} year month day
541    IGCM_config_DaysInPeriodLength
[2]542
[939]543    PeriodDateBegin=${DateBegin}
[544]544    PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
[939]545    CumulPeriod=1
[2]546
[544]547    #=================================================#
548    #              Write updated run.card             #
549    #=================================================#
[2]550
[544]551    #Correct run.card Configuration for this period
552    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
553    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
554    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
555    if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
556      IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
557    fi
[427]558
[544]559    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
[619]560
[544]561  else
562    #================================================#
563    #         The file run.card allready exist       #
564    #================================================#
[2]565
[544]566    #Test state of run in run.card
567    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
568    if ( [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
[619]569      echo
[544]570      IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
571      IGCM_debug_Print 1 "Try running ${libIGCM}/clean_month.job to fix this"
572      echo
573      IGCM_debug_Exit
574      IGCM_debug_Verif_Exit
575    fi
[2]576
[544]577    #===================================#
[786]578    #        Read updated run.card      #
[544]579    #===================================#
[2]580
[544]581    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
582    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
583    IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
[2]584
[544]585    PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
586    PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
[939]587    CumulPeriod=${run_Configuration_CumulPeriod}
[2]588
[1028]589    LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
[471]590
[544]591    if [ ${Period} = 1 ]; then
[910]592      # save last Job output and current run.card
593      typeset Potential
594      IGCM_sys_Cd ${SUBMIT_DIR}
595      #
596      IGCM_debug_Print 2 "Save previous ksh job output"
597      for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
598        if [ X${Pack} = Xtrue ] ; then
[911]599          ( IGCM_sys_TestFileBuffer  ${R_BUF_KSH}/${Potential} ) || IGCM_sys_PutBuffer_Out ${Potential} ${R_BUF_KSH}/${Potential}.$$ rw
[910]600        else
[911]601          ( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) || IGCM_sys_Put_Out ${Potential} ${R_OUT_KSH}/${Potential}.$$ rw
[910]602        fi
603      done
604      #
605      IGCM_debug_Print 2 "Save current run.card"
[544]606      IGCM_card_CheckConflict run.card
[634]607      if [ X${Pack} = Xtrue ] ; then
[910]608        IGCM_sys_PutBuffer_Out ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card rw
[634]609      else
[910]610        IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card rw
[634]611      fi
[910]612      #
[544]613      IGCM_sys_Cd ${RUN_DIR}
614    else
615      unset FileToBeDeleted
616    fi
[2]617
[892]618    # Determine number of day(s) in PeriodLength
[544]619    IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
[890]620    IGCM_config_DaysInPeriodLength
[2]621
[892]622    # Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
623    IGCM_config_DateCoherency
624    # And EXIT if not OK
625    IGCM_debug_Verif_Exit
626
[544]627    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
[890]628  fi
[2]629
[544]630  # BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
631  # Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
632  year_m1=$(( year - 1 ))
[619]633  year_p1=$(( year + 1 ))
[544]634  # Compute month_m1 (month minus 1M)
635  # Compute yyyymm_m1 (yyyymm minus 1M)
[829]636  month_m1=$(( 10#${month} - 1 ))
[544]637  if [ ${month_m1} = 0 ]; then
638    month_m1=12
639    yyyymm_m1=${year_m1}12
640  elif [ ${month_m1} -le 9 ]; then
641    month_m1=0${month_m1}
642    yyyymm_m1=${year}${month_m1}
643  else
644    yyyymm_m1=${year}${month_m1}
645  fi
646  # Compute month_p1 (month plus 1M)
647  # Compute yyyymm_p1 (yyyymm plus 1M)
[829]648  month_p1=$(( 10#${month} + 1 ))
[544]649  if [ ${month_p1} = 13 ]; then
650    month_p1=01
651    yyyymm_p1=${year_p1}01
652  elif [ ${month_p1} -le 9 ]; then
653    month_p1=0${month_p1}
654    yyyymm_p1=${year}${month_p1}
655  else
656    yyyymm_p1=${year}${month_p1}
657  fi
658  #IGCM_debug_Print 1 "jg 1 month_m1 = ${month_m1} month_p1 = ${month_p1} "
659  #IGCM_debug_Print 1 "jg 1 calculate yyyymm_m1 = ${yyyymm_m1} "
660  #IGCM_debug_Print 1 "jg 1 calculate yyyymm_p1 = ${yyyymm_p1} "
[522]661
[799]662  #===================================================================#
663  # Calculate CyclicYear to be used for looping over a given forcing  #
664  # period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
665  #===================================================================#
666
667  # To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
668  # CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
669  if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
670    CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
[819]671    CyclicYear_p1=NOTDEFINED
672
673    # For current year
674    yeartmp=$year
675    diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
676    while [ $diffy -lt 0 ] ; do
677      yeartmp=$(( ${yeartmp} + ${CycleNb} ))
678      diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
679    done
680    CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
681
682    # For next coming year
683    yeartmp=$(( $year + 1 ))
684    diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
685    while [ $diffy -lt 0 ] ; do
686      yeartmp=$(( ${yeartmp} + ${CycleNb} ))
687      diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
688    done
689    CyclicYear_p1=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
690
[812]691    IGCM_debug_Print 1 "CyclicYear   = ${CyclicYear}, CyclicYear_p1 = ${CyclicYear_p1}, current year=$year"
[799]692  else
693    CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
[812]694    CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
[819]695    IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
[799]696  fi
697
[544]698  # END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
[522]699
[544]700  #===================================================================#
701  # Prepare variables available for ${COMP}.card and ${COMP}.driver   #
[786]702  #             But available to any son functions                    #
[544]703  #===================================================================#
[2]704
[544]705  # Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
706  (( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
[2]707
[544]708  # Debug Print :
[731]709  echo
[544]710  IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
711  IGCM_debug_Print 1 "Year of simulation      : ${year}"
712  IGCM_debug_Print 1 "Month of simulation     : ${month}"
713  IGCM_debug_Print 1 "PeriodLengthInDays      : ${PeriodLengthInDays}"
714  IGCM_debug_Print 1 "PeriodDateBegin         : ${PeriodDateBegin}"
715  IGCM_debug_Print 1 "PeriodDateEnd           : ${PeriodDateEnd}"
716  IGCM_debug_Print 1 "SimulationLengthInDays  : ${SimulationLengthInDays}"
717  IGCM_debug_Print 1 "ExperienceLengthInDays  : ${ExperienceLengthInDays}"
[2]718
[544]719  #================================================================#
720  #         Prepare variables available for comp_finalyze          #
721  #================================================================#
[2]722
[544]723  # Period for save files
[939]724  DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
[2]725
[544]726  # Prefix for save files of this period
[939]727  PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
[2]728
[544]729  # List of files that will be deleted in RUN_DIR after run
730  [ -f stack ] && FileToBeDeleted[0]="stack"
[2]731
[544]732  # Test if the same run as already been saved :
733  if [ X${JobType} = XRUN ] ; then
734    if [ ${DRYRUN} -le 0 ] ; then
735      if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
[619]736        IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
737        IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
738        IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
739        IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
740        IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
741        IGCM_debug_Verif_Exit
[544]742      fi
[2]743    fi
[544]744  else
745    if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
746      IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
747    fi
748  fi
[128]749
[939]750  #================================================================#
751  #       Prepare variables available for binary execution         #
[1009]752  #       Call function for creation of run script                 #
753  #       Only done once per job                                   #
[939]754  #================================================================#
[616]755
[1009]756  if [ ${Period} -eq 1 ]; then
757    typeset ExeNameIn ExeNameFirst CompNameFirst comp i j
758    typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
[616]759
[1009]760    PROCESSUS_NUMBER=0
761    NbExec=0
762    i=0
[939]763
[1009]764    OK_PARA_MPI=false
765    OK_PARA_OMP=false
766    OK_PARA_NOD=false
767    OK_PARA_MPMD=false
[616]768
[1009]769    for comp in ${config_ListOfComponents[*]} ; do
[616]770
[1009]771      IGCM_debug_Print 1 ${comp}
[616]772
[1009]773      eval ExeNameIn=\${config_Executable_${comp}[0]}
[616]774
[1009]775      # NO order in config.card for parallelized values !
776      # just use suffix : MPI , OMP and NOD (for number of NODes.)
[616]777
[1009]778      # NOD is the number of NODes allocated
779      eval ${comp}_PROC_NOD=0
[616]780
[1009]781      # MPI is the number of MPI processus per nodes
782      eval ${comp}_PROC_MPI=0
[616]783
[1009]784      # OMP is the number of OpenMP threads per MPI processus
785      eval ${comp}_PROC_OMP=0
[619]786
[1009]787      # Only if we really have an executable for the component :
788      if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
[616]789
[1009]790        # Keep the first executable found and the first CompName
791        ExeNameFirst=${ExeNameIn}
792        CompNameFirst=${comp}
[616]793
[1009]794        # Are we a second executable?
795        (( NbExec = NbExec + 1 ))
[616]796
[1009]797        # set 1 MPI task, 1 OpenMP thread and 1 node as default
798        eval ${comp}_PROC_MPI=1
799        eval ${comp}_PROC_OMP=1
800        eval ${comp}_PROC_NOD=1
[616]801
[1009]802        eval NbElts=\${#config_Executable_${comp}[@]}
[616]803
[1009]804        if [ ${NbElts} -ge 2 ] ; then
805          (( j = 2 ))
806          while [ $j -lt ${NbElts} ] ; do
807            eval tempvar=\${config_Executable_${comp}[${j}]}
808            IGCM_debug_Print 2 ${tempvar}
[616]809
[1009]810            if [ X${tempvar} = X ] ; then
811              IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
812              IGCM_debug_Exit "Check your config.card. Exit now"
813              IGCM_debug_Verif_Exit
814            fi
[616]815
[1009]816            case ${tempvar} in
817            *[mM][pP][iI]*)
818              # Read MPI parameter for composante
819              eval ${comp}_PROC_MPI=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/MPI//" )
820              OK_PARA_MPI=true;;
821            *[oO][mM][pP]*)
822              # Read OMP parameter for composante
823              eval ${comp}_PROC_OMP=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/OMP//" )
824              OK_PARA_OMP=true;;
825            *[nN][oO][dD]*)
826              # Read NOD (NumBer of Nodes) parameter for composante
827              eval ${comp}_PROC_NOD=$( echo ${tempvar} | tr '[a-z]' '[A-Z]' | sed -e "s/NOD//" )
828              OK_PARA_NOD=true
829              OK_PARA_MPI=true;;
830            esac
831            (( j = j + 1 ))
832          done
833        fi
834        eval tempvarMPI=\${${comp}_PROC_MPI}
835        eval tempvarNOD=\${${comp}_PROC_NOD}
836        eval tempvarOMP=\${${comp}_PROC_OMP}
837
838        (( PROCESSUS_NUMBER = PROCESSUS_NUMBER + tempvarMPI * tempvarNOD * tempvarOMP ))
[616]839      fi
[1009]840      (( i=i+1 ))
841    done
[616]842
[1009]843    # set MPMD mode if more than 2 executable names.
844    [ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
[616]845
[1009]846    # Verification of BATCH_NUM_PROC_TOT total number of processors set in job header.
847    if [ X${BATCH_NUM_PROC_TOT} != X ] ; then
848      # BATCH_NUM_PROC_TOT is set
849      if ( ${OK_PARA_MPI} ) ; then
850        IGCM_debug_Print 1 "MPI/OMP/NOD found into config.card and BATCH_NUM_PROC_TOT = ${BATCH_NUM_PROC_TOT} "
851      else
852        # with previous method.
853        if [ ${BATCH_NUM_PROC_TOT} -gt 1 ] ; then
854          # with more than 1 proc
855          if ( ${OK_PARA_MPMD} ) ; then
856            # with MPMD ie CPL/oasis method
857            IGCM_debug_Print 2 "Use default number of MPI tasks for this machine : "
858            IGCM_debug_Print 2 "${DEFAULT_NUM_PROC_OCE} for OCE"
859            IGCM_debug_Print 2 "${DEFAULT_NUM_PROC_CPL} for CPL"
860            IGCM_debug_Print 2 "${DEFAULT_NUM_PROC_ATM} for ATM"
861            OK_PARA_MPI=true
862            CPL_PROC_MPI=${DEFAULT_NUM_PROC_CPL}
863            OCE_PROC_MPI=${DEFAULT_NUM_PROC_OCE}
864            ATM_PROC_MPI=${DEFAULT_NUM_PROC_ATM}
865            PROCESSUS_NUMBER=${DEFAULT_NUM_PROC_TOTAL}
866          else
867            # with have only one executable
868            IGCM_debug_Print 2 "Use ${BATCH_NUM_PROC_TOT} MPI tasks for ${CompNameFirst} : ${ExeNameFirst} "
869            OK_PARA_MPI=true
870            eval ${CompNameFirst}_PROC_MPI=${BATCH_NUM_PROC_TOT}
871            PROCESSUS_NUMBER=${BATCH_NUM_PROC_TOT}
872          fi
[616]873        else
[1009]874          PROCESSUS_NUMBER=1
[616]875        fi
[1009]876      fi
877      # Verification with PBS parameter
878      if [ ${BATCH_NUM_PROC_TOT} -ne ${PROCESSUS_NUMBER} ] ; then
879        IGCM_debug_Exit "ERROR with parallelization parameters !"
880        IGCM_debug_Print 2 "Job header variable BATCH_NUM_PROC_TOT = ${BATCH_NUM_PROC_TOT} "
881        IGCM_debug_Print 2 "is the total number of _processors_ reserved."
882        IGCM_debug_Print 2 "It is not equal to the sum of _processus_  = ${PROCESSUS_NUMBER}."
883        IGCM_debug_Verif_Exit
884      fi
885      NUM_PROC_CPL=${CPL_PROC_MPI} # for backward compatibility
886      NUM_PROC_OCE=${OCE_PROC_MPI} # for backward compatibility
887      NUM_PROC_ATM=${ATM_PROC_MPI} # for backward compatibility
888    else # BATCH_NUM_PROC_TOT=""
889      if ( ${OK_PARA_MPI} ) ; then
890        IGCM_debug_Exit "ERROR : missing value for ${BATCH_NUM_PROC_TOT} processors,"
891        IGCM_debug_Print 2 "You have parallel parameters in config->Executable->list."
892        IGCM_debug_Print 2 "Please add BATCH_NUM_PROC_TOT variable in job header as well."
893        IGCM_debug_Exit "Exit now."
894        IGCM_debug_Verif_Exit
[619]895      else
[1009]896        # sequential case !
897        if [ ${PROCESSUS_NUMBER} -eq 0 ] ; then
898          (( PROCESSUS_NUMBER = 1 ))
899          IGCM_debug_Print 2 "PROCESSUS_NUMBER is all 0 (sequential use of old definition in config->Executable->list)."
900          IGCM_debug_Print 2 "We set it to 1."
901        fi
[616]902      fi
903    fi
904
[1009]905    IGCM_debug_Print 1 "MPI/OMP treatment PROCESSUS_NUMBER = ${PROCESSUS_NUMBER}"
[616]906
[1009]907    IGCM_sys_build_execution_scripts
908  fi
[616]909
[544]910  ExecutionFail=false
[473]911
[544]912  IGCM_debug_PopStack "IGCM_config_PeriodStart"
[2]913}
914
915#===================================
[523]916function IGCM_config_SaveSourceModifications
917{
[544]918  IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
[523]919
[544]920  typeset ExeOutDateMax listVarEnv
921  ExeOutDateMax=$1
[523]922
[544]923  listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
924  IGCM_sys_RshMaster "\
[545]925    . ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
[544]926    export ExeOutDateMax=${ExeOutDateMax};\
[545]927    export R_OUT_EXE=${R_OUT_EXE};\
928    export PREFIX=${PREFIX};\
929    export SUBMIT_DIR=${SUBMIT_DIR};\
930    export listVarEnv=${listVarEnv};\
931    Script_Output=out_SaveSourceModifications;\
932    IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
[523]933
[544]934  IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
[523]935}
936
937#===================================
[2]938function IGCM_config_PeriodEnd
939{
[544]940  IGCM_debug_PushStack "IGCM_config_PeriodEnd"
[2]941
[544]942  echo
943  IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
944  echo
[2]945
[622]946  if [ ${DRYRUN} -le 1 ] ; then
[457]947
[622]948    IGCM_debug_Print 1 "Check components binary : size and creation date"
[2]949
[622]950    typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
951    typeset comp i
952    typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
[2]953
[622]954    #==================================#
955    #        Get last Exe Size         #
956    #==================================#
[2]957
[622]958    (( i=0 ))
959    if ( ${FirstInitialize} ) ; then
960      run_Log_LastExeSize=""
961      for comp in ${config_ListOfComponents[*]} ; do
962        run_Log_LastExeSize[$i]=0
963        (( i=i+1 ))
964      done
965    else
966      IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
967    fi
968    #==================================#
969    #         And Build ExeDate        #
970    #==================================#
[2]971
[622]972    # ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
973    # Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
974    # ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
975    ExeDate=""
976    NextExeSize="( "
977    (( i=0 ))
978    UpdateExe=false
979    (( ExeSecDateMax = 0 ))
980    for comp in ${config_ListOfComponents[*]} ; do
[2]981
[622]982      IGCM_debug_Print 3 ${comp}
[2]983
[622]984      eval ExeNameIn=\${config_Executable_${comp}[0]}
985      eval ExeNameOut=\${config_Executable_${comp}[1]}
986      # Only if we really have an executable for the component :
987      if [ X${ExeNameIn} = X\"\" ] ; then
988        # If there is no exe file for this component
989        (( ExeSize=0 ))
[619]990      else
[939]991        LS_bin=${R_EXE}/${ExeNameIn}
[622]992        IGCM_sys_FileSize ${LS_bin} ExeSize
993
994        set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
995        if [ X${ExeDate} = X ] ; then
996          # First component exe date
997          ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
998        else
999          ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1000        fi
1001        ExeDate=${ExeDate}_${LS_comp[7]}
[619]1002      fi
[2]1003
[622]1004      if [ ${i} -eq 0 ] ; then
1005        # First component
1006        NextExeSize="( "${ExeSize}
[619]1007      else
[622]1008        NextExeSize=${NextExeSize}", "${ExeSize}
[619]1009      fi
[939]1010      LastCompExeSize=${run_Log_LastExeSize[$i]}
[622]1011      (( i=i+1 ))
[523]1012
[622]1013      if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1014        if ( ${FirstInitialize} ) ; then
1015          IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1016        else
1017          IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1018          IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
[939]1019          FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
[622]1020        fi
[939]1021        IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
[622]1022        UpdateExe=true
1023
[691]1024        # SD : switch off for now
1025        #IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1026        #if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1027        #  ExeSecDateMax=$ExeSecDate
1028        #fi
[619]1029      fi
[622]1030    done
[2]1031
[622]1032    # SD : switch off for now
1033    #if ( ${UpdateExe} ) ; then
1034    #  echo "Launch SaveSourceModifications."
1035    #  IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1036    #fi
[523]1037
[622]1038    NextExeSize=${NextExeSize}" )"
1039    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
[457]1040
[622]1041    if [ ${DRYRUN} -le 1 ] ; then
1042      tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1043      ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1044      RET=$?
1045      if [ $RET -eq 0 ] ; then
1046        # ExeCpuLog variable contents 5 fields
1047        echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" |   \
1048          gawk '{printf("# %11d | %15s | %15s | %19s | %19s | %15.5f | %15.5f | %15.5f | %s\n", \
[544]1049            $1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
[622]1050      fi
[939]1051      FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
[619]1052    fi
[622]1053
[619]1054  fi
[2]1055
[1041]1056  #==================================#
1057  #         Save Job output          #
1058  #==================================#
1059  if [ X${Pack} = Xtrue ] ; then
1060    IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1061  else
1062    IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1063  fi
1064  FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1065
[622]1066  # All was right ? no ? then we stop.
[619]1067  IGCM_debug_Verif_Exit
[2]1068
[622]1069  # If all was OK, we can delete all files not necessary for next Job
[619]1070  echo
1071  IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
[2]1072
[619]1073  if [ ${DRYRUN} -le 2 ] ; then
1074    for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1075  else
1076    echo ${FileToBeDeleted[@]}
1077  fi
[2]1078
[922]1079  # Send some accounting element to the user if CumulPeriod=3
1080  if [ ${CumulPeriod} -eq 3 ] ; then
1081    echo
1082    IGCM_debug_Print 1 "Send email containing some accounting information : "
1083
1084    RealCpuTime=$( echo ${ExeCpuLog} | gawk '{print $3}' )
1085
1086    consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${PROCESSUS_NUMBER}/3600" | bc )
1087
1088    consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" | bc )
1089
1090    recommendedPeriodNb=$( echo "scale=6;24/${consumeHoursPerPeriod}*${PROCESSUS_NUMBER}" | bc )
1091
1092    IGCM_sys_SendMail Accounting
1093  fi
1094
[622]1095  #=================================================#
1096  #         Modification of libIGCM behaviour       #
1097  #=================================================#
[119]1098
[622]1099  # To use this function, one must copy libIGCM.card from ${libIGCM} directory
1100  # and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1101  # variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
[619]1102  if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1103    echo
1104    echo "########################################################################"
1105    echo "!!!                 Modification of libIGCM behaviour                !!!"
1106    echo
[119]1107
[619]1108    IGCM_debug_Print 1 "DefineArrayFromOption  : libIGCM_UserChanges in libIGCM.card"
1109    IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1110    IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
[119]1111
[622]1112    # Special treatments for libIGCM internals
[619]1113    for option in ${libIGCM_UserChanges[*]} ; do
1114      IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
[119]1115
[619]1116      echo "We will change : ${option}."
1117      eval echo "Previous value : " \${${option}}
1118      eval echo "Change to : " \${libIGCM_UserChanges_${option}}
[119]1119
[619]1120      eval ${option}=\${libIGCM_UserChanges_${option}}
[119]1121
[619]1122      case ${option} in
1123      config_UserChoices_DateEnd)
1124        IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1125        DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1126
[622]1127        # Period Length In Days between DateBegin and DateEnd
[619]1128        (( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} )  + 1 ))
1129        if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1130          IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
[939]1131          IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
[619]1132          IGCM_debug_Verif_Exit
1133        fi
1134        ;;
1135      config_UserChoices_PeriodLength)
1136        IGCM_debug_Print 1  "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1137        ;;
1138      PeriodNb)
1139        IGCM_debug_Print 1  "Loop in main Job with ${PeriodNb} period(s)"
1140        ;;
1141      config_Post_RebuildFrequency)
1142        IGCM_debug_Print 1  "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1143        IGCM_post_Configure
1144        ;;
1145      config_Post_TimeSeriesFrequency)
[653]1146        IGCM_debug_Print 1  "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
[619]1147        IGCM_post_Configure
1148        ;;
1149      config_Post_SeasonalFrequency)
[653]1150        IGCM_debug_Print 1  "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
[619]1151        IGCM_post_Configure
1152        ;;
1153      esac
1154    done
[119]1155
[619]1156    echo
1157    echo "########################################################################"
1158    echo
[544]1159  fi
[119]1160
[544]1161  #=================================================#
1162  #         Determine next computed period          #
1163  #=================================================#
[2]1164
[544]1165  PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1166  IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1167  year_m1=$(( year - 1 ))
1168  year_p1=$(( year + 1 ))
[890]1169  IGCM_config_DaysInPeriodLength
[544]1170  PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
[2]1171
[544]1172  # Debug Print :
1173  echo
1174  IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1175  IGCM_debug_Print 1 "PeriodDateBegin       : ${PeriodDateBegin}"
1176  IGCM_debug_Print 1 "PeriodDateEnd         : ${PeriodDateEnd}"
1177  IGCM_debug_Print 1 "PeriodLengthInDays    : ${PeriodLengthInDays}"
[2]1178
[544]1179  PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1180  PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
[2]1181
[544]1182  (( CumulPeriod = CumulPeriod + 1 ))
[2]1183
[544]1184  # Debug Print :
1185  echo
1186  IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1187  IGCM_debug_Print 3 "PeriodDateEnd Human   : ${PeriodDateEnd}"
1188  IGCM_debug_Print 3 "CumulPeriod           : ${CumulPeriod}"
[2]1189
[544]1190  #=================================================#
[786]1191  #             Write updated run.card              #
[544]1192  #=================================================#
[2]1193
[544]1194  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1195  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1196  IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
[2]1197
[544]1198  if ( ${FirstInitialize} ) ; then
1199    # It's no more the first time
1200    FirstInitialize=false
1201  fi
[446]1202
[544]1203  IGCM_debug_PopStack "IGCM_config_PeriodEnd"
[2]1204}
1205
1206#===================================
1207function IGCM_config_Finalize
1208{
[544]1209  IGCM_debug_PushStack "IGCM_config_Finalize"
[2]1210
[544]1211  echo
1212  IGCM_debug_Print 1 "IGCM_config_Finalize"
1213  echo
[2]1214
[873]1215  if ( $DEBUG_debug ) ; then
1216    # Inform the rabbitMQ queue
1217    if [ X${ActivateBigBro} = Xtrue ] ; then
[920]1218      # RabbitMQ message code
[873]1219      code=1100
[920]1220      # RabbitMQ message body
[1051]1221      Body=$( echo "{${genericSimulationID},\"msgCode\":\"${code}\",\"msgUID\":\"$(uuidgen)\",\"msgTimestamp\":\"$( date +"%Y-%m-%d-%T-%N" )\"}" )
[920]1222      # Fill the rabbitMQ queue
1223      IGCM_debug_sendAMQP
[873]1224    fi
1225  fi
1226
[544]1227  if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1228    #==========================#
1229    # End of entire simulation #
1230    #==========================#
[2]1231
[544]1232    # Mail notification
1233    IGCM_sys_SendMail
1234    #
1235    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
[920]1236    #
[544]1237    IGCM_debug_Print 1 "Normal End of computation."
[873]1238
[544]1239    if ( $DEBUG_debug ) ; then
[873]1240      if [ X${ActivateBigBro} = Xtrue ] ; then
[920]1241        # RabbitMQ message code
[873]1242        code=0100
[920]1243        # RabbitMQ message body
[1051]1244        Body=$( echo "{${genericSimulationID},\"msgCode\":\"${code}\",\"msgUID\":\"$(uuidgen)\",\"msgTimestamp\":\"$( date +"%Y-%m-%d-%T-%N" )\"}" )
[920]1245        # Fill the rabbitMQ queue
1246        IGCM_debug_sendAMQP
[873]1247      fi
[544]1248      echo
1249      IGCM_debug_Print 1 "Your files on ${R_SAVE} :"
1250      IGCM_sys_Tree ${R_SAVE}
[2]1251
[544]1252      IGCM_debug_Print 1 "Your files on ${R_BUFR} :"
1253      IGCM_sys_Tree ${R_BUFR}
1254    fi
1255  else
1256    #=================#
1257    # Submit next job #
1258    #=================#
[2]1259
[544]1260    IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
[2]1261
[544]1262    # Name of next Ksh Script output :
[939]1263    Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
[2]1264
[544]1265    IGCM_debug_Print 1 "Submit next job"
[922]1266    # SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1267    IGCM_sys_Cd ${SUBMIT_DIR}
[544]1268    # Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1269    ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null | wc -l )
1270    [ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? | head -$(( ${ScriptTot} - 5 )) )
[861]1271    # Submit next job and come back
[544]1272    IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
[861]1273    IGCM_sys_Cd -
[544]1274  fi
[154]1275
[544]1276  # Supress Non Deleted Boundary files
[804]1277  #if [ ${DRYRUN} -le 2 ] ; then
1278  #  IGCM_comp_DelFixeBoundaryFiles
1279  #  ls -la
1280  #fi
[619]1281
[804]1282  # Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
[544]1283  # Only for production run (No clean up in DEV or DEB mode)
1284  # and command sent from .. directory.
1285  IGCM_sys_Cd ..
[943]1286  [ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
[544]1287
[997]1288  # To be sure we flush message buffer before exiting the job
1289  FlushAMQP=TRUE
1290
[544]1291  IGCM_debug_PopStack "IGCM_config_Finalize"
[2]1292}
1293
1294#===================================
Note: See TracBrowser for help on using the repository browser.