source: trunk/libIGCM/AA_job @ 809

Last change on this file since 809 was 809, checked in by sdipsl, 11 years ago

Activate IGCM_sys_Chmod from DRYRUN=3 and add BoundaryFiles? list in FileToBeDeleted list at each Period.
Whitespace cleanup (from emacs macro) on libIGCM_sys_iitm.ksh
see #105
  • Property licence set to
    The following licence information concerns ONLY the libIGCM tools
    ==================================================================

    Copyright © Centre National de la Recherche Scientifique CNRS
    Commissariat à l'Énergie Atomique CEA

    libIGCM : Library for Portable Models Computation of IGCM Group.

    IGCM Group is the french IPSL Global Climate Model Group.

    This library is a set of shell scripts and functions whose purpose is
    the management of the initialization, the launch, the transfer of
    output files, the post-processing and the monitoring of datas produce
    by any numerical program on any plateforme.

    This software is governed by the CeCILL license under French law and
    abiding by the rules of distribution of free software. You can use,
    modify and/ or redistribute the software under the terms of the CeCILL
    license as circulated by CEA, CNRS and INRIA at the following URL
    "http://www.cecill.info".

    As a counterpart to the access to the source code and rights to copy,
    modify and redistribute granted by the license, users are provided only
    with a limited warranty and the software's author, the holder of the
    economic rights, and the successive licensors have only limited
    liability.

    In this respect, the user's attention is drawn to the risks associated
    with loading, using, modifying and/or developing or reproducing the
    software by the user in light of its specific status of free software,
    that may mean that it is complicated to manipulate, and that also
    therefore means that it is reserved for developers and experienced
    professionals having in-depth computer knowledge. Users are therefore
    encouraged to load and test the software's suitability as regards their
    requirements in conditions enabling the security of their systems and/or
    data to be ensured and, more generally, to use and operate it in the
    same conditions as regards security.

    The fact that you are presently reading this means that you have had
    knowledge of the CeCILL license and that you accept its terms.
  • Property svn:keywords set to Revision Author Date
File size: 20.4 KB
Line 
1#-Q- curie #!/bin/ksh
2#-Q- curie ######################
3#-Q- curie ## CURIE   TGCC/CEA ##
4#-Q- curie ######################
5#-Q- curie #MSUB -r ::Jobname::       # nom de la requete
6#-Q- curie #MSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
7#-Q- curie #MSUB -e Script_Output_::Jobname::.000001    # nom du fichier de sortie
8#-Q- curie #MSUB -eo
9#-Q- curie #MSUB -n ::JobNumProcTot:: # reservation des processeurs pour le job
10#-Q- curie #MSUB -T 86400             # Limite temps (en secondes)
11#-Q- curie #MSUB -q standard
12#-Q- curie ##MSUB -A gen****
13#-Q- curie BATCH_NUM_PROC_TOT=$BRIDGE_MSUB_NPROC
14#-Q- curie set +x
15#-Q- titane #!/usr/bin/ksh
16#-Q- titane ##################
17#-Q- titane ## TITANE   CEA ##
18#-Q- titane ##################
19#-Q- titane #MSUB -r ::Jobname::       # nom de la requete
20#-Q- titane #MSUB -o Script_Output_::Jobname::.000001    # nom du fichier de sortie
21#-Q- titane #MSUB -e Script_Output_::Jobname::.000001    # nom du fichier de sortie
22#-Q- titane #MSUB -eo
23#-Q- titane #MSUB -n ::JobNumProcTot:: # reservation des processeurs pour le job
24#-Q- titane #MSUB -T 86400             # Limite temps (en secondes)
25#-Q- titane #MSUB -p gen2211
26#-Q- titane BATCH_NUM_PROC_TOT=$BRIDGE_MSUB_NPROC
27#-Q- ada #!/bin/ksh
28#-Q- ada # ######################
29#-Q- ada # ##   ADA IDRIS   ##
30#-Q- ada # ######################
31#-Q- ada # Nom de la requete
32#-Q- ada # @ job_name = ::Jobname::
33#-Q- ada # Type de travail
34#-Q- ada # @ job_type = parallel
35#-Q- ada # Fichier de sortie standard
36#-Q- ada # @ output = Script_Output_::Jobname::.000001
37#-Q- ada # Fichier de sortie erreur (le meme)
38#-Q- ada # @ error = Script_Output_::Jobname::.000001
39#-Q- ada # Nombre de processus demandes
40#-Q- ada # @ total_tasks = ::JobNumProcTot::
41#-Q- ada # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
42#-Q- ada # Temps CPU max. par processus MPI hh:mm:ss
43#-Q- ada # @ wall_clock_limit = 1:00:00
44#-Q- ada # Nombre de taches OpenMP/pthreads par processus MPI
45#-Q- ada ### @ parallel_threads = 4
46#-Q- ada # Fin de l entete
47#-Q- ada # @ queue
48#-Q- sx9mercure #!/bin/ksh
49#-Q- sx9mercure ######################
50#-Q- sx9mercure ## SX9MERCURE   CEA ##
51#-Q- sx9mercure ######################
52#-Q- sx9mercure #PBS -N ::Jobname::           # nom de la requete
53#-Q- sx9mercure #PBS -m a                     # message a la fin du job
54#-Q- sx9mercure #PBS -j o                     # regroupement des sorties du job : standard error et output
55#-Q- sx9mercure #PBS -o Script_Output_::Jobname::.000001   # nom du fichier de sortie
56#-Q- sx9mercure #PBS -S /usr/bin/ksh          # shell de soumission
57#-Q- sx9mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
58#-Q- sx9mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT}          # reservation des processeurs pour le job
59#-Q- sx9mercure #PBS -l memsz_job=16.0gb      # limite memoire
60#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # limite en temps elapsed
61#-Q- sx9mercure #PBS -A gen6178               # groupe obligatoire sur SX9
62#-Q- sx9mercure #PBS -q para-sx9              # para-sx9 or prio-sx9 (for CMIP5 top priorities), bckg-sx9 (run with free slot) and test-sx9 (if elapsed<1h)
63#-Q- lxiv8 ######################
64#-Q- lxiv8 ## OBELIX      LSCE ##
65#-Q- lxiv8 ######################
66#-Q- lxiv8 #PBS -N ::Jobname::
67#-Q- lxiv8 #PBS -m a
68#-Q- lxiv8 #PBS -j oe
69#-Q- lxiv8 #PBS -q medium
70#-Q- lxiv8 #PBS -o Script_Output_::Jobname::.000001
71#-Q- lxiv8 #PBS -S /bin/ksh
72#-Q- lxiv8 #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
73#-Q- lxiv8 #PBS -l nodes=1:ppn=::JobNumProcTot::
74#-Q- default #!/bin/ksh
75#-Q- default ##################
76#-Q- default ## DEFAULT HOST ##
77#-Q- default ##################
78#-Q- default #For MPI use, uncomment next line :
79#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
80
81#**************************************************************
82# Author: Sebastien Denvil
83# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
84# $Revision::                                          $ Revision of last commit
85# $Author::                                            $ Author of last commit
86# $Date::                                              $ Date of last commit
87# IPSL (2006)
88#  This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
89#
90#**************************************************************
91
92#set -eu
93#set -vx
94
95date
96echo
97echo "#######################################"
98echo "#       ANOTHER GREAT SIMULATION      #"
99echo "#######################################"
100echo
101
102#D--------------------------------------------------------------------==
103#D--------------------------------------------------------------------==
104#D-
105#D-                      Job to launch IGCM models
106#D-
107#D--------------------------------------------------------------------==
108#D--------------------------------------------------------------------==
109#D-
110
111MODIPSL=::modipsl::
112libIGCM=${MODIPSL}/libIGCM
113
114#D- Task type (computing or post-processing)
115TaskType=computing
116
117#D--------------------------------------------------------------------==
118#D-                   -1. User modification place
119#D-                      - Job Verbosity
120#D-                      - PeriodNb
121#D-                      - Experience type : DEB(ug), DEV(elopment), RUN
122#D--------------------------------------------------------------------==
123
124#D- Increased verbosity (1, 2, 3)
125Verbosity=3
126#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
127JobType=RUN
128#D- Number of execution in one job
129PeriodNb=1
130#-Q- sx9mercure PeriodNb=60
131
132#D-
133# --------------------------------------------------------------------------------------------------#
134#D-                          ! Optionnal features /!\ Use with care !                               #
135# --------------------------------------------------------------------------------------------------#
136
137
138#D- Low level debug : to bypass lib test checks and stack construction
139#D- Default=true
140DEBUG_debug=false
141
142#D- Set DEBUG_sys to false to disable output calls of function
143#D- Default=true
144#DEBUG_sys=false
145
146#D- Define running directory
147#D- Default=${TMPDIR} ie temporary batch directory
148#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
149
150#D- Define submit directory
151#D- Default= where you launch qsub (variable from scheduler)
152#SUBMIT_DIR=$( pwd )
153
154#D- Define input file root directory
155#D- Default=/IGCMG/common/account/of/this/machine
156#R_IN=/u/rech/por/rpor111/DATA
157
158#D- Turn in dry run mode ? (0,1,2,3)
159#D- Default=0
160#DRYRUN=3
161# YOU HAVE TO COMPILE YOUR EXE FILES to USE DRYRUN MODE !
162# -------------------------------------------------------------------------------------
163# | DRYRUN=  |  Date computations, | sys_Get    |  Exe    | sys_Put_Out; sys_Put_Rest |
164# |          |  Cp/Exe/param/files |            |         |                           |
165# |          |  Chmod Qsub         |            |         |                           |
166# -------------------------------------------------------------------------------------
167# |    0     |       yes           |    yes     |  yes    |      yes                  |
168# -------------------------------------------------------------------------------------
169# |    1     |       yes           |    yes     |  yes    |      no                   |
170# -------------------------------------------------------------------------------------
171# |    2     |       yes           |    yes     |  no     |      no                   |
172# -------------------------------------------------------------------------------------
173# |    3     |       yes           |    no      |  no     |      no                   |
174# -------------------------------------------------------------------------------------
175
176#D-
177# --------------------------------------------------------------------------------------------------#
178#D- ! End of optionnal features /!\ Do not modify anything below unless you know what you are doing #
179# --------------------------------------------------------------------------------------------------#
180
181#D-
182#D--------------------------------------------------------------------==
183#D-                   0. System Environment
184#D-                      - Define variables for Number of processors
185#D-                      - Define MPI variables
186#D-                      - Define batch scheduler variables
187#D-                      - Source IGCM Library
188#D-                      - Get RESOLution in .resol file (temporary)
189#D--------------------------------------------------------------------==
190
191#D--------------------------------------------------------------------==
192
193. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
194       ( ${DEBUG_debug} ) && IGCM_debug_Check
195. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
196       ( ${DEBUG_debug} ) && IGCM_card_Check
197. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
198       ( ${DEBUG_debug} ) && IGCM_date_Check
199#-------
200. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
201. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
202. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
203. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
204
205#D--------------------------------------------------------------------==
206
207#-- Resolution FLAG for oasis input
208#RESOL="ORCA2xLMD9671"
209[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
210#-- Chemistrie FLAG to ease chemistries configurations management
211#CHEM="AER"
212[ -f ${SUBMIT_DIR}/../.chimie ] && eval $(grep CHEM ${SUBMIT_DIR}/../.chimie) 
213
214#D-
215#D--------------------------------------------------------------------==
216#D-           1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
217#D--------------------------------------------------------------------==
218
219# --------------------------------------------------------------------==
220#D- Initialize configuration :
221#D-              - Simulation configuration
222#D-              - Simulation parameters
223#D-              - Execution parameter
224#D-              - Define input files directory
225#D-              - Read or initialize CumulPeriod
226#D-              - run.card
227#D-
228# --------------------------------------------------------------------==
229IGCM_config_Initialize
230
231# --------------------------------------------------------------------==
232#D- Define, create and cd RUN_DIR
233#D- 
234# --------------------------------------------------------------------==
235RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
236IGCM_sys_MkdirWork ${RUN_DIR}
237IGCM_sys_Cd ${RUN_DIR}
238
239# ------------------------------------------------------------------
240# Test if all was right before proceeding further
241# ------------------------------------------------------------------
242IGCM_debug_Verif_Exit
243
244# --------------------------------------------------------------------==
245#D- Initialize each components taking part
246#D- into simulation configuration :
247#D-         - Define component executable
248#D-         - Define Write Frequency by component "1M" "1D" ...
249#D-         - Source ${comp}.driver
250#D-         - Define Storage Places by component
251#D-         - ${comp}_Initialize :
252#D-         - Patterns to sed
253#D-         - Variables from namelist
254#D-
255# --------------------------------------------------------------------==
256IGCM_comp_Initialize
257
258# --------------------------------------------------------------------==
259#D- Verify compatibility of period length, write frequencies, ...
260#D- Verify compatibility of rebuild choices and post-process choices
261#D- Will stop here if something is wrong
262# --------------------------------------------------------------------==
263IGCM_config_Check
264
265# ------------------------------------------------------------------
266#D- Test if all was right before entering the period loop
267# ------------------------------------------------------------------
268IGCM_debug_Verif_Exit
269
270#D--------------------------------------------------------------------==
271#D-                  2. ENTER THE DEEP EXECUTION LOOP
272#D-                  !!!! FASTEN YOUR SEAT BELTS !!!!
273#D-                  !!!!!      PLEASE.          !!!!
274#D-                  !! WE WILL CROSS TURBULENCES. !!
275#D--------------------------------------------------------------------==
276
277Period=1
278
279PeriodContinue=false
280if [ ${Period} -le ${PeriodNb} ]; then
281    PeriodContinue=true
282fi
283while ( ${PeriodContinue} ) ; do
284
285  echo
286  echo "Starting iteration ${Period} / ${PeriodNb}"
287
288  #D-
289  # ------------------------------------------------------------------
290  #D- COMPUTE AND DEFINE DATE STUFF.
291  # ------------------------------------------------------------------
292  IGCM_config_PeriodStart
293  #D- => ${PeriodDateBegin}
294  #D- => ${PeriodDateEnd}
295  #D- => ${CumulPeriod}
296  #D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
297  #D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
298  #D- => update run.card value
299
300  #D-
301  # ------------------------------------------------------------------
302  #D- BEGIN COMPUTATION ONLY IF NEEDED
303  # ------------------------------------------------------------------
304  if [ ${SimulationLengthInDays} -gt ${ExperienceLengthInDays} ] ; then
305    IGCM_debug_Print 1 "break because '${SimulationLengthInDays} -gt ${ExperienceLengthInDays}' : ${SimulationLengthInDays} -gt ${ExperienceLengthInDays}"
306    break ;
307  fi
308
309  #D-
310  # ------------------------------------------------------------------
311  #D- Optionnal function in driver
312  #D- to set special variables used in forward lists (Param, Init or Bc).
313  # ------------------------------------------------------------------
314  IGCM_comp_PeriodStart
315
316  #D-
317  # ------------------------------------------------------------------
318  #D- Get parameters text files updated by job (.def, namelist ...)
319  #D- READ AND USE BY GCM AT EACH EXECUTION.
320  # ------------------------------------------------------------------
321  IGCM_comp_GetInputParametersFiles
322
323  #D-
324  # ------------------------------------------------------------------
325  #D- Get initial state (Etat0, carteveg,relief...)
326  #D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
327  #D- AND ???_Restart=NO
328  #D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
329  # ------------------------------------------------------------------
330  IGCM_comp_GetInputInitialStateFiles
331
332  #D-
333  # ------------------------------------------------------------------
334  #D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
335  #D- READ AND USE BY GCM AT EACH EXECUTION.
336  # ------------------------------------------------------------------
337  IGCM_comp_GetInputBoundaryFiles
338
339  #D-
340  # ------------------------------------------------------------------
341  #D- Get SmoothFiles Conditions (SST, WIND[X,Y,Z], LAI ...)
342  #D- READ AND USE BY GCM AT EACH EXECUTION but varying in time
343  # ------------------------------------------------------------------
344  IGCM_comp_GetInputSmoothFiles
345
346  #D-
347  # ------------------------------------------------------------------
348  #D- Get restart files (restartphy.nc, orca_restart.nc ...)
349  #D- READ AND USE BY GCM AT EACH EXECUTION.
350  #D-
351  #D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
352  #D- ONLY IF CumulPeriod= (=> NUMERO) == 1
353  #D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
354  #D-
355  #D- IF CumulPeriod == 1
356  #D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
357  #D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
358  #D-
359  #D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
360  # ------------------------------------------------------------------
361  IGCM_comp_GetInputRestartFiles
362
363  #D-
364  # ------------------------------------------------------------------
365  #D- Test if all was right before Update
366  # ------------------------------------------------------------------
367  IGCM_debug_Verif_Exit
368
369  # ------------------------------------------------------------------
370  #D- Activate running environnement variables
371  # ------------------------------------------------------------------
372  IGCM_sys_activ_variables
373
374  #D-
375  # ------------------------------------------------------------------
376  #D- UPDATE ParametersFiles (.def, namelist ...) with
377  #D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
378  # ------------------------------------------------------------------
379  IGCM_comp_Update
380
381  #D-
382  # ------------------------------------------------------------------
383  #D- Test if all was right before execution
384  # ------------------------------------------------------------------
385  IGCM_debug_Verif_Exit
386
387  #D-
388  #D----------------------------------------------------------------==
389  #D-                          3. Execution
390  #D----------------------------------------------------------------==
391
392  echo
393  echo "#######################################"
394  echo "#      DIR BEFORE RUN EXECUTION       #"
395  echo "#######################################"
396  echo
397  ls -lrt
398
399  echo "========================================================================"
400  if [ ${DRYRUN} -le 1 ] ; then
401    REAL_DATE_INIT=$( date )
402    echo                                                                                   > ${Exe_Output}
403    echo "#######################################"                                        >> ${Exe_Output}
404    echo "EXECUTION of : ${EXECUTION}"
405    echo "EXECUTION of : ${EXECUTION}"                                                    >> ${Exe_Output}
406    echo                                                                                  >> ${Exe_Output}
407    typeset RET
408    RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
409    ${EXECUTION}  >> ${Exe_Output} 2>&1
410    RET=$?
411    RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
412    if [ ${RET} -gt 0 ] ; then
413      echo "Return code of executable :" ${RET}
414      IGCM_debug_Exit "EXECUTABLE"
415      IGCM_sys_Mkdir ${SUBMIT_DIR}/Debug
416      IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/Debug/${PREFIX}_${Exe_Output}_error
417      ExecutionFail=true
418    fi
419    echo                                            >> ${Exe_Output}
420    echo "#######################################"  >> ${Exe_Output}
421    echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}"   >> ${Exe_Output}
422    echo "libIGCM RunDateEnd=${RUN_DATE_END}"       >> ${Exe_Output}
423    echo                                            >> ${Exe_Output}
424  else
425    echo "EXECUTION of : ${EXECUTION} simulated for DRYRUN = " $DRYRUN
426    echo "EXECUTION of : ${EXECUTION} simulated for DRYRUN = " $DRYRUN > ${Exe_Output}
427    if ( $DEBUG_debug ) ; then
428      echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
429    fi
430  fi
431  echo "========================================================================"
432
433  echo
434  echo "#######################################"
435  echo "#       DIR AFTER RUN EXECUTION       #"
436  echo "#######################################"
437  echo
438  ls -lrt
439
440  #D-
441  # ------------------------------------------------------------------
442  #D- Desactivate running environnement variables
443  # ------------------------------------------------------------------
444  IGCM_sys_desactiv_variables
445
446  #D-
447  #D----------------------------------------------------------------==
448  #D-                    4. Post execution operations
449  #D----------------------------------------------------------------==
450
451  #D-
452  # ------------------------------------------------------------------
453  #D- Save on archive machine netcdf outputs, restarts
454  #D- and text files of models
455  # ------------------------------------------------------------------
456  IGCM_comp_Finalize
457
458  #D-
459  # ------------------------------------------------------------------
460  #D- Test if all was right after Finalize
461  # ------------------------------------------------------------------
462  IGCM_debug_Verif_Exit
463
464  #D-
465  # ------------------------------------------------------------------
466  #D- Configure asynchronous post-treatment (Level 0) when necessary (rebuild)
467  #D- Configure classic post-treatment (Level 1) when necessary (TimeSeries/Seasonal)
468  # ------------------------------------------------------------------
469  IGCM_post_Configure
470
471  #D-
472  # ------------------------------------------------------------------
473  #D- Launch remote rebuild or/and post-treatment process if necessary
474  # ------------------------------------------------------------------
475  IGCM_post_Submit
476
477  #D-
478  # ------------------------------------------------------------------
479  #D- Save job output
480  #D- Manage executable size to save only different binary
481  #D- Write in run.card user, system and elapse time
482  #D- Check that everything went well ! No ? then we stop.
483  #D- Determine next computed period
484  # ------------------------------------------------------------------
485  IGCM_config_PeriodEnd
486
487  echo "Ending iteration ${Period}"
488  (( Period = Period + 1 ))
489
490  # End loop if date end is reached
491  if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
492    break ;
493  fi
494
495  PeriodContinue=false
496  if [ ${Period} -le ${PeriodNb} ]; then
497    PeriodContinue=true
498  fi
499done
500
501#D-
502#D--------------------------------------------------------------------==
503#D-     5. Submit next job or send mail if simulation over.
504#D--------------------------------------------------------------------==
505IGCM_config_Finalize
506
507date
Note: See TracBrowser for help on using the repository browser.