Context Navigation

source: trunk/libIGCM/AA_job @ 157

Last change on this file since 157 was 157, checked in by sdipsl, 15 years ago
RebuildFromWorkdir? is back. Will use BIG_DIR as temp storage directory ideally RUN_DIR_PATH and BIG_DIR are on the same filesystem (mv REBUILD_DIR BIG_DIR) save lot of time between iteration for IO intensive simulation up to 3 minutes by iteration for CMIP5 like IO. rebuild.ksh will be usable as it is (in case you need to run it by hand) clean Up : if rebuild run smoothly we clean everything (either FromArchive? or FromWorkdir?) Watch your quota on the BIG_DIR (SCRATCHDIR OR WORKDIR depending on machine) before you play with that.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Date Author Revision`
File size: 17.4 KB

Line
1	#-Q- platine #!/usr/bin/ksh
2	#-Q- platine ##################
3	#-Q- platine ## PLATINE CEA ##
4	#-Q- platine ##################
5	#-Q- platine #BSUB -J ::Jobname:: # nom de la requete
6	#-Q- platine #BSUB -N # message a la fin du job
7	#-Q- platine #BSUB -o Script_Output_::Jobname::.000001 # nom du fichier de sortie
8	#-Q- platine #BSUB -n ::JobNumProcTot:: # reservation des processeurs pour le job
9	#-Q- platine #BSUB -W 1:00 # Limite temps
10	#-Q- platine BATCH_NUM_PROC_TOT=$SLURM_NPROCS
11	#-Q- sx8brodie #######################
12	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
13	#-Q- sx8brodie #######################
14	#-Q- sx8brodie #PBS -N ::Jobname:: # nom de la requete
15	#-Q- sx8brodie #PBS -m a # message a la fin du job
16	#-Q- sx8brodie #PBS -j o # regroupement des sorties du job : standard error et output
17	#-Q- sx8brodie #PBS -o Script_Output_::Jobname::.000001 # nom du fichier de sortie
18	#-Q- sx8brodie #PBS -S /usr/bin/ksh # shell de soumission
19	#-Q- sx8brodie #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
20	#-Q- sx8brodie #PBS -q ::JobClass:: # classe multi pour BATCH_NUM_PROC_TOT superieur a 2, mono sinon
21	#-Q- sx8brodie #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT} # reservation des processeurs pour le job
22	#-Q- sx8brodie #PBS -l memsz_job=8.0gb # Limite memoire
23	#-Q- sx8brodie #PBS -l cputim_job=1:00:00 # Limite temps
24	#-Q- aix6 #!/usr/bin/ksh
25	#-Q- aix6 # ######################
26	#-Q- aix6 # ## VARGAS IDRIS ##
27	#-Q- aix6 # ######################
28	#-Q- aix6 # Nom de la requete
29	#-Q- aix6 # @ job_name = ::Jobname::
30	#-Q- aix6 # Type de travail
31	#-Q- aix6 # @ job_type = parallel
32	#-Q- aix6 # Fichier de sortie standard
33	#-Q- aix6 # @ output = $(job_name).$(jobid)
34	#-Q- aix6 # Fichier de sortie erreur (le meme)
35	#-Q- aix6 # @ error = $(job_name).$(jobid)
36	#-Q- aix6 # Nombre de processus demandes
37	#-Q- aix6 # @ total_tasks = ::JobNumProcTot::
38	#-Q- aix6 # @ environment = "BATCH_NUM_PROC_TOT=::JobNumProcTot::"
39	#-Q- aix6 # Temps CPU max. par processus MPI hh:mm:ss
40	#-Q- aix6 # @ wall_clock_limit = 1:00:00
41	#-Q- aix6 # Memoire max. utilisee par processus
42	#-Q- aix6 # @ data_limit = 3.2gb
43	#-Q- aix6 # Memoire stack demandee
44	#-Q- aix6 # @ stack_limit = 0.3gb,0.3gb
45	#-Q- aix6 # Pas d OpenMP
46	#-Q- aix6 # @ resources = ConsumableCpus(1)
47	#-Q- aix6 # Fin de l entete
48	#-Q- aix6 # @ queue
49	#-Q- sx8mercure #!/bin/ksh
50	#-Q- sx8mercure ######################
51	#-Q- sx8mercure ## SX8MERCURE CEA ##
52	#-Q- sx8mercure ######################
53	#-Q- sx8mercure #PBS -N ::Jobname:: # nom de la requete
54	#-Q- sx8mercure #PBS -m a # message a la fin du job
55	#-Q- sx8mercure #PBS -j o # regroupement des sorties du job : standard error et output
56	#-Q- sx8mercure ####PBS -A gen6178
57	#-Q- sx8mercure #PBS -o Script_Output_::Jobname::.000001 # nom du fichier de sortie
58	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
59	#-Q- sx8mercure #PBS -v BATCH_NUM_PROC_TOT=::JobNumProcTot::
60	#-Q- sx8mercure #PBS -l cpunum_job=${BATCH_NUM_PROC_TOT} # reservation des processeurs pour le job
61	#-Q- sx8mercure #PBS -l memsz_job=8.0gb # limite memoire
62	#-Q- sx8mercure #PBS -l elapstim_req=01:00:00 # limite en temps elapsed
63	#-Q- default #!/bin/ksh
64	#-Q- default ##################
65	#-Q- default ## DEFAULT HOST ##
66	#-Q- default ##################
67	#-Q- default #For MPI use, uncomment next line :
68	#-Q- default #BATCH_NUM_PROC_TOT=::JobNumProcTot::
69
70	# $Date$
71	# $Author$
72	# $Revision$
73	# IPSL (2006)
74	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
75
76	#set -eu
77	#set -vx
78
79	date
80	echo
81	echo "#######################################"
82	echo "# ANOTHER GREAT SIMULATION #"
83	echo "#######################################"
84	echo
85
86	#D--------------------------------------------------------------------==
87	#D--------------------------------------------------------------------==
88	#D-
89	#D- Job to launch IGCM models
90	#D-
91	#D--------------------------------------------------------------------==
92	#D--------------------------------------------------------------------==
93	#D-
94
95	MODIPSL=::modipsl::
96	libIGCM=${MODIPSL}/libIGCM
97
98	#D--------------------------------------------------------------------==
99	#D- -1. User modification place
100	#D- - Job Verbosity
101	#D- - PeriodNb
102	#D- - Experience type : DEB(ug), DEV(elopment), RUN
103	#D--------------------------------------------------------------------==
104
105	#D- Increased verbosity (1, 2, 3)
106	Verbosity=3
107	#D- Number of execution in one job
108	PeriodNb=1
109
110	#D-
111	# --------------------------------------------------------------------
112	#D- ! Optionnal features /!\ Use with care !
113	# --------------------------------------------------------------------
114
115	#D- Low level debug : to bypass lib test checks and stack construction
116	#D- Default=true
117	DEBUG_debug=false
118
119	#D- Set DEBUG_sys to false to disable output calls of function
120	#D- Default=true
121	#DEBUG_sys=false
122
123	#D- Experience type : DEB(ug), DEV(elopment), RUN (default)
124	JobType=RUN
125
126	#D- Define running directory
127	#D- Default=${TMPDIR} ie temporary batch directory
128	#RUN_DIR_PATH=/workdir/or/scratchdir/of/this/machine
129
130	#D- Define submit directory
131	#D- Default= where you launch qsub (variable from scheduler)
132	#SUBMIT_DIR=$( pwd )
133
134	#D- Define BIG_DIR directory
135	#D- Default=(strongly machine dependant/see libIGCM_sys for details)
136	#BIG_DIR=/somewhere/with/lot/of/space/scratchdir/may/be
137
138	#D- Turn in dry run mode ? (0,1,2,3)
139	#D- Default=0
140	#DRYRUN=3
141	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
142	# -------------------------------------------------------------------------------------
143	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
144	# \| \| Cp/Exe param files \| \| Chmod \| \|
145	# \| \| Qsub \| \| \| \|
146	# -------------------------------------------------------------------------------------
147	# \| 0 \| yes \| yes \| yes \| yes \|
148	# -------------------------------------------------------------------------------------
149	# \| 1 \| yes \| yes \| yes \| no \|
150	# -------------------------------------------------------------------------------------
151	# \| 2 \| yes \| yes \| no \| no \|
152	# -------------------------------------------------------------------------------------
153	# \| 3 \| yes \| no \| no \| no \|
154	# -------------------------------------------------------------------------------------
155
156	#D- Define input file root directory
157	#D- Default=/IGCMG/common/account/of/this/machine
158	#R_IN=/u/rech/por/rpor111/DATA
159
160	#D-
161	#D--------------------------------------------------------------------==
162	#D- 0. System Environment
163	#D- - Define variables for Number of processors
164	#D- - Define MPI variables
165	#D- - Define batch scheduler variables
166	#D- - Source IGCM Library
167	#D- - Get RESOLution in .resol file (temporary)
168	#D--------------------------------------------------------------------==
169
170	#D--------------------------------------------------------------------==
171
172	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
173	( ${DEBUG_debug} ) && IGCM_debug_Check
174	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
175	( ${DEBUG_debug} ) && IGCM_card_Check
176	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
177	( ${DEBUG_debug} ) && IGCM_date_Check
178	#-------
179	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
180	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
181	. ${libIGCM}/libIGCM_comp/libIGCM_comp.ksh
182	. ${libIGCM}/libIGCM_post/libIGCM_post.ksh
183
184	#D--------------------------------------------------------------------==
185
186	#-- Resolution FLAG for oasis input
187	[ -f ${SUBMIT_DIR}/../.resol ] && RESOL=$(head -1 ${SUBMIT_DIR}/../.resol)
188	#RESOL="ORCA2xLMD9671"
189
190	#D-
191	#D--------------------------------------------------------------------==
192	#D- 1. INITIALIZE CONFIGURATION AND THEN COMPONENTS
193	#D--------------------------------------------------------------------==
194
195	# --------------------------------------------------------------------==
196	#D- Initialize configuration :
197	#D- - Simulation configuration
198	#D- - Simulation parameters
199	#D- - Execution parameter
200	#D- - Define input files directory
201	#D- - Read or initialize CumulPeriod
202	#D- - run.card
203	#D-
204	# --------------------------------------------------------------------==
205	IGCM_config_Initialize
206
207	# --------------------------------------------------------------------==
208	#D- Define, create and cd RUN_DIR
209	#D-
210	# --------------------------------------------------------------------==
211	RUN_DIR=${RUN_DIR_PATH}/${config_UserChoices_TagName}/${config_UserChoices_JobName}.${$}
212	IGCM_sys_MkdirWork ${RUN_DIR}
213	IGCM_sys_Cd ${RUN_DIR}
214
215	# --------------------------------------------------------------------==
216	#D- Initialize each components taking part
217	#D- into simulation configuration :
218	#D- - Define component executable
219	#D- - Define Write Frequency by component "1M" "1D" ...
220	#D- - Source ${comp}.driver
221	#D- - Define Storage Places by component
222	#D- - ${comp}_Initialize :
223	#D- - Patterns to sed
224	#D- - Variables from namelist
225	#D-
226	# --------------------------------------------------------------------==
227	IGCM_comp_Initialize
228
229	# --------------------------------------------------------------------==
230	#D- Verify compatibility of period length, write frequencies, ...
231	#D- Verify compatibility of rebuild choices and post-process choices
232	#D- Will stop here if something is wrong
233	# --------------------------------------------------------------------==
234	#IGCM_config_Check
235
236	#D--------------------------------------------------------------------==
237	#D- 2. ENTER THE DEEP EXECUTION LOOP
238	#D- !!!! FASTEN YOUR SEAT BELTS !!!!
239	#D- !!!!! PLEASE. !!!!
240	#D- !! WE WILL CROSS TURBULENCES. !!
241	#D--------------------------------------------------------------------==
242
243	Period=1
244
245	PeriodContinue=false
246	if [ ${Period} -le ${PeriodNb} ]; then
247	PeriodContinue=true
248	fi
249	while ( ${PeriodContinue} ) ; do
250
251	echo
252	echo "Starting iteration ${Period} / ${PeriodNb}"
253
254	#D-
255	# ------------------------------------------------------------------
256	#D- COMPUTE AND DEFINE DATE STUFF.
257	# ------------------------------------------------------------------
258	IGCM_config_PeriodStart
259	#D- => ${PeriodDateBegin}
260	#D- => ${PeriodDateEnd}
261	#D- => ${CumulPeriod}
262	#D- => ${DatesPeriod}=${PeriodDateBegin}_${PeriodDateEnd}
263	#D- => ${PeriodLength} => JOURS=31 , 28, 29, 30 ; MOIS=0 ; ANS=0
264	#D- => update run.card value
265
266	#D-
267	# ------------------------------------------------------------------
268	#D- Get parameters text files updated by job (.def, namelist ...)
269	#D- READ AND USE BY GCM AT EACH EXECUTION.
270	# ------------------------------------------------------------------
271	IGCM_comp_GetInputParametersFiles
272
273	#D-
274	# ------------------------------------------------------------------
275	#D- Get initial state (Etat0, carteveg,relief...)
276	#D- NECESSARY ONLY IF CumulPeriod= (=> NUMERO) == 1
277	#D- AND ???_Restart=NO
278	#D- READ AND USE BY GCM FOR ONLY ONE EXECUTION.
279	# ------------------------------------------------------------------
280	IGCM_comp_GetInputInitialStateFiles
281
282	#D-
283	# ------------------------------------------------------------------
284	#D- Get Boundaries Conditions (SST, WIND[X,Y,Z], LAI ...)
285	#D- READ AND USE BY GCM AT EACH EXECUTION.
286	# ------------------------------------------------------------------
287	IGCM_comp_GetInputBoundaryFiles
288
289	#D-
290	# ------------------------------------------------------------------
291	#D- Get restart files (restartphy.nc, orca_restart.nc ...)
292	#D- READ AND USE BY GCM AT EACH EXECUTION.
293	#D-
294	#D- IF A COMPONENT DO NOT RESTART FROM PREVIOULSY COMPUTED RESTART
295	#D- ONLY IF CumulPeriod= (=> NUMERO) == 1
296	#D- MUST EXECUTE CREATE ETAT0_LIMIT (TYPICALLY LMDZ AND ./create_etat0_limit.e)
297	#D-
298	#D- IF CumulPeriod == 1
299	#D- AND A COMPONENT RESTART FROM PREVIOULSY COMPUTED RESTART
300	#D- MUST BRING THIS RESTART IN ${RUN_DIR_PATH}
301	#D-
302	#D- ELSE BRING RESTART FROM ${JobName} IN ${RUN_DIR_PATH}
303	# ------------------------------------------------------------------
304	IGCM_comp_GetInputRestartFiles
305
306	#D-
307	# ------------------------------------------------------------------
308	#D- Test if all was right before Update
309	# ------------------------------------------------------------------
310	IGCM_debug_Verif_Exit
311
312	# ------------------------------------------------------------------
313	#D- Activate running environnement variables
314	# ------------------------------------------------------------------
315	IGCM_sys_activ_variables
316
317	#D-
318	# ------------------------------------------------------------------
319	#D- UPDATE ParametersFiles (.def, namelist ...) with
320	#D- current value of (ORCA_NIT00, ORCA_LRSTAR, RAZ_DATE, ...)
321	# ------------------------------------------------------------------
322	IGCM_comp_Update
323
324	#D-
325	# ------------------------------------------------------------------
326	#D- Test if all was right before execution
327	# ------------------------------------------------------------------
328	IGCM_debug_Verif_Exit
329
330	#D-
331	#D----------------------------------------------------------------==
332	#D- 3. Execution
333	#D----------------------------------------------------------------==
334
335	echo
336	echo "#######################################"
337	echo "# DIR BEFORE RUN EXECUTION #"
338	echo "#######################################"
339	echo
340	ls -lrt
341
342	echo "========================================================================"
343	if ( [ X${BATCH_NUM_PROC_TOT} != X ] && [ "${BATCH_NUM_PROC_TOT}" -gt 1 ] ) ; then
344	MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND}
345	eval MPIRUN_OPTIONS=${config_UserChoices_JobRunOptions}
346	else
347	MPIRUN_COMMAND=${MPIRUN_COMMAND:="time "}
348	MPIRUN_OPTIONS=${MPIRUN_OPTIONS:=""}
349	fi
350
351	if [ ${DRYRUN} -le 1 ] ; then
352	REAL_DATE_INIT=$( date )
353	echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1"
354	typeset RET
355	RUN_DATE_BEGIN=$( date '+%Y-%m-%dT%H:%M:%S' )
356	${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} > ${Exe_Output} 2>&1
357	RET=$?
358	RUN_DATE_END=$( date '+%Y-%m-%dT%H:%M:%S' )
359	if [ ${RET} -gt 0 ] ; then
360	echo "Return code of executable :" ${RET}
361	IGCM_debug_Exit "EXECUTABLE"
362	IGCM_sys_Cp ${Exe_Output} ${SUBMIT_DIR}/${PREFIX}_${Exe_Output}_error
363	fi
364	echo "libIGCM RunDateBegin=${RUN_DATE_BEGIN}" >> ${Exe_Output}
365	echo "libIGCM RunDateEnd=${RUN_DATE_END}" >> ${Exe_Output}
366	else
367	echo "EXECUTION of : ${MPIRUN_COMMAND} ${MPIRUN_OPTIONS} ./${config_Executable_Name} simulated for DRYRUN = " $DRYRUN
368	if ( $DEBUG_debug ) ; then
369	echo "FOR EXECUTION DRYRUN mode = " $DRYRUN >> stack
370	fi
371	fi
372	echo "========================================================================"
373
374	echo
375	echo "#######################################"
376	echo "# DIR AFTER RUN EXECUTION #"
377	echo "#######################################"
378	echo
379	ls -lrt
380
381	#D-
382	# ------------------------------------------------------------------
383	#D- Desactivate running environnement variables
384	# ------------------------------------------------------------------
385	IGCM_sys_desactiv_variables
386
387	#D-
388	#D----------------------------------------------------------------==
389	#D- 4. Post execution operations
390	#D----------------------------------------------------------------==
391
392	#D-
393	# ------------------------------------------------------------------
394	#D- Save on archive machine netcdf outputs, restarts
395	#D- and text files of models
396	# ------------------------------------------------------------------
397	IGCM_comp_Finalize
398
399	#D-
400	# ------------------------------------------------------------------
401	#D- Test if all was right after Finalize
402	# ------------------------------------------------------------------
403	IGCM_debug_Verif_Exit
404
405	#D-
406	# ------------------------------------------------------------------
407	#D- Launch remote rebuild or/and post-treatment process if necessary
408	# ------------------------------------------------------------------
409	IGCM_post_Configure
410
411	#D-
412	# ------------------------------------------------------------------
413	#D- Save job output
414	#D- Manage executable size to save only different binary
415	#D- Write in run.card user, system and elapse time
416	#D- Check that everything went well ! No ? then we stop.
417	#D- Determine next computed period
418	# ------------------------------------------------------------------
419	IGCM_config_PeriodEnd
420
421	echo "Ending iteration ${Period}"
422	(( Period = Period + 1 ))
423
424	# End loop if date end is reached
425	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
426	break ;
427	fi
428
429	PeriodContinue=false
430	if [ ${Period} -le ${PeriodNb} ]; then
431	PeriodContinue=true
432	fi
433	done
434
435	#D-
436	#D--------------------------------------------------------------------==
437	#D- 5. Submit next job or send mail if simulation over.
438	#D--------------------------------------------------------------------==
439	IGCM_config_Finalize
440
441	date

Note: See TracBrowser for help on using the repository browser.

Download in other formats: