Context Navigation

libIGCM_config.ksh @ 1335

Last change on this file since 1335 was 1335, checked in by sdipsl, 8 years ago

In config.card/[Post], add a keyword to launch (or not) metrics jobs. FALSE if the keyword is not there.

[Post]
...
MetricsPCMDI=TRUE|FALSE

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 52.2 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configCardPath=$1
25
26	#==================================
27	typeset option auxprint
28
29	#==================================
30	# Read UserChoices section:
31	[ ${Verbosity} -gt 0 ] && echo
32	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
33
34	IGCM_card_DefineArrayFromSection ${configCardPath} UserChoices
35	for option in ${config_UserChoices[*]} ; do
36	IGCM_card_DefineVariableFromOption ${configCardPath} UserChoices ${option}
37	eval auxprint=\${config_UserChoices_${option}}
38	IGCM_debug_Print 3 "${option} : ${auxprint}"
39	done
40
41	#==================================
42	# Read Ensemble section:
43	[ ${Verbosity} -gt 0 ] && echo
44	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
45
46	IGCM_card_DefineArrayFromSection ${configCardPath} Ensemble
47	for option in ${config_Ensemble[*]} ; do
48	IGCM_card_DefineVariableFromOption ${configCardPath} Ensemble ${option}
49	eval auxprint=\${config_Ensemble_${option}}
50	IGCM_debug_Print 3 "${option} : ${auxprint}"
51	done
52
53	#==================================
54	# Read Post section:
55	[ ${Verbosity} -gt 0 ] && echo
56	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
57
58	IGCM_card_DefineArrayFromSection ${configCardPath} Post
59	for option in ${config_Post[*]} ; do
60	IGCM_card_DefineVariableFromOption ${configCardPath} Post ${option}
61	eval auxprint=\${config_Post_${option}}
62	IGCM_debug_Print 3 "${option} : ${auxprint}"
63	done
64	[ ${Verbosity} -gt 0 ] && echo
65
66	#==================================
67	# Define default value to keep compatibility with previous card: means before changes due to TGCC
68	# Apply some overrules to ensure proper usage of computing centres resources
69	#
70	if [ X${PackDefault} = Xtrue ] ; then
71	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
72	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
73	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
74	config_Post_PackFrequency=NONE
75	else
76	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
77	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
78	fi
79	else
80	# If we DO NOT apply pack in this computing center
81	config_Post_PackFrequency=NONE
82	fi
83
84	#====================================================
85	# Define ARCHIVE : Dedicated to large files
86	# Define STORAGE : Dedicated to small/medium files
87	# Define R_OUT : Output tree located on ARCHIVE
88	# Define R_BUF : Output tree located on STORAGE (files waiting treatment, or file lcoation when SpaceName=!PROD)
89	# Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
90	# Define R_TMP : A temporary space used by IGCM_debug_send_AMQP_msg__MAILTUNNEL. Must be persistent in between jobs
91	IGCM_sys_defineArchives
92
93	#====================================================
94	# R_SAVE : Job output directory
95	# R_BUFR : Job output buffered directory
96
97	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
98	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
99	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
100	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
101	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
102	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
103	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
104	else
105	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
106	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
107	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
108	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
109	fi
110	else
111	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
112	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
113	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
114	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
115	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
116	else
117	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
118	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
119	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
120	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
121	fi
122	fi
123
124	#====================================================
125	# Define R_OUT_KSH : Storage place for job output
126	# Define R_OUT_EXE : Storage place for binary used during simulation
127	R_OUT_KSH=${R_SAVE}/Out
128	R_OUT_EXE=${R_SAVE}/Exe
129
130	#====================================================
131	# Define R_BUF_KSH : Buffer place for job output
132	# Define R_BUF_EXE : Buffer place for binary used during simulation
133	R_BUF_KSH=${R_BUFR}/Out
134	R_BUF_EXE=${R_BUFR}/Exe
135
136	#====================================================
137	# Define REBUILD_DIR : where we store files needing rebuild process
138	REBUILD_DIR=${R_BUFR}/REBUILD
139	if [ ! X${TaskType} = Xchecking ] ; then
140	IGCM_sys_MkdirWork ${REBUILD_DIR}
141	fi
142
143	#====================================================
144	# DodsCopy : apply default value if not defined
145	if ( [ X${config_Post_DodsCopy} = X${NULL_STR} ] \|\| [ X${config_Post_DodsCopy} = X ] ) ; then
146	config_Post_DodsCopy=TRUE
147	fi
148
149	#====================================================
150	# MetricsPCMDI : apply default value if not defined
151	if ( [ X${config_Post_MetricsPCMDI} = X${NULL_STR} ] \|\| [ X${config_Post_MetricsPCMDI} = X ] ) ; then
152	config_Post_MetricsPCMDI=FALSE
153	fi
154
155	#====================================================
156	# IgnoreNonMonotonic : apply default value if not defined
157	if ( [ X${config_Post_IgnoreNonMonotonic} = X${NULL_STR} ] \|\| [ X${config_Post_IgnoreNonMonotonic} = X ] ) ; then
158	config_Post_IgnoreNonMonotonic=FALSE
159	fi
160
161	#====================================================
162	# Define StackFileLocation : directory where we store stack files
163	# Define StackFileName : stack file containing call tree and instrumentation
164	# Stack file containing call tree will be stored there.
165	if ( $DEBUG_debug ) ; then
166	StackFileLocation=${StackFileLocation:=${R_BUF_KSH}}
167	[ ! -d ${StackFileLocation} ] && mkdir -p ${StackFileLocation}
168	if [ X${TaskType} = Xcomputing ]; then
169	StackFileName=computing.stack.$$
170	elif [ X${TaskType} = Xpost-processing ]; then
171	StackFileName=${Script_Post_Output}.stack.$$
172	elif [ X${TaskType} = Xchecking ]; then
173	StackFileName=checking.stack.$$
174	else
175	IGCM_debug_Exit "IGCM_config_CommonConfiguration unknown TaskType : ${TaskType}"
176	IGCM_debug_Verif_Exit
177	fi
178
179	# This boolean will trigger the filling of the stack
180	# Only now we know where things should be ...
181	# We don't fill the stack when we perform checking task
182	if [ ! X${TaskType} = Xchecking ] ; then
183	ActivateStackFilling=true
184	fi
185	fi
186
187	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
188	}
189
190	#===================================
191	function IGCM_config_Initialize
192	{
193	IGCM_debug_PushStack "IGCM_config_Initialize"
194
195	# Debug Print :
196	echo
197	IGCM_debug_Print 1 "IGCM_config_Initialize"
198
199	# Test modipsl tree existence.
200	IGCM_sys_TestDir ${MODIPSL}
201	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
202	IGCM_sys_TestDir ${libIGCM}
203	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
204	IGCM_sys_TestDir ${R_EXE}
205	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
206	IGCM_sys_TestDir ${SUBMIT_DIR}
207	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
208
209	if ( $DEBUG_debug ) ; then
210	echo "Keep trace of inital SUBMIT_DIR : "
211	ls -lta ${SUBMIT_DIR}
212	fi
213
214	#==================================
215	# Read ListOfComponents section:
216	echo
217	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
218
219	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
220	for comp in ${config_ListOfComponents[*]} ; do
221	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
222	done
223	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
224
225	#==================================
226	# Read Executable section:
227	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
228
229	#==================================
230	# Read Restarts section:
231	# Restarts : Gerneral rule or local for each component.
232	echo
233	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
234
235	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
236	for option in ${config_Restarts[*]} ; do
237	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
238	eval auxprint=\${config_Restarts_${option}}
239	IGCM_debug_Print 3 "${option} : ${auxprint}"
240	done
241
242	#==================================
243	# Define Job Outputs Name
244	echo
245	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
246	Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
247	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
248	Exe_Output=out_execution
249	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
250
251	#===================================================================#
252	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
253	# But available to any son functions #
254	#===================================================================#
255
256	# Convert yyyy-mm-dd date to gregorian yyyymmdd
257	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
258	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
259
260	# Period Length In Days between DateBegin and DateEnd
261	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
262	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
263	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
264	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
265	IGCM_debug_Verif_Exit
266	fi
267
268	# Day and Year of Initial State (Given in julian format)
269	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
270	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
271
272	#================================================================#
273	# Test and Prepare directories #
274	#================================================================#
275
276	# ==> 4 kinds of input files :
277	# 1) R_INIT : Initial State Files (Etat0, carteveg)
278	# 2) R_BC : Boundary Conditions (Forcages, lai)
279	# 3) Parameters files (allready define through ${SUBMIT_DIR})
280	# 4) Restarts files (allready define in IGCM_config_Initialize)
281
282	# Here we offer the possibility to redefine R_INIT, R_BC
283	# and PeriodNb through config.card
284	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
285	echo
286	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
287	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
288	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
289	IGCM_debug_Print 3 "R_BC=${R_BC}"
290	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
291	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
292
293	# SD ADA SPECIFIC #
294	# TO FIX #
295	#IGCM_sys_TestDirArchive ${R_IN}
296	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
297
298	if ( ${FirstInitialize} ) ; then
299	IGCM_sys_MkdirArchive ${R_SAVE}
300	[ ! ${config_Post_PackFrequency} = NONE ] && IGCM_sys_Mkdir ${R_BUFR}
301	else
302	IGCM_sys_TestDirArchive ${R_SAVE}
303	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
304
305	if [ ! ${config_Post_PackFrequency} = NONE ] ; then
306	IGCM_sys_TestDir ${R_BUFR}
307	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
308	fi
309
310	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
311	IGCM_config_StateCheck
312
313	# And EXIT if not OK
314	IGCM_debug_Verif_Exit
315	fi
316
317	#====================================================
318	# Experience type : DEB(ug), DEV(elopment), RUN
319	if [ X${JobType} != XRUN ] ; then
320	echo
321	echo "===================================================="
322	echo "libIGCM JOB is NOT in RUN type mode."
323	echo "!! OUTPUT files will NOT be PROTECTED !!"
324	echo "Be carefull : you can ERASE the result of this job !"
325
326	case ${JobType} in
327	DEB)
328	echo "DEBUG mode : activation of 'set -vx' mode."
329	echo "DEBUG mode : no protection for output files."
330	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
331	;;
332	DEV)
333	echo "DEVelopment mode : no protection for output files."
334	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
335	;;
336	esac
337
338	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
339	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
340	echo "------------"
341	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
342	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
343	echo "------------"
344	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
345	fi
346	fi
347	echo "===================================================="
348	echo
349	fi
350
351	IGCM_debug_PopStack "IGCM_config_Initialize"
352	}
353
354	#===================================
355	function IGCM_config_DaysInPeriodLength
356	{
357	IGCM_debug_PushStack "IGCM_config_DaysInPeriodLength"
358
359	typeset i
360
361	# Determine number of day(s) in PeriodLength :
362	case ${config_UserChoices_PeriodLength} in
363	Y\|y)
364	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
365	echo
366	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
367	PeriodLengthInDays=0
368	i=0
369	until [ $i -ge $PeriodLengthInYears ] ; do
370	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
371	(( i=i+1 ))
372	done
373	;;
374	M\|m)
375	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
376	echo
377	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
378	PeriodLengthInDays=0
379	i=0
380	until [ $i -ge $PeriodLengthInMonths ] ; do
381	if [ $(( 10#${month} + ${i} )) -lt 13 ] ; then
382	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} )) ) ))
383	else
384	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( 10#${month} + ${i} - 12 )) ) ))
385	fi
386	(( i=i+1 ))
387	done
388	;;
389	D\|d)
390	PeriodLengthInMonths=0
391	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
392	echo
393	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
394	*)
395	IGCM_debug_Exit "IGCM_config_DaysInPeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
396	IGCM_debug_Verif_Exit ;;
397	esac
398
399	IGCM_debug_PopStack "IGCM_config_DaysInPeriodLength"
400	}
401
402	#===================================
403	function IGCM_config_DateCoherency
404	{
405	IGCM_debug_PushStack "IGCM_config_DateCoherency"
406
407	echo
408	IGCM_debug_Print 1 "IGCM_config_DateCoherency"
409	echo
410
411	typeset Length VerifiedPeriodDateBegin VerifiedPeriodDateEnd
412
413	# check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
414	# DateBegin + CumulPeriod*PeriodLength = PeriodDateBegin
415	echo
416
417	case ${config_UserChoices_PeriodLength} in
418	Y\|y)
419	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInYears} ))Y )
420	;;
421	M\|m)
422	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInMonths} ))M )
423	;;
424	D\|d)
425	Length=$( IGCM_date_DaysInCurrentPeriod ${DateBegin} $(( ${CumulPeriod} * ${PeriodLengthInDays} ))D )
426	;;
427	esac
428	VerifiedPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} ${Length}-1 )
429
430	if [ ${VerifiedPeriodDateEnd} != ${PeriodDateEnd} ] ; then
431	IGCM_debug_Print 1 "From run.card PeriodDateEnd is not consistent with DateBegin and CumulPeriod."
432	IGCM_debug_Print 1 "We have DateBegin = ${DateBegin}"
433	IGCM_debug_Print 1 "We have CumulPeriod = ${CumulPeriod}"
434	IGCM_debug_Print 1 "We have PeriodDateEnd = ${PeriodDateEnd}"
435	IGCM_debug_Print 1 "We have VerifiedPeriodDateEnd = ${VerifiedPeriodDateEnd}"
436	IGCM_debug_Print 1 "You must have change run.card in an inconsistent way."
437
438	IGCM_debug_Exit "STOP here to avoid further issues."
439	fi
440
441	# PeriodDateBegin + PeriodLength = PeriodDateEnd
442	VerifiedPeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${VerifiedPeriodDateEnd} $(( ${PeriodLengthInDays} * -1 )) )
443
444	IGCM_debug_PopStack "IGCM_config_DateCoherency"
445	}
446
447	#===================================
448	function IGCM_config_StateCheck
449	{
450	IGCM_debug_PushStack "IGCM_config_StateCheck"
451
452	#Test state of run in run.card
453	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
454
455	if [ ${run_Configuration_PeriodState} = "Fatal" ] ; then
456	echo
457	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
458	IGCM_debug_Print 1 "Check the overall status of your simulation by visiting this page:"
459	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
460	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_month.job to rerun one period"
461	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_year.job to rerun more."
462	IGCM_debug_Exit
463	elif [ $( echo ${run_Configuration_PeriodState} \| grep Fatal \| wc -l ) -eq 1 ] ; then
464	echo
465	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
466	IGCM_debug_Print 1 "Compute jobs has been stop because at least the above mentionned post-processing jobs fails."
467	IGCM_debug_Print 1 "Check post-processing jobs carefully by visiting this page:"
468	IGCM_debug_Print 1 "http://prodiguer-test-web.ipsl.fr/static/simulation.detail.html?uid=${simuid}"
469	IGCM_debug_Print 1 "Please visit that page to see how to fix issues:"
470	IGCM_debug_Print 1 "https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocGmonitor"
471	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_month.job to rerun one period"
472	IGCM_debug_Print 1 "Then try running ${libIGCM}/clean_year.job to rerun more."
473	IGCM_debug_Exit
474	fi
475
476	IGCM_debug_PopStack "IGCM_config_StateCheck"
477	}
478
479	#===================================
480	function IGCM_config_Check
481	{
482	IGCM_debug_PushStack "IGCM_config_Check"
483
484	# If one of the following modulo is not zero :
485	# we will issue an error then explain and exit in
486	# AA_job IGCM_debug_Verif_Exit call before binary submission
487
488	echo
489	IGCM_debug_Print 1 "IGCM_config_Check"
490	echo
491
492	typeset i
493
494	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
495	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
496	AsynchronousRebuild=true
497	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
498	echo
499	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
500	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
501	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
502	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
503	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
504	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
505	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
506	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
507	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
508	else
509	AsynchronousRebuild=false
510	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
511	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
512	echo
513	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
514	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
515	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
516	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
517	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
518	fi
519
520	# Check PackFrequency against other key frequencies
521	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
522	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
523	Pack=true
524	#
525	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
526	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
527	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
528	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
529	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
530	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
531	else
532	Pack=false
533	fi
534
535	# modulo (TimeSeriesFrequency and all Chunck2D) must be zero
536	NbJob=${#CHUNCK2D_SIZE[@]}
537	i=0
538	until [ $i -ge $NbJob ]; do
539	value=${CHUNCK2D_SIZE[${i}]}
540	IGCM_debug_Print 1 "Check coherence between ${CHUNCK2D_NAME[${i}]} Chunck2D frequency and TimeSeriesFrequency"
541	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
542	case ${value} in
543	Y\|y) ;;
544	*)
545	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
546	IGCM_debug_Exit "This will stop the job" ;;
547	esac
548	(( i=i+1 ))
549	done
550
551	# modulo (TimeSeriesFrequency and all Chunck3D) must be zero
552	NbJob=${#CHUNCK3D_SIZE[@]}
553	i=0
554	until [ $i -ge $NbJob ]; do
555	value=${CHUNCK3D_SIZE[${i}]}
556	IGCM_debug_Print 1 "Check coherence between ${CHUNCK3D_NAME[${i}]} Chunck3D frequency and TimeSeriesFrequency"
557	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
558	case ${value} in
559	Y\|y) ;;
560	*)
561	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
562	IGCM_debug_Exit "This will stop the job" ;;
563	esac
564	(( i=i+1 ))
565	done
566
567	# check to be sure there is enough space on temporary filesystems to run
568	echo
569	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
570	IGCM_sys_check_quota
571
572	# check to be sure that RUN_DIR_PATH, that will be removed is not pointing to an important directory
573	echo
574	IGCM_debug_Print 1 "Check where RUN_DIR_PATH variable is pointing to"
575	IGCM_sys_check_path
576
577
578	IGCM_debug_PopStack "IGCM_config_Check"
579	}
580
581	#===================================
582	function IGCM_config_ConfigureExecution
583	{
584	IGCM_debug_PushStack " IGCM_config_ConfigureExecution"
585
586	#echo
587	IGCM_debug_Print 1 " IGCM_config_ConfigureExecution"
588	#echo
589
590	typeset ExeNameIn ExeNameFirst CompNameFirst configCardPath comp i
591	typeset tempvar tempvarMPI tempvarNOD NbElts NbExec
592
593	# config.card path
594	configCardPath=$1
595
596	coreNumber=0
597	mpiTasks=0
598	openMPthreads=0
599	NbExec=0
600
601	OK_PARA_MPI=false
602	OK_PARA_OMP=false
603	OK_PARA_NOD=false
604	OK_PARA_MPMD=false
605
606	for comp in ${config_ListOfComponents[*]} ; do
607
608	# Manage component executable
609	IGCM_card_DefineArrayFromOption ${configCardPath} Executable ${comp}
610
611	eval ExeNameIn=\${config_Executable_${comp}[0]}
612
613	# NO order in config.card for parallelized values !
614	# just use suffix : MPI , OMP and NOD (for number of NODes.)
615
616	# NOD is the number of NODes allocated
617	eval ${comp}_PROC_NOD=0
618
619	# MPI is the number of MPI processus per nodes
620	eval ${comp}_PROC_MPI=0
621
622	# OMP is the number of OpenMP threads per MPI processus
623	eval ${comp}_PROC_OMP=0
624
625	# Only if we really have an executable for the component :
626	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
627
628	IGCM_debug_Print 1 ${comp}
629
630	# Keep the first executable found and the first CompName
631	ExeNameFirst=${ExeNameIn}
632	CompNameFirst=${comp}
633
634	# Are we a second executable?
635	(( NbExec = NbExec + 1 ))
636
637	# set 1 MPI task, 1 OpenMP thread and 1 node as default
638	eval ${comp}_PROC_MPI=1
639	eval ${comp}_PROC_OMP=1
640	eval ${comp}_PROC_NOD=1
641
642	eval NbElts=\${#config_Executable_${comp}[@]}
643
644	if [ ${NbElts} -gt 2 ] ; then
645	#
646	# CURRENT METHOD TO SPECIFY MPI AND OMP RESSOURCES
647	#
648	i=2
649	while [ ${i} -lt ${NbElts} ] ; do
650	eval tempvar=\${config_Executable_${comp}[${i}]}
651	IGCM_debug_Print 2 ${tempvar}
652
653	if [ X${tempvar} = X ] ; then
654	IGCM_debug_Print 2 "Error reading MPI/OMP parameters !!!"
655	IGCM_debug_Exit "Check your config.card. Exit now"
656	IGCM_debug_Verif_Exit
657	fi
658
659	case ${tempvar} in
660	[mM][pP][iI])
661	# Read MPI parameter for composante
662	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
663	OK_PARA_MPI=true;;
664	[oO][mM][pP])
665	# Read OMP parameter for composante
666	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
667	;;
668	[nN][oO][dD])
669	# Read NOD (NumBer of Nodes) parameter for composante
670	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
671	OK_PARA_NOD=true
672	OK_PARA_MPI=true
673	;;
674	esac
675	(( i = i + 1 ))
676	done
677	else
678	#
679	# BACKWARD COMPATIBILITY NOT SUPPORTED ANYMORE
680	#
681	IGCM_debug_Exit "You are using a deprecated ressources specification mechanism in your config.card"
682	IGCM_debug_Exit "Please check : https://forge.ipsl.jussieu.fr/igcmg_doc/wiki/DocEsetup#ThesectionExecutable"
683	IGCM_debug_Exit "Please modify ${configCardPath}"
684	exit
685	fi
686	eval tempvarMPI=\${${comp}_PROC_MPI}
687	eval tempvarNOD=\${${comp}_PROC_NOD}
688	eval tempvarOMP=\${${comp}_PROC_OMP}
689
690	# set OMP mode if more than 1 OMP thread.
691	[ ${tempvarOMP} -ge 2 ] && OK_PARA_OMP=true
692
693	# Number of OMP threads
694	[ ${openMPthreads} -lt ${tempvarOMP} ] && openMPthreads=${tempvarOMP}
695
696	# SUM UP NUMBER OF CORES
697	(( coreNumber = coreNumber + tempvarMPI * tempvarNOD * tempvarOMP ))
698
699	# SUM UP NUMBER OF MPI TASKS
700	(( mpiTasks = mpiTasks + tempvarMPI * tempvarNOD ))
701	fi
702	done
703
704	# MANDATORY FOR THE OPA9.DRIVER. USED TO EDIT OPA NAMELIST
705	# WE SHOULD PLANIFY NUM_PROC_??? DEPRECATION
706	NUM_PROC_CPL=${CPL_PROC_MPI}
707	NUM_PROC_OCE=${OCE_PROC_MPI}
708	NUM_PROC_ATM=${ATM_PROC_MPI}
709
710	# set MPMD mode if more than 2 executable names.
711	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
712
713	# Define the execution type we are running in
714	if ( ${OK_PARA_MPMD} ) ; then
715	if ( ${OK_PARA_MPI} ) ; then
716	# MPMD always implies MPI
717	executionType=1
718	fi
719	if ( ${OK_PARA_OMP} ) ; then
720	# MPMD + MPI/OMP
721	executionType=2
722	fi
723	else
724	if ( ( ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
725	# SPMD + MPI/OMP
726	executionType=3
727	elif ( ( ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
728	# SPMD + MPI only
729	executionType=4
730	elif ( ( ! ${OK_PARA_MPI} ) && ( ${OK_PARA_OMP} ) ) ; then
731	# SPMD + OMP only
732	executionType=5
733	elif ( ( ! ${OK_PARA_MPI} ) && ( ! ${OK_PARA_OMP} ) ) ; then
734	# SEQUENTIAL THEN
735	executionType=6
736	coreNumber=1
737	fi
738	fi
739
740	IGCM_debug_Print 1 "MPI/OMP treatment coreNumber = ${coreNumber}"
741	IGCM_debug_Print 1 "MPI/OMP treatment mpiTasks = ${mpiTasks}"
742	IGCM_debug_Print 1 "MPI/OMP treatment openMPthreads = ${openMPthreads}"
743	IGCM_debug_Print 1 "MPI/OMP treatment executionType = ${executionType}"
744
745	IGCM_debug_PopStack "IGCM_config_ConfigureExecution"
746	}
747
748	#===================================
749	function IGCM_config_PeriodStart
750	{
751	IGCM_debug_PushStack "IGCM_config_PeriodStart"
752
753	echo
754	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
755	echo
756
757	if ( ${FirstInitialize} ) ; then
758	#================================================#
759	# Initialize date/period information #
760	#================================================#
761
762	IGCM_date_GetYearMonthDay ${DateBegin} year month day
763	IGCM_config_DaysInPeriodLength
764
765	PeriodDateBegin=${DateBegin}
766	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
767	CumulPeriod=1
768
769	#=================================================#
770	# Write updated run.card #
771	#=================================================#
772
773	#Correct run.card Configuration for this period
774	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
775	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
776	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
777	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
778	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
779	fi
780
781	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
782
783	else
784	#================================================#
785	# The file run.card allready exist #
786	#================================================#
787
788	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
789	IGCM_config_StateCheck
790	# And EXIT if not OK
791	IGCM_debug_Verif_Exit
792
793	#===================================#
794	# Read updated run.card #
795	#===================================#
796
797	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
798	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
799	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
800
801	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
802	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
803	CumulPeriod=${run_Configuration_CumulPeriod}
804
805	LastPeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate $( IGCM_date_ConvertFormatToGregorian ${PeriodDateBegin} ) -1 )
806
807	if [ ${Period} = 1 ]; then
808	# save last Job output and current run.card
809	typeset Potential
810	IGCM_sys_Cd ${SUBMIT_DIR}
811	#
812	IGCM_debug_Print 2 "Save previous ksh job output"
813	for Potential in $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.[0-9][0-9][0-9][0-9][0-9][0-9] ) ; do
814	if [ X${Pack} = Xtrue ] ; then
815	( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_BUF_KSH}/${Potential}.$$
816	else
817	( IGCM_sys_TestFileArchive ${R_OUT_KSH}/${Potential} ) \|\| IGCM_sys_Cp ${Potential} ${R_OUT_KSH}/${Potential}.$$
818	fi
819	done
820	#
821	IGCM_debug_Print 2 "Save current run.card"
822	IGCM_card_CheckConflict run.card
823	if [ X${Pack} = Xtrue ] ; then
824	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card
825	else
826	IGCM_sys_Cp ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card
827	fi
828	#
829	IGCM_sys_Cd ${RUN_DIR}
830	else
831	unset FileToBeDeleted
832	fi
833
834	# Determine number of day(s) in PeriodLength
835	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
836	IGCM_config_DaysInPeriodLength
837
838	# Check coherency between (PeriodDateBegin, PeriodDateEnd) and (DateBegin, CumulPeriod, PeriodLength)
839	IGCM_config_DateCoherency
840	# And EXIT if not OK
841	IGCM_debug_Verif_Exit
842
843	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
844	IGCM_config_StateCheck
845	# And EXIT if not OK
846	IGCM_debug_Verif_Exit
847
848	# We can say we are "Running" now.
849	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
850	fi
851
852	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
853	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
854	year_m1=$(( year - 1 ))
855	year_p1=$(( year + 1 ))
856	# Compute month_m1 (month minus 1M)
857	# Compute yyyymm_m1 (yyyymm minus 1M)
858	month_m1=$(( 10#${month} - 1 ))
859	if [ ${month_m1} = 0 ]; then
860	month_m1=12
861	yyyymm_m1=${year_m1}12
862	elif [ ${month_m1} -le 9 ]; then
863	month_m1=0${month_m1}
864	yyyymm_m1=${year}${month_m1}
865	else
866	yyyymm_m1=${year}${month_m1}
867	fi
868	# Compute month_p1 (month plus 1M)
869	# Compute yyyymm_p1 (yyyymm plus 1M)
870	month_p1=$(( 10#${month} + 1 ))
871	if [ ${month_p1} = 13 ]; then
872	month_p1=01
873	yyyymm_p1=${year_p1}01
874	elif [ ${month_p1} -le 9 ]; then
875	month_p1=0${month_p1}
876	yyyymm_p1=${year}${month_p1}
877	else
878	yyyymm_p1=${year}${month_p1}
879	fi
880
881	#===================================================================#
882	# Calculate CyclicYear to be used for looping over a given forcing #
883	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
884	#===================================================================#
885
886	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
887	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
888	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
889	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
890	CyclicYear_p1=NOTDEFINED
891
892	# For current year
893	yeartmp=$year
894	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
895	while [ $diffy -lt 0 ] ; do
896	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
897	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
898	done
899	CyclicYear=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
900
901	# For next coming year
902	yeartmp=$(( $year + 1 ))
903	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
904	while [ $diffy -lt 0 ] ; do
905	yeartmp=$(( ${yeartmp} + ${CycleNb} ))
906	diffy=$(( $yeartmp - ${config_UserChoices_CyclicBegin} ))
907	done
908	CyclicYear_p1=$(( ( ${diffy} % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
909
910	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear}, CyclicYear_p1 = ${CyclicYear_p1}, current year=$year"
911	else
912	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
913	CyclicYear_p1="ERROR_CyclicYear_p1_Variable_Not_Defined"
914	IGCM_debug_Print 1 "CyclicYear wont be use without adding CyclicBegin and CyclicEnd in config.card"
915	fi
916
917	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
918
919	#===================================================================#
920	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
921	# But available to any son functions #
922	#===================================================================#
923
924	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
925	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
926
927	# Debug Print :
928	echo
929	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
930	IGCM_debug_Print 1 "Year of simulation : ${year}"
931	IGCM_debug_Print 1 "Month of simulation : ${month}"
932	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
933	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
934	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
935	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
936	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
937
938	#================================================================#
939	# Prepare variables available for comp_finalyze #
940	#================================================================#
941
942	# Period for save files
943	DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd}
944
945	# Prefix for save files of this period
946	PREFIX=${config_UserChoices_JobName}_${DatesPeriod}
947
948	# List of files that will be deleted in RUN_DIR after run
949	[ -f stack ] && FileToBeDeleted[0]="stack"
950
951	# Test if the same run as already been saved :
952	if [ X${JobType} = XRUN ] ; then
953	if [ ${DRYRUN} -le 0 ] ; then
954	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
955	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
956	IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
957	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
958	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
959	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
960	IGCM_debug_Verif_Exit
961	fi
962	fi
963	else
964	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
965	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
966	fi
967	fi
968
969	#================================================================#
970	# Prepare variables available for binary execution #
971	# Call function for creation of run script #
972	# Only done once per job #
973	#================================================================#
974
975	if [ ${Period} -eq 1 ]; then
976	# Define the execution context (MPMD, SPMD, MPI/OMP ...)
977	IGCM_config_ConfigureExecution ${SUBMIT_DIR}/config.card
978	# Create the execution script for the current context
979	IGCM_sys_build_execution_scripts
980	fi
981
982	ExecutionFail=false
983
984	# Update the rabbitMQ queue
985	IGCM_debug_BigBro_Update
986
987	IGCM_debug_PopStack "IGCM_config_PeriodStart"
988	}
989
990	#===================================
991	function IGCM_config_SaveSourceModifications
992	{
993	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
994
995	typeset ExeOutDateMax listVarEnv
996	ExeOutDateMax=$1
997
998	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
999	IGCM_sys_RshMaster "\
1000	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
1001	export ExeOutDateMax=${ExeOutDateMax};\
1002	export R_OUT_EXE=${R_OUT_EXE};\
1003	export PREFIX=${PREFIX};\
1004	export SUBMIT_DIR=${SUBMIT_DIR};\
1005	export listVarEnv=${listVarEnv};\
1006	Script_Output=out_SaveSourceModifications;\
1007	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
1008
1009	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
1010	}
1011
1012	#===================================
1013	function IGCM_config_PeriodEnd
1014	{
1015	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
1016
1017	echo
1018	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
1019	echo
1020
1021	if [ ${DRYRUN} -le 1 ] ; then
1022
1023	IGCM_debug_Print 1 "Check components binary : size and creation date"
1024
1025	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
1026	typeset comp i
1027	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
1028
1029	#==================================#
1030	# Get last Exe Size #
1031	#==================================#
1032
1033	(( i=0 ))
1034	if ( ${FirstInitialize} ) ; then
1035	run_Log_LastExeSize=""
1036	for comp in ${config_ListOfComponents[*]} ; do
1037	run_Log_LastExeSize[$i]=0
1038	(( i=i+1 ))
1039	done
1040	else
1041	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
1042	fi
1043	#==================================#
1044	# And Build ExeDate #
1045	#==================================#
1046
1047	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
1048	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
1049	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
1050	ExeDate=""
1051	NextExeSize="( "
1052	(( i=0 ))
1053	UpdateExe=false
1054	(( ExeSecDateMax = 0 ))
1055	for comp in ${config_ListOfComponents[*]} ; do
1056
1057	IGCM_debug_Print 3 ${comp}
1058
1059	eval ExeNameIn=\${config_Executable_${comp}[0]}
1060	eval ExeNameOut=\${config_Executable_${comp}[1]}
1061	# Only if we really have an executable for the component :
1062	if [ X${ExeNameIn} = X\"\" ] ; then
1063	# If there is no exe file for this component
1064	(( ExeSize=0 ))
1065	else
1066	LS_bin=${R_EXE}/${ExeNameIn}
1067	IGCM_sys_FileSize ${LS_bin} ExeSize
1068
1069	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
1070	if [ X${ExeDate} = X ] ; then
1071	# First component exe date
1072	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
1073	else
1074	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
1075	fi
1076	ExeDate=${ExeDate}_${LS_comp[7]}
1077	fi
1078
1079	if [ ${i} -eq 0 ] ; then
1080	# First component
1081	NextExeSize="( "${ExeSize}
1082	else
1083	NextExeSize=${NextExeSize}", "${ExeSize}
1084	fi
1085	LastCompExeSize=${run_Log_LastExeSize[$i]}
1086	(( i=i+1 ))
1087
1088	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
1089	if ( ${FirstInitialize} ) ; then
1090	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
1091	else
1092	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
1093	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
1094	FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1095	fi
1096	IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1097	UpdateExe=true
1098
1099	# SD : switch off for now
1100	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1101	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1102	# ExeSecDateMax=$ExeSecDate
1103	#fi
1104	fi
1105	done
1106
1107	# SD : switch off for now
1108	#if ( ${UpdateExe} ) ; then
1109	# echo "Launch SaveSourceModifications."
1110	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1111	#fi
1112
1113	NextExeSize=${NextExeSize}" )"
1114	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1115
1116	if [ ${DRYRUN} -le 1 ] ; then
1117	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1118	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1119	RET=$?
1120	if [ $RET -eq 0 ] ; then
1121	# ExeCpuLog variable contents 5 fields
1122	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1123	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1124	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1125	fi
1126	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}_tail.txt
1127	fi
1128
1129	fi
1130
1131	#==================================#
1132	# Save Job output #
1133	#==================================#
1134	if [ X${Pack} = Xtrue ] ; then
1135	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
1136	else
1137	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
1138	fi
1139	FileToBeDeleted[${#FileToBeDeleted[@]}]=${Exe_Output}
1140
1141	# All was right ? no ? then we stop.
1142	IGCM_debug_Verif_Exit
1143
1144	# If all was OK, we can delete all files not necessary for next Job
1145	echo
1146	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1147
1148	if [ ${DRYRUN} -le 2 ] ; then
1149	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1150	else
1151	echo ${FileToBeDeleted[@]}
1152	fi
1153
1154	# Send some accounting element to the user if CumulPeriod=3
1155	if [ ${CumulPeriod} -eq 3 ] ; then
1156	echo
1157	IGCM_debug_Print 1 "Send email containing some accounting information : "
1158
1159	RealCpuTime=$( echo ${ExeCpuLog} \| gawk '{print $3}' )
1160
1161	consumeHoursPerPeriod=$( echo "scale=6;${RealCpuTime}*${coreNumber}/3600" \| bc )
1162
1163	consumeHoursPerWholeSimulation=$( echo "scale=6;${consumeHoursPerPeriod}/${PeriodLengthInDays}*${ExperienceLengthInDays}" \| bc )
1164
1165	recommendedPeriodNb=$( echo "scale=6;${jobWarningDelay}/3600/${consumeHoursPerPeriod}*${coreNumber}" \| bc )
1166
1167	IGCM_sys_SendMail Accounting
1168	fi
1169
1170	#=================================================#
1171	# Modification of libIGCM behaviour #
1172	#=================================================#
1173
1174	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1175	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1176	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1177	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1178	echo
1179	echo "########################################################################"
1180	echo "!!! Modification of libIGCM behaviour !!!"
1181	echo
1182
1183	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1184	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1185	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1186
1187	# Special treatments for libIGCM internals
1188	for option in ${libIGCM_UserChanges[*]} ; do
1189	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1190
1191	echo "We will change : ${option}."
1192	eval echo "Previous value : " \${${option}}
1193	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1194
1195	eval ${option}=\${libIGCM_UserChanges_${option}}
1196
1197	case ${option} in
1198	config_UserChoices_DateEnd)
1199	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1200	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1201
1202	# Period Length In Days between DateBegin and DateEnd
1203	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1204	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1205	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1206	IGCM_debug_Exit "IGCM_PeriodEnd have wrong dates."
1207	IGCM_debug_Verif_Exit
1208	fi
1209	;;
1210	config_UserChoices_PeriodLength)
1211	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1212	;;
1213	PeriodNb)
1214	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1215	;;
1216	config_Post_RebuildFrequency)
1217	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1218	IGCM_post_Configure
1219	;;
1220	config_Post_TimeSeriesFrequency)
1221	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1222	IGCM_post_Configure
1223	;;
1224	config_Post_SeasonalFrequency)
1225	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1226	IGCM_post_Configure
1227	;;
1228	esac
1229	done
1230
1231	echo
1232	echo "########################################################################"
1233	echo
1234	fi
1235
1236	#=================================================#
1237	# Determine next computed period #
1238	#=================================================#
1239
1240	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1241	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1242	year_m1=$(( year - 1 ))
1243	year_p1=$(( year + 1 ))
1244	IGCM_config_DaysInPeriodLength
1245	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1246
1247	# Debug Print :
1248	echo
1249	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1250	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1251	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1252	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1253
1254	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1255	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1256
1257	(( CumulPeriod = CumulPeriod + 1 ))
1258
1259	# Debug Print :
1260	echo
1261	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1262	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1263	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1264
1265	#=================================================#
1266	# Write updated run.card #
1267	#=================================================#
1268
1269	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1270	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1271	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1272
1273	if ( ${FirstInitialize} ) ; then
1274	# It's no more the first time
1275	FirstInitialize=false
1276	fi
1277
1278	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1279	}
1280
1281	#===================================
1282	function IGCM_config_Finalize
1283	{
1284	IGCM_debug_PushStack "IGCM_config_Finalize"
1285
1286	echo
1287	IGCM_debug_Print 1 "IGCM_config_Finalize"
1288	echo
1289
1290	# Test state of run in run.card. Will schedule an exit if another process setted it to "Fatal"
1291	IGCM_config_StateCheck
1292
1293	# And EXIT if not OK
1294	IGCM_debug_Verif_Exit
1295
1296	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1297	#==========================#
1298	# End of entire simulation #
1299	#==========================#
1300	simulationIsOver=true
1301
1302	# Mail notification
1303	IGCM_sys_SendMail
1304	#
1305	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1306	#
1307	IGCM_debug_Print 1 "Normal End of computation."
1308
1309	else
1310	#=================#
1311	# Submit next job #
1312	#=================#
1313	simulationIsOver=false
1314
1315	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1316
1317	# Name of next Ksh Script output :
1318	Script_Output=${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )
1319
1320	IGCM_debug_Print 1 "Submit next job"
1321	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1322	IGCM_sys_Cd ${SUBMIT_DIR}
1323	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1324	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1325	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1326	# Submit next job and come back
1327	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1328	IGCM_sys_Cd -
1329	fi
1330
1331	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1332	# Only for production run (No clean up in DEV or DEB mode)
1333	# and command sent from .. directory.
1334	IGCM_sys_Cd ..
1335	[ X${JobType} = XRUN ] && IGCM_sys_RmRunDir -rf ${RUN_DIR_PATH}
1336
1337	# Inform the rabbitMQ queue
1338	IGCM_debug_BigBro_Finalize
1339
1340	IGCM_debug_PopStack "IGCM_config_Finalize"
1341	}
1342
1343	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 1335

Download in other formats: