Context Navigation

libIGCM_config.ksh @ 804

Last change on this file since 804 was 804, checked in by sdipsl, 11 years ago

Files waiting for rebuild are stored within the relevant simulation tree. see #87

--> /IGCM_OUT/TagName/SpaceName/ExperiementName/JobName/REBUILD

Using only 1 proc and having RebuildFrequency? != NONE won't be an issue. see #88
Homogenize RUN_DIR PATH during computing and post-processing. see #92
Save text files before restarts and outputs. (more chance to have a clue of what went wrong) see #103
Remove useless variable BIG_DIR
Cosmectics

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 48.5 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip
5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr Martial.Mancip__at__ipsl.jussieu.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#===================================
15	function IGCM_config_CommonConfiguration
16	{
17	IGCM_debug_PushStack "IGCM_config_CommonConfiguration" $@
18
19	# Debug Print :
20	[ ${Verbosity} -gt 0 ] && echo
21	IGCM_debug_Print 1 "IGCM_config_CommonConfiguration" $@
22
23	# config.card path
24	configPath=$1
25
26	#==================================
27	typeset option auxprint CompatibilityTag
28
29	#==================================
30	# Read libIGCM compatibility version in config.card
31	IGCM_card_DefineVariableFromOption ${configPath} Compatibility libIGCM
32	eval CompatibilityTag=${config_Compatibility_libIGCM} > /dev/null 2>&1
33
34	if [ ! "${CompatibilityTag}" = "${libIGCM_CurrentTag}" ] ; then
35	IGCM_debug_Exit "config.card is not compatible with libIGCM version ${libIGCM_CurrentTag} see libIGCM FAQ http://wiki.ipsl.jussieu.fr/wiki_ipsl/IGCMG/libIGCM/DocUtilisateur/FAQ ."
36	fi
37
38	#==================================
39	# Read UserChoices section:
40	[ ${Verbosity} -gt 0 ] && echo
41	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
42
43	IGCM_card_DefineArrayFromSection ${configPath} UserChoices
44	for option in ${config_UserChoices[*]} ; do
45	IGCM_card_DefineVariableFromOption ${configPath} UserChoices ${option}
46	eval auxprint=\${config_UserChoices_${option}}
47	IGCM_debug_Print 3 "${option} : ${auxprint}"
48	done
49
50	#==================================
51	# Read Ensemble section:
52	[ ${Verbosity} -gt 0 ] && echo
53	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
54
55	IGCM_card_DefineArrayFromSection ${configPath} Ensemble
56	for option in ${config_Ensemble[*]} ; do
57	IGCM_card_DefineVariableFromOption ${configPath} Ensemble ${option}
58	eval auxprint=\${config_Ensemble_${option}}
59	IGCM_debug_Print 3 "${option} : ${auxprint}"
60	done
61
62	#==================================
63	# Read Post section:
64	[ ${Verbosity} -gt 0 ] && echo
65	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
66
67	IGCM_card_DefineArrayFromSection ${configPath} Post
68	for option in ${config_Post[*]} ; do
69	IGCM_card_DefineVariableFromOption ${configPath} Post ${option}
70	eval auxprint=\${config_Post_${option}}
71	IGCM_debug_Print 3 "${option} : ${auxprint}"
72	done
73	[ ${Verbosity} -gt 0 ] && echo
74
75	#==================================
76	# Define default value to keep compatibility with previous card: means before changes due to TGCC
77	# Apply some overrules to ensure proper usage of computing centres resources
78	#
79	if [ X${PackDefault} = Xtrue ] ; then
80	# If we DO apply pack in this computing center
81	IGCM_debug_Print 1 "Due to inodes policy OVERRULE RebuildFromArchive value to NONE"
82	config_Post_RebuildFromArchive=NONE
83	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
84	# TEST simulations will not be packed and will stay on SCRATCHDIR filesystem
85	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE PackFrequency to NONE"
86	IGCM_debug_Print 1 "SpaceName=TEST. OVERRULE destination path directories"
87	config_Post_PackFrequency=NONE
88	IGCM_sys_ChangeArchive
89	else
90	# Default to RebuildFrequency if nothing has been set up related to PackFrequency
91	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
92	fi
93	else
94	# If we DO NOT apply pack in this computing center
95	config_Post_PackFrequency=NONE
96	fi
97
98	#====================================================
99	# R_SAVE : Job output directory
100	# R_BUFR : Job output buffered directory
101
102	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
103	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
104	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
105	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
106	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
107	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
108	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
109	else
110	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
111	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
112	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
113	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
114	fi
115	else
116	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
117	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
118	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
119	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
120	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
121	else
122	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
123	R_FIGR=${R_FIG}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
124	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
125	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
126	fi
127	fi
128
129	#====================================================
130	# Define REBUILD_DIR : where we store files needing rebuild process
131	if [ X${config_Post_RebuildFromArchive} = Xtrue ] ; then
132	REBUILD_DIR=${R_SAVE}/REBUILD
133	IGCM_sys_MkdirArchive ${REBUILD_DIR}
134	else
135	REBUILD_DIR=${R_BUFR}/REBUILD
136	IGCM_sys_MkdirWork ${REBUILD_DIR}
137	fi
138
139	IGCM_debug_PopStack "IGCM_config_CommonConfiguration"
140	}
141
142	#===================================
143	function IGCM_config_Initialize
144	{
145	IGCM_debug_PushStack "IGCM_config_Initialize"
146
147	# Debug Print :
148	echo
149	IGCM_debug_Print 1 "IGCM_config_Initialize"
150
151	if [ ! -r ${SUBMIT_DIR}/run.card ]; then
152	#================================================#
153	# The file run.card doesn't exist #
154	#================================================#
155	FirstInitialize=true
156	#copy initial run.card
157	IGCM_sys_Cp ${SUBMIT_DIR}/run.card.init ${SUBMIT_DIR}/run.card
158	IGCM_debug_Print 2 "run.card copied from run.card.init"
159	else
160	FirstInitialize=false
161	IGCM_debug_Print 2 "run.card exists"
162	fi
163
164	# Test modipsl tree existence.
165	IGCM_sys_TestDir ${MODIPSL}
166	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
167	IGCM_sys_TestDir ${libIGCM}
168	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
169	IGCM_sys_TestDir ${R_EXE}
170	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
171	IGCM_sys_TestDir ${SUBMIT_DIR}
172	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir"
173
174	if ( $DEBUG_debug ) ; then
175	echo "Keep trace of inital SUBMIT_DIR : "
176	ls -lta ${SUBMIT_DIR}
177	fi
178
179	#==================================
180	# Read libIGCM compatibility version in config.card
181	# Read UserChoices section
182	# Read Ensemble section
183	# Read Post section
184	# Define all netcdf output directories
185	#==================================
186	IGCM_config_CommonConfiguration ${SUBMIT_DIR}/config.card
187
188	#==================================
189	# Read ListOfComponents section:
190	echo
191	IGCM_debug_Print 1 "DefineArrayFromSection : ListOfComponents"
192
193	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card ListOfComponents
194	for comp in ${config_ListOfComponents[*]} ; do
195	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/config.card ListOfComponents ${comp}
196	done
197	IGCM_debug_Print 3 ${config_ListOfComponents[*]}
198
199	#==================================
200	# Read Executable section:
201	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Executable
202
203	#==================================
204	# Read Restarts section:
205	# Restarts : Gerneral rule or local for each component.
206	echo
207	IGCM_debug_Print 1 "DefineArrayFromOption : config_Restarts"
208
209	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/config.card Restarts
210	for option in ${config_Restarts[*]} ; do
211	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/config.card Restarts ${option}
212	eval auxprint=\${config_Restarts_${option}}
213	IGCM_debug_Print 3 "${option} : ${auxprint}"
214	done
215
216	#==================================
217	# Define Job Outputs Name
218	echo
219	IGCM_debug_Print 2 "Define Script_Output_Prefix and Exe_Output"
220	eval Script_Output_Prefix=${config_UserChoices_Script_Output_Prefix:='Script_Output'}
221	IGCM_debug_Print 3 "Script_Output_Prefix = ${Script_Output_Prefix}"
222	eval Exe_Output=out_execution
223	IGCM_debug_Print 3 "Exe_Output = ${Exe_Output}"
224
225	#===================================================================#
226	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
227	# But available to any son functions #
228	#===================================================================#
229
230	# Convert yyyy-mm-dd date to gregorian yyyymmdd
231	DateBegin=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )
232	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
233
234	# Period Length In Days between DateBegin and DateEnd
235	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
236	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
237	IGCM_debug_Print 1 "Problem with dates in config.card : ${DateEnd} < ${DateBegin} ! You must check that."
238	IGCM_debug_Exit "IGCM_config_Initialize" " Wrong Dates."
239	IGCM_debug_Verif_Exit
240	fi
241
242	# Day and Year of Initial State (Given in julian format)
243	InitDay=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) % 1000 ))
244	InitYear=$(( $( IGCM_date_ConvertGregorianDateToJulian $DateBegin ) / 1000 ))
245
246	#================================================================#
247	# Test and Prepare directories #
248	#================================================================#
249
250	# ==> 4 kinds of input files :
251	# 1) R_INIT : Initial State Files (Etat0, carteveg)
252	# 2) R_BC : Boundary Conditions (Forcages, lai)
253	# 3) Parameters files (allready define through ${SUBMIT_DIR})
254	# 4) Restarts files (allready define in IGCM_config_Initialize)
255
256	# Here we offer the possibility to redefine R_INIT, R_BC
257	# and PeriodNb through config.card
258	R_INIT=${config_UserChoices_R_INIT:=${R_IN}/INIT}
259	echo
260	IGCM_debug_Print 2 "(Re)Define R_INIT, R_BC and PeriodNb"
261	IGCM_debug_Print 3 "R_INIT=${R_INIT}"
262	R_BC=${config_UserChoices_R_BC:=${R_IN}/BC}
263	IGCM_debug_Print 3 "R_BC=${R_BC}"
264	PeriodNb=${config_UserChoices_PeriodNb:=${PeriodNb}}
265	IGCM_debug_Print 3 "Loop in main Job with ${PeriodNb} period(s)"
266
267	# Test Archive input/output.
268	echo
269	IGCM_sys_TestDirArchive ${ARCHIVE}
270	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
271
272	# SD ADA SPECIFIC #
273	# TO FIX #
274	#IGCM_sys_TestDirArchive ${R_IN}
275	#[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive"
276
277	if ( ${FirstInitialize} ) ; then
278	IGCM_sys_MkdirArchive ${R_SAVE}
279	IGCM_sys_Mkdir ${R_BUFR}
280	else
281	IGCM_sys_TestDirArchive ${R_SAVE}
282	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDirArchive ${R_SAVE}"
283
284	IGCM_sys_TestDir ${R_BUFR}
285	[ $? != 0 ] && IGCM_debug_Exit "IGCM_sys_TestDir ${R_BUFR}"
286
287	#Test state of run in run.card
288	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
289
290	if ( [ ${run_Configuration_PeriodState} != "Start" ] && [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
291	echo
292	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
293	IGCM_debug_Print 1 "Try running ${libIGCM}/clean_month.job to fix this"
294	IGCM_debug_Exit
295	IGCM_debug_Verif_Exit
296	fi
297	fi
298
299	#====================================================
300	# Experience type : DEB(ug), DEV(elopment), RUN
301	if [ X${JobType} != XRUN ] ; then
302	echo
303	echo "===================================================="
304	echo "libIGCM JOB is NOT in RUN type mode."
305	echo "!! OUTPUT files will NOT be PROTECTED !!"
306	echo "Be carefull : you can ERASE the result of this job !"
307
308	case ${JobType} in
309	DEB)
310	echo "DEBUG mode : activation of 'set -vx' mode."
311	echo "DEBUG mode : no protection for output files."
312	echo "DEBUG mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
313	;;
314	DEV)
315	echo "DEVelopment mode : no protection for output files."
316	echo "DEVelopment mode : if active force asynchronous rebuild frequency to PeriodLength frequency."
317	;;
318	esac
319
320	if ( [ X${config_Post_RebuildFrequency} != XNONE ] && [ ${DRYRUN} -eq 0 ] ) ; then
321	if [ X${config_Post_RebuildFrequency} != X${config_UserChoices_PeriodLength} ] ; then
322	echo "------------"
323	echo "WARNING : Job is NOT in RUN mode then we will force REBUILD Frequency"
324	echo " to PeriodLength : ${config_UserChoices_PeriodLength}"
325	echo "------------"
326	config_Post_RebuildFrequency=${config_UserChoices_PeriodLength}
327	fi
328	fi
329	echo "===================================================="
330	echo
331	fi
332
333	#====================================================
334	#R_OUT_KSH : Storage place for job output
335	#R_OUT_EXE : Storage place for binary used during simulation
336	R_OUT_KSH=${R_SAVE}/Out
337	R_OUT_EXE=${R_SAVE}/Exe
338
339	#====================================================
340	#R_BUF_KSH : Buffer place for job output
341	#R_BUF_EXE : Buffer place for binary used during simulation
342	R_BUF_KSH=${R_BUFR}/Out
343	R_BUF_EXE=${R_BUFR}/Exe
344
345	IGCM_debug_PopStack "IGCM_config_Initialize"
346	}
347
348	#===================================
349	function IGCM_config_Analyse_PeriodLength
350	{
351	IGCM_debug_PushStack "IGCM_config_Analyse_PeriodLength"
352
353	typeset i
354
355	# Determine number of day(s) in PeriodLength :
356	case ${config_UserChoices_PeriodLength} in
357	Y\|y)
358	PeriodLengthInYears=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[yY]//' )
359	echo
360	IGCM_debug_Print 2 "Number of years for PeriodLength : ${PeriodLengthInYears}"
361	PeriodLengthInDays=0
362	i=0
363	until [ $i -ge $PeriodLengthInYears ] ; do
364	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInYear $(( year + i )) ) ))
365	(( i=i+1 ))
366	done
367	;;
368	M\|m)
369	PeriodLengthInMonths=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[mM]//' )
370	echo
371	IGCM_debug_Print 2 "Number of months for PeriodLength : ${PeriodLengthInMonths}"
372	PeriodLengthInDays=0
373	i=0
374	until [ $i -ge $PeriodLengthInMonths ] ; do
375	if [ $(( month + i )) -lt 13 ] ; then
376	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i )) ) ))
377	else
378	(( PeriodLengthInDays = PeriodLengthInDays + $( IGCM_date_DaysInMonth $year $(( month + i - 12 )) ) ))
379	fi
380	(( i=i+1 ))
381	done
382	;;
383	D\|d)
384	PeriodLengthInMonths=0
385	PeriodLengthInDays=$( echo ${config_UserChoices_PeriodLength} \| sed -e 's/[dD]//' )
386	echo
387	IGCM_debug_Print 2 "Number of days for PeriodLength : ${PeriodLengthInDays}";;
388	*)
389	IGCM_debug_Exit "IGCM_config_Analyse_PeriodLength " ${config_UserChoices_PeriodLength} " invalid period length : choose in Y, M, *D."
390	IGCM_debug_Verif_Exit ;;
391	esac
392
393	IGCM_debug_PopStack "IGCM_config_Analyse_PeriodLength"
394	}
395
396	#===================================
397	function IGCM_config_Check
398	{
399	IGCM_debug_PushStack "IGCM_config_Check"
400
401	# If one of the following modulo is not zero :
402	# we will issue an error then explain and exit in
403	# AA_job IGCM_debug_Verif_Exit call before binary submission
404
405	echo
406	IGCM_debug_Print 1 "IGCM_config_Check"
407	echo
408
409	typeset i
410
411	# When Cyclic is active, the simulation MUST start with the same year as the CyclicBegin year.
412	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
413	IGCM_debug_Print 1 "Check coherence between CyclicBegin year and DateBegin year"
414	if [ ${InitYear} -ne ${config_UserChoices_CyclicBegin} ] ; then
415	IGCM_debug_Print 1 "The simulation MUST start with the same year as the CyclicBegin year"
416	IGCM_debug_Exit "Check your CyclicBegin option in config.card UserChoices section"
417	fi
418	fi
419
420	# Check RebuildFrequency against key frequencies : PeriodLength ; PackFrequency ; TimeSeriesFrequency ; SeasonalFrequency
421	if ( [ ! X${config_Post_RebuildFrequency} = X${NULL_STR} ] && [ ! X${config_Post_RebuildFrequency} = XNONE ] ) ; then
422	AsynchronousRebuild=true
423	IGCM_debug_Print 1 "Asynchronous rebuild has been activated."
424	echo
425	# modulo (RebuildFrequency and PeriodLength/TimeSeriesFrequency/SeasonalFrequency) must be zero
426	IGCM_debug_Print 1 "Check coherence between RebuildFrequency and PeriodLength"
427	IGCM_post_CheckModuloFrequency config_Post_RebuildFrequency config_UserChoices_PeriodLength
428	IGCM_debug_Print 1 "Check coherence between PackFrequency and RebuildFrequency"
429	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_Post_RebuildFrequency
430	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and RebuildFrequency"
431	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_RebuildFrequency
432	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and RebuildFrequency"
433	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_RebuildFrequency
434	else
435	AsynchronousRebuild=false
436	IGCM_debug_Print 1 "Asynchronous rebuild has not been activated"
437	IGCM_debug_Print 1 "Proceed with standard post-treatment pathway"
438	echo
439	#modulo (PeriodLength and TimeSeriesFrequency/SeasonalFrequency) must be zero
440	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PeriodLength"
441	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_UserChoices_PeriodLength
442	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PeriodLength"
443	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_UserChoices_PeriodLength
444	fi
445
446	# Check PackFrequency against other key frequencies
447	# Modulo (PackFrequency and TimeSeriesFrequency/SeasonalFrequency and PeriodLenght) must be zero
448	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
449	Pack=true
450	#
451	IGCM_debug_Print 1 "Check coherence between PackFrequency and PeriodLength"
452	IGCM_post_CheckModuloFrequency config_Post_PackFrequency config_UserChoices_PeriodLength
453	IGCM_debug_Print 1 "Check coherence between TimeSeriesFrequency and PackFrequency"
454	IGCM_post_CheckModuloFrequency config_Post_TimeSeriesFrequency config_Post_PackFrequency
455	IGCM_debug_Print 1 "Check coherence between SeasonalFrequency and PackFrequency"
456	IGCM_post_CheckModuloFrequency config_Post_SeasonalFrequency config_Post_PackFrequency
457	else
458	Pack=false
459	fi
460
461	#modulo (TimeSeriesFrequency and all Chunck2D) must be zero
462	NbJob=${#CHUNCK2D_SIZE[@]}
463	i=0
464	until [ $i -ge $NbJob ]; do
465	value=${CHUNCK2D_SIZE[${i}]}
466	IGCM_debug_Print 1 "Check coherence between All Chunck2D frequency and TimeSeriesFrequency"
467	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
468	case ${value} in
469	Y\|y) ;;
470	*)
471	IGCM_debug_Print 1 "All ChunckJob2D frequency must be expressed in year Y\|y in comp.card"
472	IGCM_debug_Exit "This will stop the job" ;;
473	esac
474	(( i=i+1 ))
475	done
476
477	#modulo (TimeSeriesFrequency and all Chunck3D) must be zero
478	NbJob=${#CHUNCK3D_SIZE[@]}
479	i=0
480	until [ $i -ge $NbJob ]; do
481	value=${CHUNCK3D_SIZE[${i}]}
482	IGCM_debug_Print 1 "Check coherence between All Chunck3D frequency and TimeSeriesFrequency"
483	IGCM_post_CheckModuloFrequency value config_Post_TimeSeriesFrequency
484	case ${value} in
485	Y\|y) ;;
486	*)
487	IGCM_debug_Print 1 "All ChunckJob3D frequency must be expressed in year Y\|y in comp.card"
488	IGCM_debug_Exit "This will stop the job" ;;
489	esac
490	(( i=i+1 ))
491	done
492
493	#check to be sure there is enough space on temporary filesystems to run
494	echo
495	IGCM_debug_Print 1 "Check if there is enough space on temporary filesystem"
496	IGCM_sys_check_quota
497
498	IGCM_debug_PopStack "IGCM_config_Check"
499	}
500
501	#===================================
502	function IGCM_config_PeriodStart
503	{
504	IGCM_debug_PushStack "IGCM_config_PeriodStart"
505
506	echo
507	IGCM_debug_Print 1 "IGCM_config_PeriodStart"
508	echo
509
510	if ( ${FirstInitialize} ) ; then
511	#================================================#
512	# Initialize date/period information #
513	#================================================#
514
515	IGCM_date_GetYearMonthDay $DateBegin year month day
516	IGCM_config_Analyse_PeriodLength
517
518	eval PeriodDateBegin=${DateBegin} > /dev/null 2>&1
519	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( ${PeriodLengthInDays} - 1 )) )
520	eval CumulPeriod=1 > /dev/null 2>&1
521
522	#=================================================#
523	# Write updated run.card #
524	#=================================================#
525
526	#Correct run.card Configuration for this period
527	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
528	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
529	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
530	if [ X$( grep "SubmitPath" ${SUBMIT_DIR}/run.card ) != X ] ; then
531	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration SubmitPath ${SUBMIT_DIR}
532	fi
533
534	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
535
536	else
537	#================================================#
538	# The file run.card allready exist #
539	#================================================#
540
541	#Test state of run in run.card
542	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodState
543	if ( [ ${run_Configuration_PeriodState} != "Running" ] && [ ${run_Configuration_PeriodState} != "OnQueue" ] && [ ${run_Configuration_PeriodState} != "Continue" ] ) ; then
544	echo
545	IGCM_debug_Print 1 "!! Error in run.card with PeriodState : " ${run_Configuration_PeriodState} "!!"
546	IGCM_debug_Print 1 "Try running ${libIGCM}/clean_month.job to fix this"
547	echo
548	IGCM_debug_Exit
549	IGCM_debug_Verif_Exit
550	fi
551
552	#===================================#
553	# Read updated run.card #
554	#===================================#
555
556	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration OldPrefix
557	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin
558	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd
559	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod
560
561	PeriodDateBegin=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateBegin} )
562	PeriodDateEnd=$( IGCM_date_ConvertFormatToGregorian ${run_Configuration_PeriodDateEnd} )
563	eval CumulPeriod="${run_Configuration_CumulPeriod}" > /dev/null 2>&1
564
565	LastPeriodDateEnd=$( echo ${run_Configuration_OldPrefix} \| sed -e "s/${config_UserChoices_JobName}_//" )
566
567	typeset LastDate
568	set +A LastDate -- $( echo ${LastPeriodDateEnd} \| sed -e "s/$[0-9]\{${dY}\}$$[0-9]\{2\}$$[0-9]\{2\}$/\1 \2 \3/" )
569	LastYearDateEnd=${LastDate[0]}
570	LastMonthDateEnd=${LastDate[1]}
571	LastDayDateEnd=${LastDate[2]}
572
573	if [ ${Period} = 1 ]; then
574	# save last Job output
575	typeset LastPeriod iLastPeriod Last_Script_Output_Name
576	(( LastPeriod=CumulPeriod-PeriodNb ))
577	iLastPeriod=$( printf "%06d" ${LastPeriod} )
578	Last_Script_Output_Name="${Script_Output_Prefix}_${config_UserChoices_JobName}.${iLastPeriod}"
579	Last_run_card="run.card.${iLastPeriod}"
580	#eval ls -l ${SUBMIT_DIR}/${Last_Script_Output_Name}
581	#eval IGCM_sys_Put_Out ${SUBMIT_DIR}/${Last_Script_Output_Name} ${R_BUF_KSH}/${Last_Script_Output_Name} > /dev/null 2>&1
582	IGCM_debug_Print 2 "Try to save previous ksh job output"
583	IGCM_sys_Cd ${SUBMIT_DIR}/
584	if [ X${Pack} = Xtrue ] ; then
585	IGCM_sys_PutBuffer_Out ${Last_Script_Output_Name} ${R_BUF_KSH}/${Last_Script_Output_Name}.$$ rw
586	else
587	IGCM_sys_Put_Out ${Last_Script_Output_Name} ${R_OUT_KSH}/${Last_Script_Output_Name}.$$ rw
588	fi
589
590	IGCM_card_CheckConflict run.card
591	if [ X${Pack} = Xtrue ] ; then
592	IGCM_sys_PutBuffer_Out ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/${Last_run_card}.$$ rw
593	else
594	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/${Last_run_card}.$$ rw
595	fi
596	IGCM_sys_Cd ${RUN_DIR}
597	else
598	unset FileToBeDeleted
599	fi
600
601	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
602	IGCM_config_Analyse_PeriodLength
603
604	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Running"
605
606	fi
607	OldPrefix=${config_UserChoices_JobName}_${PeriodDateEnd}
608
609	# BEGIN: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
610	# Compute year_m1 and year_p1 (year minus 1Y and year plus 1Y)
611	year_m1=$(( year - 1 ))
612	year_p1=$(( year + 1 ))
613	# Compute month_m1 (month minus 1M)
614	# Compute yyyymm_m1 (yyyymm minus 1M)
615	month_m1=$(( month - 1 ))
616	if [ ${month_m1} = 0 ]; then
617	month_m1=12
618	yyyymm_m1=${year_m1}12
619	elif [ ${month_m1} -le 9 ]; then
620	month_m1=0${month_m1}
621	yyyymm_m1=${year}${month_m1}
622	else
623	yyyymm_m1=${year}${month_m1}
624	fi
625	# Compute month_p1 (month plus 1M)
626	# Compute yyyymm_p1 (yyyymm plus 1M)
627	month_p1=$(( month + 1 ))
628	if [ ${month_p1} = 13 ]; then
629	month_p1=01
630	yyyymm_p1=${year_p1}01
631	elif [ ${month_p1} -le 9 ]; then
632	month_p1=0${month_p1}
633	yyyymm_p1=${year}${month_p1}
634	else
635	yyyymm_p1=${year}${month_p1}
636	fi
637	#IGCM_debug_Print 1 "jg 1 month_m1 = ${month_m1} month_p1 = ${month_p1} "
638	#IGCM_debug_Print 1 "jg 1 calculate yyyymm_m1 = ${yyyymm_m1} "
639	#IGCM_debug_Print 1 "jg 1 calculate yyyymm_p1 = ${yyyymm_p1} "
640
641	#===================================================================#
642	# Calculate CyclicYear to be used for looping over a given forcing #
643	# period. Add CyclicBegin and CyclicEnd in config.card UserChoices. #
644	#===================================================================#
645
646	# To use the variable CyclicYear, one must add in config.card CyclicBegin and CyclicEnd.
647	# CyclicBegin is the first year in the cycle. CyclicEnd is the last year included in the cycle.
648	# The simulation MUST start with the same year as the CyclicBegin year.
649	if ( [ ! X${config_UserChoices_CyclicBegin} = X ] && [ ! X${config_UserChoices_CyclicEnd} = X ] ) ; then
650	CycleNb=$(( ${config_UserChoices_CyclicEnd} - ${config_UserChoices_CyclicBegin} + 1 ))
651	CyclicYear=$(( ( ( ${year} - ${config_UserChoices_CyclicBegin} ) % ${CycleNb} ) + ${config_UserChoices_CyclicBegin} ))
652	IGCM_debug_Print 1 "CyclicYear = ${CyclicYear} current year=$year"
653	else
654	CyclicYear="ERROR_CyclicYear_Variable_Not_Defined"
655	IGCM_debug_Print 1 "CyclicYear not possible to use without adding CyclicBegin and CyclicEnd in config.card"
656	fi
657
658	# END: SHOULD GO IN A FUNCTION FROM libIGCM_date.ksh
659
660	#===================================================================#
661	# Prepare variables available for ${COMP}.card and ${COMP}.driver #
662	# But available to any son functions #
663	#===================================================================#
664
665	# Period Length In Days between DateBegin and DateCurrent (at end of period == PeriodDateEnd !)
666	(( SimulationLengthInDays = $( IGCM_date_DaysBetweenGregorianDate ${PeriodDateEnd} ${DateBegin} ) + 1 ))
667
668	# Debug Print :
669	echo
670	IGCM_debug_Print 1 "IGCM_config_PeriodStart : Before Execution"
671	IGCM_debug_Print 1 "Year of simulation : ${year}"
672	IGCM_debug_Print 1 "Month of simulation : ${month}"
673	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
674	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
675	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
676	IGCM_debug_Print 1 "SimulationLengthInDays : ${SimulationLengthInDays}"
677	IGCM_debug_Print 1 "ExperienceLengthInDays : ${ExperienceLengthInDays}"
678
679	#================================================================#
680	# Prepare variables available for comp_finalyze #
681	#================================================================#
682
683	# Period for save files
684	eval DatesPeriod=${PeriodDateBegin}_${PeriodDateEnd} > /dev/null 2>&1
685
686	# Prefix for save files of this period
687	eval PREFIX=${config_UserChoices_JobName}_${DatesPeriod} > /dev/null 2>&1
688
689	# List of files that will be deleted in RUN_DIR after run
690	[ -f stack ] && FileToBeDeleted[0]="stack"
691
692	# Test if the same run as already been saved :
693	if [ X${JobType} = XRUN ] ; then
694	if [ ${DRYRUN} -le 0 ] ; then
695	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
696	IGCM_debug_Exit "IGCM_config_PeriodStart" "You are currently RErunning an old job."
697	IGCM_debug_Print 1 "Because of readonly permissions, you can't RErun a job when saved files"
698	IGCM_debug_Print 1 " are still in the ARCHIVE directory. You must deleted those files, or "
699	IGCM_debug_Print 1 " the whole ${R_SAVE} tree. See clean_month.job in ${libIGCM} directory."
700	IGCM_debug_Print 1 " This exit has been initiated because at least ${R_BUF_KSH}/${PREFIX}_${Exe_Output} exists."
701	IGCM_debug_Verif_Exit
702	fi
703	fi
704	else
705	if ( IGCM_sys_TestFileBuffer ${R_BUF_KSH}/${PREFIX}_${Exe_Output} ) ; then
706	IGCM_debug_Print 1 "IGCM_config_PeriodStart" "RErun an old job. Allowed in DEBUG or DEV mode."
707	fi
708	fi
709
710	typeset ExeNameIn ExeNameFirst CompNameFirst
711	typeset comp i
712	typeset tempvar tempvarMPI tempvarNOD NbElts j NbExec
713
714	(( PROCESSUS_NUMBER = 0 ))
715	(( i=0 ))
716	(( NbExec=0 ))
717
718	OK_PARA_MPI=false
719	OK_PARA_OMP=false
720	OK_PARA_NOD=false
721	OK_PARA_MPMD=false
722
723	for comp in ${config_ListOfComponents[*]} ; do
724
725	IGCM_debug_Print 1 ${comp}
726
727	eval ExeNameIn=\${config_Executable_${comp}[0]}
728
729	# NO order in config.card for parallelized values !
730	# just use suffix : MPI , OMP and NOD (for number of NODes.)
731
732	# NOD is the number of NODes allocated
733	eval ${comp}_PROC_NOD=0
734
735	# MPI is the number of MPI processus per nodes
736	eval ${comp}_PROC_MPI=0
737
738	# OMP is the number of OpenMP threads per MPI processus
739	eval ${comp}_PROC_OMP=0
740
741	# Only if we really have an executable for the component :
742	### if [ X${ExeNameIn} != X\"\" ] ; then
743	if ( [ "X${ExeNameIn}" != X\"\" ] && [ "X${ExeNameIn}" != "Xinca.dat" ] ) ; then
744
745	# Keep the first executable found and the first CompName
746	ExeNameFirst=${ExeNameIn}
747	CompNameFirst=${comp}
748
749	# Are we a second executable?
750	(( NbExec = NbExec + 1 ))
751
752	# set 1 MPI task, 1 OpenMP thread and 1 node as default
753	eval ${comp}_PROC_MPI=1
754	eval ${comp}_PROC_OMP=1
755	eval ${comp}_PROC_NOD=1
756
757	eval NbElts=\${#config_Executable_${comp}[@]}
758
759	if [ ${NbElts} -ge 2 ] ; then
760	(( j = 2 ))
761
762	while [ $j -lt ${NbElts} ] ; do
763	eval tempvar=\${config_Executable_${comp}[${j}]}
764
765	if [ X${tempvar} = X ] ; then
766	echo "Error reading MPI/OMP parameters !!!"
767	echo "Check your config.card."
768	exit 1
769	fi
770
771	eval IGCM_debug_Print 2 ${tempvar}
772
773	case ${tempvar} in
774	[mM][pP][iI])
775	# Read MPI parameter for composante
776	eval ${comp}_PROC_MPI=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/MPI//" )
777	OK_PARA_MPI=true;;
778	[oO][mM][pP])
779	# Read OMP parameter for composante
780	eval ${comp}_PROC_OMP=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/OMP//" )
781	OK_PARA_OMP=true;;
782	[nN][oO][dD])
783	# Read NOD (NumBer of Nodes) parameter for composante
784	eval ${comp}_PROC_NOD=$( echo ${tempvar} \| tr '[a-z]' '[A-Z]' \| sed -e "s/NOD//" )
785	OK_PARA_NOD=true
786	OK_PARA_MPI=true;;
787	esac
788
789	(( j = j + 1 ))
790	done
791
792	fi
793
794	eval tempvarMPI=\${${comp}_PROC_MPI}
795	eval tempvarNOD=\${${comp}_PROC_NOD}
796	eval tempvarOMP=\${${comp}_PROC_OMP}
797
798	(( PROCESSUS_NUMBER = PROCESSUS_NUMBER + tempvarMPI * tempvarNOD * tempvarOMP ))
799
800	fi
801
802	(( i=i+1 ))
803	done
804
805	# set MPMD mode if more than 2 executable names.
806	[ ${NbExec} -ge 2 ] && OK_PARA_MPMD=true
807
808	# Verification of BATCH_NUM_PROC_TOT total number of processors set in job header.
809	if [ X${BATCH_NUM_PROC_TOT} != X ] ; then
810	# BATCH_NUM_PROC_TOT is set
811	if ( ${OK_PARA_MPI} ) ; then
812	IGCM_debug_Print 1 "MPI/OMP/NOD found into config.card and BATCH_NUM_PROC_TOT = ${BATCH_NUM_PROC_TOT} "
813	else
814	# with previous method.
815	if [ ${BATCH_NUM_PROC_TOT} -gt 1 ] ; then
816	# with more than 1 proc
817	if ( ${OK_PARA_MPMD} ) ; then
818	# with MPMD ie CPL/oasis method
819	echo "We will use default number of MPI tasks for this machine : "
820	echo "${DEFAULT_NUM_PROC_OCE} for OCE; "
821	echo "${DEFAULT_NUM_PROC_CPL} for CPL; "
822	echo "${DEFAULT_NUM_PROC_ATM} for ATM; "
823	echo "${DEFAULT_NUM_PROC_TOTAL} for the total number of processors; "
824	OK_PARA_MPI=true
825	CPL_PROC_MPI=${DEFAULT_NUM_PROC_CPL}
826	OCE_PROC_MPI=${DEFAULT_NUM_PROC_OCE}
827	ATM_PROC_MPI=${DEFAULT_NUM_PROC_ATM}
828	PROCESSUS_NUMBER=${DEFAULT_NUM_PROC_TOTAL}
829	else
830	# with have only one executable
831	echo "We will use ${BATCH_NUM_PROC_TOT} MPI tasks for ${CompNameFirst} : ${ExeNameFirst} "
832	OK_PARA_MPI=true
833	eval ${CompNameFirst}_PROC_MPI=${BATCH_NUM_PROC_TOT}
834	PROCESSUS_NUMBER=${BATCH_NUM_PROC_TOT}
835	fi
836	else
837	PROCESSUS_NUMBER=1
838	fi
839	fi
840	# Verification with PBS parameter
841	if [ ${BATCH_NUM_PROC_TOT} -ne ${PROCESSUS_NUMBER} ] ; then
842	echo "Warning with parallelization parameters !"
843	echo "Job header variable BATCH_NUM_PROC_TOT = ${BATCH_NUM_PROC_TOT} "
844	echo "is the total number of _processors_ reserved."
845	echo "It is not equal to the sum of _processus_ = ${PROCESSUS_NUMBER}."
846	echo "We stop now."
847	exit 1
848	fi
849	NUM_PROC_CPL=${CPL_PROC_MPI} # for backward compatibility
850	NUM_PROC_OCE=${OCE_PROC_MPI} # for backward compatibility
851	NUM_PROC_ATM=${ATM_PROC_MPI} # for backward compatibility
852	else # BATCH_NUM_PROC_TOT=""
853	if ( ${OK_PARA_MPI} ) ; then
854	echo "You have not given a value of ${BATCH_NUM_PROC_TOT} processors,"
855	echo "but you have filled the parallel parameter in config->Executable->list."
856	echo "Please add BATCH_NUM_PROC_TOT variable in job header."
857	echo "We stop now."
858	exit 1
859	else
860	# sequential case !
861	if [ ${PROCESSUS_NUMBER} -eq 0 ] ; then
862	(( PROCESSUS_NUMBER = 1 ))
863	echo "PROCESSUS_NUMBER is all 0 (sequential use of old definition in config->Executable->list)."
864	echo "We set it to 1."
865	fi
866	fi
867	fi
868
869	IGCM_debug_Print 1 "MPI/OMP treatment PROCESSUS_NUMBER = ${PROCESSUS_NUMBER}"
870
871	IGCM_sys_build_execution_scripts
872
873	ExecutionFail=false
874
875	IGCM_debug_PopStack "IGCM_config_PeriodStart"
876	}
877
878	#===================================
879	function IGCM_config_SaveSourceModifications
880	{
881	IGCM_debug_PushStack "IGCM_config_SaveSourceModifications"
882
883	typeset ExeOutDateMax listVarEnv
884	ExeOutDateMax=$1
885
886	listVarEnv="ExeOutDateMax,R_OUT_EXE,PREFIX,SUBMIT_DIR"
887	IGCM_sys_RshMaster "\
888	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh; \
889	export ExeOutDateMax=${ExeOutDateMax};\
890	export R_OUT_EXE=${R_OUT_EXE};\
891	export PREFIX=${PREFIX};\
892	export SUBMIT_DIR=${SUBMIT_DIR};\
893	export listVarEnv=${listVarEnv};\
894	Script_Output=out_SaveSourceModifications;\
895	IGCM_sys_Qsub ${libIGCM}/SaveSourceModifications.job ${ExeOutDateMax} ${R_OUT_EXE} ${PREFIX} ${SUBMIT_DIR}"
896
897	IGCM_debug_PopStack "IGCM_config_SaveSourceModifications"
898	}
899
900	#===================================
901	function IGCM_config_PeriodEnd
902	{
903	IGCM_debug_PushStack "IGCM_config_PeriodEnd"
904
905	echo
906	IGCM_debug_Print 1 "IGCM_config_PeriodEnd"
907	echo
908
909	#==================================#
910	# Save Job output #
911	#==================================#
912	if [ X${Pack} = Xtrue ] ; then
913	IGCM_sys_PutBuffer_Out ${Exe_Output} ${R_BUF_KSH}/${PREFIX}_${Exe_Output}
914	else
915	IGCM_sys_Put_Out ${Exe_Output} ${R_OUT_KSH}/${PREFIX}_${Exe_Output}
916	fi
917	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}"
918
919	IGCM_card_CheckConflict run.card
920	if [ X${Pack} = Xtrue ] ; then
921	IGCM_sys_PutBuffer_Out ${SUBMIT_DIR}/run.card ${R_BUF_KSH}/run.card rw
922	else
923	IGCM_sys_Put_Out ${SUBMIT_DIR}/run.card ${R_OUT_KSH}/run.card rw
924	fi
925
926	if [ ${DRYRUN} -le 1 ] ; then
927
928	IGCM_debug_Print 1 "Check components binary : size and creation date"
929
930	typeset LS_comp LS_bin ExeDate ExeCpuLog NextExeSize LastCompExeSize
931	typeset comp i
932	typeset ExeNameIn ExeNameOut UpdateExe ExeSecDateMax
933
934	#==================================#
935	# Get last Exe Size #
936	#==================================#
937
938	(( i=0 ))
939	if ( ${FirstInitialize} ) ; then
940	run_Log_LastExeSize=""
941	for comp in ${config_ListOfComponents[*]} ; do
942	run_Log_LastExeSize[$i]=0
943	(( i=i+1 ))
944	done
945	else
946	IGCM_card_DefineArrayFromOption ${SUBMIT_DIR}/run.card Log LastExeSize
947	fi
948	#==================================#
949	# And Build ExeDate #
950	#==================================#
951
952	# ExeDate = ATM_Jun_12_09:34-SRF_Jun_12_09:34-OCE_Jun_12_09:34-ICE_Jun_12_09:34-CPL_Jun_12_09:33
953	# Would be nice to have next line but no way to format ls output (need to ls -l --time-style "+%Y-%m-%dT%H:%M")
954	# ExeDate = ATM_2009-06-12T09:34+SRF_2009-06-12T09:34+OCE_2009-06-12T09:34+ICE_2009-06-12T09:34+CPL_2009-06-12T09:34
955	ExeDate=""
956	NextExeSize="( "
957	(( i=0 ))
958	UpdateExe=false
959	(( ExeSecDateMax = 0 ))
960	for comp in ${config_ListOfComponents[*]} ; do
961
962	IGCM_debug_Print 3 ${comp}
963
964	eval ExeNameIn=\${config_Executable_${comp}[0]}
965	eval ExeNameOut=\${config_Executable_${comp}[1]}
966	# Only if we really have an executable for the component :
967	if [ X${ExeNameIn} = X\"\" ] ; then
968	# If there is no exe file for this component
969	(( ExeSize=0 ))
970	else
971	eval LS_bin=${R_EXE}/${ExeNameIn}
972	IGCM_sys_FileSize ${LS_bin} ExeSize
973
974	set +A LS_comp -- $( LC_TIME=en_US ls -l ${LS_bin} )
975	if [ X${ExeDate} = X ] ; then
976	# First component exe date
977	ExeDate=${comp}_${LS_comp[5]}_${LS_comp[6]}
978	else
979	ExeDate=${ExeDate}-${comp}_${LS_comp[5]}_${LS_comp[6]}
980	fi
981	ExeDate=${ExeDate}_${LS_comp[7]}
982	fi
983
984	if [ ${i} -eq 0 ] ; then
985	# First component
986	NextExeSize="( "${ExeSize}
987	else
988	NextExeSize=${NextExeSize}", "${ExeSize}
989	fi
990	eval LastCompExeSize=${run_Log_LastExeSize[$i]}
991	(( i=i+1 ))
992
993	if [ ${ExeSize} -ne ${LastCompExeSize} ] ; then
994	if ( ${FirstInitialize} ) ; then
995	IGCM_debug_Print 1 "Save first ${ExeNameIn} in ${R_OUT_EXE} !"
996	else
997	IGCM_debug_Print 1 "${ExeNameIn} has changed in ${R_EXE} !"
998	IGCM_debug_Print 1 "Save latest ${ExeNameIn} in ${R_OUT_EXE} !"
999	eval FileToBeDeleted[${#FileToBeDeleted[@]}]=${ExeNameOut}
1000	fi
1001	eval IGCM_sys_Put_Out ${ExeNameOut} ${R_OUT_EXE}/${PREFIX}_${ExeNameIn} rw
1002	UpdateExe=true
1003
1004	# SD : switch off for now
1005	#IGCM_sys_GetDate_FichWork ${LS_bin} ExeSecDate
1006	#if [ $ExeSecDateMax -lt $ExeSecDate ] ; then
1007	# ExeSecDateMax=$ExeSecDate
1008	#fi
1009	fi
1010	done
1011
1012	# SD : switch off for now
1013	#if ( ${UpdateExe} ) ; then
1014	# echo "Launch SaveSourceModifications."
1015	# IGCM_config_SaveSourceModifications ${ExeSecDateMax}
1016	#fi
1017
1018	NextExeSize=${NextExeSize}" )"
1019	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Log LastExeSize "${NextExeSize}"
1020
1021	if [ ${DRYRUN} -le 1 ] ; then
1022	tail -1500 ${Exe_Output} > ${Exe_Output}_tail.txt
1023	ExeCpuLog=$( gawk -f ${libIGCM}/libIGCM_sys/IGCM_add_out.awk ${Exe_Output}_tail.txt )
1024	RET=$?
1025	if [ $RET -eq 0 ] ; then
1026	# ExeCpuLog variable contents 5 fields
1027	echo "${CumulPeriod} ${PeriodDateBegin} ${PeriodDateEnd} ${ExeCpuLog} ${ExeDate}" \| \
1028	gawk '{printf("# %11d \| %15s \| %15s \| %19s \| %19s \| %15.5f \| %15.5f \| %15.5f \| %s\n", \
1029	$1,$2,$3,$4,$5,$6,$7,$8,$9)}' >> ${SUBMIT_DIR}/run.card
1030	fi
1031	eval FileToBeDeleted[${#FileToBeDeleted[@]}]="${Exe_Output}_tail.txt"
1032	fi
1033
1034	fi
1035
1036	# All was right ? no ? then we stop.
1037	IGCM_debug_Verif_Exit
1038
1039	# If all was OK, we can delete all files not necessary for next Job
1040	echo
1041	IGCM_debug_Print 1 "Files that will be deleted before next period-run : "
1042
1043	if [ ${DRYRUN} -le 2 ] ; then
1044	for f in ${FileToBeDeleted[@]} ; do [ -f ${f} ] && ls -la $f ; [ -f ${f} ] && rm -f $f ; done
1045	else
1046	echo ${FileToBeDeleted[@]}
1047	fi
1048
1049	#=================================================#
1050	# Modification of libIGCM behaviour #
1051	#=================================================#
1052
1053	# To use this function, one must copy libIGCM.card from ${libIGCM} directory
1054	# and put it in ${SUBMIT_DIR} directory. After modifications of ${SUBMIT_DIR}/libIGCM.card,
1055	# variables define inside [UserChanges] will be modified for next Period of libIGCM main loop.
1056	if [ -f ${SUBMIT_DIR}/libIGCM.card ] ; then
1057	echo
1058	echo "########################################################################"
1059	echo "!!! Modification of libIGCM behaviour !!!"
1060	echo
1061
1062	IGCM_debug_Print 1 "DefineArrayFromOption : libIGCM_UserChanges in libIGCM.card"
1063	IGCM_card_DefineArrayFromSection ${SUBMIT_DIR}/libIGCM.card UserChanges
1064	IGCM_debug_Print 2 "libIGCM_UserChanges" ${libIGCM_UserChanges[*]}
1065
1066	# Special treatments for libIGCM internals
1067	for option in ${libIGCM_UserChanges[*]} ; do
1068	IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/libIGCM.card UserChanges ${option}
1069
1070	echo "We will change : ${option}."
1071	eval echo "Previous value : " \${${option}}
1072	eval echo "Change to : " \${libIGCM_UserChanges_${option}}
1073
1074	eval ${option}=\${libIGCM_UserChanges_${option}}
1075
1076	case ${option} in
1077	config_UserChoices_DateEnd)
1078	IGCM_debug_PrintVariables 1 config_UserChoices_DateEnd
1079	DateEnd=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )
1080
1081	# Period Length In Days between DateBegin and DateEnd
1082	(( ExperienceLengthInDays=$( IGCM_date_DaysBetweenGregorianDate ${DateEnd} ${DateBegin} ) + 1 ))
1083	if [ ${ExperienceLengthInDays} -lt 0 ] ; then
1084	IGCM_debug_Print 1 "Problem with dates in libIGCM.card : ${DateEnd} < ${DateBegin} ! You must check that."
1085	IGCM_debug_Exit "IGCM_PeriodEnd" " Wrong Dates."
1086	IGCM_debug_Verif_Exit
1087	fi
1088	;;
1089	config_UserChoices_PeriodLength)
1090	IGCM_debug_Print 1 "Change config_UserChoices_PeriodLength=${config_UserChoices_PeriodLength}"
1091	;;
1092	PeriodNb)
1093	IGCM_debug_Print 1 "Loop in main Job with ${PeriodNb} period(s)"
1094	;;
1095	config_Post_RebuildFrequency)
1096	IGCM_debug_Print 1 "Change config_Post_RebuildFrequency=${config_Post_RebuildFrequency} : IGCM_post_Configure"
1097	IGCM_post_Configure
1098	;;
1099	config_Post_TimeSeriesFrequency)
1100	IGCM_debug_Print 1 "Change config_Post_TimeSeriesFrequency = ${config_Post_TimeSeriesFrequency} : IGCM_post_Configure"
1101	IGCM_post_Configure
1102	;;
1103	config_Post_SeasonalFrequency)
1104	IGCM_debug_Print 1 "Change config_Post_SeasonalFrequency = ${config_Post_SeasonalFrequency} : IGCM_post_Configure"
1105	IGCM_post_Configure
1106	;;
1107	esac
1108	done
1109
1110	echo
1111	echo "########################################################################"
1112	echo
1113	fi
1114
1115	#=================================================#
1116	# Determine next computed period #
1117	#=================================================#
1118
1119	PeriodDateBegin=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateEnd} 1 )
1120	IGCM_date_GetYearMonthDay $PeriodDateBegin year month day
1121	year_m1=$(( year - 1 ))
1122	year_p1=$(( year + 1 ))
1123	IGCM_config_Analyse_PeriodLength
1124	PeriodDateEnd=$( IGCM_date_AddDaysToGregorianDate ${PeriodDateBegin} $(( ${PeriodLengthInDays} - 1 )) )
1125
1126	# Debug Print :
1127	echo
1128	IGCM_debug_Print 1 "IGCM_config_PeriodEnd : Preparing Next Execution"
1129	IGCM_debug_Print 1 "PeriodDateBegin : ${PeriodDateBegin}"
1130	IGCM_debug_Print 1 "PeriodDateEnd : ${PeriodDateEnd}"
1131	IGCM_debug_Print 1 "PeriodLengthInDays : ${PeriodLengthInDays}"
1132
1133	PeriodDateBegin=$( IGCM_date_ConvertFormatToHuman ${PeriodDateBegin} )
1134	PeriodDateEnd=$( IGCM_date_ConvertFormatToHuman ${PeriodDateEnd} )
1135
1136	(( CumulPeriod = CumulPeriod + 1 ))
1137
1138	# Debug Print :
1139	echo
1140	IGCM_debug_Print 3 "PeriodDateBegin Human : ${PeriodDateBegin}"
1141	IGCM_debug_Print 3 "PeriodDateEnd Human : ${PeriodDateEnd}"
1142	IGCM_debug_Print 3 "CumulPeriod : ${CumulPeriod}"
1143
1144	#=================================================#
1145	# Write updated run.card #
1146	#=================================================#
1147
1148	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration OldPrefix ${OldPrefix}
1149	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateBegin ${PeriodDateBegin}
1150	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodDateEnd ${PeriodDateEnd}
1151	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration CumulPeriod ${CumulPeriod}
1152
1153	if ( ${FirstInitialize} ) ; then
1154	# It's no more the first time
1155	FirstInitialize=false
1156	fi
1157
1158	IGCM_debug_PopStack "IGCM_config_PeriodEnd"
1159	}
1160
1161	#===================================
1162	function IGCM_config_Finalize
1163	{
1164	IGCM_debug_PushStack "IGCM_config_Finalize"
1165
1166	echo
1167	IGCM_debug_Print 1 "IGCM_config_Finalize"
1168	echo
1169
1170	if [ ${SimulationLengthInDays} -ge ${ExperienceLengthInDays} ] ; then
1171	#==========================#
1172	# End of entire simulation #
1173	#==========================#
1174
1175	# Mail notification
1176	IGCM_sys_SendMail
1177
1178	#
1179	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "Completed"
1180	IGCM_debug_Print 1 "Normal End of computation."
1181	if ( $DEBUG_debug ) ; then
1182	echo
1183	IGCM_debug_Print 1 "Your files on ${R_SAVE} :"
1184	IGCM_sys_Tree ${R_SAVE}
1185
1186	IGCM_debug_Print 1 "Your files on ${R_BUFR} :"
1187	IGCM_sys_Tree ${R_BUFR}
1188	fi
1189	else
1190	#=================#
1191	# Submit next job #
1192	#=================#
1193
1194	IGCM_card_WriteOption ${SUBMIT_DIR}/run.card Configuration PeriodState "OnQueue"
1195
1196	# Name of next Ksh Script output :
1197	eval Script_Output="${Script_Output_Prefix}_${config_UserChoices_JobName}.$( printf "%06d" ${CumulPeriod} )"
1198
1199	IGCM_debug_Print 1 "Submit next job"
1200	# SUBMIT NEXT JOB from SUBMIT_DIR and come back in RUN_DIR
1201	IGCM_sys_Cd ${SUBMIT_DIR}
1202	# Keep only the 5 latest ${Script_Output_Prefix}_${config_UserChoices_JobName}
1203	ScriptTot=$( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? 2>/dev/null \| wc -l )
1204	[ ${ScriptTot} -gt 5 ] && rm -f $( ls ${Script_Output_Prefix}_${config_UserChoices_JobName}.?????? \| head -$(( ${ScriptTot} - 5 )) )
1205	# Submit next job and come back
1206	IGCM_sys_Qsub ${SUBMIT_DIR}/Job_${config_UserChoices_JobName}
1207	IGCM_sys_Cd -
1208	fi
1209
1210	# Supress Non Deleted Boundary files
1211	#if [ ${DRYRUN} -le 2 ] ; then
1212	# IGCM_comp_DelFixeBoundaryFiles
1213	# ls -la
1214	#fi
1215
1216	# Clean ${RUN_DIR}=${RUN_DIR_PATH}/${config_UserChoices_JobName}.${$}
1217	# Only for production run (No clean up in DEV or DEB mode)
1218	# and command sent from .. directory.
1219	IGCM_sys_Cd ..
1220	[ X${JobType} = XRUN ] && IGCM_sys_Rm -rf ${RUN_DIR}
1221
1222	IGCM_debug_PopStack "IGCM_config_Finalize"
1223	}
1224
1225	#===================================

Note: See TracBrowser for help on using the repository browser.

Context Navigation

source: trunk/libIGCM/libIGCM_config/libIGCM_config.ksh @ 804

Download in other formats: