Context Navigation

AA_create_ts @ 1488

Last change on this file since 1488 was 1468, checked in by mafoipsl, 6 years ago

On curie : set output files for all jobs.

On irene :

set output files for all jobs like curie
add the possibility to use xlarge nodes (for free) for post-processing or skylake nodes
add a question to set project used for post-processing and delete -A option in libIGCM_sys_irene
display only available projects for computing or post-processing on irene

Property licence set to
The following licence information concerns ONLY the libIGCM tools
==================================================================

Copyright © Centre National de la Recherche Scientifique CNRS
Commissariat à l'Énergie Atomique CEA

libIGCM : Library for Portable Models Computation of IGCM Group.

IGCM Group is the french IPSL Global Climate Model Group.

This library is a set of shell scripts and functions whose purpose is
the management of the initialization, the launch, the transfer of
output files, the post-processing and the monitoring of datas produce
by any numerical program on any plateforme.

This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".

As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.

In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.

The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
Property svn:keywords set to Revision Author Date

File size: 30.6 KB

Line
1	#-Q- curie ######################
2	#-Q- curie ## CURIE TGCC/CEA ##
3	#-Q- curie ######################
4	#-Q- curie #MSUB -r TS # Nom du job
5	#-Q- curie #MSUB -o TS.out_%I
6	#-Q- curie #MSUB -e TS.out_%I
7	#-Q- curie #MSUB -n 1 # Reservation du processus
8	#-Q- curie #MSUB -T 80000 # Limite de temps elapsed du job
9	#-Q- curie #MSUB -q ::default_node::
10	#-Q- curie #MSUB -c ::default_core::
11	#-Q- curie #MSUB -Q normal
12	#-Q- curie #MSUB -A ::default_project::
13	#-Q- curie set +x
14	#-Q- irene ######################
15	#-Q- irene ## IRENE TGCC/CEA ##
16	#-Q- irene ######################
17	#-Q- irene #MSUB -r TS # Job name
18	#-Q- irene #MSUB -o TS.out_%I
19	#-Q- irene #MSUB -e TS.out_%I
20	#-Q- irene #MSUB -n 1 # Number of cores
21	#-Q- irene #MSUB -T 80000 # Maximum elapsed time
22	#-Q- irene #MSUB -q ::default_node::
23	#-Q- irene #MSUB -c ::default_core::
24	#-Q- irene #MSUB -Q normal
25	#-Q- irene #MSUB -A ::default_post_project::
26	#-Q- irene #MSUB -m store,work,scratch
27	#-Q- irene set +x
28	#-Q- ada #!/bin/ksh
29	#-Q- ada #######################
30	#-Q- ada ## ADA IDRIS ##
31	#-Q- ada #######################
32	#-Q- ada # @ job_type = mpich
33	#-Q- ada # @ requirements = (Feature == "prepost")
34	#-Q- ada # Temps Elapsed max. d'une requete hh:mm:ss
35	#-Q- ada # @ wall_clock_limit = 20:00:00
36	#-Q- ada # Nom du travail LoadLeveler
37	#-Q- ada # @ job_name = TS
38	#-Q- ada # Fichier de sortie standard du travail
39	#-Q- ada # @ output = $(job_name).$(jobid)
40	#-Q- ada # Fichier de sortie d'erreur du travail
41	#-Q- ada # @ error = $(job_name).$(jobid)
42	#-Q- ada # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
43	#-Q- ada # @ notification = error
44	#-Q- ada # @ environment = $DEBUG_debug ; $BigBrother ; $postProcessingStopLevel ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $POST_DIR ; $Script_Post_Output ; $MASTER ; $RebuildFrequency ; $DateBegin ; $PeriodDateEnd ; $StandAlone ; $CompletedFlag ; $TsTask ; $CompToRead ; $FlagToRead ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; wall_clock_limit=$(wall_clock_limit)
45	#-Q- ada # @ queue
46	#-Q- lxiv8 ######################
47	#-Q- lxiv8 ## OBELIX LSCE ##
48	#-Q- lxiv8 ######################
49	#-Q- lxiv8 #PBS -N TS
50	#-Q- lxiv8 #PBS -m a
51	#-Q- lxiv8 #PBS -j oe
52	#-Q- lxiv8 #PBS -q medium
53	#-Q- lxiv8 #PBS -o TS.$$
54	#-Q- lxiv8 #PBS -S /bin/ksh
55	#-Q- ifort_CICLAD ######################
56	#-Q- ifort_CICLAD ## CICLAD IPSL ##
57	#-Q- ifort_CICLAD ######################
58	#-Q- ifort_CICLAD #PBS -N TS
59	#-Q- ifort_CICLAD #PBS -m a
60	#-Q- ifort_CICLAD #PBS -j oe
61	#-Q- ifort_CICLAD ###PBS -q h12 # Queue for 12 hours at ciclad only
62	#-Q- ifort_CICLAD #PBS -S /bin/ksh
63	#-Q- default #!/bin/ksh
64	#-Q- default ##################
65	#-Q- default ## DEFAULT HOST ##
66	#-Q- default ##################
67
68	#**************************************************************
69	# Author: Sebastien Denvil
70	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
71	# $Revision:: $ Revision of last commit
72	# $Author:: $ Author of last commit
73	# $Date:: $ Date of last commit
74	# IPSL (2006)
75	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
76	#
77	#**************************************************************
78
79	#set -eu
80	#set -vx
81
82	date
83
84	#-Q- ada export OMP_NUM_THREADS=1
85
86	#D- Task type DO NOT CHANGE (computing, post-processing or checking)
87	TaskType=post-processing
88
89	########################################################################
90
91	#D- Flag to determine if this job in a standalone mode
92	#D- Default : value from AA_job if any
93	StandAlone=${StandAlone:=true}
94
95	#D- Path to libIGCM
96	#D- Default : value from AA_job if any
97	libIGCM=${libIGCM:=::modipsl::/libIGCM}
98
99	#D- Flag to determine allready produced time series. Empty if you start from the beginning
100	#D- Default : Value from AA_job if any. Usefull in StandAlone case if you want to continue a time series
101	CompletedFlag=${CompletedFlag:=}
102
103	#D- Flag to determine task type this script will perform.
104	#D- Possible Value : 2D, 3D, Chunck2D, Chunck3D
105	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
106	TsTask=${TsTask:=2D}
107	if [ X${TsTask} = Xempty ] ; then
108	TsTask=""
109	fi
110
111	#D- Flag to determine if rebuild process has been performed asynchronously
112	#D- Possible value true or false.
113	#D- If true raw files has allready been patched by rebuild job
114	#D- If false create_ts will patch the raw files
115	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
116	RebuildFrequency=${RebuildFrequency:=true}
117
118	#D- Flag to determine job's output directory
119	#D- Default : value from libIGCM_post.ksh if any
120	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
121
122	#D- Increased verbosity (1, 2, 3)
123	#D- Default : value from AA_job if any
124	Verbosity=${Verbosity:=3}
125
126	#D- Low level debug : to bypass lib test checks and stack construction
127	#D- Default : value from AA_job if any
128	DEBUG_debug=${DEBUG_debug:=false}
129
130	#D- TEMPORARY Flag to determine atmospheric resolution
131	#D- Default : value from atmospheric driver if any
132	RESOL_ATM=ALL
133
134	#D- Flag to determine surface resolution
135	#D- Default : value from surface driver if any
136	RESOL_SRF=ALL
137
138	#D- Flag to determine surface resolution
139	#D- Default : value from surface driver if any
140	RESOL_SBG=ALL
141
142	#D- TEMPORARY Flag to determine ocean resolution
143	#D- Default : value from ocean driver if any
144	RESOL_OCE=${RESOL_OCE:=ORCA2}
145
146	#D- TEMPORARY Flag to determine ice resolution
147	#D- Default : value from ice driver if any
148	RESOL_ICE=${RESOL_OCE:=ORCA2}
149
150	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
151	#D- Default : value from ice driver if any
152	RESOL_MBG=${RESOL_OCE:=ORCA2}
153
154	########################################################################
155
156	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
157	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
158	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
159	#-------
160	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
161	. ${libIGCM}/libIGCM_config/libIGCM_config.ksh
162	#-------
163	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
164	IGCM_sys_Cd ${RUN_DIR_PATH}
165	#-------
166	( ${DEBUG_debug} ) && IGCM_debug_Check
167	( ${DEBUG_debug} ) && IGCM_card_Check
168	( ${DEBUG_debug} ) && IGCM_date_Check
169
170	########################################################################
171
172	#set -vx
173
174	# Useful cleaning function
175	MENAGE ()
176	{
177	typeset ExistFlag
178	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1/$3 ] && echo 0 \|\| echo 1" )
179	[ ${ExistFlag} -eq 0 ] && ( IGCM_sys_RshArchive rm -f $1/$2 ;)
180	}
181
182	########################################################################
183
184	# ------------------------------------------------------------------
185	# Test if all was right before proceeding further
186	# ------------------------------------------------------------------
187	IGCM_debug_Verif_Exit
188
189	if [ ${StandAlone} = true ] ; then
190	CARD_DIR=${SUBMIT_DIR}
191	else
192	CARD_DIR=${RUN_DIR_PATH}
193
194	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
195
196	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
197
198	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
199
200	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
201	fi
202
203	#==================================
204	# First of all
205	#
206	# Read libIGCM compatibility version in config.card
207	# Read UserChoices section
208	# Read Ensemble section
209	# Read Post section
210	# Define all netcdf output directories
211	#==================================
212	IGCM_config_CommonConfiguration ${CARD_DIR}/config.card
213
214	# ------------------------------------------------------------------
215	# Activate BigBrother so as to supervise this job
216	# ------------------------------------------------------------------
217	IGCM_debug_BigBro_Initialize
218
219	#==================================
220	# Define default value to keep compatibility with previous card: means before changes due to TGCC
221	if [ X${PackDefault} = Xtrue ] ; then
222	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
223	else
224	config_Post_PackFrequency=NONE
225	fi
226
227	#==================================
228	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
229	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
230	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
231	fi
232
233	#
234	# Determine component to take care of depending on kind of task create_ts will perform
235	#
236	[ ${CompToRead} ] && set -A config_ListOfComponents ${CompToRead} \|\| IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
237	#
238	# Determine period for time series. Default : value from AA_job or from launch_TS if any
239	#
240	[ X${CompletedFlag} = X ] \|\| CompletedFlag=$( IGCM_date_ConvertFormatToGregorian ${CompletedFlag} )
241
242	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
243	#
244	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
245	#
246	# Determine Dimension of the produced time series : 2D or 3D
247	if ( [ X${TsTask} = XChunck2D ] \|\| [ X${TsTask} = X2D ] ) ; then
248	Dimension=2D
249	elif ( [ X${TsTask} = XChunck3D ] \|\| [ X${TsTask} = X3D ] ) ; then
250	Dimension=3D
251	else
252	Dimension=""
253	fi
254
255	# ------------------------------------------------------------------
256	#D- Test if all was right before entering the loop
257	# ------------------------------------------------------------------
258	IGCM_debug_Verif_Exit
259
260	########################################################################
261	# Depending on requested time series fill following variables :
262	#
263	# - LISTE_FILE_${comp}[*]
264	# - LISTE_VARS_${file}[*]
265	# - LISTE_AXIS_${file}[*]
266	# - LISTE_PATCH_${file}[*]
267	########################################################################
268
269	#
270	# For each selected component determine which files need post-processing
271	#
272	DoJob=false
273	for comp in ${config_ListOfComponents[*]} ; do
274	#
275	ActiveComp=false
276	#
277	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
278	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
279	#
280	card=${CARD_DIR}/COMP/${compname}.card
281
282	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
283	#
284	ListFilesName=${compname}_OutputFiles_List
285	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
286	#
287	if [ X${FileName0} != X${NULL_STR} ] ; then
288	#
289	#IGCM_debug_Print 1 "Component : ${compname}"
290	#
291	# INITIALISATION
292	#
293	typeset i
294	if [ ${FlagToRead} ] ; then
295	((NbFiles=${FlagToRead}))
296	((i=${NbFiles}))
297	else
298	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
299	i=2
300	fi
301	#
302	until [ $i -gt $NbFiles ]; do
303	#
304	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
305	#
306	if [ X${flag_post} != XNONE ] ; then
307	#
308	# For each comp and each file who needs post-processing
309	# determine which variables need time series.
310	#
311	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
312	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} TimeSeriesVars${Dimension}
313	IGCM_card_DefineVariableFromOption ${card} Post_${FILE} ChunckJob${Dimension}
314	#
315	# If TimeSeriesVars list is empty we skip
316	#
317	if [ $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}} ) = ${NULL_STR} ] ; then
318	#
319	IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
320	IGCM_debug_Print 3 "TimeSeriesVars${Dimension} for ${FILE} is empty"
321	(( i=i+3 ))
322	continue
323	#
324	fi
325	#
326	# If files/variables has been switch off by ChunckJob${Dimension}=0 we skip
327	#
328	if [ X$( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = XOFF ] ; then
329	IGCM_debug_Print 3 "Nothing to do for ${FILE} according to ${card}"
330	IGCM_debug_Print 3 "ChunckJob${Dimension}=OFF for ${FILE} so it has been switched off"
331	(( i=i+3 ))
332	continue
333	fi
334	#
335	# If we are not a Chunck type task, we skip files/variables handled by chunck task
336	#
337	if ( [ ! X${TsTask} = XChunck${Dimension} ] && [ ! X${TsTask} = X ] && [ ! XChunck${Dimension} = XChunck ] ) ; then
338	if [ ! $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} ) = NONE ] ; then
339	IGCM_debug_Print 3 "Nothing to do for ${FILE} TimeSeriesVars${Dimension} according to ${card}"
340	IGCM_debug_Print 3 "Let another chunck task do the job $( eval echo \${${compname}_Post_${FILE}_ChunckJob${Dimension}} )"
341	(( i=i+3 ))
342	continue
343	fi
344	fi
345	#
346	# Now on we know we have things to do
347	#
348
349	#
350	# We need coordinates axis to properly fill produced time series
351	#
352	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} GatherWithInternal
353	#
354	if [ $( eval echo \${${compname}_Post_${FILE}_GatherWithInternal} ) = ${NULL_STR} ] ; then
355	#
356	IGCM_debug_Print 1 "ERROR !! We miss coordinate axis for ${FILE} according to ${card}"
357	IGCM_debug_Print 1 "So no time series will be produced for ${FILE} !!"
358	(( i=i+3 ))
359	continue
360	#
361	fi
362	#
363	# We need LIST of variables not allready produced (useful for standalone mode)
364	#
365	FlagDir=$( echo ${FILE} \| awk -F "_" '{print $1}' )
366	case ${FlagDir} in
367	*Y) TS_Dir=TS_YE ;;
368	*M) TS_Dir=TS_MO ;;
369	*D) TS_Dir=TS_DA ;;
370	HF) TS_Dir=TS_HF ;;
371	3H) TS_Dir=TS_HF ;;
372	INS) TS_Dir=TS_INS ;;
373	esac
374	#
375	count=0
376	for var in $( eval echo \${${compname}_Post_${FILE}_TimeSeriesVars${Dimension}[*]} ) ; do
377	#
378	DIRECTORY=${R_SAVE}/${comp}/Analyse/${TS_Dir}
379	TestedFile=${config_UserChoices_JobName}_${DateBegin}_${PeriodDateEnd}_${FlagDir}_${var}.nc
380	#
381	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
382	if [ ! $? = 0 ] ; then
383	eval set +A LISTE_VARS_${FILE} \${LISTE_VARS_${FILE}[*]} ${var}
384	ActiveComp=true
385	(( count = count + 1 ))
386	fi
387	done
388	#
389	if [ ${count} -gt 0 ] ; then
390	#
391	eval set +A LISTE_FILE_${comp} \${LISTE_FILE_${comp}[*]} ${FILE}
392	eval set +A LISTE_AXIS_${FILE} \${${compname}_Post_${FILE}_GatherWithInternal[*]}
393	#
394	if [ ! X${RebuildFrequency} = Xtrue ] ; then
395	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
396	eval set +A LISTE_PATCHES_${FILE} \${${compname}_Post_${FILE}_Patches[*]}
397	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
398	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
399	if [ Xload${Patch} != X ] ; then
400	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
401	eval load${Patch}=loaded
402	fi
403	done
404	fi
405	fi
406	fi
407	fi
408	(( i=i+3 ))
409	done
410	fi
411	#
412	if [ ${ActiveComp} = true ] ; then
413	set +A ActiveListOfComponents ${ActiveListOfComponents[*]} ${comp}
414	fi
415	done
416
417	set -A config_ListOfComponents ${ActiveListOfComponents[*]}
418	[ ! -z "${ActiveListOfComponents}" ] && DoJob=true
419
420	########################################################################
421	#
422	# IS THERE SOME ALLREADY PRODUCED TIME SERIES ?
423	# IF SO BRING THEM IN THE WORKING DIRECTORY
424	#
425	# PS : Keep in mind that IGCM_sys_Get here is a weak link :
426	# - especially for IDRiS
427	# - no special protection against "command too long" !
428	# - libIGCM_sys should handle that or modify following code part
429	#
430	########################################################################
431
432	if [ ${StandAlone} != true ] ; then
433	Running_Flag=$( IGCM_sys_RshMaster "IGCM_card_DefineVariableFromOption \
434	${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning ; \
435	echo \${run_PostProcessing_TimeSeriesRunning}" )
436
437	#if [ X${Running_Flag} = Xy ] ; then
438	# # Time Series Job allready running
439	# IGCM_debug_Print 1 "Time Series Job allready running exit"
440	# exit
441	#fi
442	# Now run.card know we are running
443
444
445
446
447	IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "y"
448
449
450
451
452
453	# Get information from last execution
454	#CompletedFlag=$( IGCM_sys_RshMaster \
455	# "IGCM_card_DefineVariableFromOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted ;\
456	# echo \${run_PostProcessing_TimeSeriesCompleted}" )
457	fi
458
459	# Is it the first submission or not ?
460
461	if ( [ X${CompletedFlag} = X ] \|\| [ X${CompletedFlag} = X$( IGCM_date_AddDaysToGregorianDate ${DateBegin} -1 ) ] ) ; then
462	# First Time Series Submission
463	FIRST_PASS=TRUE
464	Length=$( IGCM_date_DaysInPreviousPeriod ${DateBegin} ${config_UserChoices_PeriodLength} begin)
465	DATE_FIN_JOB_B=$( IGCM_date_AddDaysToGregorianDate ${DateBegin} $(( Length - 1 )) )
466	else
467	# Last Completed Time Series
468	FIRST_PASS=FALSE
469	DATE_FIN_JOB_B=${CompletedFlag}
470	date=${DateBegin}_${DATE_FIN_JOB_B}
471	#
472	for comp in ${config_ListOfComponents[*]} ; do
473	#
474	IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
475	#
476	IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
477	#
478	unset liste_file
479	#
480	file_path=${R_SAVE}/${comp}/Analyse
481	#
482	i=0
483	file=${NULL_STR}
484	#
485	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
486	#
487	# Determine in which output can we find file
488	# IE : Analyse/TS or Analyse/DA2TS or ...
489	#
490	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
491	case ${FlagDir} in
492	*Y) TS_Dir=TS_YE ;;
493	*M) TS_Dir=TS_MO ;;
494	*D) TS_Dir=TS_DA ;;
495	HF) TS_Dir=TS_HF ;;
496	3H) TS_Dir=TS_HF ;;
497	INS) TS_Dir=TS_INS ;;
498	esac
499	for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
500	#
501	IGCM_sys_TestFileArchive ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
502	if [ $? = 0 ] ; then
503	liste_file[${#liste_file[*]}]=${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc
504	else
505	IGCM_debug_Print 1 "WARNING ${file_path}/${TS_Dir}/${config_UserChoices_JobName}_${date}_${FlagDir}_${var}.nc dont exist"
506	fi
507	#
508	done
509	done
510	if [ X${file} != X${NULL_STR} ] ; then
511	IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
512	IGCM_debug_Verif_Exit
513	fi
514	done
515	#
516	LEVEL=0
517	#
518	fi
519
520	########################################################################
521	#
522	# DEFINE LOOP PARAMETERS
523	#
524	########################################################################
525
526	DATE_COUNT=${DATE_FIN_JOB_B}
527	DATE_FIN_JOB_B_LOOP=${DATE_FIN_JOB_B}
528
529	( [ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] ) && NBRE_FILE_TOT=0 \|\| NBRE_FILE_TOT=1
530
531	if [ ${DoJob} = true ] ; then
532	while [ ${DATE_COUNT} -lt ${PeriodDateEnd} ] ; do
533	(( NBRE_FILE_TOT = NBRE_FILE_TOT + 1 ))
534	Length=$( IGCM_date_DaysInNextPeriod ${DATE_COUNT} ${config_UserChoices_PeriodLength} )
535	DATE_COUNT=$( IGCM_date_AddDaysToGregorianDate ${DATE_COUNT} ${Length} )
536	done
537	else
538	NBRE_FILE_TOT=0
539	fi
540
541	if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
542	NBRE_TOUR=1
543	elif [ ${NBRE_FILE_TOT}%${UNIX_MAX_LIMIT} -eq 0 ] ; then
544	let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}
545	else
546	let NBRE_TOUR=${NBRE_FILE_TOT}/${UNIX_MAX_LIMIT}+1
547	fi
548
549	CURRENT_LOOP=1
550	# THIS LOOP GIVES PARAMETERS FOR THE DATE'S LOOP ie : DATE_COURANTE, DATE_FIN.
551	# WE HAVE 3 LEVELS
552	# - LEVEL 1 : JUST ONE LOOP TO COMPLETE
553	# - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
554	# - LEVEL 3 : WE ARE IN THE LAST LOOP
555
556	while [ ${CURRENT_LOOP} -le ${NBRE_TOUR} ] ; do
557
558	if [ ${NBRE_FILE_TOT} -le ${UNIX_MAX_LIMIT} ] ; then
559	# - LEVEL 1 : JUST ONE LOOP TO COMPLETE
560	NBRE_FILE_LOOP=${NBRE_FILE_TOT}
561
562	if [ ${FIRST_PASS} = TRUE ] ; then
563	DATE_COURANTE=${DATE_FIN_JOB_B_LOOP}
564	else
565	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
566	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
567	fi
568
569	DATE_FIN=${PeriodDateEnd}
570	DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP}
571
572	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
573	[ -n "${LEVEL}" ] && FLAG_B="TRUE" \|\| FLAG_B="FALSE"
574
575	LEVEL=1
576	DEPOT="TRUE"
577	elif [ ${CURRENT_LOOP} -ne ${NBRE_TOUR} ] ; then
578	# - LEVEL 2 : WE ARE IN THE "DEEP" LOOP
579	NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
580
581	if [ ! ${FIRST_PASS} = TRUE ] && [ ${CURRENT_LOOP} -eq 1 ] ; then
582	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN_JOB_B_LOOP} ${config_UserChoices_PeriodLength} )
583	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN_JOB_B_LOOP} ${Length} )
584	elif [ -z "${DATE_COURANTE}" ] ; then
585	DATE_COURANTE=${DATE_FIN_JOB_B}
586	else
587	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
588	DATE_COURANTE=$( IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
589	fi
590
591	(( TotLength = 0 ))
592	COMPTEUR=1
593	DATE_LOOP=${DATE_COURANTE}
594
595	while [ ${COMPTEUR} -lt ${NBRE_FILE_LOOP} ] ; do
596	#
597	Length=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
598	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length} )
599	(( TotLength = TotLength + Length ))
600	(( COMPTEUR = COMPTEUR + 1 ))
601	#
602	done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
603	DATE_FIN=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} ${TotLength} )
604
605	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
606	[ -n "${LEVEL}" ] && [ ${LEVEL} -eq 0 ] && FLAG_B="TRUE" \|\| [ ${CURRENT_LOOP} -gt 1 ] && FLAG_B="TRUE" \|\| FLAG_B="FALSE"
607	DEPOT="FALSE"
608	LEVEL=2
609	else
610	# - LEVEL 3 : WE ARE IN THE LAST LOOP
611	NBRE_FILE_LOOP=$(( ${NBRE_FILE_TOT} % ( ${UNIX_MAX_LIMIT} * ( ${CURRENT_LOOP} - 1 ) ) ))
612	[ ${NBRE_FILE_LOOP} -eq 0 ] && NBRE_FILE_LOOP=${UNIX_MAX_LIMIT}
613
614	Length=$( IGCM_date_DaysInNextPeriod ${DATE_FIN} ${config_UserChoices_PeriodLength} )
615	DATE_COURANTE=$(IGCM_date_AddDaysToGregorianDate ${DATE_FIN} ${Length} )
616	DATE_FIN=${PeriodDateEnd}
617	[ -n "${DATE_LOOP}" ] && DATE_FIN_JOB_B_LOOP_PREC=${DATE_FIN_JOB_B_LOOP} ; DATE_FIN_JOB_B_LOOP=${DATE_LOOP}
618
619	LEVEL=3
620	DEPOT="TRUE"
621	FLAG_B="TRUE"
622	fi
623
624	# THEN FOR EACH FILE TYPE WE LOOP BETWEEN DATE_COURANTE AND DATE_FIN.
625	for comp in ${config_ListOfComponents[*]} ; do
626	#
627	IGCM_sys_Mkdir ${RUN_DIR_PATH}/${comp}
628	#
629	IGCM_sys_Cd ${RUN_DIR_PATH}/${comp}
630	#
631	eval R_OUT_${comp}=${R_SAVE}/${comp}
632	#
633	i=0
634	#
635	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
636	#
637	# Determine in which output we can find file
638	# IE : Output/MO or Output/DA or ...
639	#
640	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
641	case ${FlagDir} in
642	*Y) TS_Dir=TS_YE ; FreqDir=YE ;;
643	*M) TS_Dir=TS_MO ; FreqDir=MO ;;
644	*D) TS_Dir=TS_DA ; FreqDir=DA ;;
645	HF) TS_Dir=TS_HF ; FreqDir=HF ;;
646	3H) TS_Dir=TS_HF ; FreqDir=HF ;;
647	INS) TS_Dir=TS_INS ; FreqDir=INS ;;
648	*) IGCM_debug_Print 1 "Error in FlagDir=${FlagDir} for file ${file}."
649	IGCM_debug_Exit "Job create_ts" ;;
650	esac
651
652	# Initialize array
653	unset liste_file
654	unset liste_file_tmp
655	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
656
657	COMPTEUR=1
658	MissingFile=FALSE
659	DATE_LOOP=${DATE_COURANTE}
660
661	while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] ; do
662	#
663	Length1=$( IGCM_date_DaysInPreviousPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} end)
664	DATE_TAB=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} $(( 1 - Length1 )) )_${DATE_LOOP}
665
666	Length2=$( IGCM_date_DaysInNextPeriod ${DATE_LOOP} ${config_UserChoices_PeriodLength} )
667	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} ${Length2} )
668
669	IGCM_sys_TestFileArchive ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
670	if [ $? = 0 ] ; then
671	liste_file[${#liste_file[*]}]=${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
672	liste_file_tmp[${#liste_file_tmp[*]}]=${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc
673	else
674	IGCM_debug_Print 1 "WARNING ${file_path}${config_UserChoices_JobName}_${DATE_TAB}_${file}.nc dont exist"
675	MissingFile=TRUE
676	fi
677	(( COMPTEUR = COMPTEUR + 1 ))
678	#
679	done # while [ ${COMPTEUR} -le ${NBRE_FILE_LOOP} ] done
680
681	# We skip this file type if one of them is missing.
682	if [ "${MissingFile}" = "TRUE" ] ; then
683	IGCM_debug_Print 1 "WARNING We skip ${file} type processing"
684	continue
685	fi
686
687	DATE_LOOP=$( IGCM_date_AddDaysToGregorianDate ${DATE_LOOP} -${Length2} )
688
689	# Get selected files
690	IGCM_sys_Get /l liste_file[*] ${RUN_DIR_PATH}/${comp}
691	IGCM_debug_Verif_Exit
692
693	# Apply IGCM_Patch if needed
694	if [ ! X${RebuildFrequency} = Xtrue ] ; then
695	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
696	for file_t in $( eval echo ${liste_file_tmp[*]} ); do
697	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
698	IGCM_${Patch} ${file_t}
699	IGCM_debug_Verif_Exit
700	done
701	done
702	fi
703	fi
704
705	# WE CAN CONCATENATE FILES AT THIS POINT
706	[ "${FLAG_B}" = "TRUE" ] && DATE_BUILD_B=${DateBegin}_${DATE_FIN_JOB_B_LOOP_PREC}
707	Length=$( IGCM_date_DaysInPreviousPeriod ${DATE_COURANTE} ${config_UserChoices_PeriodLength} end)
708	DATE_BUILD1=$( IGCM_date_AddDaysToGregorianDate ${DATE_COURANTE} -$(( Length - 1 )) )
709	DATE_BUILD=${DATE_BUILD1}_${DATE_FIN}
710	DATE_BUILD_END=${DateBegin}_${DATE_FIN}
711
712	liste_coord=" "
713	for axis in $( eval echo \${LISTE_AXIS_${file}[*]} ); do
714	liste_coord=${liste_coord}${axis}","
715	done
716
717	# Time axis must be the last dimension
718	time_axis=${axis}
719
720	# SWITCH BETWEEN MODEL TO FIT PARTICULAR CASE
721	for var in $( eval echo \${LISTE_VARS_${file}[*]} ) ; do
722	#
723	# We skip variables not in the first file of the list.
724	# Seems overkill to test thel all... but will do if needed
725	ncdump -hv ${var} ${liste_file_tmp[0]} > /dev/null 2>&1
726	if [ ! $? = 0 ] ; then
727	IGCM_debug_Print 1 "WARNING We skip ${var}, not present within ${liste_file_tmp[0]}"
728	continue
729	fi
730	#
731	if [ "${FLAG_B}" = "TRUE" ] ; then
732	# WE CONCATENATE WITH EXISTING FILES
733	file1=${config_UserChoices_JobName}_${DATE_BUILD_B}_${FlagDir}_${var}.nc
734	file_out=${config_UserChoices_JobName}_${DATE_BUILD_END}_${FlagDir}_${var}.nc
735	IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${file1} ${liste_file_tmp[*]} ${file_out}
736	IGCM_debug_Verif_Exit
737	#
738	IGCM_sys_Rm ${file1}
739	if [ ! "${DEPOT}" = "TRUE" ] ; then
740	eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
741	# "${file1} = file_before ?
742	[ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
743	fi
744	else
745	# OR NOT
746	file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
747	IGCM_sys_ncrcat --hst -v ${liste_coord}${var} ${liste_file_tmp[*]} ${file_out}
748	IGCM_debug_Verif_Exit
749	fi
750
751	#
752	if [ "${DEPOT}" = "TRUE" ] ; then
753	# WE PUT FINAL FILE ON FILE SYSTEM
754	[ "${FLAG_B}" = "FALSE" ] && file_out=${config_UserChoices_JobName}_${DATE_BUILD}_${FlagDir}_${var}.nc
755	file_before=${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc
756	#
757	eval IGCM_sys_Put_Out ${file_out} \${R_OUT_${comp}}/Analyse/${TS_Dir}/${file_out}
758	[ "${FLAG_B}" = "TRUE" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file_before} ${file_out}
759	if [ ! "X${file1}" = "X" ] ; then
760	[ ! "${file1}" = "${config_UserChoices_JobName}_${DateBegin}_${DATE_FIN_JOB_B}_${FlagDir}_${var}.nc" ] && eval MENAGE \${R_OUT_${comp}}/Analyse/${TS_Dir} ${file1} ${file_out}
761	fi
762	#
763	[ -f ${file_out} ] && IGCM_sys_Rm ${file_out}
764	#
765	fi
766	#
767	done # for var in ${liste_var} ; do
768	IGCM_sys_Rm ${liste_file_tmp[*]}
769	done # for file in ${liste_file_brut} do
770	done # for comp in ${config_ListOfComponents[*]} ; do
771	# GO AHEAD IN DEEPEST LOOP
772	(( CURRENT_LOOP = CURRENT_LOOP + 1 ))
773	done # while [ ${CURRENT_LOOP} -le ${NBRE_TOUR} ] do
774	# FINISH
775
776	# DODS copy
777	for comp in ${config_ListOfComponents[*]} ; do
778	for TS_Dir in TS_YE TS_MO TS_DA TS_HF TS_INS ; do
779	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/${TS_Dir}
780	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/${TS_Dir}
781	done
782	done
783
784	export DEBUG_debug; export BigBrother ; export libIGCM; export SUBMIT_DIR; export POST_DIR; export Script_Output ;
785	export StandAlone ; export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ;
786	export RESOL_SBG
787	listVarEnv="DEBUG_debug,BigBrother,libIGCM,SUBMIT_DIR,POST_DIR,Script_Post_Output,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG"
788	export listVarEnv
789
790	#D-
791	# --------------------------------------------------------------------
792	#D- Test if all was right
793	# --------------------------------------------------------------------
794	IGCM_debug_Verif_Exit
795	# --------------------------------------------------------------------
796	#D- Submit MONITORING
797	# --------------------------------------------------------------------
798	if ( [ X${TsTask} = X2D ] \|\| [ X${TsTask} = X3D ] \|\| [ X${TsTask} = X ] ) ; then
799	# From 2D and 3D it is the last one to finish that will submit the monitoring.
800	if [ ! "X$( grep "IGCM_sys_RmRunDir" ${POST_DIR}/create_ts.${PeriodDateEnd}.[23]D.out 2> /dev/null )" = "X" ] ; then
801	Script_Post_Output=monitoring.${PeriodDateEnd}
802	IGCM_sys_QsubPost monitoring
803	fi
804	fi
805
806	#D-
807	# --------------------------------------------------------------------
808	#D- Update the run.card
809	# --------------------------------------------------------------------
810	if [ ${StandAlone} != true ] ; then
811	# Put in run.card end period of time series
812	[ X"${DATE_FIN}" = X ] \|\| IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesCompleted "${DATE_FIN}"
813	# We have finish the job !
814	IGCM_sys_RshMaster IGCM_card_WriteOption ${SUBMIT_DIR}/run.card PostProcessing TimeSeriesRunning "n"
815	fi
816
817	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
818	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}
819
820	# ------------------------------------------------------------------
821	# Finalize BigBrother to inform that the jobs end
822	# ------------------------------------------------------------------
823	IGCM_debug_BigBro_Finalize
824
825	date

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format