Context Navigation

source: trunk/libIGCM/AA_create_se @ 721

Last change on this file since 721 was 712, checked in by aclsce, 12 years ago
Added TaskType? variable in jobs to specify the type of the job (computing or post-processing). Added the use of TaskType? variable in Curie libsys to source the appropriate .atlas_env* configuration file.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 31.8 KB

Line
1	#-Q- cesium #!/bin/ksh
2	#-Q- cesium ######################
3	#-Q- cesium ## CESIUM CEA ##
4	#-Q- cesium ######################
5	#-Q- cesium #MSUB -r SE # Nom du job
6	#-Q- cesium #MSUB -N 1 # Reservation du noeud
7	#-Q- cesium #MSUB -n 1 # Reservation du processus
8	#-Q- cesium #MSUB -T 86400 # Limite de temps elapsed du job
9	#-Q- cesium #MSUB -E "-j o"
10	#-Q- cesium #MSUB -E "-S /bin/ksh"
11	#-Q- curie ######################
12	#-Q- curie ## CURIE TGCC/CEA ##
13	#-Q- curie ######################
14	#-Q- curie #MSUB -r SE # Nom du job
15	#-Q- curie #MSUB -eo
16	#-Q- curie #MSUB -n 1 # Reservation du processus
17	#-Q- curie #MSUB -T 86400 # Limite de temps elapsed du job
18	#-Q- curie #MSUB -E '--no-requeue'
19	#-Q- curie #MSUB -q large
20	#-Q- curie #MSUB -Q normal
21	#-Q- curie set +x
22	#-Q- platine #!/usr/bin/ksh
23	#-Q- platine ###################
24	#-Q- platine ## PLATINE CEA ##
25	#-Q- platine ###################
26	#-Q- platine #BSUB -J SE # Nom du job
27	#-Q- platine #BSUB -N # message a la fin du job
28	#-Q- platine #BSUB -n 1 # reservation des processeurs pour le job
29	#-Q- platine #BSUB -W 1:00 # Limite temps
30	#-Q- platine #BSUB -q post # Passage en queue post
31	#-Q- platine #BSUB -L /bin/ksh
32	#-Q- sx8brodie #!/bin/ksh
33	#-Q- sx8brodie #######################
34	#-Q- sx8brodie ## SX8BRODIE IDRIS ##
35	#-Q- sx8brodie #######################
36	#-Q- sx8brodie # Temps Elapsed max. d'une requete hh:mm:ss
37	#-Q- sx8brodie # @ wall_clock_limit = 10:00:00
38	#-Q- sx8brodie # Nom du travail LoadLeveler
39	#-Q- sx8brodie # @ job_name = SE
40	#-Q- sx8brodie # Fichier de sortie standard du travail
41	#-Q- sx8brodie # @ output = $(job_name).$(jobid)
42	#-Q- sx8brodie # Fichier de sortie d'erreur du travail
43	#-Q- sx8brodie # @ error = $(job_name).$(jobid)
44	#-Q- sx8brodie # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
45	#-Q- sx8brodie # @ notification = error
46	#-Q- sx8brodie # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
47	#-Q- sx8brodie # @ queue
48	#-Q- aix6 #!/bin/ksh
49	#-Q- aix6 #######################
50	#-Q- aix6 ## VARGAS IDRIS ##
51	#-Q- aix6 #######################
52	#-Q- aix6 # Temps Elapsed max. d'une requete hh:mm:ss
53	#-Q- aix6 # @ wall_clock_limit = 10:00:00
54	#-Q- aix6 # Nom du travail LoadLeveler
55	#-Q- aix6 # @ job_name = SE
56	#-Q- aix6 # Fichier de sortie standard du travail
57	#-Q- aix6 # @ output = $(job_name).$(jobid)
58	#-Q- aix6 # Fichier de sortie d'erreur du travail
59	#-Q- aix6 # @ error = $(job_name).$(jobid)
60	#-Q- aix6 # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
61	#-Q- aix6 # @ notification = error
62	#-Q- aix6 # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
63	#-Q- aix6 # @ queue
64	#-Q- ulam #!/bin/ksh
65	#-Q- ulam #######################
66	#-Q- ulam ## ULAM IDRIS ##
67	#-Q- ulam #######################
68	#-Q- ulam # Temps Elapsed max. d'une requete hh:mm:ss
69	#-Q- ulam # @ wall_clock_limit = 10:00:00
70	#-Q- ulam # Nom du travail LoadLeveler
71	#-Q- ulam # @ job_name = SE
72	#-Q- ulam # Fichier de sortie standard du travail
73	#-Q- ulam # @ output = $(job_name).$(jobid)
74	#-Q- ulam # Fichier de sortie d'erreur du travail
75	#-Q- ulam # @ error = $(job_name).$(jobid)
76	#-Q- ulam # pour recevoir un mail en cas de depassement du temps Elapsed (ou autre pb.)
77	#-Q- ulam # @ notification = error
78	#-Q- ulam # @ environment = $DEBUG_debug ; $MODIPSL ; $libIGCM ; $libIGCM_SX ; $SUBMIT_DIR ; $REBUILD_DIR ; $RebuildFromArchive ; $POST_DIR ; $MASTER ; $DateBegin ; $PeriodDateBegin ; $PeriodDateEnd ; $NbRebuildDir ; $StandAlone ; $RESOL_ATM ; $RESOL_OCE ; $RESOL_ICE ; $RESOL_MBG ; $RESOL_SRF ; $RESOL_SBG ; $MASTER
79	#-Q- ulam # @ queue
80	#-Q- sx8mercure #!/bin/ksh
81	#-Q- sx8mercure ######################
82	#-Q- sx8mercure ## SX8MERCURE CEA ##
83	#-Q- sx8mercure ######################
84	#-Q- sx8mercure #PBS -N SE # Nom du job
85	#-Q- sx8mercure #PBS -j o # regroupement des stdout et stderr
86	#-Q- sx8mercure #PBS -S /usr/bin/ksh # shell de soumission
87	#-Q- sx8mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
88	#-Q- sx8mercure #PBS -l cputim_job=24:00:00 # Limite temps a 1 heures
89	#-Q- sx8mercure #PBS -q scalaire
90	#-Q- sx8mercure #PBS -r n
91	#-Q- sx9mercure #!/bin/ksh
92	#-Q- sx9mercure ######################
93	#-Q- sx9mercure ## SX9MERCURE CCRT ##
94	#-Q- sx9mercure ######################
95	#-Q- sx9mercure #PBS -N SE # Nom du job
96	#-Q- sx9mercure #PBS -j o # regroupement des stdout et stderr
97	#-Q- sx9mercure #PBS -S /usr/bin/ksh # shell de soumission
98	#-Q- sx9mercure #PBS -l memsz_job=8gb # Limite memoire a 1 Go
99	#-Q- sx9mercure #PBS -l elapstim_req=24:00:00 # Limite temps a 1 heures
100	#-Q- sx9mercure #PBS -q scalaire
101	#-Q- sx9mercure #PBS -r n
102	#-Q- titane #!/bin/ksh
103	#-Q- titane ######################
104	#-Q- titane ## TITANE CEA ##
105	#-Q- titane ######################
106	#-Q- titane #MSUB -r SE # Nom du job
107	#-Q- titane #MSUB -eo
108	#-Q- titane #MSUB -n 1 # Reservation du processus
109	#-Q- titane #MSUB -T 86400 # Limite de temps elapsed du job
110	#-Q- titane #MSUB -q mono
111	#-Q- titane #MSUB -E '-rn'
112	#-Q- lxiv8 ######################
113	#-Q- lxiv8 ## OBELIX LSCE ##
114	#-Q- lxiv8 ######################
115	#-Q- lxiv8 #PBS -N SE
116	#-Q- lxiv8 #PBS -m a
117	#-Q- lxiv8 #PBS -j oe
118	#-Q- lxiv8 #PBS -q medium
119	#-Q- lxiv8 #PBS -o SE.$$
120	#-Q- lxiv8 #PBS -S /bin/ksh
121	#-Q- default #!/bin/ksh
122	#-Q- default ##################
123	#-Q- default ## DEFAULT HOST ##
124	#-Q- default ##################
125
126	#**************************************************************
127	# Author: Sebastien Denvil
128	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
129	# $Revision:: $ Revision of last commit
130	# $Author:: $ Author of last commit
131	# $Date:: $ Date of last commit
132	# IPSL (2006)
133	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
134	#
135	#**************************************************************
136
137	#set -eu
138	#set -vx
139
140	date
141
142	#-Q- sx8brodie export OMP_NUM_THREADS=1
143	#-Q- aix6 export OMP_NUM_THREADS=1
144	#-Q- ulam export OMP_NUM_THREADS=1
145
146	#D- Task type (computing or post-processing)
147	TaskType=post-processing
148
149	########################################################################
150
151	#D- Flag to determine if this job in a standalone mode
152	#D- Default : value from AA_job if any
153	StandAlone=${StandAlone:=true}
154
155	#D- Path to libIGCM
156	#D- Default : value from AA_job if any
157	# WARNING For StandAlone use : To run this script on some machine (ulam and cesium)
158	# WARNING you must check MirrorlibIGCM variable in sys library.
159	# WARNING If this variable is true, you must use libIGCM_POST path instead
160	# WARNING of your running libIGCM directory.
161	libIGCM=${libIGCM:=::modipsl::/libIGCM}
162
163	#D- Flag to determine if rebuild process has been performed asynchronously
164	#D- Possible value true or false.
165	#D- If true raw files has allready been patched by rebuild job
166	#D- If false create_ts will patch the raw files
167	#D- Default : Value from AA_job if any. Usefull in StandAlone case.
168	RebuildFrequency=${RebuildFrequency:=true}
169
170	#D- Flag to determine atlas job's output directory
171	#D- Default : value from libIGCM_post.ksh if any
172	POST_DIR=${POST_DIR:=${PBS_O_WORKDIR:=$(pwd)}}
173
174	#D- Increased verbosity (1, 2, 3)
175	#D- Default : value from AA_job if any
176	Verbosity=${Verbosity:=3}
177
178	#D- Low level debug : to bypass lib test checks and stack construction
179	#D- Default : value from AA_job if any
180	DEBUG_debug=${DEBUG_debug:=false}
181
182	#D- TEMPORARY Flag to determine atmospheric resolution
183	#D- Default : value from atmospheric driver if any
184	RESOL_ATM=${RESOL_ATM:=ALL}
185
186	#D- Flag to determine surface resolution
187	#D- Default : value from surface driver if any
188	RESOL_SRF=ALL
189
190	#D- Flag to determine surface resolution
191	#D- Default : value from surface driver if any
192	RESOL_SBG=ALL
193
194	#D- TEMPORARY Flag to determine ocean resolution
195	#D- Default : value from ocean driver if any
196	RESOL_OCE=${RESOL_OCE:=ORCA2}
197
198	#D- TEMPORARY Flag to determine ice resolution
199	#D- Default : value from ice driver if any
200	RESOL_ICE=${RESOL_ICE:=ORCA2}
201
202	#D- TEMPORARY Flag to determine marine biogeochemistry resolution
203	#D- Default : value from ice driver if any
204	RESOL_MBG=${RESOL_MBG:=ORCA2}
205
206	########################################################################
207
208	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
209	( ${DEBUG_debug} ) && IGCM_debug_Check
210	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
211	( ${DEBUG_debug} ) && IGCM_card_Check
212	. ${libIGCM}/libIGCM_date/libIGCM_date.ksh
213	( ${DEBUG_debug} ) && IGCM_date_Check
214	#-------
215	. ${libIGCM}/libIGCM_sys/libIGCM_sys.ksh
216
217	########################################################################
218
219	#set -vx
220
221	IGCM_sys_MkdirWork ${RUN_DIR_PATH}
222	IGCM_sys_Cd ${RUN_DIR_PATH}
223
224	# ------------------------------------------------------------------
225	# Test if all was right before proceeding further
226	# ------------------------------------------------------------------
227	IGCM_debug_Verif_Exit_Post
228
229	if [ ${StandAlone} = true ] ; then
230	CARD_DIR=${SUBMIT_DIR}
231	else
232	CARD_DIR=${RUN_DIR_PATH}
233	IGCM_sys_Get_Master ${SUBMIT_DIR}/config.card ${RUN_DIR_PATH}
234	IGCM_sys_Get_Master ${SUBMIT_DIR}/run.card ${RUN_DIR_PATH}
235	IGCM_sys_Get_Master ${SUBMIT_DIR}/COMP ${RUN_DIR_PATH}
236	IGCM_sys_Get_Master ${SUBMIT_DIR}/POST ${RUN_DIR_PATH}
237	fi
238
239	#==================================
240	# First of all
241	#
242	typeset option
243
244	#==================================
245	# Read UserChoices section:
246	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card UserChoices
247	echo
248	IGCM_debug_Print 1 "DefineArrayFromOption : config_UserChoices"
249	for option in ${config_UserChoices[*]} ; do
250	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card UserChoices ${option}
251	eval auxprint=\${config_UserChoices_${option}}
252	IGCM_debug_Print 3 "${option} : ${auxprint}"
253	done
254	#
255	echo
256
257	#==================================
258	# Read Ensemble section:
259	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card Ensemble
260	echo
261	IGCM_debug_Print 1 "DefineArrayFromOption : config_Ensemble"
262	for option in ${config_Ensemble[*]} ; do
263	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Ensemble ${option}
264	eval auxprint=\${config_Ensemble_${option}}
265	IGCM_debug_Print 3 "${option} : ${auxprint}"
266	done
267	#
268	echo
269
270	#==================================
271	# Read Post section:
272	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card Post
273	echo
274	IGCM_debug_Print 1 "DefineArrayFromOption : config_Post"
275	for option in ${config_Post[*]} ; do
276	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post ${option}
277	eval auxprint=\${config_Post_${option}}
278	IGCM_debug_Print 3 "${option} : ${auxprint}"
279	done
280	#
281	echo
282
283	#==================================
284	# Define default value to keep compatibility with previous card: means before changes due to TGCC
285	if [ X${PackDefault} = Xtrue ] ; then
286	[ X${config_Post_PackFrequency} = X ] && config_Post_PackFrequency=${config_Post_RebuildFrequency}
287	else
288	config_Post_PackFrequency=NONE
289	fi
290
291	#==================================
292	# If pack is active then PackFrequency overule the config_UserChoices_PeriodLength
293	if ( [ ! X${config_Post_PackFrequency} = X${NULL_STR} ] && [ ! X${config_Post_PackFrequency} = XNONE ] ) ; then
294	config_UserChoices_PeriodLength=${config_Post_PackFrequency}
295	fi
296
297	#==================================
298	#R_SAVE : Job output directory
299	#R_BUFR : Job output buffered directory
300	if ( [ ! X${config_UserChoices_SpaceName} = X ] && [ ! X${config_UserChoices_ExperimentName} = X ] ) ; then
301	FreeName=$( echo ${config_UserChoices_JobName} \| sed 's/.*_//' )
302	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
303	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
304	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
305	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${FreeName}
306	else
307	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
308	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
309	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_SpaceName}/${config_UserChoices_ExperimentName}/${FreeName}
310	fi
311	else
312	if ( [ ! X${config_Ensemble_EnsembleName} = X ] && [ ! X${config_Ensemble_EnsembleDate} = X ] ) ; then
313	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
314	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
315	R_DODS=${config_UserChoices_TagName}/${config_Ensemble_EnsembleName}/${config_Ensemble_EnsembleDate}/${config_UserChoices_JobName}
316	else
317	R_SAVE=${R_OUT}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
318	R_BUFR=${R_BUF}/${config_UserChoices_TagName}/${config_UserChoices_JobName}
319	R_DODS=${config_UserChoices_TagName}/${config_UserChoices_JobName}
320	fi
321	fi
322	#
323	IGCM_card_DefineArrayFromSection ${CARD_DIR}/config.card ListOfComponents
324	#
325	IGCM_card_DefineVariableFromOption ${CARD_DIR}/config.card Post SeasonalFrequency
326	#
327	# Determine period for seasonnal average. Default : value from AA_job if any
328	#
329	DateBegin=${DateBegin:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateBegin} )}
330	#
331	PeriodDateEnd=${PeriodDateEnd:=$( IGCM_date_ConvertFormatToGregorian ${config_UserChoices_DateEnd} )}
332	#
333	freq=$( echo ${config_Post_SeasonalFrequency} \| sed -e "s/[yY]//" )
334	#
335	an_fin=$(( $( IGCM_date_ConvertGregorianDateToJulian ${PeriodDateEnd} ) / 1000 ))
336	an_deb=$( IGCM_date_YearDigit $(( an_fin - freq + 1 )) )
337	an_fin=$( IGCM_date_YearDigit ${an_fin} )
338	#
339	# For each component selected determine which files need post-processing
340	#
341	for comp in ${config_ListOfComponents[*]} ; do
342	#
343	IGCM_card_DefineArrayFromOption ${CARD_DIR}/config.card ListOfComponents ${comp}
344	eval compname=\${config_ListOfComponents_${comp}[0]} > /dev/null 2>&1
345	#
346	card=${CARD_DIR}/COMP/${compname}.card
347	IGCM_card_DefineArrayFromOption ${card} OutputFiles List
348	#
349	ListFilesName=${compname}_OutputFiles_List
350	eval FileName0=\${${ListFilesName}[0]} > /dev/null 2>&1
351	#
352	if [ X${FileName0} != X${NULL_STR} ] ; then
353	#
354	# INITIALISATION
355	#
356	eval NbFiles=\${#${ListFilesName}[@]} > /dev/null 2>&1
357	typeset i=2
358	#
359	until [ $i -gt $NbFiles ]; do
360	#
361	eval flag_post=\${${ListFilesName}[$i]} > /dev/null 2>&1
362	#
363	if [ X${flag_post} != XNONE ] ; then
364	#
365	# Check if seasonal is required for this file
366	#
367	IGCM_card_DefineArrayFromSection ${card} ${flag_post}
368	#
369	# Seasonal case : If option Seasonal is not found (old cards) put SEASONAL ON by default
370	#
371	# variable option allready typeset above
372	for option in $( eval echo \${${compname}_${flag_post}[*]} ) ; do
373	if [ ${option} = Seasonal ] ; then
374	FoundSeasonal=true
375	IGCM_card_DefineVariableFromOption ${card} ${flag_post} Seasonal
376	fi
377	done
378	#
379	if [ ! X${FoundSeasonal} = Xtrue ] ; then
380	eval ${compname}_${flag_post}_Seasonal=ON
381	fi
382	#
383	if [ ! X$( eval echo \${${compname}_${flag_post}_Seasonal} ) = XON ] ; then
384	(( i=i+3 ))
385	continue
386	fi
387	#
388	# CREATE LIST
389	#
390	FILE=$( echo ${flag_post} \| awk "-FPost_" '{print $2}' )
391	#
392	DIRECTORY=${R_SAVE}/${comp}/Analyse/SE
393	#
394	TestedFile=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${FILE}.nc
395	#
396	IGCM_sys_TestFileArchive ${DIRECTORY}/${TestedFile}
397	if [ ! $? = 0 ] ; then
398	eval set +A LISTE_FILE_${comp} \$\{LISTE_FILE_${comp}[*]} ${FILE}
399	if [ ! ${RebuildFrequency} = true ] ; then
400	IGCM_card_DefineArrayFromOption ${card} Post_${FILE} Patches
401	eval set +A LISTE_PATCHES_${FILE} \$\{${compname}_Post_${FILE}_Patches[*]}
402	if [ X$( eval echo \${LISTE_PATCHES_${FILE}[0]} ) != X${NULL_STR} ]; then
403	for Patch in $( eval echo \${LISTE_PATCHES_${FILE}[*]} ); do
404	if [ Xload${Patch} != X ] ; then
405	. ${libIGCM}/libIGCM_post/IGCM_${Patch}.ksh
406	eval load${Patch}=loaded
407	fi
408	done
409	fi
410	fi
411	fi
412	fi
413	(( i=i+3 ))
414	done # until [ $i -eq $NbFiles ]
415	fi
416	done # for comp
417
418	for comp in ${config_ListOfComponents[*]} ; do
419	#
420	eval R_OUT_${comp}=${R_SAVE}/${comp}
421	#
422	i=0
423	for file in $( eval echo \${LISTE_FILE_${comp}[*]} ); do
424	#
425	# Determine in which output can we find file
426	# IE : Output/MO or Output/DA or ...
427	#
428	FlagDir=$( echo ${file} \| awk -F "_" '{print $1}' )
429	case ${FlagDir} in
430	1Y) FreqDir=YE ;;
431	1M) FreqDir=MO ;;
432	5D\|1D) FreqDir=DA ;;
433	HF) FreqDir=HF ;;
434	3H) FreqDir=HF ;;
435	INS) FreqDir=INS ;;
436	esac
437	#
438	RESULT_SE=${config_UserChoices_JobName}_SE_${an_deb}_${an_fin}_${file}.nc
439	# Do not redoo an existing SE
440	IGCM_sys_TestFileArchive ${R_SAVE}/${comp}/Analyse/SE/${RESULT_SE}
441	if [ $? = 0 ] ; then
442	continue
443	fi
444	#
445	eval file_path=\${R_OUT_${comp}}/Output/${FreqDir}/
446	#
447	case ${config_UserChoices_PeriodLength} in
448	# For Yearly simulation :
449	1Y\|1y)
450	(( an = an_deb ))
451	# Initialize array
452	#set -A liste_file_se
453	#set -A liste_file_se_tmp
454	unset liste_file_se
455	unset liste_file_se_tmp
456	while [ ${an} -le ${an_fin} ] ; do
457	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
458	#
459	an4=$(printf "%04i\n" ${an} )
460	#
461	diy=$( IGCM_date_DaysInYear $an4 )
462	afin=$( IGCM_date_AddDaysToGregorianDate ${an4}0101 $(( diy - 1 )) )
463	date=${an4}0101_${afin}
464	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
465	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
466	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
467	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
468	(( an = an + 1 ))
469	done
470	#
471	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
472	IGCM_debug_Verif_Exit_Post
473
474	# Apply IGCM_Patch if needed
475	if [ ! ${RebuildFrequency} = true ] ; then
476	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
477	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
478	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
479	IGCM_${Patch} ${file_t}
480	done
481	done
482	fi
483	fi
484
485	# Search of common list of variables to be treated
486	nbfile=0
487	for file_common_var in ${liste_file_se_tmp[*]} ; do
488	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
489	(( nbfile = nbfile + 1 ))
490	done
491	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
492	IGCM_sys_Rm tmpfile_ncra
493
494	# detect time counter : time or time_counter ie unlimited variable
495	var_unlim=$(ncdump -h ${liste_file_se_tmp[1]}\|grep UNLIMITED\|awk '{print $1}')
496
497	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
498	#
499	(( nb = ${mois} - 1 ))
500	#
501	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
502	IGCM_sys_ncra -O -v ${list_var_final_ncra} -d ${var_unlim},${nb},,12 ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
503	done
504	IGCM_sys_Rm ${liste_file_se_tmp[*]}
505	;;
506	# For Monthly simulation :
507	1M\|1m)
508	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
509	#
510	(( nb = ${mois} - 1 ))
511	#
512	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
513	#
514	(( an = an_deb ))
515	# Initialize array
516	#set -A liste_file_se
517	#set -A liste_file_se_tmp
518	unset liste_file_se
519	unset liste_file_se_tmp
520	while [ ${an} -le ${an_fin} ] ; do
521	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
522	#
523	an4=$(printf "%04i\n" ${an} )
524	#
525	date=${an4}${mois}01_${an4}${mois}${moisL}
526	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
527	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
528	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
529	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
530
531	(( an = an + 1 ))
532	done
533	#
534	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
535	IGCM_debug_Verif_Exit_Post
536
537	# Apply IGCM_Patch if needed
538	if [ ! ${RebuildFrequency} = true ] ; then
539	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
540	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
541	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
542	IGCM_${Patch} ${file_t}
543	done
544	done
545	fi
546	fi
547
548	# Search of common list of variables to be treated
549	nbfile=0
550	for file_common_var in ${liste_file_se_tmp[*]} ; do
551	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
552	(( nbfile = nbfile + 1 ))
553	done
554	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
555	IGCM_sys_Rm tmpfile_ncra
556
557	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
558
559	IGCM_sys_Rm ${liste_file_se_tmp[*]}
560	done
561	;;
562	D\|d)
563	for mois in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
564	#
565	(( nb = ${mois} - 1 ))
566	#
567	OUT_SE[${nb}]=${config_UserChoices_JobName}_SE${mois}_${an_deb}_${an_fin}_${file}.nc
568	#
569	(( an = an_deb ))
570	# Initialize array
571	#set -A liste_file_se
572	#set -A liste_file_se_tmp
573	unset liste_file_se
574	unset liste_file_se_tmp
575	while [ ${an} -le ${an_fin} ] ; do
576	moisL=$( IGCM_date_DaysInMonth ${an} ${mois} )
577	#
578	an4=$(printf "%04i\n" ${an} )
579	#
580	length_d=$(( ${config_UserChoices_PeriodLength} - 1 ))
581	# for one day
582	if [ ${length_d} -eq 1 ] ; then
583	(( length_d = 0 ))
584	fi
585	(( day = 1 ))
586	while [ ${day} -le ${moisL} ] ; do
587	day2=$(printf "%02i\n" ${day} )
588	day_beg=${an4}${mois}${day2}
589	day_end=$( IGCM_date_AddDaysToGregorianDate ${day_beg} ${length_d} )
590	date=${an4}${mois}${day_beg}_${an4}${mois}${day_end}
591	#set +A liste_file_se ${liste_file_se[*]} ${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
592	#set +A liste_file_se_tmp ${liste_file_se_tmp[*]} ${config_UserChoices_JobName}_${date}_${file}.nc
593	liste_file_se[${#liste_file_se[*]}]=${file_path}${config_UserChoices_JobName}_${date}_${file}.nc
594	liste_file_se_tmp[${#liste_file_se_tmp[*]}]=${config_UserChoices_JobName}_${date}_${file}.nc
595	(( day = day + 1 ))
596	done
597	(( an = an + 1 ))
598	done
599	#
600	IGCM_sys_Get /l liste_file_se[*] ${RUN_DIR_PATH}
601	IGCM_debug_Verif_Exit_Post
602
603	# Apply IGCM_Patch if needed
604	if [ ! ${RebuildFrequency} = true ] ; then
605	if [ X$( eval echo \${LISTE_PATCHES_${file}[0]} ) != X${NULL_STR} ]; then
606	for file_t in $( eval echo ${liste_file_se_tmp[*]} ); do
607	for Patch in $( eval echo \${LISTE_PATCHES_${file}[*]} ); do
608	IGCM_${Patch} ${file_t}
609	done
610	done
611	fi
612	fi
613
614	# Search of common list of variables to be treated
615	nbfile=0
616	for file_common_var in ${liste_file_se_tmp[*]} ; do
617	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncra
618	(( nbfile = nbfile + 1 ))
619	done
620	list_var_final_ncra=`cat tmpfile_ncra \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
621	IGCM_sys_Rm tmpfile_ncra
622
623	IGCM_sys_ncra -O -v ${list_var_final_ncra} ${liste_file_se_tmp[*]} ${OUT_SE[${nb}]}
624	IGCM_sys_Rm ${liste_file_se_tmp[*]}
625	done
626	;;
627	esac
628	#
629
630	# Search of common list of variables to be treated
631	nbfile=0
632	for file_common_var in ${OUT_SE[*]} ; do
633	ncdump -h ${file_common_var} \| grep -E 'float\|double' \| cut -f 1 -d '(' \| cut -f 2 -d ' ' >> tmpfile_ncrcat
634	(( nbfile = nbfile + 1 ))
635	done
636	list_var_final_ncrcat=`cat tmpfile_ncrcat \| sort \| uniq -c \| awk -v nbfile=$nbfile '{if ($1 == nbfile) {print $2}}' \| paste -s -d ','`
637	IGCM_sys_Rm tmpfile_ncrcat
638
639	IGCM_sys_ncrcat -O -v ${list_var_final_ncrcat} ${OUT_SE[*]} ${RESULT_SE}
640	#
641	IGCM_sys_Rm ${OUT_SE[*]}
642	#
643	# detect time counter : time or time_counter ie unlimited variable
644	var_unlim=$(ncdump -h ${RESULT_SE}\|grep UNLIMITED\| cut -d ' ' -f 1)
645
646	# Put correct climatological axis following calendar
647	case ${config_UserChoices_CalendarType} in
648	360d\|360_day)
649	# 360 days
650	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15, 45, 75, 105, 135, 165, 195, 225, 255, 285, 315, 345.} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 30, 30, 60, 60, 90, 90, 120, 120, 150, 150, 180, 180, 210, 210, 240, 240, 270, 270, 300, 300, 330, 330, 360.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"360_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
651	;;
652	noleap\|365_day)
653	# 365 days
654	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45, 74.5, 105, 135.5, 166, 196.5, 227.5, 258, 288.5, 319, 349.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59, 59, 90, 90, 120, 120, 151, 151, 181, 181, 212, 212, 243, 243, 273, 273, 304, 304, 334, 334, 365.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"365_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
655	;;
656	all_leap\|366_day)
657	# 366 days
658	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.5, 75.5, 106, 136.5, 167, 197.5, 228.5, 259, 289.5, 320, 350.5} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 60, 60, 91, 91, 121, 121, 152, 152, 182, 182, 213, 213, 244, 244, 274, 274, 305, 305, 335, 335, 366.} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"366_day\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
659	;;
660	leap\|gregorian\|standard)
661	# 365.2425 days
662	ncap2cmd="'defdim(\"tbnds\",2) ; ${var_unlim}[${var_unlim}]={15.5, 45.12125, 74.7425, 105.2425, 135.7425, 166.2425, 196.7425, 227.7425, 258.2425, 288.7425, 319.2425, 349.7425} ; ${var_unlim}_bnds[${var_unlim},tbnds]={0, 31, 31, 59.2425, 59.2425, 90.2425, 90.2425, 120.2425, 120.2425, 151.2425, 151.2425, 181.2425, 181.2425, 212.2425, 212.2425, 243.2425, 243.2425, 273.2425, 273.2425, 304.2425, 304.2425, 334.2425, 334.2425, 365.2425} ; ${var_unlim}@units=\"days since 0000-01-01 00:00:00\" ; ${var_unlim}@time_origin=\"01-JAN-0000 00:00:00\" ; ${var_unlim}@calendar=\"gregorian\" ; ${var_unlim}@bounds=\"${var_unlim}_bnds\"'"
663	;;
664	esac
665	eval IGCM_sys_ncap2 -Oh -s $ncap2cmd ${RESULT_SE} tmp.nc
666	IGCM_sys_Rm ${RESULT_SE}
667	IGCM_sys_Mv tmp.nc ${RESULT_SE}
668	#
669	eval IGCM_sys_Put_Out ${RESULT_SE} \${R_OUT_${comp}}/Analyse/SE/${RESULT_SE}
670	IGCM_sys_Rm ${RESULT_SE}
671	#
672	done
673	done
674
675	# DODS copy
676	for comp in ${config_ListOfComponents[*]} ; do
677	IGCM_sys_TestDirArchive ${R_SAVE}/${comp}/Analyse/SE
678	[ $? = 0 ] && IGCM_sys_Put_Dods ${comp}/Analyse/SE
679	done
680
681	YEARS=${an_deb}_${an_fin}
682	DIR_OUT=SE
683
684	export POST_DIR; export SUBMIT_DIR; export libIGCM; export R_INIT; export R_BC; export StandAlone
685	export RESOL_ATM; export RESOL_OCE ; export RESOL_ICE ; export RESOL_MBG ; export RESOL_SRF ; export RESOL_SBG
686	export R_SAVE; export config_UserChoices_JobName; export config_UserChoices_TagName; export YEARS; export DIR_OUT
687	listVarEnv="POST_DIR,SUBMIT_DIR,libIGCM,R_INIT,R_BC,StandAlone,RESOL_ATM,RESOL_OCE,RESOL_ICE,RESOL_MBG,RESOL_SRF,RESOL_SBG,R_SAVE,config_UserChoices_JobName,config_UserChoices_TagName,YEARS,DIR_OUT"
688	export listVarEnv;
689
690	#D-
691	# --------------------------------------------------------------------
692	#D- Test if all was right before ATLAS
693	# --------------------------------------------------------------------
694	IGCM_debug_Verif_Exit_Post
695
696	if [ ${config_UserChoices_TagName} = "IPSLCM4_v1_OASIS3" ] \|\| \
697	[ ${config_UserChoices_TagName} = "IPSLCM4_v2" ] \|\| \
698	[ ${config_UserChoices_TagName} = "IPSL_ESM_V1" ] ; then
699	#
700	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
701	IGCM_sys_QsubPost atlas_ORCHIDEE ;
702	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
703	IGCM_sys_QsubPost atlas_LMDZ ;
704	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
705	IGCM_sys_QsubPost atlas_ORCA_LIM ;
706	Script_Post_Output=additionnal.${PeriodDateEnd} ;
707	IGCM_sys_QsubPost additionnal ;
708	#
709	elif [ ${config_UserChoices_TagName} = "IPSLCM5" ] \|\| \
710	[ ${config_UserChoices_TagName} = "IPSLCM5A" ] \|\| \
711	[ ${config_UserChoices_TagName} = "IPSLCM5B" ] \|\| \
712	[ ${config_UserChoices_TagName} = "IPSLCM5A-MR" ] ; then
713	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
714	IGCM_sys_QsubPost atlas_ORCHIDEE ;
715	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
716	IGCM_sys_QsubPost atlas_LMDZ ;
717	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
718	IGCM_sys_QsubPost atlas_ORCA_LIM ;
719	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
720	IGCM_sys_QsubPost atlas_PISCES ;
721	#
722	elif [ ${config_UserChoices_TagName} = "LMDZ4OR" ] \|\| \
723	[ ${config_UserChoices_TagName} = "LMDZOR" ] \|\| \
724	[ ${config_UserChoices_TagName} = "LMDZ4OR_v2" ] ; then
725	#
726	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
727	IGCM_sys_QsubPost atlas_LMDZ ;
728	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
729	IGCM_sys_QsubPost atlas_ORCHIDEE ;
730	#
731	elif [ ${config_UserChoices_TagName} = "LMDZ" ] ; then
732	#
733	Script_Post_Output=atlas_LDMZ.${PeriodDateEnd} ;
734	IGCM_sys_QsubPost atlas_LMDZ ;
735	#
736	elif [ ${config_UserChoices_TagName} = "OL" ] \|\| \
737	[ ${config_UserChoices_TagName} = "OL2" ] ; then
738	#
739	Script_Post_Output=atlas_ORCHIDEE.${PeriodDateEnd} ;
740	IGCM_sys_QsubPost atlas_ORCHIDEE ;
741	#
742	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM" ] \|\|
743	[ ${config_UserChoices_TagName} = "ORCA2_LIM2" ] ; then
744	#
745	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
746	IGCM_sys_QsubPost atlas_ORCA_LIM ;
747	elif [ ${config_UserChoices_TagName} = "ORCA2_LIM2_PISCES" ] ; then
748	#
749	Script_Post_Output=atlas_ORCA_LIM.${PeriodDateEnd} ;
750	IGCM_sys_QsubPost atlas_ORCA_LIM ;
751	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
752	IGCM_sys_QsubPost atlas_PISCES ;
753	elif [ ${config_UserChoices_TagName} = "ORCA2_OFF_PISCES" ] ; then
754	#
755	Script_Post_Output=atlas_PISCES.${PeriodDateEnd} ;
756	IGCM_sys_QsubPost atlas_PISCES ;
757	#
758	fi
759
760	# Clean RUN_DIR_PATH (necessary for cesium and titane only)
761	IGCM_sys_RmRunDir -Rf ${RUN_DIR_PATH}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: