Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_curie.ksh @ 1060

Last change on this file since 1060 was 1060, checked in by sdipsl, 10 years ago
seems more natural to have upper case here
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 92.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sebastien Denvil, Martial Mancip, Arnaud Caubel
5	# Contact: Arnaud.Caubel__at__lsce.ipsl.fr
6	# $Revision:: $ Revision of last commit
7	# $Author:: $ Author of last commit
8	# $Date:: $ Date of last commit
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	#
12	#**************************************************************
13
14	#=========================================================
15	# The documentation of this file can be automatically generated
16	# if you use the prefix #D- for comments to be extracted.
17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
18	#=========================================================
19
20	#D-#==================================================
21	#D-LibIGCM_sys for Curie
22	#D-#==================================================
23	#D-
24	#D- This ksh library if a layer under some usefull
25	#D-environment variables and shell commands.
26	#D-All those definitions depend on host particularities.
27	#D-It manages a stack mechanism and test validity of operations.
28	#D-All function described bellow must be prefixed by IGCM_sys.
29
30	#====================================================
31	# libIGCM_sys PARAMETERS
32	#====================================================
33
34	#====================================================
35	# set DEBUG_sys to true to output calls of function
36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
37
38	#====================================================
39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
40	typeset -r DRYRUN=${DRYRUN:=0}
41
42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
43	# -------------------------------------------------------------------------------------
44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
45	# \| \| Cp/Exe/param/files \| \| \| \|
46	# \| \| Chmod Qsub \| \| \| \|
47	# -------------------------------------------------------------------------------------
48	# \| 0 \| yes \| yes \| yes \| yes \|
49	# -------------------------------------------------------------------------------------
50	# \| 1 \| yes \| yes \| yes \| no \|
51	# -------------------------------------------------------------------------------------
52	# \| 2 \| yes \| yes \| no \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 3 \| yes \| no \| no \| no \|
55	# -------------------------------------------------------------------------------------
56
57	#=====================================================
58	# Global Variables :
59	#=====================================================
60	# Language : "fr" or "en"
61	typeset -r MYLANG="fr"
62
63	#=====================================================
64	# Host and user names
65	# $hostname ou hostname
66	typeset HOST=${HOST:=$( hostname )}
67	# $username ou whoami
68	typeset LOGIN=${LOGIN:=$( whoami )}
69	# $hostname of the MASTER job
70	typeset MASTER=curie
71	# add default project on curie
72	typeset DEFAULT_PROJECT=gen0826
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# Submit command
80	typeset SUBMIT=${SUBMIT:=ccc_msub}
81	# rsync with path
82	typeset -r RSYNC=/usr/bin/rsync
83	# RSYNC_opt args to rsync
84	typeset -r RSYNC_opt="-va"
85	# ie storage filesystem
86	typeset -r STOREHOST=${MASTER}
87
88	#====================================================
89	# Source default environment
90	#====================================================
91	##. /etc/profile
92
93	#====================================================
94	# Set environment tools (ferret, nco, cdo)
95	#====================================================
96	if [ X${TaskType} = Xcomputing ] ; then
97	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
98	# to run with netcdf 3.6.3 ie compilation done before 17/2/2014
99	# uncomment 2 lines :
100	# module unload netcdf
101	# module load netcdf/3.6.3
102	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddNoise/src_X64_CURIE/bin
103	export PATH=${PATH}:/ccc/cont003/home/dsm/p86ipsl/AddPerturbation/src_X64_CURIE/bin
104	else
105	. /ccc/cont003/home/dsm/p86ipsl/.atlas_env_netcdf4_curie_ksh > /dev/null 2>&1
106	fi
107
108	#====================================================
109	# Specific for ocean additionnal diagnostic
110	export FER_GO="$FER_GO /home/cont003/p86denv/IGCM_POST_UTIL/JNL /home/cont003/p86denv/GRAF /home/cont003/p86denv/GRAF/GO"
111	export FER_PALETTE="$FER_PALETTE /home/cont003/p86denv/GRAF/PALET"
112
113	#====================================================
114	# Host specific DIRECTORIES
115	#====================================================
116
117	# ============ CESIUM START ============ #
118
119	#====================================================
120	#- Mirror libIGCM from titane to cesium if needed
121	#ROOTSYS=$( echo ${libIGCM} \| gawk -F"/" '{print $3}' )
122	#if [ ! ${ROOTSYS} = "home" ] ; then
123	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
124	#else
125	# typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
126	#fi
127
128	#====================================================
129	#- libIGCM_POST
130	#if ( ${MirrorlibIGCM} ) ; then
131	# PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
132	# typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
133	#else
134	# typeset -r libIGCM_POST=${libIGCM}
135	#fi
136
137	# ============ CESIUM END ============ #
138
139	#====================================================
140	#- MirrorlibIGCM for frontend
141	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
142
143	#====================================================
144	#- libIGCM_POST for frontend
145	typeset -r libIGCM_POST=${libIGCM}
146
147	#====================================================
148	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
149	typeset -r R_EXE="${MODIPSL}/bin"
150
151	#====================================================
152	#- SUBMIT_DIR : submission dir
153	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${BRIDGE_MSUB_PWD}}
154
155	#====================================================
156	#- IN
157	typeset -r R_IN=${R_IN:=/ccc/work/cont003/dsm/p86ipsl/IGCM}
158	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/ccc/work/cont003/dsm/p24data}
159
160	#====================================================
161	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
162	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${SCRATCHDIR}/RUN_DIR/${BRIDGE_MSUB_JOBID}_${$}}
163
164	#====================================================
165	#- HOST_MPIRUN_COMMAND
166	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time ccc_mprun -E-K1"}
167
168	#====================================================
169	#- Max number of arguments passed to nco operator or demigration command
170	UNIX_MAX_LIMIT=360
171
172	#====================================================
173	#- set PackDefault true on curie
174	PackDefault=true
175
176	#====================================================
177	#- Number of core per node (max number of OpenMP task)
178	NUM_COREPERNODE=8
179
180	#====================================================
181	#- Default number of MPI task for IPSL coupled model
182	#- required for backward compatibility
183	#-
184	DEFAULT_NUM_PROC_OCE=5
185	DEFAULT_NUM_PROC_CPL=1
186	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
187	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
188
189	#D-#==================================================
190	#D-function IGCM_sys_defineArchives
191	#D-* Purpose:
192	#D-* Define ARCHIVE : Dedicated to large files
193	#D-* Define STORAGE : Dedicated to small/medium files
194	#D-* Define R_OUT : Output tree located on ARCHIVE
195	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
196	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
197	#D-* if SpaceName=TEST everything is stored on SCRATCHDIR
198	#D-* Examples:
199	#D-
200	function IGCM_sys_defineArchives {
201	IGCM_debug_PushStack "IGCM_sys_defineArchives"
202
203	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
204	#====================================================
205	#- ARCHIVE (dedicated to large files)
206	typeset ARCHIVE=${config_UserChoices_ARCHIVE}
207	else
208	#====================================================
209	#- ARCHIVE (dedicated to large files)
210	typeset ARCHIVE=${CCCSTOREDIR}
211	fi
212
213	if [ ! X${config_UserChoices_STORAGE} = X ]; then
214	#====================================================
215	#- STORAGE (dedicated to small/medium files)
216	typeset STORAGE=${config_UserChoices_STORAGE}
217	else
218	#====================================================
219	#- STORAGE (dedicated to small/medium files)
220	typeset STORAGE=${CCCWORKDIR}
221	fi
222
223	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
224	#====================================================
225	#- R_OUT
226	typeset R_OUT=${SCRATCHDIR}/IGCM_OUT
227
228	#====================================================
229	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
230	typeset R_FIG=${SCRATCHDIR}/IGCM_OUT
231
232	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
233
234	else
235	#====================================================
236	#- R_OUT
237	typeset R_OUT=${ARCHIVE}/IGCM_OUT
238
239	#====================================================
240	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
241	typeset R_FIG=${STORAGE}/IGCM_OUT
242	fi
243
244	#====================================================
245	#- R_BUF (ONLY FOR double copy an scratch)
246	typeset -r R_BUF=${SCRATCHDIR}/IGCM_OUT
247
248	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
249	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
250	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
251
252	IGCM_debug_PopStack "IGCM_sys_defineArchives"
253	}
254
255	#D-#==================================================
256	#D-function IGCM_sys_RshMaster
257	#D-* Purpose: Connection to frontend machine.
258	#D-* Examples:
259	#D-
260	function IGCM_sys_RshMaster {
261	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
262	/bin/ksh <<-EOF
263	export libIGCM=${libIGCM}
264	export DEBUG_debug=${DEBUG_debug}
265	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
266	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
267	${@}
268	EOF
269	if [ $? -gt 0 ] ; then
270	echo "IGCM_sys_RshMaster : erreur."
271	IGCM_debug_Exit "IGCM_sys_RshMaster"
272	fi
273	IGCM_debug_PopStack "IGCM_sys_RshMaster"
274	}
275
276	#D-#==================================================
277	#D-function IGCM_sys_RshArchive
278	#D-* Purpose: Archive rsh command
279	#D-* Examples:
280	#D-
281	function IGCM_sys_RshArchive {
282	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
283	/bin/ksh <<-EOF
284	${@}
285	EOF
286	status=$?
287	if [ ${status} -gt 0 ] ; then
288	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
289	IGCM_debug_Exit "IGCM_sys_RshArchive"
290	fi
291	IGCM_debug_PopStack "IGCM_sys_RshArchive"
292	}
293
294	#D-#==================================================
295	#D-function IGCM_sys_RshArchive_NoError
296	#D-* Purpose: Archive rsh command, without error
297	#D-* used only in monitoring.job
298	#D-* Examples:
299	#D-
300	function IGCM_sys_RshArchive_NoError {
301	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
302	/bin/ksh <<-EOF
303	${@} 2> \dev\null
304	EOF
305	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
306	}
307
308	#D-#==================================================
309	#D-function IGCM_sys_RshPost
310	#D-* Purpose: Post-process rsh command
311	#D-* Examples:
312	#D-
313	function IGCM_sys_RshPost {
314	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
315	if ( $DEBUG_sys ) ; then
316	echo "IGCM_sys_RshPost :" $@
317	fi
318	# keep standard input (stdin) for the loop onto temporary file
319	cat >/tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
320
321	/bin/ksh </tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
322	if [ $? -gt 0 ] ; then
323	echo "IGCM_sys_RshPost : erreur."
324	IGCM_debug_Exit "IGCM_sys_RshPost"
325	fi
326	# delete temporary file
327	\rm /tmp/tmp_IGCM_sys_RshPost_$$_${LOGNAME}
328
329	# ============ FRONTEND END ============ #
330
331	# ============ CESIUM START ============ #
332	# typeset NB_ESSAI DELAI status i
333	# if [ "X$( grep rebuild_from tmp_IGCM_sys_RshPost_$$ )" != "X" ] ; then
334	# #little hack so that rebuild submission is done on titane not an cesium
335	#
336	# libIGCM_POST_sed=$( echo $libIGCM_POST \| sed 's/\//\\\//g' )
337	# POST_DIR_sed=$( echo ${POST_DIR} \| sed 's/\//\\\//g' )
338	# sed "s/IGCM_sys_QsubPost/IGCM_sys_Qsub/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
339	# sed "s/ rebuild_fromWorkdir/ ${libIGCM_POST_sed}\/rebuild_fromWorkdir.job/g" tmp.txt > tmp_IGCM_sys_RshPost_$$
340	# sed "s/ rebuild_fromArchive/ ${libIGCM_POST_sed}\/rebuild_fromArchive.job/g" tmp_IGCM_sys_RshPost_$$ > tmp.txt
341	# sed "s/Script_Post_Output=/Script_Output=${POST_DIR_sed}\//g" tmp.txt > tmp_IGCM_sys_RshPost_$$
342	# \mv tmp.txt tmp_IGCM_sys_RshPost_$$
343	#
344	# echo cat tmp_IGCM_sys_RshPost_$$ AFTER
345	# cat tmp_IGCM_sys_RshPost_$$
346	#
347	# /bin/ksh <tmp_IGCM_sys_RshPost_$$
348	# if [ $? -gt 0 ] ; then
349	# echo "IGCM_sys_RshPost : erreur."
350	# IGCM_debug_Exit "IGCM_sys_RshPost"
351	# fi
352	# # delete temporary file
353	# \rm tmp_IGCM_sys_RshPost_$$
354	#
355	# else
356	# # number of tentative
357	# NB_ESSAI=10
358	# # time delay between tentative
359	# DELAI=10
360	# i=0
361	# while [ $i -ne $NB_ESSAI ] ; do
362	# ssh -t titane996 ssh cesium /bin/ksh <tmp_IGCM_sys_RshPost_$$
363	# status=$?
364	# if [ ${status} -ne 0 ];
365	# then
366	# sleep $DELAI
367	# else
368	# break
369	# fi
370	# let i=$i+1
371	# done
372	# # delete temporary file
373	# \rm tmp_IGCM_sys_RshPost_$$
374	#
375	# if [ ${status} -gt 0 ] ; then
376	# echo "IGCM_sys_RshPost : erreur."
377	# IGCM_debug_Exit "IGCM_sys_RshPost"
378	# fi
379	# fi
380
381	# ============ CESIUM END ============ #
382
383	IGCM_debug_PopStack "IGCM_sys_RshPost"
384	}
385
386	#D-#==================================================
387	#D-function IGCM_sys_SendMail
388	#D-* Purpose: Send mail when simulation is over
389	#D-* Examples:
390	#D-
391	function IGCM_sys_SendMail {
392	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
393
394	if ( $DEBUG_sys ) ; then
395	echo "IGCM_sys_SendMail :" $@
396	fi
397
398	if [ X${1} = XAccounting ] ; then
399	status=Accounting
400	mailText=jobAccounting.mail
401	elif ( ${ExitFlag} ) ; then
402	status=failed
403	mailText=jobEnd.mail
404	else
405	status=completed
406	mailText=jobEnd.mail
407	fi
408
409	# Update selected mail template
410	while read -r line; do
411	eval echo $line >> mail.txt ;
412	done < ${libIGCM}/libIGCM_sys/${mailText}
413
414	if [ ! -z ${config_UserChoices_MailName} ] ; then
415	mail -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
416	elif [ -f ~/.forward ] ; then
417	mail -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
418	fi
419
420	sleep 10
421	rm -f mail.txt
422
423	IGCM_debug_PopStack "IGCM_sys_SendMail"
424	}
425
426	#D-#==================================================
427	#D-function IGCM_sys_Mkdir
428	#D-* Purpose: Master locale mkdir command
429	#D-* Examples:
430	#D-
431	function IGCM_sys_Mkdir {
432	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
433	if ( $DEBUG_sys ) ; then
434	echo "IGCM_sys_Mkdir :" $@
435	fi
436	if [ ! -d ${1} ]; then
437	\mkdir -p $1
438	if [ $? -gt 0 ] ; then
439	echo "IGCM_sys_Mkdir : erreur."
440	IGCM_debug_Exit "IGCM_sys_Mkdir"
441	fi
442	fi
443	# vérification :
444	if [ ! -d ${1} ] ; then
445	echo "IGCM_sys_Mkdir : erreur."
446	IGCM_debug_Exit "IGCM_sys_Mkdir"
447	fi
448	IGCM_debug_PopStack "IGCM_sys_Mkdir"
449	}
450
451	#D-#==================================================
452	#D-function IGCM_sys_MkdirArchive
453	#D-* Purpose: Mkdir on Archive
454	#D-* Examples:
455	#D-
456	function IGCM_sys_MkdirArchive {
457	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
458	if ( $DEBUG_sys ) ; then
459	echo "IGCM_sys_MkdirArchive :" $@
460	fi
461	#- creation de repertoire sur le serveur fichier
462	if [ ! -d ${1} ]; then
463	\mkdir -p $1
464	status=$?
465
466	if [ ${status} -gt 0 ] ; then
467	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
468	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
469	fi
470	fi
471	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
472	}
473
474	#D-#==================================================
475	#D-function IGCM_sys_MkdirWork
476	#D-* Purpose: Mkdir on Work
477	#D-* Examples:
478	#D-
479	function IGCM_sys_MkdirWork {
480	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
481	if ( $DEBUG_sys ) ; then
482	echo "IGCM_sys_MkdirWork :" $@
483	fi
484	#- creation de repertoire sur le serveur fichier
485	if [ ! -d ${1} ]; then
486	\mkdir -p $1
487	if [ $? -gt 0 ] ; then
488	echo "IGCM_sys_MkdirWork : erreur."
489	IGCM_debug_Exit "IGCM_sys_MkdirWork"
490	fi
491	fi
492	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
493	}
494
495	#D-#==================================================
496	#D-function IGCM_sys_Cd
497	#D-* Purpose: master cd command
498	#D-* Examples:
499	#D-
500	function IGCM_sys_Cd {
501	IGCM_debug_PushStack "IGCM_sys_Cd" $@
502	if ( $DEBUG_sys ) ; then
503	echo "IGCM_sys_Cd :" $@
504	fi
505	\cd $1
506	if [ $? -gt 0 ] ; then
507	echo "IGCM_sys_Cd : erreur."
508	IGCM_debug_Exit "IGCM_sys_Cd"
509	fi
510	IGCM_debug_PopStack "IGCM_sys_Cd"
511	}
512
513	#D-#==================================================
514	#D-function IGCM_sys_Chmod
515	#D-* Purpose: Chmod
516	#D-* Examples:
517	#D-
518	function IGCM_sys_Chmod {
519	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
520	if ( $DEBUG_sys ) ; then
521	echo "IGCM_sys_Chmod :" $@
522	fi
523	\chmod $@
524	if [ $? -gt 0 ] ; then
525	echo "IGCM_sys_Chmod : erreur."
526	IGCM_debug_Exit "IGCM_sys_Chmod"
527	fi
528	IGCM_debug_PopStack "IGCM_sys_Chmod"
529	}
530
531	#D-#==================================================
532	#D-function IGCM_sys_FileSize
533	#D-* Purpose: Filesize
534	#D-* Examples:
535	#D-
536	function IGCM_sys_FileSize {
537	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
538
539	typeset sizeF
540	set +A sizeF -- $( ls -la ${1} )
541	if [ $? -gt 0 ] ; then
542	IGCM_debug_Exit "IGCM_sys_FileSize"
543	fi
544	eval ${2}=${sizeF[4]}
545
546	IGCM_debug_PopStack "IGCM_sys_FileSize"
547	}
548
549	#D-#==================================================
550	#D-function IGCM_sys_TestDir
551	#D-* Purpose: Test Directory that must exists
552	#D-* Examples:
553	#D-
554	function IGCM_sys_TestDir {
555	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
556	if ( $DEBUG_sys ) ; then
557	echo "IGCM_sys_TestDir :" $@
558	fi
559	typeset ExistFlag
560	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
561	IGCM_debug_PopStack "IGCM_sys_TestDir"
562
563	return ${ExistFlag}
564	}
565
566	#D-#==================================================
567	#D-function IGCM_sys_TestDirArchive
568	#D-* Purpose: Test Directory that must exists on Archive
569	#D-* Examples:
570	#D-
571	function IGCM_sys_TestDirArchive {
572	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
573	if ( $DEBUG_sys ) ; then
574	echo "IGCM_sys_TestDirArchive :" $@
575	fi
576	typeset ExistFlag
577	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
578	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
579	return ${ExistFlag}
580	}
581
582	#D-#==================================================
583	#D-function IGCM_sys_IsFileArchived
584	#D-* Purpose: Test file that must NOT EXISTS on Archive
585	#D-* Examples:
586	#D-
587	function IGCM_sys_IsFileArchived {
588	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
589	if ( $DEBUG_sys ) ; then
590	echo "IGCM_sys_IsFileArchived :" $@
591	fi
592	typeset IsArchivedFlag
593	IsArchivedFlag=$( [ X$( echo $@ \| grep ^\/ccc\/store ) != X ] && echo 0 \|\| echo 1 )
594	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
595
596	return ${IsArchivedFlag}
597	}
598
599	#D-#==================================================
600	#D-function IGCM_sys_TestFileArchive
601	#D-* Purpose: Test file that must NOT EXISTS on Archive
602	#D-* Examples:
603	#D-
604	function IGCM_sys_TestFileArchive {
605	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
606	typeset ExistFlag
607	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
608	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
609
610	return ${ExistFlag}
611	}
612
613	#D-#==================================================
614	#D-function IGCM_sys_TestFileBuffer
615	#D-* Purpose: Test file that must NOT EXISTS on Buffer
616	#D-* Examples:
617	#D-
618	function IGCM_sys_TestFileBuffer {
619	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
620	typeset ExistFlag
621	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
622	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
623
624	return ${ExistFlag}
625	}
626
627	#D-#==================================================
628	#D-function IGCM_sys_CountFileArchive
629	#D-* Purpose: Count files on Archive filesystem
630	#D-* Examples:
631	#D-
632	function IGCM_sys_CountFileArchive {
633	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
634	ls ${@} 2>/dev/null \| wc -l
635	if [ $? -gt 0 ] ; then
636	echo "IGCM_sys_CountFileArchive : erreur."
637	fi
638	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
639	}
640
641	#D-#==================================================
642	#D-function IGCM_sys_CountFileBuffer
643	#D-* Purpose: Count files on Scratch filesystem
644	#D-* Examples:
645	#D-
646	function IGCM_sys_CountFileBuffer {
647	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
648	ls ${@} 2>/dev/null \| wc -l
649	if [ $? -gt 0 ] ; then
650	echo "IGCM_sys_CountFileBuffer : erreur."
651	fi
652	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
653	}
654
655	#D-#==================================================
656	#D-function IGCM_sys_Tree
657	#D-* Purpose: Tree directories with files on ${ARCHIVE}
658	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
659	#D-
660	function IGCM_sys_Tree {
661	IGCM_debug_PushStack "IGCM_sys_Tree" $@
662	if ( $DEBUG_sys ) ; then
663	echo "IGCM_sys_Tree :" $@
664	fi
665
666	\ls -lR ${@}
667
668	IGCM_debug_PopStack "IGCM_sys_Tree"
669	}
670
671	#D-#==================================================
672	#D-function IGCM_sys_Tar
673	#D-* Purpose: master tar command
674	#D-* Examples:
675	#D-
676	function IGCM_sys_Tar {
677	IGCM_debug_PushStack "IGCM_sys_Tar" $@
678	if ( $DEBUG_sys ) ; then
679	echo "IGCM_sys_Tar :" $@
680	fi
681	\tar cf $@
682	if [ $? -gt 0 ] ; then
683	echo "IGCM_sys_Tar : erreur."
684	IGCM_debug_Exit "IGCM_sys_Tar"
685	fi
686	IGCM_debug_PopStack "IGCM_sys_Tar"
687	}
688
689	#D-#==================================================
690	#D-function IGCM_sys_UnTar
691	#D-* Purpose: master un-tar command
692	#D-* Examples:
693	#D-
694	function IGCM_sys_UnTar {
695	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
696	if ( $DEBUG_sys ) ; then
697	echo "IGCM_sys_UnTar :" $@
698	fi
699	\tar xvf $1
700	if [ $? -gt 0 ] ; then
701	echo "IGCM_sys_UnTar : erreur."
702	IGCM_debug_Exit "IGCM_sys_UnTar"
703	fi
704	IGCM_debug_PopStack "IGCM_sys_UnTar"
705	}
706
707	#D-#==================================================
708	#D-function IGCM_sys_Qsub
709	#D-* Purpose: Qsub new job
710	#D-* Examples:
711	#D-
712	function IGCM_sys_Qsub {
713	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
714	if ( $DEBUG_sys ) ; then
715	echo "IGCM_sys_Qsub :" $@
716	fi
717	typeset options status
718	options="-o ${SUBMIT_DIR}/${Script_Output} -e ${SUBMIT_DIR}/${Script_Output}"
719
720	/usr/bin/ccc_msub ${options} $1 > /tmp/out_command.$$ 2>&1
721	status=$?
722
723	cat /tmp/out_command.$$
724	if [ ${status} -gt 0 ] ; then
725	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
726	IGCM_debug_Exit "IGCM_sys_Qsub"
727	else
728	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
729	fi
730	IGCM_debug_PopStack "IGCM_sys_Qsub"
731	}
732
733	#D-#==================================================
734	#D-function IGCM_sys_QsubPost
735	#D-* Purpose: Qsub new job on scalaire
736	#D-* Examples:
737	#D-
738	function IGCM_sys_QsubPost {
739	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
740	if ( $DEBUG_sys ) ; then
741	echo "IGCM_sys_QsubPost :" $@
742	fi
743	typeset options status
744	MY_PROJECT=$(echo ${BRIDGE_MSUB_PROJECT:=${DEFAULT_PROJECT}} \| cut -d@ -f1 )
745	options="-Q normal -A ${MY_PROJECT} -o ${POST_DIR}/${Script_Post_Output}.out -e ${POST_DIR}/${Script_Post_Output}.out"
746
747	/usr/bin/ccc_msub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command.$$ 2>&1
748	status=$?
749
750	cat /tmp/out_command.$$
751	if [ ${status} -gt 0 ] ; then
752	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
753	IGCM_debug_Exit "IGCM_sys_QsubPost"
754	else
755	JobID=$( gawk {'print $4'} /tmp/out_command.$$ )
756	fi
757	IGCM_debug_PopStack "IGCM_sys_QsubPost"
758	}
759
760	#D-*************************
761	#D- File transfer functions
762	#D-*************************
763	#D-
764
765	#D-#==================================================
766	#D-function IGCM_sys_Rsync_out
767	#D-* Purpose: treat return val of rsync
768	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
769	#D- Error values and explanations can depend on your system version.
770	function IGCM_sys_Rsync_out {
771	status=$1
772	if [ ! $status ] ; then
773	echo "rsync error !"
774	fi
775
776	if [ $MYLANG = "fr" ]; then
777	case $status in
778	0) return ;;
779	1) echo "Erreur de rsync ; RERR_SYNTAX : "
780	echo "Erreur de syntaxe ou d'utilisation."
781	return;;
782	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
783	echo "Incompatibilité de protocole."
784	return;;
785	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
786	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
787	echo "répertoires"
788	return;;
789	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
790	echo "Action demandée non supportée : une tentative de manipulation de"
791	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
792	echo "été faite ; ou une option qui est supportée par le client mais"
793	echo "pas par le serveur a été spécifiée."
794	return;;
795	10) echo "Erreur de rsync ; RERR_SOCKETIO"
796	echo "Erreur dans le socket d'entrée sortie"
797	return;;
798	11) echo "Erreur de rsync ; RERR_FILEIO"
799	echo "Erreur d'entrée sortie fichier"
800	return;;
801	12) echo "Erreur de rsync ; RERR_STREAMIO"
802	echo "Erreur dans flux de donnée du protocole rsync"
803	return;;
804	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
805	echo "Erreur avec les diagnostics du programme"
806	return;;
807	14) echo "Erreur de rsync ; RERR_IPC"
808	echo "Erreur dans le code IPC"
809	return;;
810	20) echo "Erreur de rsync ; RERR_SIGNAL"
811	echo "SIGUSR1 ou SIGINT reçu"
812	return;;
813	21) echo "Erreur de rsync ; RERR_WAITCHILD"
814	echo "Une erreur retournée par waitpid()"
815	return;;
816	22) echo "Erreur de rsync ; RERR_MALLOC"
817	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
818	return;;
819	23) echo ""
820	echo "Erreur fichier inexistant"
821	return;;
822	30) echo "Erreur de rsync ; RERR_TIMEOUT"
823	echo "Temps d'attente écoulé dans l'envoi/réception de données"
824	return;;
825	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
826	return;;
827	esac
828	elif [ $MYLANG = "en" ] ; then
829	case $status in
830	0) return;;
831	1) echo "rsync error : Syntax or usage error "
832	return;;
833	2) echo "rsync error : Protocol incompatibility "
834	return;;
835	3) echo "rsync error : Errors selecting input/output files, dirs"
836	return;;
837	4) echo "rsync error : Requested action not supported: an attempt"
838	echo "was made to manipulate 64-bit files on a platform that cannot support"
839	echo "them; or an option was specified that is supported by the client and"
840	echo "not by the server."
841	return;;
842	5) echo "rsync error : Error starting client-server protocol"
843	return;;
844	10) echo "rsync error : Error in socket I/O "
845	return;;
846	11) echo "rsync error : Error in file I/O "
847	return;;
848	12) echo "rsync error : Error in rsync protocol data stream "
849	return;;
850	13) echo "rsync error : Errors with program diagnostics "
851	return;;
852	14) echo "rsync error : Error in IPC code "
853	return;;
854	20) echo "rsync error : Received SIGUSR1 or SIGINT "
855	return;;
856	21) echo "rsync error : Some error returned by waitpid() "
857	return;;
858	22) echo "rsync error : Error allocating core memory buffers "
859	return;;
860	23) echo "rsync error : Partial transfer due to error"
861	return;;
862	24) echo "rsync error : Partial transfer due to vanished source files"
863	return;;
864	30) echo "rsync error : Timeout in data send/receive "
865	return;;
866	*) echo "rsync error : return code of rsync unknown :" $status
867	return;;
868	esac
869	else
870	echo "unknown language $MYLANG."
871	return
872	fi
873	}
874
875	#D-#==================================================
876	#D-function IGCM_sys_Miror_libIGCM
877	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
878	#D-* Examples:
879	#D-
880	function IGCM_sys_Mirror_libIGCM {
881	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
882	if ( $DEBUG_sys ) ; then
883	echo "IGCM_sys_Mirror_libIGCM"
884	fi
885
886	typeset status
887
888	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
889
890	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command.$$ 2>&1
891	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command.$$ 2>&1
892	status=$?
893
894	if [ ${status} -gt 0 ] ; then
895	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
896	cat /tmp/out_command.$$
897	fi
898	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
899	}
900
901	#D-#==================================================
902	#D-function IGCM_sys_Cp
903	#D-* Purpose: generic cp
904	#D-* Examples:
905	#D-
906	function IGCM_sys_Cp {
907	IGCM_debug_PushStack "IGCM_sys_Cp" $@
908	if ( $DEBUG_sys ) ; then
909	echo "IGCM_sys_Cp :" $@
910	fi
911
912	typeset status
913
914	echo cp $@ > /tmp/out_command.$$ 2>&1
915	\cp $@ >> /tmp/out_command.$$ 2>&1
916	status=$?
917
918	if [ ${status} -gt 0 ] ; then
919	echo "IGCM_sys_Cp : error code ${status}"
920	cat /tmp/out_command.$$
921	IGCM_debug_Exit "IGCM_sys_Cp"
922	else
923	\rm /tmp/out_command.$$
924	fi
925	IGCM_debug_PopStack "IGCM_sys_Cp"
926	}
927
928	#D-#==================================================
929	#D-function IGCM_sys_Rm
930	#D-* Purpose: generic rm
931	#D-* Examples:
932	#D-
933	function IGCM_sys_Rm {
934	IGCM_debug_PushStack "IGCM_sys_Rm" $@
935	if ( $DEBUG_sys ) ; then
936	echo "IGCM_sys_Rm :" $@
937	fi
938
939	typeset status
940
941	echo rm $@ > /tmp/out_command.$$ 2>&1
942	\rm $@ >> /tmp/out_command.$$ 2>&1
943	status=$?
944
945	if [ ${status} -gt 0 ] ; then
946	echo "IGCM_sys_Rm : error code ${status}"
947	cat /tmp/out_command.$$
948	IGCM_debug_Exit "IGCM_sys_Rm"
949	else
950	\rm /tmp/out_command.$$
951	fi
952	IGCM_debug_PopStack "IGCM_sys_Rm"
953	}
954
955	#D-#==================================================
956	#D-function IGCM_sys_RmRunDir
957	#D-* Purpose: rm tmpdir (dummy function most of the time batch
958	#D- scheduler will do the job)
959	#D-* Examples:
960	#D-
961	function IGCM_sys_RmRunDir {
962	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
963	if ( $DEBUG_sys ) ; then
964	echo "IGCM_sys_RmRunDir :" $@
965	fi
966
967	typeset status
968
969	echo rm $@ > /tmp/out_command.$$ 2>&1
970	\rm $@ >> /tmp/out_command.$$ 2>&1
971	status=$?
972
973	if [ ${status} -gt 0 ] ; then
974	echo "IGCM_sys_RmRunDir : rm error code is ${status}."
975	cat /tmp/out_command.$$
976	IGCM_debug_Exit "IGCM_sys_RmRunDir"
977	else
978	\rm /tmp/out_command.$$
979	fi
980	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
981	}
982
983	#D-#==================================================
984	#D-function IGCM_sys_Mv
985	#D-* Purpose: generic move
986	#D-* Examples:
987	#D-
988	function IGCM_sys_Mv {
989	IGCM_debug_PushStack "IGCM_sys_Mv" $@
990	if ( $DEBUG_sys ) ; then
991	echo "IGCM_sys_Mv :" $@
992	fi
993
994	if [ $DRYRUN = 0 ]; then
995
996	typeset status
997
998	echo mv $@ > /tmp/out_command.$$ 2>&1
999	\mv $@ >> /tmp/out_command.$$ 2>&1
1000	status=$?
1001
1002	if [ ${status} -gt 0 ] ; then
1003	echo "IGCM_sys_Mv : error code ${status}"
1004	cat /tmp/out_command.$$
1005	IGCM_debug_Exit "IGCM_sys_Mv"
1006	else
1007	\rm /tmp/out_command.$$
1008	fi
1009	fi
1010
1011	IGCM_debug_PopStack "IGCM_sys_Mv"
1012	}
1013
1014	#D-#==================================================
1015	#D-function IGCM_sys_Put_Dir
1016	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
1017	#D-* Examples:
1018	#D-
1019	function IGCM_sys_Put_Dir {
1020	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
1021	if ( $DEBUG_sys ) ; then
1022	echo "IGCM_sys_Put_Dir :" $@
1023	fi
1024	if [ $DRYRUN = 0 ]; then
1025	if [ ! -d ${1} ] ; then
1026	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
1027	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1028	return
1029	fi
1030
1031	typeset status
1032
1033	# Only if we use rsync
1034	#IGCM_sys_TestDirArchive $( dirname $2 )
1035	#
1036	#USUAL WAY
1037	\cp -r $1 $2 > /tmp/out_command.$$ 2>&1
1038	status=$?
1039
1040	if [ ${status} -gt 0 ] ; then
1041	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
1042	cat /tmp/out_command.$$
1043	IGCM_debug_Exit "IGCM_sys_Put_Dir"
1044	else
1045	\rm /tmp/out_command.$$
1046	fi
1047	fi
1048	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
1049	}
1050
1051	#D-#==================================================
1052	#D-function IGCM_sys_Get_Dir
1053	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
1054	#D-* Examples:
1055	#D-
1056	function IGCM_sys_Get_Dir {
1057	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
1058	if ( $DEBUG_sys ) ; then
1059	echo "IGCM_sys_Get_Dir :" $@
1060	fi
1061	if [ $DRYRUN = 0 ]; then
1062	# if [ ! -d ${1} ] ; then
1063	# echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
1064	# IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1065	# return
1066	# fi
1067
1068	typeset NB_ESSAI DELAI status i
1069	# number of tentative
1070	NB_ESSAI=3
1071	# time delay between tentative
1072	DELAI=2
1073
1074	# Only if we use rsync
1075	#IGCM_sys_TestDirArchive $( dirname $2 )
1076	#
1077	# USUAL WAY
1078	# add 'ccc_hsm get' (to demigrate all offline files) to reduce time of this command :
1079	ccc_hsm get -r $1
1080
1081	i=0
1082	while [ $i -lt $NB_ESSAI ] ; do
1083	\cp -ur $1 $2 >> /tmp/out_command.$$ 2>&1
1084	status=$?
1085	if [ ${status} -gt 0 ] ; then
1086	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
1087	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
1088	sleep $DELAI
1089	else
1090	break
1091	fi
1092	(( i = i + 1 ))
1093	done
1094
1095	if [ ${status} -gt 0 ] ; then
1096	echo "IGCM_sys_Get_Dir : error."
1097	cat /tmp/out_command.$$
1098	IGCM_debug_Exit "IGCM_sys_Get_Dir"
1099	else
1100	\rm /tmp/out_command.$$
1101	fi
1102	fi
1103	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
1104	}
1105
1106	#D-#==================================================
1107	#D-function IGCM_sys_Get_Master
1108	#D-* Purpose: Copy a complete directory from MASTER filesystem
1109	#D-* Examples:
1110	#D-
1111	function IGCM_sys_Get_Master {
1112	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
1113	if ( $DEBUG_sys ) ; then
1114	echo "IGCM_sys_Get_Master :" $@
1115	fi
1116	if [ $DRYRUN = 0 ]; then
1117	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
1118	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
1119	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1120	return
1121	fi
1122
1123	typeset NB_ESSAI DELAI status i
1124	# number of tentative
1125	NB_ESSAI=3
1126	# time delay between tentative
1127	DELAI=2
1128
1129	i=0
1130	while [ $i -lt $NB_ESSAI ] ; do
1131	\cp -urL $1 $2 > /tmp/out_command.$$ 2>&1
1132	status=$?
1133	if [ ${status} -gt 0 ]; then
1134	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
1135	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
1136	sleep $DELAI
1137	else
1138	break
1139	fi
1140	(( i = i + 1 ))
1141	done
1142
1143	if [ ${status} -gt 0 ] ; then
1144	echo "IGCM_sys_Get_Master : error."
1145	cat /tmp/out_command.$$
1146	IGCM_debug_Exit "IGCM_sys_Get_Master"
1147	else
1148	\rm /tmp/out_command.$$
1149	fi
1150	fi
1151	IGCM_debug_PopStack "IGCM_sys_Get_Master"
1152	}
1153
1154	#====================================================
1155	#- Call IGCM_sys_Mirror_libIGCM now !
1156	if ( $MirrorlibIGCM ) ; then
1157	IGCM_sys_Mirror_libIGCM
1158	fi
1159
1160	#D-#==================================================
1161	#D-function IGCM_sys_Put_Rest
1162	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
1163	#D- File and target directory must exist.
1164	#D-* Examples:
1165	#D-
1166	function IGCM_sys_Put_Rest {
1167	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
1168	if ( $DEBUG_sys ) ; then
1169	echo "IGCM_sys_Put_Rest :" $@
1170	fi
1171	if [ $DRYRUN = 0 ]; then
1172	if [ ! -f ${1} ] ; then
1173	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
1174	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1175	fi
1176
1177	typeset status
1178	#
1179	# USUAL WAY
1180	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1181	status=$?
1182
1183	# #RSYNC WITH NETWORK SSH CALL
1184	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1185	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1186
1187	# #RSYNC WITH NFS USE
1188	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1189	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1190
1191	# status=$?
1192	# IGCM_sys_Rsync_out $status
1193
1194	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1195	# (( status=status+$? ))
1196
1197	if [ ${status} -gt 0 ] ; then
1198	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
1199	[ -f ${1} ] && ls -l ${1}
1200	[ -f ${2} ] && ls -l ${2}
1201	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1202	cat /tmp/out_command.$$
1203	IGCM_debug_Exit "IGCM_sys_Put_Rest"
1204	else
1205
1206	if [ X${JobType} = XRUN ] ; then
1207	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1208	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1209	fi
1210
1211	\rm /tmp/out_command.$$
1212	fi
1213	fi
1214	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
1215	}
1216
1217	#D-#==================================================
1218	#D-function IGCM_sys_PutBuffer_Rest
1219	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
1220	#D- File and target directory must exist.
1221	#D-* Examples:
1222	#D-
1223	function IGCM_sys_PutBuffer_Rest {
1224	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
1225	if ( $DEBUG_sys ) ; then
1226	echo "IGCM_sys_PutBuffer_Rest :" $@
1227	fi
1228	if [ $DRYRUN = 0 ]; then
1229	if [ ! -f ${1} ] ; then
1230	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
1231	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1232	fi
1233
1234	typeset status
1235	#
1236	# USUAL WAY
1237	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1238	status=$?
1239
1240	if [ ${status} -gt 0 ] ; then
1241	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
1242	[ -f ${2} ] && ls -l ${2}
1243	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1244	cat /tmp/out_command.$$
1245	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
1246	else
1247
1248	if [ X${JobType} = XRUN ] ; then
1249	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1250	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1251	fi
1252
1253	\rm /tmp/out_command.$$
1254	fi
1255	fi
1256	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
1257	}
1258
1259	#D-#==================================================
1260	#D-function IGCM_sys_PrepareTaredRestart
1261	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
1262	#D-* Examples:
1263	#D-
1264	function IGCM_sys_PrepareTaredRestart {
1265	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
1266	if [ $DRYRUN = 0 ]; then
1267	[ ! -f $( basename $1 ) ] && IGCM_sys_Get $1 .
1268	fi
1269	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
1270	}
1271
1272	#D-#==================================================
1273	#D-function IGCM_sys_Put_Out
1274	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
1275	#D-* Examples:
1276	#D-
1277	function IGCM_sys_Put_Out {
1278	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
1279	if ( $DEBUG_sys ) ; then
1280	echo "IGCM_sys_Put_Out :" $@
1281	fi
1282
1283	typeset NB_ESSAI DELAI status i exist skip
1284
1285	# number of tentative
1286	NB_ESSAI=3
1287	# time delay between tentative
1288	DELAI=2
1289
1290	if [ $DRYRUN = 0 ]; then
1291	if [ ! -f ${1} ] ; then
1292	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
1293	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1294	return 1
1295	fi
1296	#
1297	IGCM_sys_MkdirArchive $( dirname $2 )
1298	#
1299
1300	#=====================================================
1301	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1302	#=====================================================
1303
1304	#echo ${2} \| grep "${R_OUT}" > /dev/null 2>&1
1305	#if [ $? -eq 0 ] ; then
1306	# typeset WORKPATH FILEPATH
1307	# WORKPATH=$( dirname $2 \| sed -e "s\|${R_OUT}\|${R_BUF}\|" )
1308	# IGCM_sys_MkdirWork ${WORKPATH}
1309	# FILEPATH=${WORKPATH}/$( basename $2 )
1310	# #
1311	# IGCM_sys_Cp ${1} ${FILEPATH}
1312	#fi
1313
1314	exist=false
1315	skip=false
1316	if [ -f $2 ] ; then
1317	IGCM_debug_Print 1 "$2 already exist"
1318	ccc_hsm get $2
1319	exist=true
1320	if [ "X$( diff $1 $2 )" = X ] ; then
1321	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1322	skip=true
1323	else
1324	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1325	skip=false
1326	fi
1327	fi
1328	#
1329	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1330	IGCM_sys_Chmod u+w $2
1331	fi
1332
1333	if [ X${skip} = Xfalse ] ; then
1334	i=0
1335	while [ $i -lt $NB_ESSAI ] ; do
1336	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1337	# USUAL WAY
1338	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1339	status=$?
1340	else
1341	# NOT SO USUAL WAY
1342	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1343	status=$?
1344	fi
1345	if [ ${status} -gt 0 ]; then
1346	IGCM_debug_Print 2 "IGCM_sys_Put_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1347	IGCM_debug_Print 2 "IGCM_sys_Put_Out : sleep ${DELAI} seconds and try again."
1348	[ -f ${1} ] && ls -l ${1}
1349	[ -f ${2} ] && ls -l ${2}
1350	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1351	sleep $DELAI
1352	else
1353	break
1354	fi
1355	(( i = i + 1 ))
1356	done
1357	fi
1358
1359	# #RSYNC WITH NETWORK SSH CALL
1360	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command.$$ 2>&1
1361	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command.$$ 2>&1
1362
1363	# #RSYNC WITH NFS USE
1364	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command.$$ 2>&1
1365	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command.$$ 2>&1
1366
1367	# status=$?
1368	# IGCM_sys_Rsync_out $status
1369
1370	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1371	# (( status=status+$? ))
1372
1373	if [ ${status} -gt 0 ] ; then
1374	echo "IGCM_sys_Put_Out : error."
1375	[ -f ${1} ] && ls -l ${1}
1376	[ -f ${2} ] && ls -l ${2}
1377	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1378	cat /tmp/out_command.$$
1379	IGCM_debug_Exit "IGCM_sys_Put_Out"
1380	else
1381
1382	if [ X${JobType} = XRUN ] ; then
1383	if [ X${3} = X ] ; then
1384	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1385	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1386	fi
1387	fi
1388
1389	\rm /tmp/out_command.$$
1390	fi
1391	fi
1392	IGCM_debug_PopStack "IGCM_sys_Put_Out"
1393	return 0
1394	}
1395
1396	#D-#==================================================
1397	#D-function IGCM_sys_PutBuffer_Out
1398	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
1399	#D-* Examples:
1400	#D-
1401	function IGCM_sys_PutBuffer_Out {
1402	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
1403	if ( $DEBUG_sys ) ; then
1404	echo "IGCM_sys_PutBuffer_Out :" $@
1405	fi
1406
1407	typeset NB_ESSAI DELAI status i exist skip
1408
1409	# number of tentative
1410	NB_ESSAI=3
1411	# time delay between tentative
1412	DELAI=2
1413
1414	if [ $DRYRUN = 0 ]; then
1415	if [ ! -f ${1} ] ; then
1416	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
1417	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1418	return 1
1419	fi
1420	#
1421	IGCM_sys_Mkdir $( dirname $2 )
1422	#
1423
1424	exist=false
1425	skip=false
1426	if [ -f $2 ] ; then
1427	IGCM_debug_Print 1 "$2 already exist"
1428	exist=true
1429	if [ "X$( diff $1 $2 )" = X ] ; then
1430	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
1431	status=0
1432	skip=true
1433	else
1434	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
1435	skip=false
1436	fi
1437	fi
1438	#
1439	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
1440	IGCM_sys_Chmod u+w $2
1441	fi
1442
1443	if [ X${skip} = Xfalse ] ; then
1444	i=0
1445	while [ $i -lt $NB_ESSAI ] ; do
1446	if [ $( stat -c %d $1 ) -ne $( stat -c %d $( dirname $2 ) ) ] ; then
1447	# USUAL WAY
1448	\cp $1 $2 > /tmp/out_command.$$ 2>&1
1449	status=$?
1450	else
1451	# NOT SO USUAL WAY
1452	\mv $1 $2 > /tmp/out_command.$$ 2>&1
1453	status=$?
1454	fi
1455	if [ ${status} -gt 0 ]; then
1456	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
1457	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
1458	[ -f ${2} ] && ls -l ${2}
1459	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1460	sleep $DELAI
1461	else
1462	break
1463	fi
1464	(( i = i + 1 ))
1465	done
1466	fi
1467
1468	if [ ${status} -gt 0 ] ; then
1469	echo "IGCM_sys_PutBuffer_Out : error."
1470	[ -f ${2} ] && ls -l ${2}
1471	[ -f ${2}/${1} ] && ls -l ${2}/${1}
1472	cat /tmp/out_command.$$
1473	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
1474	else
1475
1476	if [ X${JobType} = XRUN ] ; then
1477	if [ X${3} = X ] ; then
1478	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
1479	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
1480	fi
1481	fi
1482
1483	\rm /tmp/out_command.$$
1484	fi
1485	fi
1486	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
1487	return 0
1488	}
1489
1490	#D-#==================================================
1491	#D-function IGCM_sys_Get
1492	#D-* Purpose: Get a file from ${ARCHIVE}
1493	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
1494	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
1495	function IGCM_sys_Get {
1496	IGCM_debug_PushStack "IGCM_sys_Get" $@
1497
1498	typeset DEST dm_liste target file_work
1499	typeset NB_ESSAI DELAI status i
1500
1501	if ( $DEBUG_sys ) ; then
1502	echo "IGCM_sys_Get :" $@
1503	fi
1504
1505	# number of tentative
1506	NB_ESSAI=3
1507	# time delay between tentative
1508	DELAI=2
1509
1510	if [ $DRYRUN -le 2 ]; then
1511	if [ X${1} = X'/l' ] ; then
1512	eval set +A dm_liste \${${2}}
1513	else
1514	eval set +A dm_liste ${1}
1515	fi
1516	eval DEST=\${${#}}
1517
1518	#=====================================================
1519	# COMMENT OUT DOUBLE COPY ON SCRATCHDIR
1520	#=====================================================
1521
1522	# Is it an R_OUT file (not R_IN) ?
1523	#echo ${dm_liste[0]} \| grep "${R_OUT}" > /dev/null 2>&1
1524	#if [ $? -eq 0 ] ; then
1525	# # Yes ? then we try to get it in SCRATCHDIR
1526	# set +A file_work $( echo ${dm_liste[*]} \| sed -e "s\|${R_OUT}\|${R_BUF}\|g" )
1527	# if [ -f ${file_work[0]} ] ; then
1528	# IGCM_sys_Cp ${file_work[*]} ${DEST}
1529	# IGCM_debug_PopStack "IGCM_sys_Get"
1530	# return
1531	# fi
1532	#fi
1533
1534	ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1535	status=$?
1536	if [ ${status} -gt 0 ] ; then
1537	echo "WARNING IGCM_sys_Get : error code ${status}"
1538	cat /tmp/out_command.$$
1539	echo "WARNING IGCM_sys_Get : will stop later if the cp fails."
1540	fi
1541
1542	#if [ ${status} -gt 0 ] ; then
1543	# if [ ! "X$( grep "Lost dmusrcmd connection" /tmp/out_command.$$ )" = "X" ] ; then
1544	# cat /tmp/out_command.$$
1545	# echo "WARNING IGCM_sys_Get : Lost dmusrcmd connection : "
1546	# sleep 30
1547	# echo "We try another time"
1548	## dmget ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1549	# ccc_hsm get ${dm_liste[*]} > /tmp/out_command.$$ 2>&1
1550	# status=$?
1551	# if [ ${status} -gt 0 ] ; then
1552	# echo "ERROR IGCM_sys_Get : again demigration error :"
1553	# cat /tmp/out_command.$$
1554	# IGCM_debug_Exit "IGCM_sys_Get"
1555	# fi
1556	# else
1557	# echo "ERROR IGCM_sys_Get : demigration error :"
1558	# cat /tmp/out_command.$$
1559	# IGCM_debug_Exit "IGCM_sys_Get"
1560	# fi
1561	#fi
1562
1563	# #RSYNC WITH NETWORK SSH CALL
1564	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1565	# ${RSYNC} ${RSYNC_opt} -e ssh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1566
1567	# #RSYNC WITH NFS USE
1568	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > /tmp/out_command.$$ 2>&1
1569	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> /tmp/out_command.$$ 2>&1
1570
1571	# status=$?
1572	# IGCM_sys_Rsync_out $status
1573
1574	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command.$$
1575	# (( status=status+$? ))
1576
1577	#USUAL WAY
1578	if [ X${1} = X'/l' ] ; then
1579	for target in ${dm_liste[*]} ; do
1580	local_file=$( basename ${target} )
1581	# test if the target file is present before the loop
1582	IGCM_sys_TestFileArchive ${target}
1583	status=$?
1584	if [ ${status} -gt 0 ] ; then
1585	echo "IGCM_sys_Get, ERROR : regular file ${target} DOES NOT EXIST ."
1586	IGCM_debug_Exit "IGCM_sys_Get"
1587	else
1588	i=0
1589	while [ $i -lt $NB_ESSAI ] ; do
1590	#if [ X${DoLink} = Xtrue ] ; then
1591	# \ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1592	# status=$?
1593	# else
1594	# \cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1595	# status=$?
1596	#fi
1597	\ln -s ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1598	status=$?
1599	if [ ${status} -gt 0 ]; then
1600	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1601	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1602	sleep $DELAI
1603	else
1604	break
1605	fi
1606	(( i = i + 1 ))
1607	done
1608	if [ ${status} -gt 0 ] ; then
1609	echo "IGCM_sys_Get : error"
1610	cat /tmp/out_command.$$
1611	\rm /tmp/out_command.$$
1612	IGCM_debug_Exit "IGCM_sys_Get"
1613	else
1614	\rm /tmp/out_command.$$
1615	fi
1616	fi
1617	done
1618	else
1619	i=0
1620	while [ $i -lt $NB_ESSAI ] ; do
1621	\cp ${dm_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1622	status=$?
1623	if [ ${status} -gt 0 ]; then
1624	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status} ${i}/${NB_ESSAI}"
1625	IGCM_debug_Print 2 "IGCM_sys_Get : sleep ${DELAI} seconds and try again."
1626	sleep $DELAI
1627	else
1628	break
1629	fi
1630	(( i = i + 1 ))
1631	done
1632	if [ ${status} -gt 0 ] ; then
1633	echo "IGCM_sys_Get : error"
1634	cat /tmp/out_command.$$
1635	\rm /tmp/out_command.$$
1636	IGCM_debug_Exit "IGCM_sys_Get"
1637	else
1638	\rm /tmp/out_command.$$
1639	fi
1640	fi
1641	fi
1642	IGCM_debug_PopStack "IGCM_sys_Get"
1643	}
1644
1645	#D-#==================================================
1646	#D-function IGCM_sys_GetBuffer
1647	#D-* Purpose: Get a file from ${SCRATCHDIR}
1648	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
1649	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
1650	function IGCM_sys_GetBuffer {
1651	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
1652
1653	typeset DEST buf_liste target file_work
1654	typeset NB_ESSAI DELAI status i
1655
1656	if ( $DEBUG_sys ) ; then
1657	echo "IGCM_sys_GetBuffer :" $@
1658	fi
1659
1660	# number of tentative
1661	NB_ESSAI=3
1662	# time delay between tentative
1663	DELAI=2
1664
1665	if [ $DRYRUN -le 2 ]; then
1666	if [ X${1} = X'/l' ] ; then
1667	eval set +A buf_liste \${${2}}
1668	else
1669	eval set +A buf_liste ${1}
1670	fi
1671	eval DEST=\${${#}}
1672
1673	#USUAL WAY
1674	if [ X${1} = X'/l' ] ; then
1675	for target in ${buf_liste[*]} ; do
1676	local_file=$( basename ${target} )
1677	i=0
1678	while [ $i -lt $NB_ESSAI ] ; do
1679	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command.$$ 2>&1
1680	status=$?
1681	if [ ${status} -gt 0 ]; then
1682	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1683	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1684	sleep $DELAI
1685	else
1686	break
1687	fi
1688	(( i = i + 1 ))
1689	done
1690	if [ ${status} -gt 0 ] ; then
1691	echo "IGCM_sys_Get : error"
1692	cat /tmp/out_command.$$
1693	\rm /tmp/out_command.$$
1694	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1695	else
1696	\rm /tmp/out_command.$$
1697	fi
1698	done
1699	else
1700	i=0
1701	while [ $i -lt $NB_ESSAI ] ; do
1702	\cp ${buf_liste} ${DEST} >> /tmp/out_command.$$ 2>&1
1703	status=$?
1704	if [ ${status} -gt 0 ]; then
1705	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
1706	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
1707	sleep $DELAI
1708	else
1709	break
1710	fi
1711	(( i = i + 1 ))
1712	done
1713	if [ ${status} -gt 0 ] ; then
1714	echo "IGCM_sys_Get : error"
1715	cat /tmp/out_command.$$
1716	\rm /tmp/out_command.$$
1717	IGCM_debug_Exit "IGCM_sys_GetBuffer"
1718	else
1719	\rm /tmp/out_command.$$
1720	fi
1721	fi
1722	fi
1723	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
1724	}
1725
1726	#D-#==================================================
1727	#D-function IGCM_sys_GetDate_FichWork
1728	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
1729	#D-* Examples:
1730	#D-
1731	function IGCM_sys_GetDate_FichWork {
1732	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
1733	if ( $DEBUG_sys ) ; then
1734	echo "IGCM_sys_GetDate_FichWork :" $@
1735	fi
1736
1737	if [ $# -ge 3 ] ; then
1738	mode=$3
1739	TimeStyle=$4
1740	else
1741	mode="default"
1742	TimeStyle="%Y%m%d%H%M%S"
1743	fi
1744
1745	typeset dateF
1746	set +A dateF -- $( ls -l --full-time --time-style=+"${TimeStyle}" ${1} )
1747
1748	case $mode in
1749	"default")
1750	eval ${2}=${dateF[5]}
1751	;;
1752	"SplitFields")
1753	eval ${2}="${dateF[5]}\ ${dateF[6]}"
1754	;;
1755	esac
1756
1757
1758	# donne la date filesys d'un fichier sur la machine work
1759	IGCM_debug_PopStack "IGCM_sys_FichWork"
1760	}
1761
1762	#D-#==================================================
1763	#D-function IGCM_sys_GetDate_FichArchive
1764	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
1765	#D-* Examples:
1766	#D-
1767	function IGCM_sys_GetDate_FichArchive {
1768	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
1769	if ( $DEBUG_sys ) ; then
1770	echo "IGCM_sys_GetDate_FichArchive :" $@
1771	fi
1772	typeset dateF
1773	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
1774	eval ${2}=${dateF[5]}
1775
1776	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1777	}
1778
1779	#D-#==================================================
1780	#D-function IGCM_sys_GetDate_Monitoring
1781	#D-* Purpose: get the last year for which the monitoring has been computed
1782	#D-* Examples:
1783	#D-
1784	function IGCM_sys_GetDate_Monitoring {
1785	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
1786	if ( $DEBUG_sys ) ; then
1787	echo "IGCM_sys_GetDate_Monitoring :" $@
1788	fi
1789
1790	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
1791
1792	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
1793	}
1794
1795	#D-#==================================================
1796	#D-function IGCM_sys_Dods_Rm
1797	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
1798	#D-* Examples:
1799	#D-
1800	function IGCM_sys_Dods_Rm {
1801	if ( $DEBUG_sys ) ; then
1802	echo "IGCM_sys_Dods_Rm :" $@
1803	fi
1804	typeset status
1805	if [ $DRYRUN = 0 ]; then
1806
1807	# if [ ! -d /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} ] ; then
1808	# echo "WARNING : IGCM_sys_Dods_Rm /dmnfs/cont003/dods/public/${LOGIN}/${R_DODS}/${1} DOES NOT EXIST ."
1809	# echo "Nothing has been done."
1810	# return
1811	# fi
1812
1813	/ccc/cont003/home/dsm/p86ipsl/bin/dods_rm public/${LOGIN}/${R_DODS}/${1} # > out_dods_rm 2>&1
1814	status=$?
1815
1816	# if [ ${status} -gt 0 ] ; then
1817	# echo "IGCM_sys_Dods_Rm : error."
1818	# cat out_dods_rm
1819	# IGCM_debug_Exit "IGCM_sys_Dods_Rm"
1820	# else
1821	# rm out_dods_rm
1822	# fi
1823
1824	fi
1825	return $status
1826	}
1827
1828	#D-#==================================================
1829	#D-function IGCM_sys_Dods_Cp
1830	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
1831	#D-* Examples:
1832	#D-
1833	function IGCM_sys_Dods_Cp {
1834	if ( $DEBUG_sys ) ; then
1835	echo "IGCM_sys_Dods_Cp :" $@
1836	fi
1837	typeset status
1838	if [ $DRYRUN = 0 ]; then
1839
1840	# if [ ! -d ${R_SAVE}/${1} ] ; then
1841	# echo "WARNING : IGCM_sys_Dods_Cp ${R_SAVE}/${1} DOES NOT EXIST ."
1842	# echo "Nothing has been done."
1843	# return
1844	# fi
1845
1846	/ccc/cont003/home/dsm/p86ipsl/bin/dods_cp ${1} public/${LOGIN}/${R_DODS} # > out_dods_cp 2>&1
1847	status=$?
1848
1849	# if [ ${status} -gt 0 ] ; then
1850	# echo "IGCM_sys_Dods_Cp : error."
1851	# cat out_dods_cp
1852	# IGCM_debug_Exit "IGCM_sys_Dods_Cp"
1853	# else
1854	# rm out_dods_cp
1855	# fi
1856
1857	fi
1858	return $status
1859	}
1860
1861	#D-#==================================================
1862	#D-function IGCM_sys_Put_Dods
1863	#D-* Purpose: Put $(ARCHIVE) files on DODS internet protocole.
1864	#D-* Examples:
1865	#D-
1866	function IGCM_sys_Put_Dods {
1867	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
1868	if ( $DEBUG_sys ) ; then
1869	echo "IGCM_sys_Put_Dods :" $@
1870	fi
1871	#set -vx
1872	typeset status
1873	if [ $DRYRUN = 0 ]; then
1874	if ( [ ! -d ${R_SAVE}/${1} ] && [ ! -d ${R_FIGR}/${1} ] ) ; then
1875	echo "WARNING IGCM_sys_Put_Dods : None of the following directories exist. Exactly one should."
1876	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} DOES NOT EXIST."
1877	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} DOES NOT EXIST."
1878	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1879	return
1880	fi
1881
1882	if ( [ -d ${R_SAVE}/${1} ] && [ -d ${R_FIGR}/${1} ] ) ; then
1883	echo "WARNING IGCM_sys_Put_Dods : Both of the following directories exist. Exactly one should."
1884	echo "WARNING IGCM_sys_Put_Dods : ${R_SAVE}/${1} EXISTS."
1885	echo "WARNING IGCM_sys_Put_Dods : ${R_FIGR}/${1} EXISTS."
1886	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1887	return
1888	fi
1889	#
1890	if [ -d ${R_SAVE}/${1} ] ; then
1891	cd ${R_SAVE}
1892	elif [ -d ${R_FIGR}/${1} ] ; then
1893	cd ${R_FIGR}
1894	fi
1895
1896	IGCM_sys_Dods_Rm ${1}
1897	IGCM_sys_Dods_Cp ${1}
1898	status=0
1899
1900	if [ ${status} -gt 0 ] ; then
1901	echo "IGCM_sys_Put_Dods : error."
1902	IGCM_debug_Exit "IGCM_sys_Put_Dods"
1903	fi
1904	fi
1905	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
1906	}
1907
1908	##############################################################
1909	# REBUILD OPERATOR
1910
1911	#D-#==================================================
1912	#D-function IGCM_sys_rebuild
1913	#D-* Purpose: rebuild parallel files
1914	#D-* Examples:
1915	#D-
1916	function IGCM_sys_rebuild {
1917	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
1918	if ( $DEBUG_sys ) ; then
1919	echo "IGCM_sys_rebuild :" $@
1920	fi
1921
1922	typeset NB_ESSAI DELAI status i firstArg
1923	# number of tentative
1924	NB_ESSAI=3
1925	# time delay between tentative
1926	DELAI=2
1927
1928	i=0
1929	while [ $i -lt $NB_ESSAI ] ; do
1930	~p86ipsl/rebuild/src_X64_CURIE/modipsl_v2_2_2_netcdf4.2/bin/rebuild -f -o $@ > /tmp/out_command.$$ 2>&1
1931	status=$?
1932	if [ ${status} -gt 0 ] ; then
1933	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
1934	cat /tmp/out_command.$$
1935	\rm /tmp/out_command.$$
1936	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
1937	firstArg=${1}
1938	\rm ${firstArg}
1939	sleep $DELAI
1940	else
1941	\rm /tmp/out_command.$$
1942	break
1943	fi
1944	(( i = i + 1 ))
1945	done
1946
1947	if [ ${status} -gt 0 ] ; then
1948	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
1949	IGCM_debug_Exit "rebuild"
1950	fi
1951
1952	IGCM_debug_PopStack "IGCM_sys_rebuild"
1953	}
1954
1955	#D-#==================================================
1956	#D-function IGCM_sys_rebuild_station
1957	#D-* Purpose: rebuild parallel files describing station
1958	#D-* Examples:
1959	#D-
1960	function IGCM_sys_rebuild_station {
1961	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
1962	typeset i list_opt file_in file_out prefix_invert list_invert
1963	if ( $DEBUG_sys ) ; then
1964	echo "IGCM_sys_rebuild_station :" $@
1965	fi
1966	list_opt=$@
1967
1968	# Invert Axis : t,x -> x,t
1969	# t,pres,x -> x,t,pres
1970	# So that we can concatenate along x
1971	i=0
1972	for file_in in ${list_opt} ; do
1973	(( i = i + 1))
1974	[ ${i} = 1 ] && file_out=${file_in} && continue
1975	# detect time counter and do the job only if present
1976	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1)
1977	if [ X${var_unlim} = Xtime_counter ] ; then
1978	prefix_invert=$( basename ${file_in} .nc )
1979	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
1980	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
1981	fi
1982	done
1983
1984	# Concatenate
1985	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
1986
1987	# Re-ivert file
1988	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
1989
1990	# Station re-ordering is too expansive to be run within libIGCM
1991	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
1992	# This re-ordering must be done "in memory" by the cmorization process
1993	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
1994	# BEGIN reordering
1995
1996	# Only LMDZ text output contains the exact ordering of the station.
1997	# We isolate this in the code below:
1998	# 0 38 -157.5000000000000 70.98591549295774
1999	# 0 54 27.49999999999999 67.18309859154928
2000	# 0 56 -62.50000000000001 82.39436619718309
2001	# 0 79 12.49999999999999 78.59154929577466
2002	# 0 116 -165.0000000000000 76.05633802816901
2003	# 0 117 130.0000000000000 70.98591549295774
2004	# 0 118 110.0000000000000 87.46478873239437
2005	# 1 40 4.999999999999995 51.97183098591550
2006
2007	# typeset iStation iProc list_opt file_in file_out prefix_invert
2008	# typeset -Z4 j4
2009	# typeset -Z3 j3
2010
2011	# unset list_opt
2012	# set +A list_opt $@
2013
2014	# Filename after rebuild
2015	# file_out=${list_opt[0]}
2016	# Prefix of output files
2017	# prefix_invert=$( basename ${file_out} .nc )
2018	# Number of procs
2019	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
2020
2021	# iProc=0
2022	# while [ ${iProc} -lt ${num_proc} ] ; do
2023	# Array containing Station as a number
2024	# unset proc_stn
2025	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
2026	# Number of stations produced by processor proc
2027	# stationLast=${#proc_stn[*]}
2028	# Proc number on 4 digits
2029	# j4=${iProc}
2030	# Init
2031	# iStation=0
2032	# while [ ${iStation} -lt ${stationLast} ] ; do
2033	# Station number on 3 digits
2034	# j3=${proc_stn[${iStation}]}
2035	# Extract station
2036	# Invert Axis : t,x -> x,t
2037	# t,pres,x -> x,t,pres
2038	# So that we can concatenate along x
2039	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
2040	# (( iStation = iStation + 1 ))
2041	# done
2042	# (( iProc = iProc + 1 ))
2043	# done
2044
2045	# Concatenate all station along x
2046	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
2047
2048	# Re-invert file
2049	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
2050
2051	# END reordering
2052
2053	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
2054	}
2055
2056	############################################################
2057	# Activate Running Environnment Variables
2058
2059	#D-#==================================================
2060	#D-function IGCM_sys_desactiv_variables
2061	#D-* Purpose: set environement variables prior to execution
2062	#D-* Examples:
2063	#D-
2064	function IGCM_sys_activ_variables {
2065	IGCM_debug_PushStack "IGCM_sys_activ_variables"
2066	if ( $DEBUG_sys ) ; then
2067	echo "IGCM_sys_activ_variables"
2068	fi
2069
2070	# --------------------------------------------------------------------
2071	#D- MPI specifications
2072	# --------------------------------------------------------------------
2073
2074	# --------------------------------------------------------------------
2075	#D- Other specifications
2076	# --------------------------------------------------------------------
2077
2078	ulimit -s unlimited
2079
2080	IGCM_debug_PopStack "IGCM_sys_activ_variables"
2081	}
2082
2083	############################################################
2084	# Desactivate Running Environnment Variables
2085
2086	#D-#==================================================
2087	#D-function IGCM_sys_desactiv_variables
2088	#D-* Purpose: unset environement variables after execution
2089	#D-* Examples:
2090	#D-
2091	function IGCM_sys_desactiv_variables {
2092	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
2093	if ( $DEBUG_sys ) ; then
2094	echo "IGCM_sys_desactiv_variables"
2095	fi
2096	# --------------------------------------------------------------------
2097	#D- MPI specifications
2098	# --------------------------------------------------------------------
2099
2100	# --------------------------------------------------------------------
2101	#D- Other specifications
2102	# --------------------------------------------------------------------
2103
2104	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
2105	}
2106
2107	############################################################
2108	# Build MPI/OMP scripts run file (dummy function)
2109
2110	#D-#==================================================
2111	#D-function IGCM_sys_build_run_file
2112	#D-* Purpose: build run file (deprecated)
2113	#D-* Examples:
2114	#D-
2115	function IGCM_sys_build_run_file {
2116
2117	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
2118
2119	}
2120
2121	############################################################
2122	# Build MPI/OMP scripts
2123
2124	#D-#==================================================
2125	#D-function IGCM_sys_build_execution_scripts
2126	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
2127	#D-* Examples:
2128	#D-
2129	function IGCM_sys_build_execution_scripts
2130	{
2131	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
2132	if ( $DEBUG_sys ) ; then
2133	echo "IGCM_sys_build_execution_scripts " $@
2134	fi
2135
2136	EXECUTION=${HOST_MPIRUN_COMMAND}
2137
2138	# MPMD mode
2139
2140	if ( ${OK_PARA_MPMD} ) ; then
2141
2142	# Only MPI (MPMD)
2143	if ( ! ${OK_PARA_OMP} ) ; then
2144
2145	if [ -f run_file ] ; then
2146	IGCM_sys_Rm -f run_file
2147	fi
2148	touch run_file
2149
2150	# Build run_file
2151
2152	# First loop on the components for the coupler ie oasis (only if oasis3)
2153
2154	## the coupler ie oasis3 must be the first one
2155	for comp in ${config_ListOfComponents[*]} ; do
2156
2157	eval ExeNameIn=\${config_Executable_${comp}[0]}
2158	eval ExeNameOut=\${config_Executable_${comp}[1]}
2159
2160	# for CPL component only
2161	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
2162
2163	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2164	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2165	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2166	fi
2167	done
2168
2169	# Then second loop on the components
2170
2171	for comp in ${config_ListOfComponents[*]} ; do
2172
2173	eval ExeNameIn=\${config_Executable_${comp}[0]}
2174	eval ExeNameOut=\${config_Executable_${comp}[1]}
2175
2176	# Only if we really have an executable for the component and not the coupler ie oasis:
2177	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
2178
2179	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2180	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2181	echo "${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
2182	fi
2183	done
2184
2185	EXECUTION="${HOST_MPIRUN_COMMAND} -f ./run_file"
2186
2187	IGCM_sys_Chmod u+x run_file
2188	if ( $DEBUG_sys ) ; then
2189	echo "run_file contains : "
2190	cat run_file
2191	fi
2192
2193	# MPI-OpenMP (MPMD)
2194	else
2195
2196	# Use of mpirun instead of ccc_mprun
2197	EXECUTION="time mpirun"
2198
2199	# Hosts treatment
2200	${EXECUTION} hostname \| sort \| uniq > hosts.tmp
2201
2202	i=0
2203	rm -f hosts rankfile
2204	IGCM_debug_Print 1 "sys Curie, Hosts available :"
2205	for nodes in `cat hosts.tmp`
2206	do
2207	host[$i]=$nodes
2208	echo "${host[$i]}" >> hosts
2209	IGCM_debug_Print 1 ${host[$i]}
2210	i=$((i+1))
2211	done
2212	rm -f hosts.tmp
2213
2214	listnodes=${host[*]}
2215
2216	EXECUTION="${EXECUTION} -hostfile hosts -rankfile rankfile"
2217
2218	# Initialisation
2219	rank=0
2220	current_core=0
2221	core_per_node=16
2222	init_exec=n
2223
2224	# Loop on the components
2225	for comp in ${config_ListOfComponents[*]} ; do
2226
2227	eval ExeNameIn=\${config_Executable_${comp}[0]}
2228	eval ExeNameOut=\${config_Executable_${comp}[1]}
2229
2230	# Not possible if oasis has an executable (i.e old version of oasis3)
2231	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
2232	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
2233	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
2234	IGCM_debug_Verif_Exit
2235	fi
2236
2237	# Only if we really have an executable for the component :
2238	if [ "X${ExeNameOut}" != X\"\" ] ; then
2239
2240	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2241	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2242
2243	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2244	echo "" >> script_${ExeNameOut}.ksh
2245	if [ ${comp_proc_omp_loc} -gt 1 ] ; then
2246
2247	# Check if the number of threads is correct
2248	case ${comp_proc_omp_loc} in
2249	2\|4\|8\|16)
2250	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
2251	;;
2252	*)
2253	IGCM_debug_Exit "ERROR with OMP parameters !"
2254	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
2255	IGCM_debug_Print 2 "Only 2,4,8,16 as number of OMP threads are possible "
2256	IGCM_debug_Verif_Exit
2257	;;
2258	esac
2259	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2260	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2261	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2262	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2263	fi
2264	# echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
2265	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
2266	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2267	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2268
2269	if [ ${init_exec} = y ] ; then
2270	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2271	else
2272	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2273	init_exec=y
2274	fi
2275
2276	# Build rankfile : method used to assign cores and nodes for the MPI process
2277	# Ex :
2278	#rank 0=curie5296 slot=0,1,2,3
2279	#rank 1=curie5296 slot=4,5,6,7
2280	# Example of final command :
2281	# mpirun -hostfile hosts -rankfile rankfile -np 27 ./script_lmdz.x.ksh : -np 5 ./script_opa.xx.ksh
2282	# with script_lmdz.x.ksh :
2283	# #!/bin/ksh
2284	#export KMP_STACKSIZE=3g
2285	#export KMP_LIBRARY=turnaround
2286	#export MKL_SERIAL=YES
2287	#OMP_NUM_THREADS=4
2288	#./lmdz.x
2289	#
2290
2291	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
2292	(( index_host = current_core / core_per_node ))
2293	host_value=${host[${index_host}]}
2294	(( slot = current_core % core_per_node ))
2295	virg=","
2296	string_final=""
2297	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
2298	string=$index$virg
2299	string_final=$string_final$string
2300	done
2301	string_final=$( echo $string_final \| sed "s/.$//" )
2302	echo "rank $rank=$host_value slot=$string_final" >> rankfile
2303	(( rank = rank + 1 ))
2304	(( current_core = current_core + comp_proc_omp_loc ))
2305	done
2306	fi
2307
2308	done
2309	fi
2310
2311	# Only one executable (SPMD mode).
2312	else
2313
2314	for comp in ${config_ListOfComponents[*]} ; do
2315
2316	# Only if we really have an executable for the component :
2317	eval ExeNameOut=\${config_Executable_${comp}[1]}
2318	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
2319
2320	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
2321	echo "" >> script_${ExeNameOut}.ksh
2322	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
2323
2324	if ( ${OK_PARA_OMP} ) ; then
2325	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
2326	echo "" >> script_${ExeNameOut}.ksh
2327	echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
2328	echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
2329	echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
2330	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
2331	fi
2332
2333	if ( ${OK_PARA_MPI} ) ; then
2334	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
2335	# Default : ccc_mprun used if nb_proc gt 1
2336	# to have out/err per process on different files
2337	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${SLURM_PROCID} 2>out_${ExeNameOut}.err.\${SLURM_PROCID}" >> script_${ExeNameOut}.ksh
2338	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2339	EXECUTION="${HOST_MPIRUN_COMMAND} -n ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
2340	else
2341	# Default : ccc_mprun is NOT used if nb_proc eq 1
2342	# to have out/err per process on different files
2343	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
2344	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
2345	EXECUTION="/usr/bin/time ./script_${ExeNameOut}.ksh"
2346	fi
2347
2348	IGCM_debug_Print 1 "sys Curie : script_${ExeNameOut}.ksh contains"
2349	cat script_${ExeNameOut}.ksh
2350
2351	fi
2352
2353	done
2354
2355	fi
2356
2357	IGCM_debug_Print 1 "sys Curie : execution command is "
2358	IGCM_debug_Print 1 "$EXECUTION"
2359
2360	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
2361	}
2362
2363	#D-#==================================================
2364	#D-function IGCM_sys_check_path
2365	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
2366	#D-* do not point to an important use directory. Stop immediately in that case.
2367	#D-* Examples:
2368	#D-
2369	function IGCM_sys_check_path {
2370	IGCM_debug_PushStack "IGCM_sys_check_path"
2371	if ( $DEBUG_sys ) ; then
2372	echo "IGCM_sys_check_path"
2373	fi
2374
2375	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${SCRATCHDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCWORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${CCCSTOREDIR} ] ) ; then
2376	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
2377	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
2378	IGCM_debug_Exit "This will stop the job"
2379	fi
2380	IGCM_debug_PopStack "IGCM_sys_check_path"
2381	}
2382
2383	#D-#==================================================
2384	#D-function IGCM_sys_check_quota
2385	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
2386	#D-* Examples:
2387	#D-
2388	function IGCM_sys_check_quota {
2389	IGCM_debug_PushStack "IGCM_sys_check_quota"
2390	if ( $DEBUG_sys ) ; then
2391	echo "IGCM_sys_check_quota"
2392	fi
2393	# Limit of quota (in %)
2394	limit_quota=90
2395
2396	# Check of the volume
2397	volume_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $2}')
2398	volume_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $3}')
2399
2400	if ( [ ! X${volume_quota} = X ] && [ ! ${volume_quota} = "-" ] ) ; then
2401
2402	unit_avail=${volume_avail: -1}
2403	unit_quota=${volume_quota: -1}
2404
2405	if [ "${unit_quota}" = "*" ] ; then
2406	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2407	IGCM_debug_Print 1 "More than 100% of your quota is used"
2408	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2409	IGCM_debug_Print 1 "You must have more than 10% available to run"
2410	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2411	IGCM_debug_Verif_Exit
2412	fi
2413
2414	temp_avail=${volume_avail%%${volume_avail: -1}*}
2415	temp_quota=${volume_quota%%${volume_quota: -1}*}
2416
2417	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2418
2419	# Convertion
2420	if [ ${volume_avail: -1} = "T" ] ; then
2421	(( temp_avail = temp_avail * 1000000000000 ))
2422	elif [ ${volume_avail: -1} = "G" ] ; then
2423	(( temp_avail = temp_avail * 1000000000 ))
2424	elif [ ${volume_avail: -1} = "M" ] ; then
2425	(( temp_avail = temp_avail * 1000000 ))
2426	elif [ ${volume_avail: -1} = "k" ] ; then
2427	(( temp_avail = temp_avail * 1000 ))
2428	else
2429	(( temp_avail = volume_avail ))
2430	fi
2431	if [ ${volume_quota: -1} = "T" ] ; then
2432	(( temp_quota = temp_quota * 1000000000000 ))
2433	elif [ ${volume_quota: -1} = "G" ] ; then
2434	(( temp_quota = temp_quota * 1000000000 ))
2435	elif [ ${volume_quota: -1} = "M" ] ; then
2436	(( temp_quota = temp_quota * 1000000 ))
2437	elif [ ${volume_quota: -1} = "k" ] ; then
2438	(( temp_quota = temp_quota * 1000 ))
2439	else
2440	(( temp_quota = volume_quota ))
2441	fi
2442	fi
2443
2444	quota_volume=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2445	# echo "volume ratio is " $quota_volume
2446
2447	if [ ${quota_volume} -ge ${limit_quota} ] ; then
2448	IGCM_debug_Print 1 "Please, check your quota of volume on scratch"
2449	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
2450	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2451	IGCM_debug_Print 1 "You must have more than 10% available to run"
2452	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2453	IGCM_debug_Verif_Exit
2454	fi
2455
2456	fi
2457
2458	# Check of the number of inodes
2459
2460	inode_quota=$(ccc_quota \| grep ' scratch' \| gawk '{print $6}')
2461	inode_avail=$(ccc_quota \| grep ' scratch' \| gawk '{print $7}')
2462
2463	if ( [ ! X${inode_quota} = X ] && [ ! ${inode_quota} = "-" ] ) ; then
2464
2465	unit_avail=${inode_avail: -1}
2466	unit_quota=${inode_quota: -1}
2467
2468	if [ "${unit_quota}" = "*" ] ; then
2469	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2470	IGCM_debug_Print 1 "More than 100% of your quota is used"
2471	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2472	IGCM_debug_Print 1 "You must have more than 10% available to run"
2473	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2474	IGCM_debug_Verif_Exit
2475	fi
2476
2477	temp_avail=${inode_avail%%${inode_avail: -1}*}
2478	temp_quota=${inode_quota%%${inode_quota: -1}*}
2479
2480	if [ ! ${unit_avail} = ${unit_quota} ] ; then
2481
2482	# Convertion
2483	if [ ${inode_avail: -1} = "T" ] ; then
2484	(( temp_avail = temp_avail * 1000000000000 ))
2485	elif [ ${inode_avail: -1} = "G" ] ; then
2486	(( temp_avail = temp_avail * 1000000000 ))
2487	elif [ ${inode_avail: -1} = "M" ] ; then
2488	(( temp_avail = temp_avail * 1000000 ))
2489	elif [ ${inode_avail: -1} = "k" ] ; then
2490	(( temp_avail = temp_avail * 1000 ))
2491	else
2492	(( temp_avail = inode_avail ))
2493	fi
2494
2495	if [ ${inode_quota: -1} = "T" ] ; then
2496	(( temp_quota = temp_quota * 1000000000000 ))
2497	elif [ ${inode_quota: -1} = "G" ] ; then
2498	(( temp_quota = temp_quota * 1000000000 ))
2499	elif [ ${inode_quota: -1} = "M" ] ; then
2500	(( temp_quota = temp_quota * 1000000 ))
2501	elif [ ${inode_quota: -1} = "k" ] ; then
2502	(( temp_quota = temp_quota * 1000 ))
2503	else
2504	(( temp_quota = inode_quota ))
2505	fi
2506	fi
2507	quota_inode=$(echo "scale=2 ; $temp_quota/$temp_avail*100" \| bc)
2508	# echo "inode ratio is " $quota_inode
2509
2510	if [ ${quota_inode} -ge ${limit_quota} ] ; then
2511	IGCM_debug_Print 1 "Please, check your quota of inode on scratch"
2512	IGCM_debug_Print 1 "${quota_inode}% of your quota is used"
2513	IGCM_debug_Print 1 "Use the ccc_quota command to check"
2514	IGCM_debug_Print 1 "You must have more than 10% available to run"
2515	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
2516	IGCM_debug_Verif_Exit
2517	fi
2518	fi
2519	IGCM_debug_PopStack "IGCM_sys_check_quota"
2520	}
2521
2522	#D-#==================================================
2523	#D-function IGCM_sys_GetJobID
2524	#D-* Purpose: Check if job_name is currently
2525	#D- running or in queue
2526	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
2527	#D-
2528	function IGCM_sys_GetJobID {
2529	IGCM_debug_PushStack "IGCM_sys_GetJobID"
2530	if ( $DEBUG_sys ) ; then
2531	echo "IGCM_sys_GetJobID"
2532	fi
2533
2534	# With -f option, the full job name is given in the last column
2535	ID=$( ccc_mstat -f -u $2 \| \
2536	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
2537
2538	eval ${3}=${ID}
2539
2540	IGCM_debug_PopStack "IGCM_sys_GetJobID"
2541	}
2542
2543	#D-#==================================================
2544	#D-function IGCM_sys_CountJobInQueue
2545	#D-* Purpose: Check if job_name is currently
2546	#D- running or in queue
2547	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
2548	#D-
2549	function IGCM_sys_CountJobInQueue {
2550	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
2551	if ( $DEBUG_sys ) ; then
2552	echo "IGCM_sys_CountJobInQueue"
2553	fi
2554
2555	#NbRun=$( ccc_mstat -f \| grep -c ${JobName} )
2556
2557	# With -f option, the full job name is given in the last column
2558	NbRun=$( ccc_mstat -f \| gawk -v JobName=$1 'BEGIN { x=0 } ( $NF ~ JobName ) { x=x+1 } END { print x }' )
2559
2560	eval ${2}=${NbRun}
2561
2562	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
2563	}
2564
2565	#D-#==================================================
2566	#D-function IGCM_sys_ListJobInQueue
2567	#D-* Purpose: Check if job_name is currently
2568	#D- running or in queue
2569	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
2570	#D-
2571	function IGCM_sys_ListJobInQueue {
2572	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
2573	if ( $DEBUG_sys ) ; then
2574	echo "IGCM_sys_ListJobInQueue"
2575	fi
2576
2577	# With -f option, the full job name is given in the last column
2578	set -A JobList $( ccc_mstat -f \| gawk -v User=$1 \
2579	'( $2 == User && \
2580	$NF != /TS/ && \
2581	$NF !~ /PACK/ && \
2582	$NF !~ /REBUILD/ && \
2583	$NF !~ /pack/ ) \
2584	{ print $NF }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
2585
2586	eval set -A ${2} ${JobList[*]}
2587
2588	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
2589	}
2590
2591	##############################################################
2592	# NCO OPERATOR
2593
2594	#D-#==================================================
2595	#D-function IGCM_sys_ncap2
2596	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
2597	#D-* Examples:
2598	#D-
2599	function IGCM_sys_ncap2 {
2600	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
2601	if ( $DEBUG_sys ) ; then
2602	echo "IGCM_sys_ncap2 :" $@
2603	fi
2604
2605	typeset NB_ESSAI DELAI status i
2606	# number of tentative
2607	NB_ESSAI=3
2608	# time delay between tentative
2609	DELAI=2
2610
2611	i=0
2612	while [ $i -lt $NB_ESSAI ] ; do
2613	ncap2 "$@" > /tmp/out_command.$$ 2>&1
2614	status=$?
2615	if [ ${status} -gt 0 ] ; then
2616	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
2617	cat /tmp/out_command.$$
2618	\rm /tmp/out_command.$$
2619	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2620	sleep $DELAI
2621	else
2622	\rm /tmp/out_command.$$
2623	break
2624	fi
2625	(( i = i + 1 ))
2626	done
2627
2628	if [ ${status} -gt 0 ] ; then
2629	echo "IGCM_sys_ncap2 : ncap2 error"
2630	IGCM_debug_Exit "ncap2"
2631	fi
2632
2633	IGCM_debug_PopStack "IGCM_sys_ncap2"
2634	}
2635
2636	#D-#==================================================
2637	#D-function IGCM_sys_ncatted
2638	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
2639	#D-* Examples:
2640	#D-
2641	function IGCM_sys_ncatted {
2642	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
2643	if ( $DEBUG_sys ) ; then
2644	echo "IGCM_sys_ncatted :" $@
2645	fi
2646
2647	typeset NB_ESSAI DELAI status i
2648	# number of tentative
2649	NB_ESSAI=3
2650	# time delay between tentative
2651	DELAI=2
2652
2653	i=0
2654	while [ $i -lt $NB_ESSAI ] ; do
2655	ncatted "$@" > /tmp/out_command.$$ 2>&1
2656	status=$?
2657	if [ ${status} -gt 0 ] ; then
2658	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
2659	cat /tmp/out_command.$$
2660	\rm /tmp/out_command.$$
2661	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2662	sleep $DELAI
2663	else
2664	\rm /tmp/out_command.$$
2665	break
2666	fi
2667	(( i = i + 1 ))
2668	done
2669
2670	if [ ${status} -gt 0 ] ; then
2671	echo "IGCM_sys_ncatted : ncatted error"
2672	IGCM_debug_Exit "ncatted"
2673	fi
2674
2675	IGCM_debug_PopStack "IGCM_sys_ncatted"
2676	}
2677
2678	#D-#==================================================
2679	#D-function IGCM_sys_ncbo
2680	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
2681	#D-* Examples:
2682	#D-
2683	function IGCM_sys_ncbo {
2684	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
2685	if ( $DEBUG_sys ) ; then
2686	echo "IGCM_sys_ncbo :" $@
2687	fi
2688
2689	typeset NB_ESSAI DELAI status i
2690	# number of tentative
2691	NB_ESSAI=3
2692	# time delay between tentative
2693	DELAI=2
2694
2695	i=0
2696	while [ $i -lt $NB_ESSAI ] ; do
2697	ncbo $@ > /tmp/out_command.$$ 2>&1
2698	status=$?
2699	if [ ${status} -gt 0 ] ; then
2700	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
2701	cat /tmp/out_command.$$
2702	\rm /tmp/out_command.$$
2703	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2704	sleep $DELAI
2705	else
2706	\rm /tmp/out_command.$$
2707	break
2708	fi
2709	(( i = i + 1 ))
2710	done
2711
2712	if [ ${status} -gt 0 ] ; then
2713	echo "IGCM_sys_ncbo : ncbo error"
2714	IGCM_debug_Exit "ncbo"
2715	fi
2716
2717	IGCM_debug_PopStack "IGCM_sys_ncbo"
2718	}
2719
2720	#D-#==================================================
2721	#D-function IGCM_sys_ncdif
2722	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
2723	#D-* Examples:
2724	#D-
2725	function IGCM_sys_ncdiff {
2726	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
2727	if ( $DEBUG_sys ) ; then
2728	echo "IGCM_sys_ncdiff :" $@
2729	fi
2730
2731	typeset NB_ESSAI DELAI status i
2732	# number of tentative
2733	NB_ESSAI=3
2734	# time delay between tentative
2735	DELAI=2
2736
2737	i=0
2738	while [ $i -lt $NB_ESSAI ] ; do
2739	ncdiff $@ > /tmp/out_command.$$ 2>&1
2740	status=$?
2741	if [ ${status} -gt 0 ] ; then
2742	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
2743	cat /tmp/out_command.$$
2744	\rm /tmp/out_command.$$
2745	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2746	sleep $DELAI
2747	else
2748	\rm /tmp/out_command.$$
2749	break
2750	fi
2751	(( i = i + 1 ))
2752	done
2753
2754	if [ ${status} -gt 0 ] ; then
2755	echo "IGCM_sys_ncdiff : ncdiff error"
2756	IGCM_debug_Exit "ncdiff"
2757	fi
2758
2759	IGCM_debug_PopStack "IGCM_sys_ncdiff"
2760	}
2761
2762	#D-#==================================================
2763	#D-function IGCM_sys_ncea
2764	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
2765	#D-* Examples:
2766	#D-
2767	function IGCM_sys_ncea {
2768	IGCM_debug_PushStack "IGCM_sys_ncea" $@
2769	if ( $DEBUG_sys ) ; then
2770	echo "IGCM_sys_ncea :" $@
2771	fi
2772
2773	typeset NB_ESSAI DELAI status i
2774	# number of tentative
2775	NB_ESSAI=3
2776	# time delay between tentative
2777	DELAI=2
2778
2779	i=0
2780	while [ $i -lt $NB_ESSAI ] ; do
2781	ncea $@ > /tmp/out_command.$$ 2>&1
2782	status=$?
2783	if [ ${status} -gt 0 ] ; then
2784	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
2785	cat /tmp/out_command.$$
2786	\rm /tmp/out_command.$$
2787	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2788	sleep $DELAI
2789	else
2790	\rm /tmp/out_command.$$
2791	break
2792	fi
2793	(( i = i + 1 ))
2794	done
2795
2796	if [ ${status} -gt 0 ] ; then
2797	echo "IGCM_sys_ncea : ncea error"
2798	IGCM_debug_Exit "ncea"
2799	fi
2800
2801	IGCM_debug_PopStack "IGCM_sys_ncea"
2802	}
2803
2804	#D-#==================================================
2805	#D-function IGCM_sys_ncecat
2806	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
2807	#D-* Examples:
2808	#D-
2809	function IGCM_sys_ncecat {
2810	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
2811	if ( $DEBUG_sys ) ; then
2812	echo "IGCM_sys_ncecat :" $@
2813	fi
2814
2815	typeset NB_ESSAI DELAI status i
2816	# number of tentative
2817	NB_ESSAI=3
2818	# time delay between tentative
2819	DELAI=2
2820
2821	i=0
2822	while [ $i -lt $NB_ESSAI ] ; do
2823	ncecat $@ > /tmp/out_command.$$ 2>&1
2824	status=$?
2825	if [ ${status} -gt 0 ] ; then
2826	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
2827	cat /tmp/out_command.$$
2828	\rm /tmp/out_command.$$
2829	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2830	sleep $DELAI
2831	else
2832	\rm /tmp/out_command.$$
2833	break
2834	fi
2835	(( i = i + 1 ))
2836	done
2837
2838	if [ ${status} -gt 0 ] ; then
2839	echo "IGCM_sys_ncecat : ncecat error"
2840	IGCM_debug_Exit "ncecat"
2841	fi
2842
2843	IGCM_debug_PopStack "IGCM_sys_ncecat"
2844	}
2845
2846	#D-#==================================================
2847	#D-function IGCM_sys_ncflint
2848	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
2849	#D-* Examples:
2850	#D-
2851	function IGCM_sys_ncflint {
2852	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
2853	if ( $DEBUG_sys ) ; then
2854	echo "IGCM_sys_ncflint :" $@
2855	fi
2856
2857	typeset NB_ESSAI DELAI status i
2858	# number of tentative
2859	NB_ESSAI=3
2860	# time delay between tentative
2861	DELAI=2
2862
2863	i=0
2864	while [ $i -lt $NB_ESSAI ] ; do
2865	ncflint $@ > /tmp/out_command.$$ 2>&1
2866	status=$?
2867	if [ ${status} -gt 0 ] ; then
2868	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
2869	cat /tmp/out_command.$$
2870	\rm /tmp/out_command.$$
2871	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2872	sleep $DELAI
2873	else
2874	\rm /tmp/out_command.$$
2875	break
2876	fi
2877	(( i = i + 1 ))
2878	done
2879
2880	if [ ${status} -gt 0 ] ; then
2881	echo "IGCM_sys_ncflint : ncflint error"
2882	IGCM_debug_Exit "ncflint"
2883	fi
2884
2885	IGCM_debug_PopStack "IGCM_sys_ncflint"
2886	}
2887
2888	#D-#==================================================
2889	#D-function IGCM_sys_ncks
2890	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
2891	#D-* Examples:
2892	#D-
2893	function IGCM_sys_ncks {
2894	IGCM_debug_PushStack "IGCM_sys_ncks" $@
2895	if ( $DEBUG_sys ) ; then
2896	echo "IGCM_sys_ncks :" $@
2897	fi
2898
2899	typeset NB_ESSAI DELAI status i
2900	# number of tentative
2901	NB_ESSAI=3
2902	# time delay between tentative
2903	DELAI=2
2904
2905	i=0
2906	while [ $i -lt $NB_ESSAI ] ; do
2907	ncks $@ > /tmp/out_command.$$ 2>&1
2908	status=$?
2909	if [ ${status} -gt 0 ] ; then
2910	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
2911	cat /tmp/out_command.$$
2912	\rm /tmp/out_command.$$
2913	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2914	sleep $DELAI
2915	else
2916	\rm /tmp/out_command.$$
2917	break
2918	fi
2919	(( i = i + 1 ))
2920	done
2921
2922	if [ ${status} -gt 0 ] ; then
2923	echo "IGCM_sys_ncks : ncks error"
2924	IGCM_debug_Exit "ncks"
2925	fi
2926
2927	IGCM_debug_PopStack "IGCM_sys_ncks"
2928	}
2929
2930	#D-#==================================================
2931	#D-function IGCM_sys_ncpdq
2932	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
2933	#D-* Examples:
2934	#D-
2935	function IGCM_sys_ncpdq {
2936	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
2937	if ( $DEBUG_sys ) ; then
2938	echo "IGCM_sys_ncpdq :" $@
2939	fi
2940
2941	typeset NB_ESSAI DELAI status i
2942	# number of tentative
2943	NB_ESSAI=3
2944	# time delay between tentative
2945	DELAI=2
2946
2947	i=0
2948	while [ $i -lt $NB_ESSAI ] ; do
2949	ncpdq $@ > /tmp/out_command.$$ 2>&1
2950	status=$?
2951	if [ ${status} -gt 0 ] ; then
2952	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
2953	cat /tmp/out_command.$$
2954	\rm /tmp/out_command.$$
2955	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2956	sleep $DELAI
2957	else
2958	\rm /tmp/out_command.$$
2959	break
2960	fi
2961	(( i = i + 1 ))
2962	done
2963
2964	if [ ${status} -gt 0 ] ; then
2965	echo "IGCM_sys_ncpdq : ncpdq error"
2966	IGCM_debug_Exit "ncpdq"
2967	fi
2968
2969	IGCM_debug_PopStack "IGCM_sys_ncpdq"
2970	}
2971
2972	#D-#==================================================
2973	#D-function IGCM_sys_ncra
2974	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
2975	#D-* Examples:
2976	#D-
2977	function IGCM_sys_ncra {
2978	IGCM_debug_PushStack "IGCM_sys_ncra" $@
2979	if ( $DEBUG_sys ) ; then
2980	echo "IGCM_sys_ncra :" $@
2981	fi
2982
2983	typeset NB_ESSAI DELAI status i
2984	# number of tentative
2985	NB_ESSAI=3
2986	# time delay between tentative
2987	DELAI=2
2988
2989	i=0
2990	while [ $i -lt $NB_ESSAI ] ; do
2991	ncra $@ > /tmp/out_command.$$ 2>&1
2992	status=$?
2993	if [ ${status} -gt 0 ] ; then
2994	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
2995	cat /tmp/out_command.$$
2996	\rm /tmp/out_command.$$
2997	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
2998	sleep $DELAI
2999	else
3000	\rm /tmp/out_command.$$
3001	break
3002	fi
3003	(( i = i + 1 ))
3004	done
3005
3006	if [ ${status} -gt 0 ] ; then
3007	echo "IGCM_sys_ncra : ncra error"
3008	IGCM_debug_Exit "ncra"
3009	fi
3010
3011	IGCM_debug_PopStack "IGCM_sys_ncra"
3012	}
3013
3014	#D-#==================================================
3015	#D-function IGCM_sys_ncrcat
3016	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
3017	#D-* Examples:
3018	#D-
3019	function IGCM_sys_ncrcat {
3020	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
3021	if ( $DEBUG_sys ) ; then
3022	echo "IGCM_sys_ncrcat :" $@
3023	fi
3024
3025	typeset NB_ESSAI DELAI status i lastArg
3026	# number of tentative
3027	NB_ESSAI=3
3028	# time delay between tentative
3029	DELAI=2
3030
3031	i=0
3032	while [ $i -lt $NB_ESSAI ] ; do
3033	ncrcat $@ > /tmp/out_command.$$ 2>&1
3034	status=$?
3035	if [ ${status} -gt 0 ] ; then
3036	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
3037	cat /tmp/out_command.$$
3038	\rm /tmp/out_command.$$
3039	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3040	sleep $DELAI
3041	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command.$$ )" = "X" ] ; then
3042	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
3043	cat /tmp/out_command.$$
3044	# remove files having corrupted time axis
3045	eval lastArg=\${$#}
3046	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
3047	\rm ${lastArg}
3048	\rm /tmp/out_command.$$
3049	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3050	sleep $DELAI
3051	else
3052	\rm /tmp/out_command.$$
3053	break
3054	fi
3055	(( i = i + 1 ))
3056	done
3057
3058	if [ ${status} -gt 0 ] ; then
3059	echo "IGCM_sys_ncrcat : ncrcat error"
3060	#IGCM_debug_Exit "ncrcat"
3061	fi
3062
3063	IGCM_debug_PopStack "IGCM_sys_ncrcat"
3064	}
3065
3066	#D-#==================================================
3067	#D-function IGCM_sys_ncrename
3068	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
3069	#D-* Examples:
3070	#D-
3071	function IGCM_sys_ncrename {
3072	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
3073	if ( $DEBUG_sys ) ; then
3074	echo "IGCM_sys_ncrename :" $@
3075	fi
3076
3077	typeset NB_ESSAI DELAI status i
3078	# number of tentative
3079	NB_ESSAI=3
3080	# time delay between tentative
3081	DELAI=2
3082
3083	i=0
3084	while [ $i -lt $NB_ESSAI ] ; do
3085	ncrename $@ > /tmp/out_command.$$ 2>&1
3086	status=$?
3087	if [ ${status} -gt 0 ] ; then
3088	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
3089	cat /tmp/out_command.$$
3090	\rm /tmp/out_command.$$
3091	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3092	sleep $DELAI
3093	else
3094	\rm /tmp/out_command.$$
3095	break
3096	fi
3097	(( i = i + 1 ))
3098	done
3099
3100	if [ ${status} -gt 0 ] ; then
3101	echo "IGCM_sys_ncrename : ncrename error"
3102	IGCM_debug_Exit "ncrename"
3103	fi
3104
3105	IGCM_debug_PopStack "IGCM_sys_ncrename"
3106	}
3107
3108	#D-#==================================================
3109	#D-function IGCM_sys_ncwa
3110	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
3111	#D-* Examples:
3112	#D-
3113	function IGCM_sys_ncwa {
3114	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
3115	if ( $DEBUG_sys ) ; then
3116	echo "IGCM_sys_ncwa :" $@
3117	fi
3118
3119	typeset NB_ESSAI DELAI status i
3120	# number of tentative
3121	NB_ESSAI=3
3122	# time delay between tentative
3123	DELAI=2
3124
3125	i=0
3126	while [ $i -lt $NB_ESSAI ] ; do
3127	ncwa $@ > /tmp/out_command.$$ 2>&1
3128	status=$?
3129	if [ ${status} -gt 0 ] ; then
3130	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
3131	cat /tmp/out_command.$$
3132	\rm /tmp/out_command.$$
3133	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
3134	sleep $DELAI
3135	else
3136	\rm /tmp/out_command.$$
3137	break
3138	fi
3139	(( i = i + 1 ))
3140	done
3141
3142	if [ ${status} -gt 0 ] ; then
3143	echo "IGCM_sys_ncwa : ncwa error"
3144	IGCM_debug_Exit "ncwa"
3145	fi
3146
3147	IGCM_debug_PopStack "IGCM_sys_ncwa"
3148	}
3149
3150	##############################################################
3151	# CDO OPERATOR
3152
3153	#D-#==================================================
3154	#D-function IGCM_sys_cdo
3155	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
3156	#D-* Examples:
3157	#D-
3158	function IGCM_sys_cdo {
3159	IGCM_debug_PushStack "IGCM_sys_cdo" $@
3160	if ( $DEBUG_sys ) ; then
3161	echo "IGCM_sys_cdo :" $@
3162	fi
3163
3164	typeset status
3165
3166	\cdo $@ > /tmp/out_command.$$ 2>&1
3167	status=$?
3168	if [ ${status} -gt 0 ] ; then
3169	echo "IGCM_sys_cdo : error code ${status}"
3170	cat /tmp/out_command.$$
3171	\rm /tmp/out_command.$$
3172	IGCM_debug_PopStack "IGCM_sys_cdo"
3173	return 1
3174	else
3175	IGCM_debug_PopStack "IGCM_sys_cdo"
3176	return 0
3177	fi
3178
3179	IGCM_debug_PopStack "IGCM_sys_cdo"
3180	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: