Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_brodie.ksh @ 217

Last change on this file since 217 was 217, checked in by sdipsl, 14 years ago
Avoid side effect with user environment. Do not define read-only varible potentially defined by system environment.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Date Author Revision`
File size: 31.6 KB

Line
1	#!/bin/ksh
2
3	#**************************************************************
4	# Author: Sébastien Denvil
5	# Contact: sebastien.denvil.@ipsl.jussieu.fr
6	# $Date$
7	# $Author$
8	# $Revision$
9	# IPSL (2006)
10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
11	# History:
12	# Modification:
13	#
14	#**************************************************************
15
16	#=========================================================
17	# The documentation of this file can be automatically generated
18	# if you use the prefix #D- for comments to be extracted.
19	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
20	#=========================================================
21
22	#D-#=================================================
23	#D-LibIGCM_sys for Brodie SX8
24	#D-#=================================================
25	#D-
26	#D- This ksh library if a layer under some usefull
27	#D-environment variables and shell commands.
28	#D-All those definitions depend on host particularities.
29	#D-It manages a stack mechanism and test validity of operations.
30	#D-All function described bellow must be prefixed by IGCM_sys.
31
32	#====================================================
33	# libIGCM_sys PARAMETERS
34	#====================================================
35
36	#====================================================
37	# set DEBUG_sys to true to output calls of function
38	typeset -r DEBUG_sys=${DEBUG_sys:=true}
39
40	#====================================================
41	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
42	typeset -r DRYRUN=${DRYRUN:=0}
43
44	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
45	# -------------------------------------------------------------------------------------
46	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
47	# \| \| Cp/Exe param files \| \| Chmod \| \|
48	# \| \| Qsub \| \| \| \|
49	# -------------------------------------------------------------------------------------
50	# \| 0 \| yes \| yes \| yes \| yes \|
51	# -------------------------------------------------------------------------------------
52	# \| 1 \| yes \| yes \| yes \| no \|
53	# -------------------------------------------------------------------------------------
54	# \| 2 \| yes \| yes \| no \| no \|
55	# -------------------------------------------------------------------------------------
56	# \| 3 \| yes \| no \| no \| no \|
57	# -------------------------------------------------------------------------------------
58
59	#=====================================================
60	# Global Variables :
61	#=====================================================
62	# Language : "fr" or "en"
63	typeset -r MYLANG="fr"
64
65	#=====================================================
66	# Host and user names
67	# $hostname ou hostname
68	typeset HOST=${HOST:=$( hostname ) }
69	# $username ou whoami
70	typeset LOGIN=${LOGIN:=$( whoami ) }
71	# $hostname of the MASTER job
72	typeset -r MASTER=brodie
73
74	#D-
75	#D-#==================================================
76	#D-Program used in libIGCM
77	#D-#==================================================
78
79	# rsync with path
80	typeset -r RSYNC=/home/rech/ces/rces452/RSYNC_SX8/bin/rsync
81	# RSYNC_opt args to rsync
82	typeset -r RSYNC_opt="-va"
83	# ie storage filesystem
84	typeset -r RHOST=gaya
85	typeset -r REMOTE_RSYNC=/u/rech/ces/rces452/RSYNC/bin/rsync
86
87	#====================================================
88	# Host specific DIRECTORIES
89	#====================================================
90
91	#====================================================
92	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
93	typeset -r R_EXE="${MODIPSL}/bin"
94
95	#====================================================
96	#- SUBMIT_DIR : submission dir
97	typeset SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
98
99	#====================================================
100	#- ARCHIVE
101	typeset -r ARCHIVE=$( rsh gaya -n env \| grep PWD \| gawk "-F=" '{print $2}' )
102
103	#====================================================
104	#- Mirror libIGCM from brodie to ulam
105	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=true}
106
107	#====================================================
108	#- libIGCM_POST
109	PATHlibIGCM=$( echo ${libIGCM} \| gawk -F"${LOGIN}/" '{print $2}' \| sed -e "s&/libIGCM&&" )
110	typeset -r libIGCM_POST=${HOME}/MIRROR/${PATHlibIGCM}/libIGCM
111
112	#====================================================
113	#- IN
114	typeset -r R_IN=${R_IN:=/u/rech/psl/rpsl035/IGCM}
115
116	#====================================================
117	#- OUT
118	typeset -r R_OUT=${ARCHIVE}/IGCM_OUT
119
120	#====================================================
121	#- OUT_POST
122	typeset -r R_OUT_POST=$( rsh ulam -n echo $WORKDIR \| gawk "-F=" "{print $2}" )/IGCM_OUT
123
124	#====================================================
125	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
126	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${TMPDIR}}
127
128	#====================================================
129	#- BIG_DIR : BIG_DIR to store files waiting for rebuild
130	typeset -r BIG_DIR=${BIG_DIR:=${WORKDIR}/REBUILD}
131
132	#====================================================
133	#- HOST_MPIRUN_COMMAND
134	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="mpirun"}
135
136	#====================================================
137	#- Max number of arguments passed to nco operator or demigration command
138	UNIX_MAX_LIMIT=360
139
140	#D-#==================================================
141	#D-function IGCM_sys_RshArchive
142	#D-* Purpose: Archive rsh command
143	#D-* Examples:
144	#D-
145	function IGCM_sys_RshArchive {
146	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
147	rsh gaya exec /bin/ksh <<-EOF
148	${@}
149	EOF
150	if [ $? -gt 0 ] ; then
151	echo "IGCM_sys_RshArchive : erreur."
152	IGCM_debug_Exit "IGCM_sys_RshArchive"
153	fi
154	IGCM_debug_PopStack "IGCM_sys_RshArchive"
155	}
156
157	#D-#==================================================
158	#D-function IGCM_sys_RshPost
159	#D-* Purpose: Post-process rsh command
160	#D-* Examples:
161	#D-
162	function IGCM_sys_RshPost {
163	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
164	if ( $DEBUG_sys ) ; then
165	echo "IGCM_sys_RshPost :" $@
166	fi
167	rsh ulam exec /bin/ksh ${@}
168	if [ $? -gt 0 ] ; then
169	echo "IGCM_sys_RshPost : erreur."
170	IGCM_debug_Exit "IGCM_sys_RshPost"
171	fi
172	IGCM_debug_PopStack "IGCM_sys_RshPost"
173	}
174
175	#D-#==================================================
176	#D-function IGCM_sys_SendMail
177	#D-* Purpose: Send mail when simulation is over
178	#D-* Examples:
179	#D-
180	function IGCM_sys_SendMail {
181	IGCM_debug_PushStack "IGCM_sys_SendMailPost" $@
182	if ( $DEBUG_sys ) ; then
183	echo "IGCM_sys_SendMail :" $@
184	fi
185
186	cat << END_MAIL > job_end.mail
187	Dear ${LOGIN},
188
189	Simulation ${config_UserChoices_JobName} is finished on supercomputer `hostname`.
190	Job started : ${DateBegin}
191	Job ended : ${DateEnd}
192	Ouput files are available in ${R_SAVE}
193	END_MAIL
194
195	if [ X${config_UserChoices_MailName} != X ] ; then
196	mailx -s "${config_UserChoices_JobName} completed" ${config_UserChoices_MailName} < job_end.mail
197	elif [ -f ~/.forward ] ; then
198	mailx -s "${config_UserChoices_JobName} completed" $( cat ~/.forward ) < job_end.mail
199	else
200	mailx -s "${config_UserChoices_JobName} completed" ${USER} < job_end.mail
201	fi
202
203	if [ $? -gt 0 ] ; then
204	echo "IGCM_sys_SendMail : erreur."
205	IGCM_debug_Exit "IGCM_sys_SendMail"
206	fi
207	IGCM_debug_PopStack "IGCM_sys_SendMail"
208	}
209
210	#D-#==================================================
211	#D-function IGCM_sys_Mkdir
212	#D-* Purpose: Master locale mkdir command
213	#D-* Examples:
214	#D-
215	function IGCM_sys_Mkdir {
216	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
217	if ( $DEBUG_sys ) ; then
218	echo "IGCM_sys_Mkdir :" $@
219	fi
220	if [ ! -d ${1} ]; then
221	\mkdir -p $1
222	if [ $? -gt 0 ] ; then
223	echo "IGCM_sys_Mkdir : erreur."
224	IGCM_debug_Exit "IGCM_sys_Mkdir"
225	fi
226	fi
227	# vérification :
228	if [ ! -d ${1} ] ; then
229	echo "IGCM_sys_Mkdir : erreur."
230	IGCM_debug_Exit "IGCM_sys_Mkdir"
231	fi
232	IGCM_debug_PopStack "IGCM_sys_Mkdir"
233	}
234
235	#D-#==================================================
236	#D-function IGCM_sys_MkdirArchive
237	#D-* Purpose: Mkdir on Archive
238	#D-* Examples:
239	#D-
240	function IGCM_sys_MkdirArchive {
241	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
242	if ( $DEBUG_sys ) ; then
243	echo "IGCM_sys_MkdirArchive :" $@
244	fi
245	#- creation de repertoire sur le serveur fichier
246	rsh gaya -n mkdir -p $1
247
248	if [ $? -gt 0 ] ; then
249	echo "IGCM_sys_MkdirArchive : erreur."
250	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
251	fi
252	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
253	}
254
255	#D-#==================================================
256	#D-function IGCM_sys_MkdirWork
257	#D-* Purpose: Mkdir on Work
258	#D-* Examples:
259	#D-
260	function IGCM_sys_MkdirWork {
261	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
262	if ( $DEBUG_sys ) ; then
263	echo "IGCM_sys_MkdirWork :" $@
264	fi
265	#- creation de repertoire sur le serveur fichier
266	if [ ! -d ${1} ]; then
267	\mkdir -p $1
268	if [ $? -gt 0 ] ; then
269	echo "IGCM_sys_MkdirWork : erreur."
270	IGCM_debug_Exit "IGCM_sys_MkdirWork"
271	fi
272	fi
273	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
274	}
275
276	#D-#==================================================
277	#D-function IGCM_sys_Cd
278	#D-* Purpose: master cd command
279	#D-* Examples:
280	#D-
281	function IGCM_sys_Cd {
282	IGCM_debug_PushStack "IGCM_sys_Cd" $@
283	if ( $DEBUG_sys ) ; then
284	echo "IGCM_sys_Cd :" $@
285	fi
286	\cd $1
287	if [ $? -gt 0 ] ; then
288	echo "IGCM_sys_Cd : erreur."
289	IGCM_debug_Exit "IGCM_sys_Cd"
290	fi
291	IGCM_debug_PopStack "IGCM_sys_Cd"
292	}
293
294	#D-#==================================================
295	#D-function IGCM_sys_Chmod
296	#D-* Purpose: Chmod
297	#D-* Examples:
298	#D-
299	function IGCM_sys_Chmod {
300	IGCM_debug_PushStack "IGCM_sys_Chmod" -- $@
301	if ( $DEBUG_sys ) ; then
302	echo "IGCM_sys_Chmod :" $@
303	fi
304	if [ $DRYRUN -le 1 ]; then
305	\chmod $@
306	if [ $? -gt 0 ] ; then
307	echo "IGCM_sys_Chmod : erreur."
308	IGCM_debug_Exit "IGCM_sys_Chmod"
309	fi
310	else
311	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
312	fi
313	IGCM_debug_PopStack "IGCM_sys_Chmod"
314	}
315
316	#D-#==================================================
317	#D-function IGCM_sys_FileSize
318	#D-* Purpose: Filesize
319	#D-* Examples:
320	#D-
321	function IGCM_sys_FileSize {
322	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
323
324	typeset sizeF
325	set +A sizeF -- $( ls -la ${1} )
326	if [ $? -gt 0 ] ; then
327	IGCM_debug_Exit "IGCM_sys_FileSize"
328	fi
329	eval ${2}=${sizeF[4]}
330
331	IGCM_debug_PopStack "IGCM_sys_FileSize"
332	}
333
334	#D-#==================================================
335	#D-function IGCM_sys_TestDir
336	#D-* Purpose: Test Directory that must exists
337	#D-* Examples:
338	#D-
339	function IGCM_sys_TestDir {
340	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
341	if ( $DEBUG_sys ) ; then
342	echo "IGCM_sys_TestDir :" $@
343	fi
344	typeset ExistFlag
345	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
346	IGCM_debug_PopStack "IGCM_sys_TestDir"
347
348	return ${ExistFlag}
349	}
350
351	#D-#==================================================
352	#D-function IGCM_sys_TestDirArchive
353	#D-* Purpose: Test Directory that must exists on Archive
354	#D-* Examples:
355	#D-
356	function IGCM_sys_TestDirArchive {
357	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
358	if ( $DEBUG_sys ) ; then
359	echo "IGCM_sys_TestDirArchive :" $@
360	fi
361	typeset ExistFlag
362	ExistFlag=$( IGCM_sys_RshArchive "[ -d $1 ] && echo 0 \|\| echo 1" )
363	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
364
365	return ${ExistFlag}
366	}
367
368	#D-#==================================================
369	#D-function IGCM_sys_TestFileArchive
370	#D-* Purpose: Test file that must NOT EXISTS on Archive
371	#D-* Examples:
372	#D-
373	function IGCM_sys_TestFileArchive {
374	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
375	if ( $DEBUG_sys ) ; then
376	echo "IGCM_sys_TestFileArchive :" $@
377	fi
378	typeset ExistFlag
379	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1 ] && echo 0 \|\| echo 1" )
380	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
381
382	return ${ExistFlag}
383	}
384
385	#D-#==================================================
386	#D-function IGCM_sys_CountFileArchive
387	#D-* Purpose: Count files on Archive filesystem
388	#D-* Examples:
389	#D-
390	function IGCM_sys_CountFileArchive {
391	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
392	IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
393	if [ $? -gt 0 ] ; then
394	echo "IGCM_sys_CountFileArchive : erreur."
395	fi
396	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
397	}
398
399	#D-#==================================================
400	#D-function IGCM_sys_Tree
401	#D-* Purpose: Tree directories with files on ${ARCHIVE}
402	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
403	#D-
404	function IGCM_sys_Tree {
405	IGCM_debug_PushStack "IGCM_sys_Tree" $@
406	if ( $DEBUG_sys ) ; then
407	echo "IGCM_sys_Tree :" $@
408	fi
409
410	\mfls -r $@
411
412	IGCM_debug_PopStack "IGCM_sys_Tree"
413	}
414
415	#D-#==================================================
416	#D-function IGCM_sys_Tar
417	#D-* Purpose: master un-tar command
418	#D-* Examples:
419	#D-
420	function IGCM_sys_Tar {
421	IGCM_debug_PushStack "IGCM_sys_Tar" $@
422	if ( $DEBUG_sys ) ; then
423	echo "IGCM_sys_Tar :" $@
424	fi
425	\tar xvf $1
426	if [ $? -gt 0 ] ; then
427	echo "IGCM_sys_Tar : erreur."
428	IGCM_debug_Exit "IGCM_sys_Tar"
429	fi
430	IGCM_debug_PopStack "IGCM_sys_Tar"
431	}
432
433	#D-#==================================================
434	#D-function IGCM_sys_UnTar
435	#D-* Purpose: master un-tar command
436	#D-* Examples:
437	#D-
438	function IGCM_sys_UnTar {
439	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
440	if ( $DEBUG_sys ) ; then
441	echo "IGCM_sys_UnTar :" $@
442	fi
443	\tar xvf $1
444	if [ $? -gt 0 ] ; then
445	echo "IGCM_sys_UnTar : erreur."
446	IGCM_debug_Exit "IGCM_sys_UnTar"
447	fi
448	IGCM_debug_PopStack "IGCM_sys_UnTar"
449	}
450
451	#D-#==================================================
452	#D-function IGCM_sys_Qsub
453	#D-* Purpose: Qsub new job
454	#D-* Examples:
455	#D-
456	function IGCM_sys_Qsub {
457	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
458	if ( $DEBUG_sys ) ; then
459	echo "IGCM_sys_Qsub :" $@
460	fi
461	/usr/bin/nqsII/qsub -o ${Script_Output} -N ${config_UserChoices_JobName}.${CumulPeriod} < $1
462	if [ $? -gt 0 ] ; then
463	echo "IGCM_sys_Qsub : erreur -o ${Script_Output} -N ${config_UserChoices_JobName}.${CumulPeriod} $@."
464	IGCM_debug_Exit "IGCM_sys_Qsub"
465	fi
466	IGCM_debug_PopStack "IGCM_sys_Qsub"
467	}
468
469	#D-#==================================================
470	#D-function IGCM_sys_QsubPost
471	#D-* Purpose: Qsub new job on scalaire
472	#D-* Examples:
473	#D-
474	function IGCM_sys_QsubPost {
475	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
476	if ( $DEBUG_sys ) ; then
477	echo "IGCM_sys_QsubPost :" $@
478	fi
479	cd ${POST_DIR}
480	/opt/ibmll/LoadL/full/bin/llsubmit ${libIGCM_POST}/$1.job
481	cd -
482	if [ $? -gt 0 ] ; then
483	echo "IGCM_sys_QsubPost : erreur " $@
484	IGCM_debug_Exit "IGCM_sys_QsubPost"
485	fi
486	IGCM_debug_PopStack "IGCM_sys_QsubPost"
487	}
488
489	#D-*************************
490	#D- File transfer functions
491	#D-*************************
492	#D-
493
494	#D-#==================================================
495	#D-function IGCM_sys_Rsync_out
496	#D-* Purpose: treat return val of rsync
497	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
498	#D- Error values and explanations can depend on your system version.
499	function IGCM_sys_Rsync_out {
500	RET=$1
501	if [ ! $RET ] ; then
502	echo "rsync error !"
503	fi
504
505	if [ $MYLANG = "fr" ]; then
506	case $RET in
507	0) return ;;
508	1) echo "Erreur de rsync ; RERR_SYNTAX : "
509	echo "Erreur de syntaxe ou d'utilisation."
510	return;;
511	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
512	echo "Incompatibilité de protocole."
513	return;;
514	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
515	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
516	echo "répertoires"
517	return;;
518	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
519	echo "Action demandée non supportée : une tentative de manipulation de"
520	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
521	echo "été faite ; ou une option qui est supportée par le client mais"
522	echo "pas par le serveur a été spécifiée."
523	return;;
524	10) echo "Erreur de rsync ; RERR_SOCKETIO"
525	echo "Erreur dans le socket d'entrée sortie"
526	return;;
527	11) echo "Erreur de rsync ; RERR_FILEIO"
528	echo "Erreur d'entrée sortie fichier"
529	return;;
530	12) echo "Erreur de rsync ; RERR_STREAMIO"
531	echo "Erreur dans flux de donnée du protocole rsync"
532	return;;
533	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
534	echo "Erreur avec les diagnostics du programme"
535	return;;
536	14) echo "Erreur de rsync ; RERR_IPC"
537	echo "Erreur dans le code IPC"
538	return;;
539	20) echo "Erreur de rsync ; RERR_SIGNAL"
540	echo "SIGUSR1 ou SIGINT reçu"
541	return;;
542	21) echo "Erreur de rsync ; RERR_WAITCHILD"
543	echo "Une erreur retournée par waitpid()"
544	return;;
545	22) echo "Erreur de rsync ; RERR_MALLOC"
546	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
547	return;;
548	23) echo ""
549	echo "Erreur fichier inexistant"
550	return;;
551	30) echo "Erreur de rsync ; RERR_TIMEOUT"
552	echo "Temps d'attente écoulé dans l'envoi/réception de données"
553	return;;
554	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $RET
555	return;;
556	esac
557	elif [ $MYLANG = "en" ] ; then
558	case $RET in
559	0) return;;
560	1) echo "rsync error : Syntax or usage error "
561	return;;
562	2) echo "rsync error : Protocol incompatibility "
563	return;;
564	3) echo "rsync error : Errors selecting input/output files, dirs"
565	return;;
566	4) echo "rsync error : Requested action not supported: an attempt"
567	echo "was made to manipulate 64-bit files on a platform that cannot support"
568	echo "them; or an option was specified that is supported by the client and"
569	echo "not by the server."
570	return;;
571	5) echo "rsync error : Error starting client-server protocol"
572	return;;
573	10) echo "rsync error : Error in socket I/O "
574	return;;
575	11) echo "rsync error : Error in file I/O "
576	return;;
577	12) echo "rsync error : Error in rsync protocol data stream "
578	return;;
579	13) echo "rsync error : Errors with program diagnostics "
580	return;;
581	14) echo "rsync error : Error in IPC code "
582	return;;
583	20) echo "rsync error : Received SIGUSR1 or SIGINT "
584	return;;
585	21) echo "rsync error : Some error returned by waitpid() "
586	return;;
587	22) echo "rsync error : Error allocating core memory buffers "
588	return;;
589	23) echo "rsync error : Partial transfer due to error"
590	return;;
591	24) echo "rsync error : Partial transfer due to vanished source files"
592	return;;
593	30) echo "rsync error : Timeout in data send/receive "
594	return;;
595	*) echo "rsync error : return code of rsync unknown :" $RET
596	return;;
597	esac
598	else
599	echo "unknown language $MYLANG."
600	return
601	fi
602	}
603
604
605	#D-#==================================================
606	#D-function IGCM_sys_Miror_libIGCM
607	#D-* Purpose: Mirror libIGCM PATH and lib to ulam
608	#D-* Examples:
609	#D-
610	function IGCM_sys_Mirror_libIGCM {
611	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
612	if ( $DEBUG_sys ) ; then
613	echo "IGCM_sys_Mirror_libIGCM"
614	fi
615
616	typeset RET DEST
617
618	IGCM_sys_RshPost <<-EOF
619	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
620	EOF
621	rsh ulam -n hostname > /dev/null 2>&1
622	RET=$?
623	if [ $RET -eq 0 ] ; then
624	echo ${RSYNC} ${RSYNC_opt} -e "rsh " ${libIGCM} ulam:${HOME}/MIRROR/${PATHlibIGCM} > out_rsync 2>&1
625	${RSYNC} ${RSYNC_opt} -e "rsh " ${libIGCM} ulam:${HOME}/MIRROR/${PATHlibIGCM} >> out_rsync 2>&1
626	RET=$?
627
628	if [ ${RET} -gt 0 ] ; then
629	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on ulam."
630	cat out_rsync
631	fi
632	else
633	echo "No POST-TREATMENT avaible because ulam is down."
634	fi
635	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
636	}
637
638	#====================================================
639	#- Call IGCM_sys_Mirror_libIGCM now !
640	if ( $MirrorlibIGCM ) ; then
641	IGCM_sys_Mirror_libIGCM
642	fi
643
644	#D-#==================================================
645	#D-function IGCM_sys_Cp
646	#D-* Purpose: generic cp
647	#D-* Examples:
648	#D-
649	function IGCM_sys_Cp {
650	IGCM_debug_PushStack "IGCM_sys_Cp" $@
651	if ( $DEBUG_sys ) ; then
652	echo "IGCM_sys_Cp :" $@
653	fi
654
655	typeset RET
656
657	echo cp $@ > out_rsync 2>&1
658	\cp $@ >> out_rsync 2>&1
659	RET=$?
660
661	if [ ${RET} -gt 0 ] ; then
662	echo "IGCM_sys_Cp : error."
663	cat out_rsync
664	IGCM_debug_Exit "IGCM_sys_Cp"
665	fi
666	IGCM_debug_PopStack "IGCM_sys_Cp"
667	}
668
669	#D-#==================================================
670	#D-function IGCM_sys_Rm
671	#D-* Purpose: generic rm
672	#D-* Examples:
673	#D-
674	function IGCM_sys_Rm {
675	IGCM_debug_PushStack "IGCM_sys_Rm" -- $@
676	if ( $DEBUG_sys ) ; then
677	echo "IGCM_sys_Rm :" $@
678	fi
679
680	typeset RET
681
682	echo rm $@ > out_rsync 2>&1
683	\rm $@ >> out_rsync 2>&1
684	RET=$?
685
686	if [ ${RET} -gt 0 ] ; then
687	echo "IGCM_sys_Rm : error."
688	cat out_rsync
689	IGCM_debug_Exit "IGCM_sys_Rm"
690	fi
691	IGCM_debug_PopStack "IGCM_sys_Rm"
692	}
693
694	#D-#==================================================
695	#D-function IGCM_sys_Mv
696	#D-* Purpose: generic move
697	#D-* Examples:
698	#D-
699	function IGCM_sys_Mv {
700	IGCM_debug_PushStack "IGCM_sys_Mv" $@
701	if ( $DEBUG_sys ) ; then
702	echo "IGCM_sys_Mv :" $@
703	fi
704
705	if [ $DRYRUN = 0 ]; then
706
707	typeset RET
708
709	echo mv $@ > out_rsync 2>&1
710	\mv $@ >> out_rsync 2>&1
711	RET=$?
712
713	if [ ${RET} -gt 0 ] ; then
714	echo "IGCM_sys_Mv : error in mv."
715	cat out_rsync
716	IGCM_debug_Exit "IGCM_sys_Mv"
717	fi
718	else
719	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
720	fi
721
722	IGCM_debug_PopStack "IGCM_sys_Mv"
723	}
724
725	#D-#==================================================
726	#D-function IGCM_sys_Put_Dir
727	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
728	#D-* Examples:
729	#D-
730	function IGCM_sys_Put_Dir {
731	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
732	if ( $DEBUG_sys ) ; then
733	echo "IGCM_sys_Put_Dir :" $@
734	fi
735	if [ $DRYRUN = 0 ]; then
736	if [ ! -d ${1} ] ; then
737	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
738	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
739	return
740	fi
741
742	typeset RET
743
744	# Only if we use rsync
745	#IGCM_sys_TestDirArchive $( dirname $2 )
746	#
747	#USUAL WAY
748	rcp -r $1 gaya:$2 > out_rsync 2>&1
749	RET=$?
750
751	if [ ${RET} -gt 0 ] ; then
752	echo "IGCM_sys_Put_Dir : error."
753	cat out_rsync
754	IGCM_debug_Exit "IGCM_sys_Put_Dir"
755	fi
756	else
757	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
758	fi
759	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
760	}
761
762	#D-#==================================================
763	#D-function IGCM_sys_Get_Dir
764	#D-* Purpose: Copy a complete directory from $(ARCHIVE)
765	#D-* Examples:
766	#D-
767	function IGCM_sys_Get_Dir {
768	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
769	if ( $DEBUG_sys ) ; then
770	echo "IGCM_sys_Get_Dir :" $@
771	fi
772	if [ $DRYRUN = 0 ]; then
773	if [ ! -d ${1} ] ; then
774	echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
775	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
776	return
777	fi
778
779	typeset RET
780
781	#USUAL WAY
782	rcp -rp gaya:$1 $2 > out_rsync 2>&1
783	RET=$?
784
785	if [ ${RET} -gt 0 ] ; then
786	echo "IGCM_sys_Get_Dir : error."
787	cat out_rsync
788	IGCM_debug_Exit "IGCM_sys_Get_Dir"
789	fi
790	else
791	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
792	fi
793	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
794	}
795
796	#D-#==================================================
797	#D-function IGCM_sys_Put_Rest
798	#D-* Purpose: Put computied restarts on $(ARCHIVE).
799	#D- File and target directory must exist.
800	#D-* Examples:
801	#D-
802	function IGCM_sys_Put_Rest {
803	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
804	if ( $DEBUG_sys ) ; then
805	echo "IGCM_sys_Put_Rest :" $@
806	fi
807	if [ $DRYRUN = 0 ]; then
808	if [ ! -f ${1} ] ; then
809	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
810	IGCM_debug_Exit "IGCM_sys_Put_Rest"
811	fi
812
813	typeset RET
814	#
815	if [ X${JobType} = XRUN ] ; then
816	IGCM_sys_Chmod 444 ${1}
817	fi
818	#
819	#USUAL WAY
820	mfput $1 $2 > out_rsync 2>&1
821	RET=$?
822
823	# #RSYNC WITH NETWORK RSH CALL
824	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} > out_rsync 2>&1
825	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} >> out_rsync 2>&1
826
827	# #RSYNC WITH NFS USE
828	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > out_rsync 2>&1
829	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> out_rsync 2>&1
830
831	# RET=$?
832	# IGCM_sys_Rsync_out $RET
833
834	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
835	# (( RET=RET+$? ))
836
837	if [ ${RET} -gt 0 ] ; then
838	echo "IGCM_sys_Put_Rest : error."
839	cat out_rsync
840	IGCM_debug_Exit "IGCM_sys_Put_Rest"
841	fi
842	else
843	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
844	fi
845	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
846	}
847
848	#D-#==================================================
849	#D-function IGCM_sys_Put_Out
850	#D-* Purpose: Copy a file on $(ARCHIVE) after have chmod it in readonly
851	#D-* Examples:
852	#D-
853	function IGCM_sys_Put_Out {
854	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
855	if ( $DEBUG_sys ) ; then
856	echo "IGCM_sys_Put_Out :" $@
857	fi
858	if [ $DRYRUN = 0 ]; then
859	if [ ! -f ${1} ] ; then
860	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
861	IGCM_debug_PopStack "IGCM_sys_Put_Out"
862	return 1
863	fi
864
865	typeset RET
866	#
867	if [ X${JobType} = XRUN ] ; then
868	IGCM_sys_Chmod 444 ${1}
869	fi
870	#
871	#USUAL WAY
872	mfput $1 $2 > out_rsync 2>&1
873	RET=$?
874
875	# #RSYNC WITH NETWORK RSH CALL
876	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} > out_rsync 2>&1
877	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${RHOST}:${2} >> out_rsync 2>&1
878
879	# #RSYNC WITH NFS USE
880	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > out_rsync 2>&1
881	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> out_rsync 2>&1
882
883	# RET=$?
884	# IGCM_sys_Rsync_out $RET
885
886	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
887	# (( RET=RET+$? ))
888
889	if [ ${RET} -gt 0 ] ; then
890	echo "IGCM_sys_Put_Out : error."
891	cat out_rsync
892	IGCM_debug_Exit "IGCM_sys_Put_Out"
893	fi
894	else
895	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
896	fi
897	IGCM_debug_PopStack "IGCM_sys_Put_Out"
898	return 0
899	}
900
901	#D-#==================================================
902	#D-function IGCM_sys_Get
903	#D-* Purpose: Get a file from ${ARCHIVE}
904	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
905	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
906	function IGCM_sys_Get {
907	IGCM_debug_PushStack "IGCM_sys_Get" $@
908
909	typeset DEST RET dm_liste ifile target
910
911	if ( $DEBUG_sys ) ; then
912	echo "IGCM_sys_Get :" $@
913	fi
914	if [ $DRYRUN -le 2 ]; then
915	if [ X${1} = X'/l' ] ; then
916	# test if the first file is present in the old computation :
917	eval set +A dm_liste \${${2}}
918	else
919	dm_liste=${1}
920	fi
921	eval DEST=\${${#}}
922
923	# test if the (first) file is present in the old computation :
924	IGCM_sys_TestFileArchive ${dm_liste[0]}
925	RET=$?
926	if [ ${RET} -gt 0 ] ; then
927	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
928	IGCM_debug_Exit "IGCM_sys_Get"
929	IGCM_debug_PopStack "IGCM_sys_Get"
930	return
931	fi
932
933	# SD : dm_liste is not suited for computing job
934	# because we change filename during transfert
935	# dm_liste is better suited for post-treatment
936	#dm_liste=" "
937	#(( ifile=1 ))
938	#while [ $ifile -lt $# ] ; do
939	# dm_liste=$( eval echo ${dm_liste} " "\${${ifile}} )
940	# (( ifile = ifile + 1 ))
941	#done
942	#DEST=$( eval echo \${${#}} )
943
944	#USUAL WAY
945	mfget ${dm_liste[*]} ${DEST} > out_rsync 2>&1
946	RET=$?
947
948	# #RSYNC WITH NETWORK RSH CALL
949	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RHOST}:"${dm_liste}" ${RHOST}:${RUN_DIR}/${DEST} > out_rsync 2>&1
950	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RHOST}:"${dm_liste}" ${RHOST}:${RUN_DIR}/${DEST} >> out_rsync 2>&1
951
952	# #RSYNC WITH NFS USE
953	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > out_rsync 2>&1
954	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> out_rsync 2>&1
955
956	# RET=$?
957	# IGCM_sys_Rsync_out $RET
958
959	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk out_rsync
960	# (( RET=RET+$? ))
961
962	if [ ${RET} -gt 0 ] ; then
963	echo "IGCM_sys_Get : copy error."
964	cat out_rsync
965	IGCM_debug_Exit "IGCM_sys_Get"
966	fi
967	else
968	( ${DEBUG_debug} ) && echo "DRYRUN mode = " $DRYRUN >> stack
969	fi
970	IGCM_debug_PopStack "IGCM_sys_Get"
971	}
972
973	############################################################## A FINIR !!
974
975	#D-#==================================================
976	#D-function IGCM_sys_GetDate_FichWork
977	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
978	#D-* Examples:
979	#D-
980	function IGCM_sys_GetDate_FichWork {
981	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
982	if ( $DEBUG_sys ) ; then
983	echo "IGCM_sys_GetDate_FichWork :" $@
984	fi
985	# donne la date filesys d'un fichier sur la machine work
986	IGCM_debug_PopStack "IGCM_sys_FichWork"
987	}
988
989	#D-#==================================================
990	#D-function IGCM_sys_GetDate_FichArchive
991	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
992	#D-* Examples:
993	#D-
994	function IGCM_sys_GetDate_FichArchive {
995	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
996	if ( $DEBUG_sys ) ; then
997	echo "IGCM_sys_GetDate_FichArchive :" $@
998	fi
999	IGCM_debug_PopStack "IGCM_sys_FichArchive"
1000	}
1001
1002	##############################################################
1003	# REBUILD OPERATOR
1004
1005	function IGCM_sys_rebuild {
1006	IGCM_debug_PushStack "IGCM_sys_rebuild" -- $@
1007	if ( $DEBUG_sys ) ; then
1008	echo "IGCM_sys_rebuild :" $@
1009	fi
1010	/home/rech/psl/rpsl035/SX/bin/rebuild -f -o $@
1011	if [ $? -gt 0 ] ; then
1012	echo "IGCM_sys_rebuild : erreur ${@}."
1013	IGCM_debug_Exit "rebuild"
1014	fi
1015
1016	IGCM_debug_PopStack "IGCM_sys_rebuild"
1017	}
1018
1019	############################################################
1020	# Activate Running Environnment Variables
1021
1022	function IGCM_sys_activ_variables {
1023	IGCM_debug_PushStack "IGCM_sys_activ_variables"
1024	if ( $DEBUG_sys ) ; then
1025	echo "IGCM_sys_activ_variables"
1026	fi
1027
1028	# --------------------------------------------------------------------
1029	#D- MPI specifications
1030	# --------------------------------------------------------------------
1031
1032	#D-- MPISUSPEND
1033	export MPISUSPEND=${MPISUSPEND:=OFF}
1034
1035	#D-- MPIPROGINF #other choices : ALL_DETAIL2
1036	export MPIPROGINF=ALL
1037	#D- activate ftrace (with -ftrace)
1038	export F_FTRACE=YES
1039	#D- communication information (with -ftrace)
1040	export MPICOMMINF=DETAIL
1041
1042	# --------------------------------------------------------------------
1043	#D- Other specifications
1044	# --------------------------------------------------------------------
1045
1046	#D- max number of character/line in output job
1047	export F_SYSLEN=5000
1048	#D- number of error that can be admitted on the NEC
1049	export F_ERRCNT=0
1050	#D- global performance
1051	export F_PROGINF=DETAIL
1052
1053	#D- I/O performance (FORTRAN I/O only not netCDF)
1054	export F_FILEINF=${F_FILEINF:=NO}
1055	#D- netCDF I/O performance
1056	export NC_FILEINF=${NC_FILEINF:=NO}
1057
1058	IGCM_debug_PopStack "IGCM_sys_activ_variables"
1059	}
1060
1061	############################################################
1062	# Desactivate Running Environnment Variables
1063
1064	function IGCM_sys_desactiv_variables {
1065	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
1066	if ( $DEBUG_sys ) ; then
1067	echo "IGCM_sys_desactiv_variables"
1068	fi
1069	# --------------------------------------------------------------------
1070	#D- MPI specifications
1071	# --------------------------------------------------------------------
1072
1073	#D-- MPIPROGINF
1074	export MPIPROGINF=NO
1075
1076	# --------------------------------------------------------------------
1077	#D- Other specifications
1078	# --------------------------------------------------------------------
1079
1080	#D- global performance
1081	export F_PROGINF=NO
1082
1083	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
1084
1085	}
1086
1087	############################################################
1088	# Build run file
1089
1090	function IGCM_sys_build_run_file {
1091	IGCM_debug_PushStack "IGCM_sys_build_run_file" $@
1092	if ( $DEBUG_sys ) ; then
1093	echo "IGCM_sys_build_run_file " $@
1094	fi
1095	(( NUM_PROC_ATM = BATCH_NUM_PROC_TOT - 1 ))
1096	(( NUM_PROC_OASIS = BATCH_NUM_PROC_TOT - NUM_PROC_ATM ))
1097	(( NUM_PROC_OCE = BATCH_NUM_PROC_TOT - NUM_PROC_ATM ))
1098
1099	if [ $1 = MPI2 ]; then
1100	cat <<EOF > run_file
1101	-p 1 -np 1 -e ./oasis
1102	EOF
1103	(( NUM_PROCESS = BATCH_NUM_PROC_TOT + 1 ))
1104	config_UserChoices_JobRunOptions='"-max_np ${NUM_PROCESS} -f"'
1105
1106	elif [ $1 = MPI1 ]; then
1107	cat <<EOF > run_file
1108	-p $NUM_PROC_OASIS -e ./oasis
1109	-p $NUM_PROC_ATM -e ./lmdz.x
1110	-p $NUM_PROC_OCE -e ./opa.xx
1111	EOF
1112
1113	fi
1114
1115	IGCM_debug_PopStack "IGCM_sys_build_run_file"
1116
1117	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: