Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_obelix.ksh @ 1148

Last change on this file since 1148 was 1147, checked in by sdipsl, 10 years ago
broken ensemble. Tentative fix for Agathe. Ensemble are broken since r1057... broken rebuild station due to changes in LMDZ station files. Tentative fix for Lidia.
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 74.8 KB

Rev	Line
[657]	1	#!/bin/ksh
[16]	2
	3	#**************************************************************
	4	# Author: Sebastien Denvil, Martial Mancip, Christian Laguerre
[373]	5	# Contact: Martial.Mancip__at__ipsl.jussieu.fr
	6	# $Revision:: $ Revision of last commit
	7	# $Author:: $ Author of last commit
	8	# $Date:: $ Date of last commit
[16]	9	# IPSL (2006)
	10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
[657]	11	#
[16]	12	#**************************************************************
	13
	14	#=========================================================
	15	# The documentation of this file can be automatically generated
[657]	16	# if you use the prefix #D- for comments to be extracted.
[16]	17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
	18	#=========================================================
	19
	20	#D-#==================================================
	21	#D-LibIGCM_sys for obelix
	22	#D-#==================================================
	23	#D-
[657]	24	#D- This ksh library if a layer under some usefull
[16]	25	#D-environment variables and shell commands.
	26	#D-All those definitions depend on host particularities.
	27	#D-It manages a stack mechanism and test validity of operations.
	28	#D-All function described bellow must be prefixed by IGCM_sys.
	29
	30	#====================================================
	31	# libIGCM_sys PARAMETERS
	32	#====================================================
	33
[343]	34	#====================================================
[16]	35	# set DEBUG_sys to true to output calls of function
	36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
	37
[343]	38	#====================================================
[16]	39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
	40	typeset -r DRYRUN=${DRYRUN:=0}
	41
	42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
	43	# -------------------------------------------------------------------------------------
	44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
[809]	45	# \| \| Cp/Exe/param/files \| \| \| \|
	46	# \| \| Chmod Qsub \| \| \| \|
[16]	47	# -------------------------------------------------------------------------------------
	48	# \| 0 \| yes \| yes \| yes \| yes \|
	49	# -------------------------------------------------------------------------------------
	50	# \| 1 \| yes \| yes \| yes \| no \|
	51	# -------------------------------------------------------------------------------------
	52	# \| 2 \| yes \| yes \| no \| no \|
	53	# -------------------------------------------------------------------------------------
	54	# \| 3 \| yes \| no \| no \| no \|
	55	# -------------------------------------------------------------------------------------
	56
	57	#=====================================================
	58	# Global Variables :
	59	#=====================================================
	60	# Language : "fr" or "en"
	61	typeset -r MYLANG="fr"
	62
	63	#=====================================================
[657]	64	# Host and user names
[217]	65	# $hostname ou hostname
[800]	66	typeset HOST=${HOST:=$( hostname )}
[16]	67	# $username ou whoami
[800]	68	typeset LOGIN=${LOGIN:=$( whoami )}
[657]	69	# $hostname of the MASTER job
[742]	70	typeset MASTER=${MASTER:=$( hostname )}
[16]	71
	72	#D-
	73	#D-#==================================================
	74	#D-Program used in libIGCM
	75	#D-#==================================================
	76
[742]	77	# Submit command
	78	typeset SUBMIT=${SUBMIT:=qsub}
[16]	79	# rsync with path
	80	typeset -r RSYNC=/usr/bin/rsync
	81	# RSYNC_opt args to rsync
[282]	82	typeset -r RSYNC_opt="-va"
[657]	83	# ie storage filesystem
	84	typeset -r STOREHOST=${MASTER}
[16]	85
	86	#====================================================
[305]	87	# Set environment tools (ferret, nco, cdo)
	88	#====================================================
[1097]	89	. /home/users/igcmg/.atlas_env_asterix_ksh
[16]	90
[343]	91	#====================================================
[421]	92	# Set lf95 environment
	93	#====================================================
	94	. /usr/local/install/lf6481/bash_laheyfort_setup
	95
	96	#====================================================
[16]	97	# Host specific DIRECTORIES
	98	#====================================================
	99
	100	#====================================================
[657]	101	#- MirrorlibIGCM for frontend
	102	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
	103
	104	#====================================================
	105	#- libIGCM_POST for frontend
	106	typeset -r libIGCM_POST=${libIGCM}
	107
	108	#====================================================
[16]	109	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
	110	typeset -r R_EXE="${MODIPSL}/bin"
	111
	112	#====================================================
	113	#- SUBMIT_DIR : submission dir
[256]	114	if [ X${PBS_O_WORKDIR} != X ] ; then
[1124]	115	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
[256]	116	else
[1124]	117	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PWD}}
[256]	118	fi
[657]	119
[16]	120	#====================================================
[725]	121	#- IN
[978]	122	typeset -r R_IN=${R_IN:=/home/orchideeshare/igcmg/IGCM}
[725]	123
	124	#====================================================
[16]	125	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
[838]	126	if [ X${PBS_JOBID} != X ] ; then
	127	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/scratch/$PBS_O_LOGNAME.$PBS_JOBID}
	128	else
[839]	129	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/tmp/tmp$$}
[838]	130	fi
[16]	131
	132	#====================================================
	133	#- HOST_MPIRUN_COMMAND
	134	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="time mpirun"}
	135
[61]	136	#====================================================
	137	#- Max number of arguments passed to nco operator or demigration command
	138	UNIX_MAX_LIMIT=360
	139
[657]	140	#====================================================
	141	#- Number of core per node (max number of OpenMP task)
	142	NUM_COREPERNODE=1
	143
	144	#====================================================
	145	#- Default number of MPI task for IPSL coupled model
	146	#- required for backward compatibility
	147	#-
	148	DEFAULT_NUM_PROC_OCE=1
	149	DEFAULT_NUM_PROC_CPL=1
	150	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
	151	DEFAULT_NUM_PROC_TOTAL=${BATCH_NUM_PROC_TOT}
	152
[16]	153	#D-#==================================================
[1057]	154	#D-function IGCM_sys_defineArchives
	155	#D-* Purpose:
	156	#D-* Define ARCHIVE : Dedicated to large files
	157	#D-* Define STORAGE : Dedicated to small/medium files
	158	#D-* Define R_OUT : Output tree located on ARCHIVE
	159	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
	160	#D-* Define R_BUF : Output tree located on STORAGE hosting files waiting for rebuild or pack processes
	161	#D-* if SpaceName=TEST nothing special will hapen
[725]	162	#D-* Examples:
	163	#D-
[1057]	164	function IGCM_sys_defineArchives {
	165	IGCM_debug_PushStack "IGCM_sys_defineArchives"
	166
[1060]	167	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
[1057]	168	#====================================================
	169	#- ARCHIVE (dedicated to large files)
[1063]	170	ARCHIVE=${config_UserChoices_ARCHIVE}
[1057]	171	else
	172	#====================================================
	173	#- ARCHIVE (dedicated to large files)
[1063]	174	ARCHIVE=${ARCHIVE:=/home/scratch01/${LOGIN}}
[1057]	175	fi
	176
[1060]	177	if [ ! X${config_UserChoices_STORAGE} = X ]; then
[1057]	178	#====================================================
	179	#- STORAGE (dedicated to small/medium files)
[1063]	180	STORAGE=${config_UserChoices_STORAGE}
[1057]	181	else
	182	#====================================================
	183	#- STORAGE (dedicated to small/medium files)
[1063]	184	STORAGE=${ARCHIVE}
[1057]	185	fi
	186
	187	# ON OBELIX NO SPECIAL CASE WHEN X${config_UserChoices_SpaceName} = XTEST
	188
	189	#====================================================
	190	#- R_OUT
[1063]	191	R_OUT=${ARCHIVE}/IGCM_OUT
[1057]	192
	193	#====================================================
	194	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
[1063]	195	R_FIG=${STORAGE}/IGCM_OUT
[1057]	196
	197	#====================================================
	198	#- R_BUF (ONLY FOR double copy an scratch)
[1063]	199	R_BUF=${STORAGE}/IGCM_OUT
[1057]	200
[1140]	201	#====================================================
	202	#- R_TMP (point a space acting as /tmp)
	203	R_TMP=/tmp
	204
[1057]	205	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
	206	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
	207	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
	208
	209	IGCM_debug_PopStack "IGCM_sys_defineArchives"
[725]	210	}
	211
	212	#D-#==================================================
[16]	213	#D-function IGCM_sys_RshMaster
[800]	214	#D-* Purpose: Connection to frontend machine.
[16]	215	#D-* Examples:
	216	#D-
	217	function IGCM_sys_RshMaster {
[657]	218	IGCM_debug_PushStack "IGCM_sys_RshMaster" $@
[862]	219	/bin/ksh <<-EOF
[16]	220	export libIGCM=${libIGCM}
[98]	221	export DEBUG_debug=${DEBUG_debug}
[16]	222	. ${libIGCM}/libIGCM_debug/libIGCM_debug.ksh
	223	. ${libIGCM}/libIGCM_card/libIGCM_card.ksh
	224	${@}
	225	EOF
[657]	226	if [ $? -gt 0 ] ; then
	227	echo "IGCM_sys_RshMaster : erreur."
	228	IGCM_debug_Exit "IGCM_sys_RshMaster"
	229	fi
	230	IGCM_debug_PopStack "IGCM_sys_RshMaster"
[16]	231	}
	232
	233	#D-#==================================================
[37]	234	#D-function IGCM_sys_RshArchive
	235	#D-* Purpose: Archive rsh command
	236	#D-* Examples:
	237	#D-
	238	function IGCM_sys_RshArchive {
[657]	239	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
	240	/bin/ksh <<-EOF
[40]	241	${@}
	242	EOF
[811]	243	status=$?
	244	if [ ${status} -gt 0 ] ; then
	245	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
[657]	246	IGCM_debug_Exit "IGCM_sys_RshArchive"
	247	fi
	248	IGCM_debug_PopStack "IGCM_sys_RshArchive"
[37]	249	}
	250
	251	#D-#==================================================
[16]	252	#D-function IGCM_sys_RshPost
[282]	253	#D-* Purpose: Post-process rsh command
[16]	254	#D-* Examples:
	255	#D-
	256	function IGCM_sys_RshPost {
[657]	257	IGCM_debug_PushStack "IGCM_sys_RshPost" $@
	258	if ( $DEBUG_sys ) ; then
	259	echo "IGCM_sys_RshPost :" $@
	260	fi
[800]	261
	262	# keep standard input (stdin) for the loop onto temporary file
	263	cat >tmp_IGCM_sys_RshPost_$$
	264
	265	# ============ FRONTEND START ============ #
	266
[862]	267	/bin/ksh <tmp_IGCM_sys_RshPost_$$
[657]	268	if [ $? -gt 0 ] ; then
	269	echo "IGCM_sys_RshPost : erreur."
	270	IGCM_debug_Exit "IGCM_sys_RshPost"
	271	fi
[800]	272	# delete temporary file
	273	\rm tmp_IGCM_sys_RshPost_$$
[657]	274	IGCM_debug_PopStack "IGCM_sys_RshPost"
[16]	275	}
	276
	277	#D-#==================================================
[153]	278	#D-function IGCM_sys_SendMail
	279	#D-* Purpose: Send mail when simulation is over
	280	#D-* Examples:
	281	#D-
	282	function IGCM_sys_SendMail {
[657]	283	IGCM_debug_PushStack "IGCM_sys_SendMail" $@
	284	if ( $DEBUG_sys ) ; then
	285	echo "IGCM_sys_SendMail :" $@
	286	fi
[153]	287
[922]	288	if [ X${1} = XAccounting ] ; then
	289	status=Accounting
	290	mailText=jobAccounting.mail
	291	elif ( ${ExitFlag} ) ; then
[657]	292	status=failed
[922]	293	mailText=jobEnd.mail
[657]	294	else
	295	status=completed
[922]	296	mailText=jobEnd.mail
[657]	297	fi
[717]	298
[922]	299	# Update selected mail template
	300	while read -r line; do
	301	eval echo $line >> mail.txt ;
	302	done < ${libIGCM}/libIGCM_sys/${mailText}
[153]	303
[657]	304	if [ ! -z ${config_UserChoices_MailName} ] ; then
[922]	305	mailx -s "${config_UserChoices_JobName} ${status}" ${config_UserChoices_MailName} < mail.txt
[657]	306	elif [ -f ~/.forward ] ; then
[922]	307	mailx -s "${config_UserChoices_JobName} ${status}" $( cat ~/.forward ) < mail.txt
[657]	308	fi
[153]	309
[811]	310	sleep 10
[922]	311	rm -f mail.txt
[811]	312
[657]	313	IGCM_debug_PopStack "IGCM_sys_SendMail"
[153]	314	}
	315
	316	#D-#==================================================
[16]	317	#D-function IGCM_sys_Mkdir
	318	#D-* Purpose: Master locale mkdir command
	319	#D-* Examples:
	320	#D-
	321	function IGCM_sys_Mkdir {
[657]	322	IGCM_debug_PushStack "IGCM_sys_Mkdir" $@
	323	if ( $DEBUG_sys ) ; then
	324	echo "IGCM_sys_Mkdir :" $@
	325	fi
	326	if [ ! -d ${1} ]; then
	327	\mkdir -p $1
	328	if [ $? -gt 0 ] ; then
	329	echo "IGCM_sys_Mkdir : erreur."
	330	IGCM_debug_Exit "IGCM_sys_Mkdir"
[16]	331	fi
[657]	332	fi
[717]	333	# vérification :
[657]	334	if [ ! -d ${1} ] ; then
	335	echo "IGCM_sys_Mkdir : erreur."
	336	IGCM_debug_Exit "IGCM_sys_Mkdir"
	337	fi
	338	IGCM_debug_PopStack "IGCM_sys_Mkdir"
[16]	339	}
	340
	341	#D-#==================================================
	342	#D-function IGCM_sys_MkdirArchive
	343	#D-* Purpose: Mkdir on Archive
	344	#D-* Examples:
	345	#D-
	346	function IGCM_sys_MkdirArchive {
[657]	347	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
	348	if ( $DEBUG_sys ) ; then
	349	echo "IGCM_sys_MkdirArchive :" $@
	350	fi
[717]	351	#- creation de repertoire sur le serveur fichier
[657]	352	if [ ! -d ${1} ]; then
[811]	353	mkdir -p $1
	354	status=$?
	355
	356	if [ ${status} -gt 0 ] ; then
	357	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
[657]	358	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
[16]	359	fi
[657]	360	fi
	361	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
[16]	362	}
	363
	364	#D-#==================================================
	365	#D-function IGCM_sys_MkdirWork
	366	#D-* Purpose: Mkdir on Work
	367	#D-* Examples:
	368	#D-
	369	function IGCM_sys_MkdirWork {
[657]	370	IGCM_debug_PushStack "IGCM_sys_MkdirWork" $@
	371	if ( $DEBUG_sys ) ; then
	372	echo "IGCM_sys_MkdirWork :" $@
	373	fi
[717]	374	#- creation de repertoire sur le serveur fichier
[657]	375	if [ ! -d ${1} ]; then
	376	\mkdir -p $1
	377	if [ $? -gt 0 ] ; then
	378	echo "IGCM_sys_MkdirWork : erreur."
	379	IGCM_debug_Exit "IGCM_sys_MkdirWork"
[16]	380	fi
[657]	381	fi
	382	IGCM_debug_PopStack "IGCM_sys_MkdirWork"
[16]	383	}
	384
	385	#D-#==================================================
	386	#D-function IGCM_sys_Cd
	387	#D-* Purpose: master cd command
	388	#D-* Examples:
	389	#D-
	390	function IGCM_sys_Cd {
[657]	391	IGCM_debug_PushStack "IGCM_sys_Cd" $@
	392	if ( $DEBUG_sys ) ; then
	393	echo "IGCM_sys_Cd :" $@
	394	fi
	395	\cd $1
	396	if [ $? -gt 0 ] ; then
	397	echo "IGCM_sys_Cd : erreur."
	398	IGCM_debug_Exit "IGCM_sys_Cd"
	399	fi
	400	IGCM_debug_PopStack "IGCM_sys_Cd"
[16]	401	}
	402
	403	#D-#==================================================
	404	#D-function IGCM_sys_Chmod
	405	#D-* Purpose: Chmod
	406	#D-* Examples:
	407	#D-
	408	function IGCM_sys_Chmod {
[1057]	409	IGCM_debug_PushStack "IGCM_sys_Chmod" $@
[657]	410	if ( $DEBUG_sys ) ; then
	411	echo "IGCM_sys_Chmod :" $@
	412	fi
[809]	413	\chmod $@
	414	if [ $? -gt 0 ] ; then
	415	echo "IGCM_sys_Chmod : erreur."
	416	IGCM_debug_Exit "IGCM_sys_Chmod"
[657]	417	fi
	418	IGCM_debug_PopStack "IGCM_sys_Chmod"
[16]	419	}
	420
	421	#D-#==================================================
	422	#D-function IGCM_sys_FileSize
	423	#D-* Purpose: Filesize
	424	#D-* Examples:
	425	#D-
	426	function IGCM_sys_FileSize {
[657]	427	IGCM_debug_PushStack "IGCM_sys_FileSize" $@
[16]	428
[657]	429	typeset sizeF
	430	set +A sizeF -- $( ls -la ${1} )
	431	if [ $? -gt 0 ] ; then
	432	IGCM_debug_Exit "IGCM_sys_FileSize"
	433	fi
	434	eval ${2}=${sizeF[4]}
[16]	435
[657]	436	IGCM_debug_PopStack "IGCM_sys_FileSize"
[16]	437	}
	438
	439	#D-#==================================================
	440	#D-function IGCM_sys_TestDir
	441	#D-* Purpose: Test Directory that must exists
	442	#D-* Examples:
	443	#D-
	444	function IGCM_sys_TestDir {
[657]	445	IGCM_debug_PushStack "IGCM_sys_TestDir" $@
	446	if ( $DEBUG_sys ) ; then
	447	echo "IGCM_sys_TestDir :" $@
	448	fi
	449	typeset ExistFlag
	450	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
	451	IGCM_debug_PopStack "IGCM_sys_TestDir"
[67]	452
[657]	453	return ${ExistFlag}
[16]	454	}
	455
	456	#D-#==================================================
	457	#D-function IGCM_sys_TestDirArchive
	458	#D-* Purpose: Test Directory that must exists on Archive
	459	#D-* Examples:
	460	#D-
	461	function IGCM_sys_TestDirArchive {
[657]	462	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
	463	if ( $DEBUG_sys ) ; then
	464	echo "IGCM_sys_TestDirArchive :" $@
	465	fi
	466	typeset ExistFlag
	467	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
	468	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
	469	return ${ExistFlag}
[16]	470	}
	471
	472	#D-#==================================================
[800]	473	#D-function IGCM_sys_IsFileArchived
	474	#D-* Purpose: Test file that must NOT EXISTS on Archive
	475	#D-* Examples:
	476	#D-
	477	function IGCM_sys_IsFileArchived {
	478	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
	479	if ( $DEBUG_sys ) ; then
	480	echo "IGCM_sys_IsFileArchived :" $@
	481	fi
	482	typeset IsArchivedFlag
	483	# Never archived for this system
	484	IsArchivedFlag=1
	485	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
	486
	487	return ${IsArchivedFlag}
	488	}
	489
	490	#D-#==================================================
[16]	491	#D-function IGCM_sys_TestFileArchive
	492	#D-* Purpose: Test file that must NOT EXISTS on Archive
	493	#D-* Examples:
	494	#D-
	495	function IGCM_sys_TestFileArchive {
[657]	496	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
	497	typeset ExistFlag
	498	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
	499	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
[67]	500
[657]	501	return ${ExistFlag}
[67]	502	}
	503
	504	#D-#==================================================
[657]	505	#D-function IGCM_sys_TestFileBuffer
	506	#D-* Purpose: Test file that must NOT EXISTS on Buffer
	507	#D-* Examples:
	508	#D-
	509	function IGCM_sys_TestFileBuffer {
	510	IGCM_debug_PushStack "IGCM_sys_TestFileBuffer" $@
	511	typeset ExistFlag
	512	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
	513	IGCM_debug_PopStack "IGCM_sys_TestFileBuffer"
	514
	515	return ${ExistFlag}
	516	}
	517
	518	#D-#==================================================
[67]	519	#D-function IGCM_sys_CountFileArchive
	520	#D-* Purpose: Count files on Archive filesystem
	521	#D-* Examples:
	522	#D-
	523	function IGCM_sys_CountFileArchive {
[657]	524	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
	525	ls ${@} 2>/dev/null \| wc -l
	526	if [ $? -gt 0 ] ; then
	527	echo "IGCM_sys_CountFileArchive : erreur."
	528	fi
	529	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
[16]	530	}
	531
	532	#D-#==================================================
[657]	533	#D-function IGCM_sys_CountFileBuffer
	534	#D-* Purpose: Count files on Scratch filesystem
	535	#D-* Examples:
	536	#D-
	537	function IGCM_sys_CountFileBuffer {
	538	IGCM_debug_PushStack "IGCM_sys_CountFileBuffer" $@
	539	ls ${@} 2>/dev/null \| wc -l
	540	if [ $? -gt 0 ] ; then
	541	echo "IGCM_sys_CountFileBuffer : erreur."
	542	fi
	543	IGCM_debug_PopStack "IGCM_sys_CountFileBuffer"
	544	}
	545
	546	#D-#==================================================
[16]	547	#D-function IGCM_sys_Tree
	548	#D-* Purpose: Tree directories with files on ${ARCHIVE}
	549	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
	550	#D-
	551	function IGCM_sys_Tree {
[657]	552	IGCM_debug_PushStack "IGCM_sys_Tree" $@
	553	if ( $DEBUG_sys ) ; then
	554	echo "IGCM_sys_Tree :" $@
	555	fi
[16]	556
[657]	557	\tree -f $@
[16]	558
[657]	559	IGCM_debug_PopStack "IGCM_sys_Tree"
[16]	560	}
	561
	562	#D-#==================================================
	563	#D-function IGCM_sys_Tar
[657]	564	#D-* Purpose: master tar command
[16]	565	#D-* Examples:
	566	#D-
	567	function IGCM_sys_Tar {
[657]	568	IGCM_debug_PushStack "IGCM_sys_Tar" $@
	569	if ( $DEBUG_sys ) ; then
	570	echo "IGCM_sys_Tar :" $@
	571	fi
	572	\tar cf $@
	573	if [ $? -gt 0 ] ; then
	574	echo "IGCM_sys_Tar : erreur."
	575	IGCM_debug_Exit "IGCM_sys_Tar"
	576	fi
	577	IGCM_debug_PopStack "IGCM_sys_Tar"
[16]	578	}
	579
	580	#D-#==================================================
	581	#D-function IGCM_sys_UnTar
	582	#D-* Purpose: master un-tar command
	583	#D-* Examples:
	584	#D-
	585	function IGCM_sys_UnTar {
[657]	586	IGCM_debug_PushStack "IGCM_sys_UnTar" $@
	587	if ( $DEBUG_sys ) ; then
	588	echo "IGCM_sys_UnTar :" $@
	589	fi
	590	\tar xvf $1
	591	if [ $? -gt 0 ] ; then
	592	echo "IGCM_sys_UnTar : erreur."
	593	IGCM_debug_Exit "IGCM_sys_UnTar"
	594	fi
	595	IGCM_debug_PopStack "IGCM_sys_UnTar"
[16]	596	}
	597
	598	#D-#==================================================
	599	#D-function IGCM_sys_Qsub
	600	#D-* Purpose: Qsub new job
	601	#D-* Examples:
	602	#D-
	603	function IGCM_sys_Qsub {
[657]	604	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
	605	if ( $DEBUG_sys ) ; then
	606	echo "IGCM_sys_Qsub :" $@
	607	fi
[923]	608	typeset options status
	609	options="-o ${SUBMIT_DIR}/${Script_Output}"
[978]	610	/usr/local/bin/qsub ${options} $1 > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	611	status=$?
[949]	612
[978]	613	cat /tmp/out_command_${LOGIN}.$$
[811]	614	if [ ${status} -gt 0 ] ; then
	615	IGCM_debug_Print 2 "IGCM_sys_Qsub $1 : error code ${status}"
[657]	616	IGCM_debug_Exit "IGCM_sys_Qsub"
[949]	617	else
[978]	618	JobID=$( gawk {'print $1'} /tmp/out_command_${LOGIN}.$$ )
[657]	619	fi
	620	IGCM_debug_PopStack "IGCM_sys_Qsub"
[16]	621	}
	622
	623	#D-#==================================================
	624	#D-function IGCM_sys_QsubPost
	625	#D-* Purpose: Qsub new job on scalaire
	626	#D-* Examples:
	627	#D-
	628	function IGCM_sys_QsubPost {
[657]	629	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
	630	if ( $DEBUG_sys ) ; then
	631	echo "IGCM_sys_QsubPost :" $@
	632	fi
[923]	633	typeset options status
	634	options="-o ${POST_DIR}/${Script_Post_Output}.out -v ${listVarEnv}"
[978]	635	/usr/local/bin/qsub ${options} ${libIGCM_POST}/$1.job > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	636	status=$?
[949]	637
[978]	638	cat /tmp/out_command_${LOGIN}.$$
[811]	639	if [ ${status} -gt 0 ] ; then
[923]	640	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
[657]	641	IGCM_debug_Exit "IGCM_sys_QsubPost"
[949]	642	else
[978]	643	JobID=$( gawk {'print $1'} /tmp/out_command_${LOGIN}.$$ )
[657]	644	fi
	645	IGCM_debug_PopStack "IGCM_sys_QsubPost"
[16]	646	}
	647
	648	#D-*************************
[657]	649	#D- File transfer functions
[16]	650	#D-*************************
	651	#D-
	652
	653	#D-#==================================================
	654	#D-function IGCM_sys_Rsync_out
	655	#D-* Purpose: treat return val of rsync
	656	#D-* Examples: IGCM_sys_Rsync_out out_RET_rsync
	657	#D- Error values and explanations can depend on your system version.
	658	function IGCM_sys_Rsync_out {
[717]	659	status=$1
	660	if [ ! $status ] ; then
[657]	661	echo "rsync error !"
	662	fi
[16]	663
[657]	664	if [ $MYLANG = "fr" ]; then
[717]	665	case $status in
[657]	666	0) return ;;
	667	1) echo "Erreur de rsync ; RERR_SYNTAX : "
	668	echo "Erreur de syntaxe ou d'utilisation."
	669	return;;
	670	2) echo "Erreur de rsync ; RERR_PROTOCOL : "
	671	echo "Incompatibilité de protocole."
	672	return;;
	673	3) echo "Erreur de rsync ; RERR_FILESELECT 3"
	674	echo "Erreurs lors de la sélection des fichiers d'entrée sortie et"
	675	echo "répertoires"
	676	return;;
	677	4) echo "Erreur de rsync ; RERR_UNSUPPORTED"
	678	echo "Action demandée non supportée : une tentative de manipulation de"
	679	echo "fichiers 64-bits sur une plate-forme qui ne les supporte pas a"
	680	echo "été faite ; ou une option qui est supportée par le client mais"
	681	echo "pas par le serveur a été spécifiée."
	682	return;;
	683	10) echo "Erreur de rsync ; RERR_SOCKETIO"
	684	echo "Erreur dans le socket d'entrée sortie"
	685	return;;
	686	11) echo "Erreur de rsync ; RERR_FILEIO"
	687	echo "Erreur d'entrée sortie fichier"
	688	return;;
	689	12) echo "Erreur de rsync ; RERR_STREAMIO"
	690	echo "Erreur dans flux de donnée du protocole rsync"
	691	return;;
	692	13) echo "Erreur de rsync ; RERR_MESSAGEIO"
	693	echo "Erreur avec les diagnostics du programme"
	694	return;;
	695	14) echo "Erreur de rsync ; RERR_IPC"
	696	echo "Erreur dans le code IPC"
	697	return;;
	698	20) echo "Erreur de rsync ; RERR_SIGNAL"
	699	echo "SIGUSR1 ou SIGINT reçu"
	700	return;;
	701	21) echo "Erreur de rsync ; RERR_WAITCHILD"
	702	echo "Une erreur retournée par waitpid()"
	703	return;;
	704	22) echo "Erreur de rsync ; RERR_MALLOC"
	705	echo "Erreur lors de l'allocation des tampons de mémoire de coeur"
	706	return;;
	707	23) echo ""
	708	echo "Erreur fichier inexistant"
	709	return;;
	710	30) echo "Erreur de rsync ; RERR_TIMEOUT"
	711	echo "Temps d'attente écoulé dans l'envoi/réception de données"
	712	return;;
[717]	713	*) echo "Erreur de rsync : code de retour de rsync inconnu :" $status
[657]	714	return;;
	715	esac
	716	elif [ $MYLANG = "en" ] ; then
[717]	717	case $status in
[657]	718	0) return;;
	719	1) echo "rsync error : Syntax or usage error "
	720	return;;
	721	2) echo "rsync error : Protocol incompatibility "
	722	return;;
	723	3) echo "rsync error : Errors selecting input/output files, dirs"
	724	return;;
	725	4) echo "rsync error : Requested action not supported: an attempt"
	726	echo "was made to manipulate 64-bit files on a platform that cannot support"
	727	echo "them; or an option was specified that is supported by the client and"
	728	echo "not by the server."
	729	return;;
	730	5) echo "rsync error : Error starting client-server protocol"
	731	return;;
	732	10) echo "rsync error : Error in socket I/O "
	733	return;;
	734	11) echo "rsync error : Error in file I/O "
	735	return;;
	736	12) echo "rsync error : Error in rsync protocol data stream "
	737	return;;
	738	13) echo "rsync error : Errors with program diagnostics "
	739	return;;
	740	14) echo "rsync error : Error in IPC code "
	741	return;;
	742	20) echo "rsync error : Received SIGUSR1 or SIGINT "
	743	return;;
	744	21) echo "rsync error : Some error returned by waitpid() "
	745	return;;
	746	22) echo "rsync error : Error allocating core memory buffers "
	747	return;;
	748	23) echo "rsync error : Partial transfer due to error"
	749	return;;
	750	24) echo "rsync error : Partial transfer due to vanished source files"
	751	return;;
	752	30) echo "rsync error : Timeout in data send/receive "
	753	return;;
[717]	754	*) echo "rsync error : return code of rsync unknown :" $status
[657]	755	return;;
	756	esac
	757	else
	758	echo "unknown language $MYLANG."
	759	return
	760	fi
[16]	761	}
[657]	762
[16]	763	#D-#==================================================
[811]	764	#D-function IGCM_sys_Miror_libIGCM
	765	#D-* Purpose: Mirror libIGCM PATH and lib to frontend
	766	#D-* Examples:
	767	#D-
	768	function IGCM_sys_Mirror_libIGCM {
	769	IGCM_debug_PushStack "IGCM_sys_Mirror_libIGCM"
	770	if ( $DEBUG_sys ) ; then
	771	echo "IGCM_sys_Mirror_libIGCM"
	772	fi
	773
	774	typeset status
	775
	776	mkdir -p ${HOME}/MIRROR/${PATHlibIGCM}
	777
[978]	778	echo ${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} > /tmp/out_command_${LOGIN}.$$ 2>&1
	779	${RSYNC} ${RSYNC_opt} ${libIGCM} ${HOME}/MIRROR/${PATHlibIGCM} >> /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	780	status=$?
	781
	782	if [ ${status} -gt 0 ] ; then
	783	echo "IGCM_sys_Mirror_libIGCM Warning : no libIGCM on frontend."
[978]	784	cat /tmp/out_command_${LOGIN}.$$
[811]	785	fi
	786	IGCM_debug_PopStack "IGCM_sys_Mirror_libIGCM"
	787	}
	788
	789	#D-#==================================================
[16]	790	#D-function IGCM_sys_Cp
	791	#D-* Purpose: generic cp
	792	#D-* Examples:
	793	#D-
	794	function IGCM_sys_Cp {
[657]	795	IGCM_debug_PushStack "IGCM_sys_Cp" $@
	796	if ( $DEBUG_sys ) ; then
	797	echo "IGCM_sys_Cp :" $@
	798	fi
[16]	799
[717]	800	typeset status
[16]	801
[978]	802	echo cp $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
	803	\cp $@ >> /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	804	status=$?
[657]	805
[717]	806	if [ ${status} -gt 0 ] ; then
	807	echo "IGCM_sys_Cp : error code ${status}"
[978]	808	cat /tmp/out_command_${LOGIN}.$$
[657]	809	IGCM_debug_Exit "IGCM_sys_Cp"
	810	else
[978]	811	\rm /tmp/out_command_${LOGIN}.$$
[657]	812	fi
	813	IGCM_debug_PopStack "IGCM_sys_Cp"
[16]	814	}
	815
	816	#D-#==================================================
	817	#D-function IGCM_sys_Rm
	818	#D-* Purpose: generic rm
	819	#D-* Examples:
	820	#D-
	821	function IGCM_sys_Rm {
[1057]	822	IGCM_debug_PushStack "IGCM_sys_Rm" $@
[657]	823	if ( $DEBUG_sys ) ; then
	824	echo "IGCM_sys_Rm :" $@
	825	fi
[16]	826
[717]	827	typeset status
[16]	828
[978]	829	echo rm $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
	830	\rm $@ >> /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	831	status=$?
[657]	832
[717]	833	if [ ${status} -gt 0 ] ; then
	834	echo "IGCM_sys_Rm : error code ${status}"
[978]	835	cat /tmp/out_command_${LOGIN}.$$
[657]	836	IGCM_debug_Exit "IGCM_sys_Rm"
	837	else
[978]	838	\rm /tmp/out_command_${LOGIN}.$$
[657]	839	fi
	840	IGCM_debug_PopStack "IGCM_sys_Rm"
[16]	841	}
	842
	843	#D-#==================================================
[203]	844	#D-function IGCM_sys_RmRunDir
	845	#D-* Purpose: rm tmpdir (dummy function most of the time batch
	846	#D- scheduler will do the job)
	847	#D-* Examples:
	848	#D-
	849	function IGCM_sys_RmRunDir {
[1057]	850	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
[657]	851	if ( $DEBUG_sys ) ; then
	852	echo "IGCM_sys_RmRunDir :" $@
[964]	853	echo "Dummy call, let the scheduler do that."
[657]	854	fi
	855	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
[203]	856	}
	857
	858	#D-#==================================================
[16]	859	#D-function IGCM_sys_Mv
	860	#D-* Purpose: generic move
	861	#D-* Examples:
	862	#D-
	863	function IGCM_sys_Mv {
[657]	864	IGCM_debug_PushStack "IGCM_sys_Mv" $@
	865	if ( $DEBUG_sys ) ; then
	866	echo "IGCM_sys_Mv :" $@
	867	fi
[16]	868
[657]	869	if [ $DRYRUN = 0 ]; then
[16]	870
[717]	871	typeset status
[657]	872
[978]	873	echo mv $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
	874	\mv $@ >> /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	875	status=$?
[657]	876
[717]	877	if [ ${status} -gt 0 ] ; then
	878	echo "IGCM_sys_Mv : error code ${status}"
[978]	879	cat /tmp/out_command_${LOGIN}.$$
[657]	880	IGCM_debug_Exit "IGCM_sys_Mv"
[173]	881	else
[978]	882	\rm /tmp/out_command_${LOGIN}.$$
[16]	883	fi
[657]	884	fi
	885	IGCM_debug_PopStack "IGCM_sys_Mv"
[16]	886	}
	887
	888	#D-#==================================================
	889	#D-function IGCM_sys_Put_Dir
	890	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
	891	#D-* Examples:
	892	#D-
	893	function IGCM_sys_Put_Dir {
[657]	894	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
	895	if ( $DEBUG_sys ) ; then
	896	echo "IGCM_sys_Put_Dir :" $@
	897	fi
	898	if [ $DRYRUN = 0 ]; then
	899	if [ ! -d ${1} ] ; then
	900	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
	901	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
	902	return
[16]	903	fi
	904
[717]	905	typeset status
[16]	906
[717]	907	# Only if we use rsync
	908	#IGCM_sys_TestDirArchive $( dirname $2 )
	909	#
	910	#USUAL WAY
[978]	911	\cp -r $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	912	status=$?
[16]	913
[717]	914	if [ ${status} -gt 0 ] ; then
[811]	915	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
[978]	916	cat /tmp/out_command_${LOGIN}.$$
[657]	917	IGCM_debug_Exit "IGCM_sys_Put_Dir"
[16]	918	else
[978]	919	\rm /tmp/out_command_${LOGIN}.$$
[16]	920	fi
[657]	921	fi
	922	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
[16]	923	}
	924
	925	#D-#==================================================
	926	#D-function IGCM_sys_Get_Dir
[717]	927	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
[16]	928	#D-* Examples:
	929	#D-
	930	function IGCM_sys_Get_Dir {
[657]	931	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
	932	if ( $DEBUG_sys ) ; then
	933	echo "IGCM_sys_Get_Dir :" $@
	934	fi
	935	if [ $DRYRUN = 0 ]; then
	936	if [ ! -d ${1} ] ; then
	937	echo "WARNING : IGCM_sys_Get_Dir ${1} DOES NOT EXIST ."
	938	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
	939	return
[16]	940	fi
	941
[717]	942	typeset status
[16]	943
[719]	944	#USUAL WAY
[978]	945	\cp -ur $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	946	status=$?
[16]	947
[717]	948	if [ ${status} -gt 0 ] ; then
[811]	949	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
[978]	950	cat /tmp/out_command_${LOGIN}.$$
[657]	951	IGCM_debug_Exit "IGCM_sys_Get_Dir"
[16]	952	else
[978]	953	\rm /tmp/out_command_${LOGIN}.$$
[16]	954	fi
[657]	955	fi
	956	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
[16]	957	}
	958
	959	#D-#==================================================
	960	#D-function IGCM_sys_Get_Master
	961	#D-* Purpose: Copy a complete directory from MASTER filesystem
	962	#D-* Examples:
	963	#D-
	964	function IGCM_sys_Get_Master {
[657]	965	IGCM_debug_PushStack "IGCM_sys_Get_Master" $@
	966	if ( $DEBUG_sys ) ; then
	967	echo "IGCM_sys_Get_Master :" $@
	968	fi
	969	if [ $DRYRUN = 0 ]; then
	970	if ( [ ! -d ${1} ] && [ ! -f ${1} ] ) ; then
	971	echo "WARNING : IGCM_sys_Get_Master ${1} DOES NOT EXIST ."
	972	IGCM_debug_PopStack "IGCM_sys_Get_Master"
	973	return
[16]	974	fi
	975
[811]	976	typeset NB_ESSAI DELAI status i
	977	# number of tentative
	978	NB_ESSAI=3
	979	# time delay between tentative
	980	DELAI=2
[16]	981
[811]	982	i=0
	983	while [ $i -lt $NB_ESSAI ] ; do
[978]	984	\cp -urL $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	985	status=$?
	986	if [ ${status} -gt 0 ]; then
	987	IGCM_debug_Print 2 "IGCM_sys_Get_Master : cp failed error code ${status} ${i}/${NB_ESSAI}"
	988	IGCM_debug_Print 2 "IGCM_sys_Get_Master : sleep ${DELAI} seconds and try again."
	989	sleep $DELAI
	990	else
	991	break
	992	fi
	993	(( i = i + 1 ))
	994	done
[16]	995
[717]	996	if [ ${status} -gt 0 ] ; then
[657]	997	echo "IGCM_sys_Get_Master : error."
[978]	998	cat /tmp/out_command_${LOGIN}.$$
[657]	999	IGCM_debug_Exit "IGCM_sys_Get_Master"
[16]	1000	else
[978]	1001	\rm /tmp/out_command_${LOGIN}.$$
[16]	1002	fi
[657]	1003	fi
	1004	IGCM_debug_PopStack "IGCM_sys_Get_Master"
[16]	1005	}
	1006
[811]	1007	#====================================================
	1008	#- Call IGCM_sys_Mirror_libIGCM now !
	1009	if ( $MirrorlibIGCM ) ; then
	1010	IGCM_sys_Mirror_libIGCM
	1011	fi
	1012
[16]	1013	#D-#==================================================
	1014	#D-function IGCM_sys_Put_Rest
[717]	1015	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
[282]	1016	#D- File and target directory must exist.
[16]	1017	#D-* Examples:
	1018	#D-
	1019	function IGCM_sys_Put_Rest {
[657]	1020	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
	1021	if ( $DEBUG_sys ) ; then
	1022	echo "IGCM_sys_Put_Rest :" $@
	1023	fi
	1024	if [ $DRYRUN = 0 ]; then
	1025	if [ ! -f ${1} ] ; then
	1026	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
	1027	IGCM_debug_Exit "IGCM_sys_Put_Rest"
[16]	1028	fi
[717]	1029
	1030	typeset status
	1031	#
[657]	1032	if [ X${JobType} = XRUN ] ; then
	1033	IGCM_sys_Chmod 444 ${1}
	1034	fi
[717]	1035	#
[811]	1036	# Only if we use rsync
	1037	#IGCM_sys_MkdirArchive $( dirname $2 )
	1038	#
	1039	#USUAL WAY
[978]	1040	\cp $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	1041	status=$?
[16]	1042
[717]	1043	# #RSYNC WITH NETWORK SSH CALL
[978]	1044	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > /tmp/out_command_${LOGIN}.$$ 2>&1
	1045	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> /tmp/out_command_${LOGIN}.$$ 2>&1
[16]	1046
[717]	1047	# #RSYNC WITH NFS USE
[978]	1048	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > /tmp/out_command_${LOGIN}.$$ 2>&1
	1049	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> /tmp/out_command_${LOGIN}.$$ 2>&1
[16]	1050
[717]	1051	# status=$?
	1052	# IGCM_sys_Rsync_out $status
	1053
[978]	1054	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command_${LOGIN}.$$
[717]	1055	# (( status=status+$? ))
	1056
	1057	if [ ${status} -gt 0 ] ; then
	1058	echo "IGCM_sys_Put_Rest : cp failed error code ${status}"
[978]	1059	cat /tmp/out_command_${LOGIN}.$$
[657]	1060	IGCM_debug_Exit "IGCM_sys_Put_Rest"
	1061	else
[978]	1062	\rm /tmp/out_command_${LOGIN}.$$
[657]	1063	fi
	1064	fi
	1065	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
	1066	}
[16]	1067
[657]	1068	#D-#==================================================
	1069	#D-function IGCM_sys_PutBuffer_Rest
	1070	#D-* Purpose: Put computied restarts on ${SCRATCHDIR}.
	1071	#D- File and target directory must exist.
	1072	#D-* Examples:
	1073	#D-
	1074	function IGCM_sys_PutBuffer_Rest {
	1075	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Rest" $@
	1076	if ( $DEBUG_sys ) ; then
	1077	echo "IGCM_sys_PutBuffer_Rest :" $@
	1078	fi
	1079	if [ $DRYRUN = 0 ]; then
	1080	if [ ! -f ${1} ] ; then
	1081	echo "ERROR : IGCM_sys_PutBuffer_Rest ${1} DOES NOT EXIST ."
	1082	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
	1083	fi
[16]	1084
[717]	1085	typeset status
[657]	1086	#
	1087	if [ X${JobType} = XRUN ] ; then
	1088	IGCM_sys_Chmod 444 ${1}
	1089	fi
	1090
	1091	#
	1092	# USUAL WAY
[978]	1093	\cp $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	1094	status=$?
[657]	1095
[717]	1096	if [ ${status} -gt 0 ] ; then
	1097	echo "IGCM_sys_PutBuffer_Rest : error code ${status}"
[811]	1098	[ -f ${2} ] && ls -l ${2}
	1099	[ -f ${2}/${1} ] && ls -l ${2}/${1}
[978]	1100	cat /tmp/out_command_${LOGIN}.$$
[657]	1101	IGCM_debug_Exit "IGCM_sys_PutBuffer_Rest"
[16]	1102	else
[978]	1103	\rm /tmp/out_command_${LOGIN}.$$
[16]	1104	fi
[657]	1105	fi
	1106	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Rest"
[16]	1107	}
	1108
	1109	#D-#==================================================
[801]	1110	#D-function IGCM_sys_PrepareTaredRestart
	1111	#D-* Purpose: Prepare tared restart to be access by computing job. Identity here.
	1112	#D-* Examples:
	1113	#D-
	1114	function IGCM_sys_PrepareTaredRestart {
	1115	IGCM_debug_PushStack "IGCM_sys_PrepareTaredRestart" $@
[918]	1116
[801]	1117	IGCM_debug_PopStack "IGCM_sys_PrepareTaredRestart"
	1118	}
	1119
	1120	#D-#==================================================
[16]	1121	#D-function IGCM_sys_Put_Out
[717]	1122	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
[16]	1123	#D-* Examples:
	1124	#D-
	1125	function IGCM_sys_Put_Out {
[657]	1126	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
	1127	if ( $DEBUG_sys ) ; then
	1128	echo "IGCM_sys_Put_Out :" $@
	1129	fi
[811]	1130
	1131	typeset status
	1132
[657]	1133	if [ $DRYRUN = 0 ]; then
[811]	1134	if [ ! -f ${1} ] ; then
[657]	1135	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
	1136	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	1137	return 1
[16]	1138	fi
[719]	1139	#
[811]	1140	IGCM_sys_MkdirArchive $( dirname $2 )
	1141	#
[657]	1142	if [ X${JobType} = XRUN ] ; then
	1143	if [ X${3} = X ] ; then
	1144	IGCM_sys_Chmod 444 ${1}
	1145	fi
	1146	fi
[717]	1147	#
[16]	1148
[978]	1149	echo ${RSYNC} ${RSYNC_opt} $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
	1150	${RSYNC} ${RSYNC_opt} $1 $2 >> /tmp/out_command_${LOGIN}.$$ 2>&1
[717]	1151	status=$?
	1152	IGCM_sys_Rsync_out $status
[16]	1153
[978]	1154	${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk /tmp/out_command_${LOGIN}.$$
[717]	1155	(( status=status+$? ))
[16]	1156
[717]	1157	if [ ${status} -gt 0 ] ; then
[811]	1158	IGCM_debug_Print 2 "IGCM_sys_Put_Out : rsync failed error code ${status}"
[978]	1159	cat /tmp/out_command_${LOGIN}.$$
[657]	1160	IGCM_debug_Exit "IGCM_sys_Put_Out"
[16]	1161	else
[978]	1162	\rm /tmp/out_command_${LOGIN}.$$
[16]	1163	fi
[657]	1164	fi
	1165	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	1166	return 0
[16]	1167	}
	1168
	1169	#D-#==================================================
[657]	1170	#D-function IGCM_sys_PutBuffer_Out
	1171	#D-* Purpose: Copy a file on ${SCRATCHDIR} after having chmod it in readonly
	1172	#D-* Examples:
	1173	#D-
	1174	function IGCM_sys_PutBuffer_Out {
	1175	IGCM_debug_PushStack "IGCM_sys_PutBuffer_Out" $@
	1176	if ( $DEBUG_sys ) ; then
	1177	echo "IGCM_sys_PutBuffer_Out :" $@
	1178	fi
[719]	1179
[811]	1180	typeset NB_ESSAI DELAI status i exist skip
[719]	1181
[811]	1182	# number of tentative
	1183	NB_ESSAI=3
	1184	# time delay between tentative
	1185	DELAI=2
	1186
[657]	1187	if [ $DRYRUN = 0 ]; then
	1188	if [ ! -f ${1} ] ; then
	1189	echo "WARNING : IGCM_sys_PutBuffer_Out ${1} DOES NOT EXIST ."
	1190	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
	1191	return 1
	1192	fi
	1193	#
	1194	IGCM_sys_Mkdir $( dirname $2 )
	1195	#
	1196
[811]	1197	exist=false
	1198	skip=false
	1199	if [ -f $2 ] ; then
	1200	IGCM_debug_Print 1 "$2 already exist"
	1201	exist=true
	1202	if [ "X$( diff $1 $2 )" = X ] ; then
	1203	IGCM_debug_Print 2 "$1 and $2 are the same file, we skip the copy"
	1204	status=0
	1205	skip=true
	1206	else
	1207	IGCM_debug_Print 2 "$1 and $2 are not the same file, we force the copy"
	1208	skip=false
[657]	1209	fi
	1210	fi
	1211	#
[811]	1212	if ( [ X${exist} = Xtrue ] && [ X${skip} = Xfalse ] ) ; then
	1213	IGCM_sys_Chmod u+w $2
	1214	fi
[657]	1215
[811]	1216	if [ X${skip} = Xfalse ] ; then
	1217	i=0
	1218	while [ $i -lt $NB_ESSAI ] ; do
	1219	# USUAL WAY
[978]	1220	\cp $1 $2 > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	1221	status=$?
	1222	if [ ${status} -gt 0 ]; then
	1223	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : cp failed error code ${status} ${i}/${NB_ESSAI}"
	1224	IGCM_debug_Print 2 "IGCM_sys_PutBuffer_Out : sleep ${DELAI} seconds and try again."
	1225	[ -f ${2} ] && ls -l ${2}
	1226	[ -f ${2}/${1} ] && ls -l ${2}/${1}
	1227	sleep $DELAI
	1228	else
	1229	break
	1230	fi
	1231	(( i = i + 1 ))
	1232	done
	1233	fi
	1234
[717]	1235	if [ ${status} -gt 0 ] ; then
[657]	1236	echo "IGCM_sys_PutBuffer_Out : error."
[811]	1237	[ -f ${2} ] && ls -l ${2}
	1238	[ -f ${2}/${1} ] && ls -l ${2}/${1}
[978]	1239	cat /tmp/out_command_${LOGIN}.$$
[657]	1240	IGCM_debug_Exit "IGCM_sys_PutBuffer_Out"
	1241	else
[811]	1242
	1243	if [ X${JobType} = XRUN ] ; then
	1244	if [ X${3} = X ] ; then
	1245	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
	1246	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
	1247	fi
	1248	fi
	1249
[978]	1250	\rm /tmp/out_command_${LOGIN}.$$
[657]	1251	fi
	1252	fi
	1253	IGCM_debug_PopStack "IGCM_sys_PutBuffer_Out"
	1254	return 0
	1255	}
	1256
	1257	#D-#==================================================
[16]	1258	#D-function IGCM_sys_Get
	1259	#D-* Purpose: Get a file from ${ARCHIVE}
	1260	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
	1261	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
	1262	function IGCM_sys_Get {
[657]	1263	IGCM_debug_PushStack "IGCM_sys_Get" $@
[16]	1264
[811]	1265	typeset DEST status dm_liste
[16]	1266
[657]	1267	if ( $DEBUG_sys ) ; then
	1268	echo "IGCM_sys_Get :" $@
	1269	fi
	1270	if [ $DRYRUN -le 2 ]; then
	1271	if [ X${1} = X'/l' ] ; then
	1272	eval set +A dm_liste \${${2}}
	1273	else
	1274	dm_liste=${1}
[16]	1275	fi
[657]	1276	eval DEST=\${${#}}
[16]	1277
[717]	1278	# test if the (first) file is present in the old computation :
[657]	1279	IGCM_sys_TestFileArchive ${dm_liste[0]}
[717]	1280	status=$?
	1281	if [ ${status} -gt 0 ] ; then
[657]	1282	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
	1283	IGCM_debug_Exit "IGCM_sys_Get"
[811]	1284	return
[657]	1285	fi
[16]	1286
[811]	1287	#USUAL WAY
[978]	1288	\cp ${dm_liste[*]} ${DEST} > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	1289	status=$?
[16]	1290
[811]	1291	if [ ${status} -gt 0 ] ; then
	1292	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status}"
[978]	1293	cat /tmp/out_command_${LOGIN}.$$
[657]	1294	IGCM_debug_Exit "IGCM_sys_Get"
[16]	1295	else
[978]	1296	\rm /tmp/out_command_${LOGIN}.$$
[16]	1297	fi
[657]	1298	fi
	1299	IGCM_debug_PopStack "IGCM_sys_Get"
[16]	1300	}
	1301
	1302	#D-#==================================================
[657]	1303	#D-function IGCM_sys_GetBuffer
	1304	#D-* Purpose: Get a file from ${SCRATCHDIR}
	1305	#D-* Examples: IGCM_sys_GetBuffer myfile /destpath/myfile_with_PREFIX
	1306	#D- IGCM_sys_GetBuffer /l Array_contain_myfiles /destpath/
	1307	function IGCM_sys_GetBuffer {
	1308	IGCM_debug_PushStack "IGCM_sys_GetBuffer" $@
	1309
[811]	1310	typeset DEST buf_liste target file_work
	1311	typeset NB_ESSAI DELAI status i
[657]	1312
	1313	if ( $DEBUG_sys ) ; then
	1314	echo "IGCM_sys_GetBuffer :" $@
	1315	fi
[811]	1316
	1317	# number of tentative
	1318	NB_ESSAI=3
	1319	# time delay between tentative
	1320	DELAI=2
	1321
[657]	1322	if [ $DRYRUN -le 2 ]; then
	1323	if [ X${1} = X'/l' ] ; then
	1324	eval set +A buf_liste \${${2}}
	1325	else
	1326	eval set +A buf_liste ${1}
	1327	fi
	1328	eval DEST=\${${#}}
	1329
	1330	#USUAL WAY
	1331	if [ X${1} = X'/l' ] ; then
	1332	for target in ${buf_liste[*]} ; do
	1333	local_file=$( basename ${target} )
[811]	1334	i=0
	1335	while [ $i -lt $NB_ESSAI ] ; do
[978]	1336	\cp ${target} ${DEST}/${local_file} >> /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	1337	status=$?
	1338	if [ ${status} -gt 0 ]; then
	1339	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
	1340	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
	1341	sleep $DELAI
	1342	else
	1343	break
	1344	fi
	1345	(( i = i + 1 ))
	1346	done
	1347	if [ ${status} -gt 0 ] ; then
	1348	echo "IGCM_sys_Get : error"
[978]	1349	cat /tmp/out_command_${LOGIN}.$$
	1350	\rm /tmp/out_command_${LOGIN}.$$
[811]	1351	IGCM_debug_Exit "IGCM_sys_GetBuffer"
	1352	else
[978]	1353	\rm /tmp/out_command_${LOGIN}.$$
[811]	1354	fi
[657]	1355	done
	1356	else
[811]	1357	i=0
	1358	while [ $i -lt $NB_ESSAI ] ; do
[978]	1359	\cp ${buf_liste} ${DEST} >> /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	1360	status=$?
	1361	if [ ${status} -gt 0 ]; then
	1362	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : cp failed error code ${status} ${i}/${NB_ESSAI}"
	1363	IGCM_debug_Print 2 "IGCM_sys_GetBuffer : sleep ${DELAI} seconds and try again."
	1364	sleep $DELAI
	1365	else
	1366	break
	1367	fi
	1368	(( i = i + 1 ))
	1369	done
	1370	if [ ${status} -gt 0 ] ; then
	1371	echo "IGCM_sys_Get : error"
[978]	1372	cat /tmp/out_command_${LOGIN}.$$
	1373	\rm /tmp/out_command_${LOGIN}.$$
[811]	1374	IGCM_debug_Exit "IGCM_sys_GetBuffer"
	1375	else
[978]	1376	\rm /tmp/out_command_${LOGIN}.$$
[811]	1377	fi
[657]	1378	fi
	1379	fi
	1380	IGCM_debug_PopStack "IGCM_sys_GetBuffer"
	1381	}
	1382
	1383	#D-#==================================================
	1384	#D-function IGCM_sys_GetDate_FichWork
	1385	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem WORK
	1386	#D-* Examples:
	1387	#D-
	1388	function IGCM_sys_GetDate_FichWork {
	1389	IGCM_debug_PushStack "IGCM_sys_FichWork" $@
	1390	if ( $DEBUG_sys ) ; then
	1391	echo "IGCM_sys_GetDate_FichWork :" $@
	1392	fi
	1393	typeset dateF
	1394	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
	1395	eval ${2}=${dateF[5]}
	1396
[800]	1397	# donne la date filesys d'un fichier sur la machine work
[657]	1398	IGCM_debug_PopStack "IGCM_sys_FichWork"
	1399	}
	1400
	1401	#D-#==================================================
	1402	#D-function IGCM_sys_GetDate_FichArchive
	1403	#D-* Purpose: donne la date filesys d'un fichier sur le filesystem ARCHIVE
	1404	#D-* Examples:
	1405	#D-
	1406	function IGCM_sys_GetDate_FichArchive {
	1407	IGCM_debug_PushStack "IGCM_sys_FichArchive" $@
	1408	if ( $DEBUG_sys ) ; then
	1409	echo "IGCM_sys_GetDate_FichArchive :" $@
	1410	fi
	1411	typeset dateF
	1412	set +A dateF -- $( ls -l --full-time --time-style='+%Y%m%d%H%M%S' ${1} )
	1413	eval ${2}=${dateF[5]}
	1414
	1415	IGCM_debug_PopStack "IGCM_sys_FichArchive"
	1416	}
	1417
	1418	#D-#==================================================
[1131]	1419	#D-function IGCM_sys_GetDate_Monitoring
	1420	#D-* Purpose: get the last year for which the monitoring has been computed
	1421	#D-* Examples:
	1422	#D-
	1423	function IGCM_sys_GetDate_Monitoring {
	1424	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
	1425	if ( $DEBUG_sys ) ; then
	1426	echo "IGCM_sys_GetDate_Monitoring :" $@
	1427	fi
	1428
	1429	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
	1430
	1431	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
	1432	}
	1433
	1434	#D-#==================================================
[343]	1435	#D-function IGCM_sys_Dods_Rm
[717]	1436	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
[343]	1437	#D-* Examples:
	1438	#D-
	1439	function IGCM_sys_Dods_Rm {
[657]	1440	if ( $DEBUG_sys ) ; then
	1441	echo "IGCM_sys_Dods_Rm :" $@
	1442	fi
	1443	return 0
[343]	1444	}
	1445
	1446	#D-#==================================================
	1447	#D-function IGCM_sys_Dods_Cp
[717]	1448	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
[343]	1449	#D-* Examples:
	1450	#D-
	1451	function IGCM_sys_Dods_Cp {
[657]	1452	if ( $DEBUG_sys ) ; then
	1453	echo "IGCM_sys_Dods_Cp :" $@
	1454	fi
	1455	return 0
[343]	1456	}
	1457
	1458	#D-#==================================================
[16]	1459	#D-function IGCM_sys_Put_Dods
[811]	1460	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
[16]	1461	#D-* Examples:
	1462	#D-
	1463	function IGCM_sys_Put_Dods {
[657]	1464	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
	1465	if ( $DEBUG_sys ) ; then
	1466	echo "IGCM_sys_Put_Dods :" $@
	1467	fi
	1468	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
[16]	1469	}
	1470
[657]	1471	##############################################################
	1472	# REBUILD OPERATOR
[536]	1473
[811]	1474	#D-#==================================================
	1475	#D-function IGCM_sys_rebuild
	1476	#D-* Purpose: rebuild parallel files
	1477	#D-* Examples:
	1478	#D-
[657]	1479	function IGCM_sys_rebuild {
[1057]	1480	IGCM_debug_PushStack "IGCM_sys_rebuild" $@
[657]	1481	if ( $DEBUG_sys ) ; then
	1482	echo "IGCM_sys_rebuild :" $@
	1483	fi
[717]	1484
[811]	1485	typeset NB_ESSAI DELAI status i firstArg
	1486	# number of tentative
	1487	NB_ESSAI=3
	1488	# time delay between tentative
	1489	DELAI=2
[717]	1490
[811]	1491	i=0
	1492	while [ $i -lt $NB_ESSAI ] ; do
[978]	1493	/home/users/igcmg/rebuild/bin/rebuild -f -o $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[811]	1494	status=$?
	1495	if [ ${status} -gt 0 ] ; then
	1496	IGCM_debug_Print 2 "IGCM_sys_rebuild : error code ${status}"
[978]	1497	cat /tmp/out_command_${LOGIN}.$$
	1498	\rm /tmp/out_command_${LOGIN}.$$
[811]	1499	IGCM_debug_Print 2 "IGCM_sys_rebuild : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
	1500	firstArg=${1}
	1501	\rm ${firstArg}
	1502	sleep $DELAI
	1503	else
[978]	1504	\rm /tmp/out_command_${LOGIN}.$$
[811]	1505	break
	1506	fi
	1507	(( i = i + 1 ))
	1508	done
	1509
[717]	1510	if [ ${status} -gt 0 ] ; then
[811]	1511	echo "IGCM_sys_rebuild : rebuild error code is ${status}"
[657]	1512	IGCM_debug_Exit "rebuild"
	1513	fi
	1514
	1515	IGCM_debug_PopStack "IGCM_sys_rebuild"
[16]	1516	}
	1517
[811]	1518	#D-#==================================================
	1519	#D-function IGCM_sys_rebuild_station
[922]	1520	#D-* Purpose: rebuild parallel files describing station
[811]	1521	#D-* Examples:
	1522	#D-
[697]	1523	function IGCM_sys_rebuild_station {
[1057]	1524	IGCM_debug_PushStack "IGCM_sys_rebuild_station" $@
[697]	1525	typeset i list_opt file_in file_out prefix_invert list_invert
	1526	if ( $DEBUG_sys ) ; then
	1527	echo "IGCM_sys_rebuild_station :" $@
	1528	fi
	1529	list_opt=$@
	1530
[800]	1531	# Invert Axis : t,x -> x,t
[697]	1532	# t,pres,x -> x,t,pres
	1533	# So that we can concatenate along x
	1534	i=0
	1535	for file_in in ${list_opt} ; do
	1536	(( i = i + 1))
	1537	[ ${i} = 1 ] && file_out=${file_in} && continue
[1046]	1538	# detect time counter and do the job only if present
[1147]	1539	var_unlim=$(ncdump -h ${file_in} \| grep UNLIMITED \| cut -d ' ' -f 1 \| sed -e 's/^[ \t]//' -e 's/[ \t]$//')
[1046]	1540	if [ X${var_unlim} = Xtime_counter ] ; then
	1541	prefix_invert=$( basename ${file_in} .nc )
	1542	IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs ${file_in} ${prefix_invert}_xt.nc
	1543	list_invert[${#list_invert[*]}]=${prefix_invert}_xt.nc
	1544	fi
[697]	1545	done
	1546
	1547	# Concatenate
	1548	IGCM_sys_ncrcat ${list_invert[*]} histstn_xt.nc
	1549
	1550	# Re-ivert file
	1551	IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x histstn_xt.nc ${file_out}
	1552
[922]	1553	# Station re-ordering is too expansive to be run within libIGCM
[697]	1554	# This is due to (ncpdq - nrcat - ncpdq) I/O sequence.
	1555	# This re-ordering must be done "in memory" by the cmorization process
	1556	# Anyway this is the best sequence using (ncpdq - nrcat - ncpdq)
	1557	# BEGIN reordering
	1558
	1559	# Only LMDZ text output contains the exact ordering of the station.
	1560	# We isolate this in the code below:
	1561	# 0 38 -157.5000000000000 70.98591549295774
	1562	# 0 54 27.49999999999999 67.18309859154928
	1563	# 0 56 -62.50000000000001 82.39436619718309
	1564	# 0 79 12.49999999999999 78.59154929577466
	1565	# 0 116 -165.0000000000000 76.05633802816901
	1566	# 0 117 130.0000000000000 70.98591549295774
	1567	# 0 118 110.0000000000000 87.46478873239437
	1568	# 1 40 4.999999999999995 51.97183098591550
	1569	# typeset iStation iProc list_opt file_in file_out prefix_invert
	1570	# typeset -Z4 j4
	1571	# typeset -Z3 j3
	1572
	1573	# unset list_opt
	1574	# set +A list_opt $@
	1575
	1576	# Filename after rebuild
	1577	# file_out=${list_opt[0]}
	1578	# Prefix of output files
	1579	# prefix_invert=$( basename ${file_out} .nc )
	1580	# Number of procs
	1581	# num_proc=$( grep -i mpi_size ${PREFIX}_${Exe_Output} \| wc -l )
	1582
	1583	# iProc=0
	1584	# while [ ${iProc} -lt ${num_proc} ] ; do
	1585	# Array containing Station as a number
	1586	# unset proc_stn
[912]	1587	# set +A proc_stn $( grep "iophy_mpi rank ip lon lat $iProc" ${PREFIX}_${Exe_Output} \| sed -e "s/iophy_mpi rank ip lon lat //g" \| gawk ' {print $2}' )
[697]	1588	# Number of stations produced by processor proc
	1589	# stationLast=${#proc_stn[*]}
	1590	# Proc number on 4 digits
	1591	# j4=${iProc}
	1592	# Init
	1593	# iStation=0
	1594	# while [ ${iStation} -lt ${stationLast} ] ; do
	1595	# Station number on 3 digits
	1596	# j3=${proc_stn[${iStation}]}
	1597	# Extract station
[800]	1598	# Invert Axis : t,x -> x,t
[697]	1599	# t,pres,x -> x,t,pres
	1600	# So that we can concatenate along x
	1601	# IGCM_sys_ncpdq -a x,time_counter -a x,time_counter,presnivs -d x,$iStation,$iStation ${prefix_invert}_${j4}.nc ${prefix_invert}_stn_${j3}.nc
	1602	# (( iStation = iStation + 1 ))
	1603	# done
	1604	# (( iProc = iProc + 1 ))
	1605	# done
	1606
	1607	# Concatenate all station along x
	1608	# IGCM_sys_ncrcat ${prefix_invert}_stn_???.nc ${prefix_invert}_xt.nc
	1609
	1610	# Re-invert file
	1611	# IGCM_sys_ncpdq -a time_counter,x -a time_counter,presnivs,x ${prefix_invert}_xt.nc ${file_out}
	1612
	1613	# END reordering
	1614
	1615	IGCM_debug_PopStack "IGCM_sys_rebuild_station"
	1616	}
	1617
[657]	1618	############################################################
	1619	# Activate Running Environnment Variables
[536]	1620
[811]	1621	#D-#==================================================
	1622	#D-function IGCM_sys_desactiv_variables
	1623	#D-* Purpose: set environement variables prior to execution
	1624	#D-* Examples:
	1625	#D-
[657]	1626	function IGCM_sys_activ_variables {
	1627	IGCM_debug_PushStack "IGCM_sys_activ_variables"
	1628	if ( $DEBUG_sys ) ; then
	1629	echo "IGCM_sys_activ_variables"
	1630	fi
[811]	1631
	1632	# --------------------------------------------------------------------
	1633	#D- MPI specifications
	1634	# --------------------------------------------------------------------
	1635
	1636	# --------------------------------------------------------------------
	1637	#D- Other specifications
	1638	# --------------------------------------------------------------------
	1639
[657]	1640	IGCM_debug_PopStack "IGCM_sys_activ_variables"
[16]	1641	}
	1642
[657]	1643	############################################################
	1644	# Desactivate Running Environnment Variables
[16]	1645
[811]	1646	#D-#==================================================
	1647	#D-function IGCM_sys_desactiv_variables
	1648	#D-* Purpose: unset environement variables after execution
	1649	#D-* Examples:
	1650	#D-
[657]	1651	function IGCM_sys_desactiv_variables {
	1652	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
	1653	if ( $DEBUG_sys ) ; then
	1654	echo "IGCM_sys_desactiv_variables"
	1655	fi
[811]	1656	# --------------------------------------------------------------------
	1657	#D- MPI specifications
	1658	# --------------------------------------------------------------------
	1659
	1660	# --------------------------------------------------------------------
	1661	#D- Other specifications
	1662	# --------------------------------------------------------------------
	1663
[657]	1664	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
	1665	}
	1666
	1667	############################################################
[800]	1668	# Build MPI/OMP scripts run file (dummy function)
[657]	1669
[811]	1670	#D-#==================================================
	1671	#D-function IGCM_sys_build_run_file
	1672	#D-* Purpose: build run file (deprecated)
	1673	#D-* Examples:
	1674	#D-
[657]	1675	function IGCM_sys_build_run_file {
	1676
[664]	1677	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
[657]	1678
	1679	}
	1680
	1681	############################################################
	1682	# Build MPI/OMP scripts
[811]	1683
	1684	#D-#==================================================
	1685	#D-function IGCM_sys_build_execution_scripts
	1686	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
	1687	#D-* Examples:
	1688	#D-
[657]	1689	function IGCM_sys_build_execution_scripts
	1690	{
	1691	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
	1692	if ( $DEBUG_sys ) ; then
	1693	echo "IGCM_sys_build_execution_scripts " $@
	1694	fi
	1695	typeset nodes listnodes init_node start_num init_exec comp ExeNameIn ExeNameOut
	1696	typeset node_num_current node_current comp_proc_mpi_loc comp_proc_omp_loc
	1697	typeset num_corempi nombre_restant_node nombre_restant_comp
	1698
	1699	if [ ! -f ${SUBMIT_DIR}/Job_${config_UserChoices_JobName} ] ; then
[811]	1700	IGCM_debug_Exit "IGCM_sys_obelix build_execution_scripts : Job_${config_UserChoices_JobName} don't exist in SUBMIT_DIR : ${SUBMIT_DIR} "
[657]	1701	fi
	1702
	1703	if ( ${OK_PARA_MPMD} ) ; then
	1704
	1705	if [ -f run_file ] ; then
	1706	IGCM_sys_Rm -f run_file
[16]	1707	fi
[657]	1708	touch run_file
	1709
	1710	if ( ${OK_PARA_OMP} ) ; then
	1711
	1712	# Hosts treatment
	1713
	1714	${HOST_MPIRUN_COMMAND} hostname \| sort \| uniq > hosts.tmp
	1715
	1716	i=0
	1717	rm -f hosts
	1718	IGCM_debug_Print 1 "sys Obelix, Hosts avaible :"
	1719	for nodes in `cat hosts.tmp` ; do
	1720	host[$i]=$nodes
	1721	echo "${host[$i]} slots=1 max_slots=1" >> hosts
	1722	IGCM_debug_Print 1 ${host[$i]}
	1723	i=$((i+1))
	1724	done
	1725	rm -f hosts.tmp
	1726
	1727	listnodes=${host[*]}
	1728
	1729	EXECUTION="${HOST_MPIRUN_COMMAND} -hostfile hosts"
	1730
	1731	# Initialisation
	1732
	1733	init_node=y
	1734	node_num_current=0
	1735	start_num=0
	1736	init_exec=n
	1737
	1738	# Test : if oasis is there, we put it at the first position
	1739
	1740	for comp in ${config_ListOfComponents[*]} ; do
	1741
	1742	if [ "X${comp}" = "XCPL" ] ; then
	1743
	1744	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1745	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1746
	1747	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1748	echo "" >> script_${ExeNameOut}.ksh
	1749	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
	1750	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
	1751	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
	1752	echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
	1753	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1754
	1755	init_node=n
	1756
	1757	(( nombre_restant_node = NUM_COREPERNODE - 1 ))
	1758	node_num_current=0
	1759	node_current=${host[${node_num_current}]}
	1760
	1761	EXECUTION="${EXECUTION} -H ${node_current} -np 1 ./script_${ExeNameOut}.ksh"
	1762
	1763	init_exec=y
	1764	start_num=1
	1765
	1766	fi
	1767
	1768	done
	1769
	1770	# Then loop on the components (except for oasis)
	1771
	1772	for comp in ${config_ListOfComponents[*]} ; do
	1773
	1774	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1775	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1776
	1777	# Only if we really have an executable for the component :
	1778	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	1779
	1780	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1781	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1782
	1783	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1784	# echo "set -vx" >> script_${ExeNameOut}.ksh
	1785	echo "" >> script_${ExeNameOut}.ksh
	1786	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
	1787	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
	1788	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
	1789	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
	1790	echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
	1791	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
	1792	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1793
	1794	node_num=0
	1795
	1796	# We define the number of MPI process to be assigned for the component
	1797
	1798	nombre_restant_comp=${comp_proc_mpi_loc}
	1799
	1800	# Loop on the allocated nodes
	1801
	1802	for node in ${listnodes} ; do
	1803
	1804	# We go to the current node
	1805	if [ ${node_num} = ${node_num_current} ] ; then
	1806
	1807	node_current=${host[${node_num_current}]}
	1808
	1809	# If first time on the node : initialisation
	1810
	1811	if [ ${init_node} = y ] ; then
	1812	nombre_restant_node=${NUM_COREPERNODE}
	1813	fi
	1814
	1815	# Test on the number of OMP threads
	1816
	1817	if [ ${comp_proc_omp_loc} -gt ${nombre_restant_node} ] ; then
	1818	(( node_num = node_num + 1 ))
	1819	node_num_current=${node_num}
	1820	init_node=y
	1821	continue
	1822	fi
	1823
	1824	# Number of MPI process to assign
	1825
	1826	(( num_corempi = nombre_restant_node / comp_proc_omp_loc ))
	1827
	1828	if [ ${num_corempi} -gt ${nombre_restant_comp} ] ; then
	1829	num_corempi=${nombre_restant_comp}
	1830	fi
	1831
	1832	(( nombre_restant_node = nombre_restant_node - num_corempi * comp_proc_omp_loc ))
	1833	(( nombre_restant_comp = nombre_restant_comp - num_corempi ))
	1834
	1835	if [ ${init_exec} = y ] ; then
	1836	EXECUTION="${EXECUTION} : -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
	1837	else
	1838	EXECUTION="${EXECUTION} -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
	1839	init_exec=y
	1840	fi
	1841
	1842	(( start_num = num_corempi + start_num ))
	1843
	1844	else
	1845
	1846	(( node_num = node_num + 1 ))
	1847	continue
	1848	fi
	1849
	1850	# Test on the number of core/process remaining on the node/component
	1851
	1852	if [ ${nombre_restant_node} = 0 ] ; then
	1853	(( node_num = node_num + 1 ))
	1854	node_num_current=${node_num}
	1855	init_node=y
	1856
	1857	if [ ${nombre_restant_comp} = 0 ] ; then
	1858	break 1
	1859	fi
	1860	else
	1861
	1862	node_num_current=${node_num}
	1863	init_node=n
	1864
	1865	if [ ${nombre_restant_comp} = 0 ] ; then
	1866	break 1
	1867	fi
	1868	fi
	1869	done
	1870	fi
	1871	done
	1872
	1873	else
	1874
[664]	1875	# Then first loop on the components for the coupler ie oasis
[657]	1876
[664]	1877	## the coupler ie oasis must be the first one
	1878	for comp in ${config_ListOfComponents[*]} ; do
[657]	1879
[664]	1880	eval ExeNameOut=\${config_Executable_${comp}[1]}
[657]	1881
	1882	# for CPL component only
[664]	1883	if [ "X${comp}" = "XCPL" ] ; then
	1884	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1885	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut} " >> run_file
	1886	fi
	1887	done
[657]	1888
[811]	1889	# Then second loop on the components
[657]	1890
[664]	1891	for comp in ${config_ListOfComponents[*]} ; do
[657]	1892
[664]	1893	eval ExeNameOut=\${config_Executable_${comp}[1]}
[657]	1894
[664]	1895	# Only if we really have an executable for the component and not the coupler ie oasis:
	1896	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	1897	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1898	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
	1899	fi
	1900	done
	1901	IGCM_sys_Chmod u+x run_file
[657]	1902
[664]	1903	EXECUTION="${HOST_MPIRUN_COMMAND} --app ./run_file"
[657]	1904
[16]	1905	fi
	1906
[657]	1907	else # Only one executable. launch it.
	1908
[664]	1909	for comp in ${config_ListOfComponents[*]} ; do
[657]	1910
[664]	1911	# Only if we really have an executable for the component :
	1912	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1913	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
[657]	1914
[664]	1915	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1916	echo "" >> script_${ExeNameOut}.ksh
	1917	if ( ${OK_PARA_OMP} ) ; then
	1918	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1919	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
	1920	fi
	1921	if ( ${OK_PARA_MPI} ) ; then
	1922	# Default : mpirun used if nb_proc gt 1
	1923	# pour sortie out/err par process
	1924	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${OMPI_COMM_WORLD_RANK} 2>out_${ExeNameOut}.err.\${OMPI_COMM_WORLD_RANK}" >> script_${ExeNameOut}.ksh
	1925	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1926	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1927	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
	1928	else
	1929	# Default : mpirun is NOT used if nb_proc eq 1
	1930	# pour sortie out/err par process
	1931	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
	1932	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1933	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1934	EXECUTION="time ./script_${ExeNameOut}.ksh"
	1935	fi
[1002]	1936
	1937	IGCM_debug_Print 1 "sys Obelix : script_${ExeNameOut}.ksh contains"
	1938	cat script_${ExeNameOut}.ksh
	1939
[664]	1940	fi
	1941	done
[657]	1942
	1943	fi
	1944
[811]	1945	IGCM_debug_Print 1 "sys Obelix : execution command is"
	1946	IGCM_debug_Print 1 "$EXECUTION"
[657]	1947
	1948	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
[16]	1949	}
	1950
[933]	1951	#D-#==================================================
	1952	#D-function IGCM_sys_check_path
	1953	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
	1954	#D-* do not point to an important use directory. Stop immediately in that case.
	1955	#D-* Examples:
	1956	#D-
	1957	function IGCM_sys_check_path {
	1958	IGCM_debug_PushStack "IGCM_sys_check_path"
	1959	if ( $DEBUG_sys ) ; then
	1960	echo "IGCM_sys_check_path"
	1961	fi
[811]	1962
[933]	1963	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
	1964	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
	1965	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
[936]	1966	IGCM_debug_Exit "This will stop the job"
[933]	1967	fi
	1968	IGCM_debug_PopStack "IGCM_sys_check_path"
	1969	}
	1970
[811]	1971	#D-#==================================================
	1972	#D-function IGCM_sys_check_quota. Dummy call here
	1973	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
	1974	#D-* Examples:
	1975	#D-
[657]	1976	function IGCM_sys_check_quota {
	1977	IGCM_debug_PushStack "IGCM_sys_check_quota"
	1978	if ( $DEBUG_sys ) ; then
	1979	echo "IGCM_sys_check_quota"
	1980	fi
	1981	IGCM_debug_PopStack "IGCM_sys_check_quota"
	1982	}
	1983
[811]	1984	#D-#==================================================
[1124]	1985	#D-function IGCM_sys_GetJobID
	1986	#D-* Purpose: Check if job_name is currently
	1987	#D- running or in queue
	1988	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
	1989	#D-
	1990	function IGCM_sys_GetJobID {
	1991	IGCM_debug_PushStack "IGCM_sys_GetJobID"
	1992	if ( $DEBUG_sys ) ; then
	1993	echo "IGCM_sys_GetJobID"
	1994	fi
	1995
	1996	# With -f option, the full job name is given in the last column
	1997	ID="$( qstat -u $2 \| grep -w $1 \| gawk '-F ' '{print $10}' )"
	1998
	1999	eval ${3}=${ID}
	2000	IGCM_debug_PopStack "IGCM_sys_GetJobID"
	2001	}
	2002
	2003	#D-#==================================================
[811]	2004	#D-function IGCM_sys_CountJobInQueue
	2005	#D-* Purpose: Check if job_name is currently
	2006	#D- running or in queue
	2007	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
	2008	#D-
	2009	function IGCM_sys_CountJobInQueue {
	2010	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
	2011	if ( $DEBUG_sys ) ; then
	2012	echo "IGCM_sys_CountJobInQueue"
	2013	fi
	2014	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
	2015	}
	2016
[16]	2017	##############################################################
	2018	# NCO OPERATOR
	2019
[811]	2020	#D-#==================================================
	2021	#D-function IGCM_sys_ncap2
	2022	#D-* Purpose: encapsulate ncap2 call so as to manage error code and retry
	2023	#D-* Examples:
	2024	#D-
[377]	2025	function IGCM_sys_ncap2 {
[1057]	2026	IGCM_debug_PushStack "IGCM_sys_ncap2" $@
[657]	2027	if ( $DEBUG_sys ) ; then
	2028	echo "IGCM_sys_ncap2 :" $@
	2029	fi
[714]	2030
	2031	typeset NB_ESSAI DELAI status i
	2032	# number of tentative
	2033	NB_ESSAI=3
	2034	# time delay between tentative
	2035	DELAI=2
	2036
	2037	i=0
	2038	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2039	ncap2 -C "$@" > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2040	status=$?
	2041	if [ ${status} -gt 0 ] ; then
	2042	IGCM_debug_Print 2 "IGCM_sys_ncap2 : error code ${status}"
[978]	2043	cat /tmp/out_command_${LOGIN}.$$
	2044	\rm /tmp/out_command_${LOGIN}.$$
[714]	2045	IGCM_debug_Print 2 "IGCM_sys_ncap2 : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2046	sleep $DELAI
[714]	2047	else
[978]	2048	\rm /tmp/out_command_${LOGIN}.$$
[714]	2049	break
	2050	fi
	2051	(( i = i + 1 ))
	2052	done
	2053
	2054	if [ ${status} -gt 0 ] ; then
[811]	2055	echo "IGCM_sys_ncap2 : ncap2 error"
	2056	IGCM_debug_Exit "ncap2"
[657]	2057	fi
[16]	2058
[657]	2059	IGCM_debug_PopStack "IGCM_sys_ncap2"
[16]	2060	}
	2061
[811]	2062	#D-#==================================================
	2063	#D-function IGCM_sys_ncatted
	2064	#D-* Purpose: encapsulate ncatted call so as to manage error code and retry
	2065	#D-* Examples:
	2066	#D-
[375]	2067	function IGCM_sys_ncatted {
[1057]	2068	IGCM_debug_PushStack "IGCM_sys_ncatted" $@
[657]	2069	if ( $DEBUG_sys ) ; then
	2070	echo "IGCM_sys_ncatted :" $@
	2071	fi
[714]	2072
	2073	typeset NB_ESSAI DELAI status i
	2074	# number of tentative
	2075	NB_ESSAI=3
	2076	# time delay between tentative
	2077	DELAI=2
	2078
	2079	i=0
	2080	while [ $i -lt $NB_ESSAI ] ; do
[978]	2081	ncatted "$@" > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2082	status=$?
	2083	if [ ${status} -gt 0 ] ; then
	2084	IGCM_debug_Print 2 "IGCM_sys_ncatted : error code ${status}"
[978]	2085	cat /tmp/out_command_${LOGIN}.$$
	2086	\rm /tmp/out_command_${LOGIN}.$$
[714]	2087	IGCM_debug_Print 2 "IGCM_sys_ncatted : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2088	sleep $DELAI
[714]	2089	else
[978]	2090	\rm /tmp/out_command_${LOGIN}.$$
[714]	2091	break
	2092	fi
	2093	(( i = i + 1 ))
	2094	done
	2095
	2096	if [ ${status} -gt 0 ] ; then
[811]	2097	echo "IGCM_sys_ncatted : ncatted error"
	2098	IGCM_debug_Exit "ncatted"
[657]	2099	fi
[16]	2100
[657]	2101	IGCM_debug_PopStack "IGCM_sys_ncatted"
[375]	2102	}
[16]	2103
[811]	2104	#D-#==================================================
	2105	#D-function IGCM_sys_ncbo
	2106	#D-* Purpose: encapsulate ncbo call so as to manage error code and retry
	2107	#D-* Examples:
	2108	#D-
[16]	2109	function IGCM_sys_ncbo {
[1057]	2110	IGCM_debug_PushStack "IGCM_sys_ncbo" $@
[657]	2111	if ( $DEBUG_sys ) ; then
	2112	echo "IGCM_sys_ncbo :" $@
	2113	fi
[714]	2114
	2115	typeset NB_ESSAI DELAI status i
	2116	# number of tentative
	2117	NB_ESSAI=3
	2118	# time delay between tentative
	2119	DELAI=2
	2120
	2121	i=0
	2122	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2123	ncbo -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2124	status=$?
	2125	if [ ${status} -gt 0 ] ; then
	2126	IGCM_debug_Print 2 "IGCM_sys_ncbo : error code ${status}"
[978]	2127	cat /tmp/out_command_${LOGIN}.$$
	2128	\rm /tmp/out_command_${LOGIN}.$$
[714]	2129	IGCM_debug_Print 2 "IGCM_sys_ncbo : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2130	sleep $DELAI
[714]	2131	else
[978]	2132	\rm /tmp/out_command_${LOGIN}.$$
[714]	2133	break
	2134	fi
	2135	(( i = i + 1 ))
	2136	done
	2137
	2138	if [ ${status} -gt 0 ] ; then
[811]	2139	echo "IGCM_sys_ncbo : ncbo error"
	2140	IGCM_debug_Exit "ncbo"
[657]	2141	fi
[16]	2142
[657]	2143	IGCM_debug_PopStack "IGCM_sys_ncbo"
[16]	2144	}
	2145
[811]	2146	#D-#==================================================
	2147	#D-function IGCM_sys_ncdif
	2148	#D-* Purpose: encapsulate ncdiff call so as to manage error code and retry
	2149	#D-* Examples:
	2150	#D-
[16]	2151	function IGCM_sys_ncdiff {
[1057]	2152	IGCM_debug_PushStack "IGCM_sys_ncdiff" $@
[657]	2153	if ( $DEBUG_sys ) ; then
	2154	echo "IGCM_sys_ncdiff :" $@
	2155	fi
[714]	2156
	2157	typeset NB_ESSAI DELAI status i
	2158	# number of tentative
	2159	NB_ESSAI=3
	2160	# time delay between tentative
	2161	DELAI=2
	2162
	2163	i=0
	2164	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2165	ncdiff -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2166	status=$?
	2167	if [ ${status} -gt 0 ] ; then
	2168	IGCM_debug_Print 2 "IGCM_sys_ncdiff : error code ${status}"
[978]	2169	cat /tmp/out_command_${LOGIN}.$$
	2170	\rm /tmp/out_command_${LOGIN}.$$
[714]	2171	IGCM_debug_Print 2 "IGCM_sys_ncdiff : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2172	sleep $DELAI
[714]	2173	else
[978]	2174	\rm /tmp/out_command_${LOGIN}.$$
[714]	2175	break
	2176	fi
	2177	(( i = i + 1 ))
	2178	done
	2179
	2180	if [ ${status} -gt 0 ] ; then
[811]	2181	echo "IGCM_sys_ncdiff : ncdiff error"
	2182	IGCM_debug_Exit "ncdiff"
[657]	2183	fi
[16]	2184
[657]	2185	IGCM_debug_PopStack "IGCM_sys_ncdiff"
[16]	2186	}
	2187
[811]	2188	#D-#==================================================
	2189	#D-function IGCM_sys_ncea
	2190	#D-* Purpose: encapsulate ncea call so as to manage error code and retry
	2191	#D-* Examples:
	2192	#D-
[16]	2193	function IGCM_sys_ncea {
[1057]	2194	IGCM_debug_PushStack "IGCM_sys_ncea" $@
[657]	2195	if ( $DEBUG_sys ) ; then
	2196	echo "IGCM_sys_ncea :" $@
	2197	fi
[714]	2198
	2199	typeset NB_ESSAI DELAI status i
	2200	# number of tentative
	2201	NB_ESSAI=3
	2202	# time delay between tentative
	2203	DELAI=2
	2204
	2205	i=0
	2206	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2207	ncea -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2208	status=$?
	2209	if [ ${status} -gt 0 ] ; then
	2210	IGCM_debug_Print 2 "IGCM_sys_ncea : error code ${status}"
[978]	2211	cat /tmp/out_command_${LOGIN}.$$
	2212	\rm /tmp/out_command_${LOGIN}.$$
[714]	2213	IGCM_debug_Print 2 "IGCM_sys_ncea : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2214	sleep $DELAI
[714]	2215	else
[978]	2216	\rm /tmp/out_command_${LOGIN}.$$
[714]	2217	break
	2218	fi
	2219	(( i = i + 1 ))
	2220	done
	2221
	2222	if [ ${status} -gt 0 ] ; then
[811]	2223	echo "IGCM_sys_ncea : ncea error"
	2224	IGCM_debug_Exit "ncea"
[657]	2225	fi
[16]	2226
[657]	2227	IGCM_debug_PopStack "IGCM_sys_ncea"
[16]	2228	}
	2229
[811]	2230	#D-#==================================================
	2231	#D-function IGCM_sys_ncecat
	2232	#D-* Purpose: encapsulate ncecat call so as to manage error code and retry
	2233	#D-* Examples:
	2234	#D-
[16]	2235	function IGCM_sys_ncecat {
[1057]	2236	IGCM_debug_PushStack "IGCM_sys_ncecat" $@
[657]	2237	if ( $DEBUG_sys ) ; then
	2238	echo "IGCM_sys_ncecat :" $@
	2239	fi
[714]	2240
	2241	typeset NB_ESSAI DELAI status i
	2242	# number of tentative
	2243	NB_ESSAI=3
	2244	# time delay between tentative
	2245	DELAI=2
	2246
	2247	i=0
	2248	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2249	ncecat -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2250	status=$?
	2251	if [ ${status} -gt 0 ] ; then
	2252	IGCM_debug_Print 2 "IGCM_sys_ncecat : error code ${status}"
[978]	2253	cat /tmp/out_command_${LOGIN}.$$
	2254	\rm /tmp/out_command_${LOGIN}.$$
[714]	2255	IGCM_debug_Print 2 "IGCM_sys_ncecat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2256	sleep $DELAI
[714]	2257	else
[978]	2258	\rm /tmp/out_command_${LOGIN}.$$
[714]	2259	break
	2260	fi
	2261	(( i = i + 1 ))
	2262	done
	2263
	2264	if [ ${status} -gt 0 ] ; then
[811]	2265	echo "IGCM_sys_ncecat : ncecat error"
	2266	IGCM_debug_Exit "ncecat"
[657]	2267	fi
[16]	2268
[657]	2269	IGCM_debug_PopStack "IGCM_sys_ncecat"
[16]	2270	}
	2271
[811]	2272	#D-#==================================================
	2273	#D-function IGCM_sys_ncflint
	2274	#D-* Purpose: encapsulate ncflint call so as to manage error code and retry
	2275	#D-* Examples:
	2276	#D-
[16]	2277	function IGCM_sys_ncflint {
[1057]	2278	IGCM_debug_PushStack "IGCM_sys_ncflint" $@
[657]	2279	if ( $DEBUG_sys ) ; then
	2280	echo "IGCM_sys_ncflint :" $@
	2281	fi
[714]	2282
	2283	typeset NB_ESSAI DELAI status i
	2284	# number of tentative
	2285	NB_ESSAI=3
	2286	# time delay between tentative
	2287	DELAI=2
	2288
	2289	i=0
	2290	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2291	ncflint -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2292	status=$?
	2293	if [ ${status} -gt 0 ] ; then
	2294	IGCM_debug_Print 2 "IGCM_sys_ncflint : error code ${status}"
[978]	2295	cat /tmp/out_command_${LOGIN}.$$
	2296	\rm /tmp/out_command_${LOGIN}.$$
[714]	2297	IGCM_debug_Print 2 "IGCM_sys_ncflint : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2298	sleep $DELAI
[714]	2299	else
[978]	2300	\rm /tmp/out_command_${LOGIN}.$$
[714]	2301	break
	2302	fi
	2303	(( i = i + 1 ))
	2304	done
	2305
	2306	if [ ${status} -gt 0 ] ; then
[811]	2307	echo "IGCM_sys_ncflint : ncflint error"
	2308	IGCM_debug_Exit "ncflint"
[657]	2309	fi
[16]	2310
[657]	2311	IGCM_debug_PopStack "IGCM_sys_ncflint"
[16]	2312	}
	2313
[811]	2314	#D-#==================================================
	2315	#D-function IGCM_sys_ncks
	2316	#D-* Purpose: encapsulate ncks call so as to manage error code and retry
	2317	#D-* Examples:
	2318	#D-
[16]	2319	function IGCM_sys_ncks {
[1057]	2320	IGCM_debug_PushStack "IGCM_sys_ncks" $@
[657]	2321	if ( $DEBUG_sys ) ; then
	2322	echo "IGCM_sys_ncks :" $@
	2323	fi
[714]	2324
	2325	typeset NB_ESSAI DELAI status i
	2326	# number of tentative
	2327	NB_ESSAI=3
	2328	# time delay between tentative
	2329	DELAI=2
	2330
	2331	i=0
	2332	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2333	ncks -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2334	status=$?
	2335	if [ ${status} -gt 0 ] ; then
	2336	IGCM_debug_Print 2 "IGCM_sys_ncks : error code ${status}"
[978]	2337	cat /tmp/out_command_${LOGIN}.$$
	2338	\rm /tmp/out_command_${LOGIN}.$$
[714]	2339	IGCM_debug_Print 2 "IGCM_sys_ncks : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2340	sleep $DELAI
[714]	2341	else
[978]	2342	\rm /tmp/out_command_${LOGIN}.$$
[714]	2343	break
	2344	fi
	2345	(( i = i + 1 ))
	2346	done
	2347
	2348	if [ ${status} -gt 0 ] ; then
[811]	2349	echo "IGCM_sys_ncks : ncks error"
	2350	IGCM_debug_Exit "ncks"
[657]	2351	fi
[16]	2352
[657]	2353	IGCM_debug_PopStack "IGCM_sys_ncks"
[16]	2354	}
	2355
[811]	2356	#D-#==================================================
	2357	#D-function IGCM_sys_ncpdq
	2358	#D-* Purpose: encapsulate ncpdq call so as to manage error code and retry
	2359	#D-* Examples:
	2360	#D-
[16]	2361	function IGCM_sys_ncpdq {
[1057]	2362	IGCM_debug_PushStack "IGCM_sys_ncpdq" $@
[657]	2363	if ( $DEBUG_sys ) ; then
	2364	echo "IGCM_sys_ncpdq :" $@
	2365	fi
[714]	2366
	2367	typeset NB_ESSAI DELAI status i
	2368	# number of tentative
	2369	NB_ESSAI=3
	2370	# time delay between tentative
	2371	DELAI=2
	2372
	2373	i=0
	2374	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2375	ncpdq -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2376	status=$?
	2377	if [ ${status} -gt 0 ] ; then
	2378	IGCM_debug_Print 2 "IGCM_sys_ncpdq : error code ${status}"
[978]	2379	cat /tmp/out_command_${LOGIN}.$$
	2380	\rm /tmp/out_command_${LOGIN}.$$
[714]	2381	IGCM_debug_Print 2 "IGCM_sys_ncpdq : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2382	sleep $DELAI
[714]	2383	else
[978]	2384	\rm /tmp/out_command_${LOGIN}.$$
[714]	2385	break
	2386	fi
	2387	(( i = i + 1 ))
	2388	done
	2389
	2390	if [ ${status} -gt 0 ] ; then
[811]	2391	echo "IGCM_sys_ncpdq : ncpdq error"
	2392	IGCM_debug_Exit "ncpdq"
[657]	2393	fi
[16]	2394
[657]	2395	IGCM_debug_PopStack "IGCM_sys_ncpdq"
[16]	2396	}
	2397
[811]	2398	#D-#==================================================
	2399	#D-function IGCM_sys_ncra
	2400	#D-* Purpose: encapsulate ncra call so as to manage error code and retry
	2401	#D-* Examples:
	2402	#D-
[16]	2403	function IGCM_sys_ncra {
[1057]	2404	IGCM_debug_PushStack "IGCM_sys_ncra" $@
[657]	2405	if ( $DEBUG_sys ) ; then
	2406	echo "IGCM_sys_ncra :" $@
	2407	fi
[714]	2408
	2409	typeset NB_ESSAI DELAI status i
	2410	# number of tentative
	2411	NB_ESSAI=3
	2412	# time delay between tentative
	2413	DELAI=2
	2414
	2415	i=0
	2416	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2417	ncra -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2418	status=$?
	2419	if [ ${status} -gt 0 ] ; then
	2420	IGCM_debug_Print 2 "IGCM_sys_ncra : error code ${status}"
[978]	2421	cat /tmp/out_command_${LOGIN}.$$
	2422	\rm /tmp/out_command_${LOGIN}.$$
[714]	2423	IGCM_debug_Print 2 "IGCM_sys_ncra : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2424	sleep $DELAI
[714]	2425	else
[978]	2426	\rm /tmp/out_command_${LOGIN}.$$
[714]	2427	break
	2428	fi
	2429	(( i = i + 1 ))
	2430	done
	2431
	2432	if [ ${status} -gt 0 ] ; then
[811]	2433	echo "IGCM_sys_ncra : ncra error"
	2434	IGCM_debug_Exit "ncra"
[657]	2435	fi
[16]	2436
[657]	2437	IGCM_debug_PopStack "IGCM_sys_ncra"
[16]	2438	}
	2439
[811]	2440	#D-#==================================================
	2441	#D-function IGCM_sys_ncrcat
	2442	#D-* Purpose: encapsulate ncrcat call so as to manage error code and retry
	2443	#D-* Examples:
	2444	#D-
[16]	2445	function IGCM_sys_ncrcat {
[1057]	2446	IGCM_debug_PushStack "IGCM_sys_ncrcat" $@
[657]	2447	if ( $DEBUG_sys ) ; then
	2448	echo "IGCM_sys_ncrcat :" $@
	2449	fi
[714]	2450
[811]	2451	typeset NB_ESSAI DELAI status i lastArg
[714]	2452	# number of tentative
	2453	NB_ESSAI=3
	2454	# time delay between tentative
	2455	DELAI=2
	2456
	2457	i=0
	2458	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2459	ncrcat -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2460	status=$?
	2461	if [ ${status} -gt 0 ] ; then
	2462	IGCM_debug_Print 2 "IGCM_sys_ncrcat : error code ${status}"
[978]	2463	cat /tmp/out_command_${LOGIN}.$$
	2464	\rm /tmp/out_command_${LOGIN}.$$
[714]	2465	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2466	sleep $DELAI
[978]	2467	elif [ ! "X$( grep "WARNING Intra-file non-monotonicity" /tmp/out_command_${LOGIN}.$$ )" = "X" ] ; then
[811]	2468	IGCM_debug_Print 2 "IGCM_sys_ncrcat : WARNING Intra-file non-monotonicity"
[978]	2469	cat /tmp/out_command_${LOGIN}.$$
[811]	2470	# remove files having corrupted time axis
	2471	eval lastArg=\${$#}
	2472	IGCM_debug_Print 2 "IGCM_sys_ncrcat : Delete ${lastArg}"
	2473	\rm ${lastArg}
[978]	2474	\rm /tmp/out_command_${LOGIN}.$$
[811]	2475	IGCM_debug_Print 2 "IGCM_sys_ncrcat : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
	2476	sleep $DELAI
[714]	2477	else
[978]	2478	\rm /tmp/out_command_${LOGIN}.$$
[714]	2479	break
	2480	fi
	2481	(( i = i + 1 ))
	2482	done
	2483
	2484	if [ ${status} -gt 0 ] ; then
[811]	2485	echo "IGCM_sys_ncrcat : ncrcat error"
	2486	#IGCM_debug_Exit "ncrcat"
[657]	2487	fi
[16]	2488
[657]	2489	IGCM_debug_PopStack "IGCM_sys_ncrcat"
[16]	2490	}
	2491
[811]	2492	#D-#==================================================
	2493	#D-function IGCM_sys_ncrename
	2494	#D-* Purpose: encapsulate ncrename call so as to manage error code and retry
	2495	#D-* Examples:
	2496	#D-
[16]	2497	function IGCM_sys_ncrename {
[1057]	2498	IGCM_debug_PushStack "IGCM_sys_ncrename" $@
[657]	2499	if ( $DEBUG_sys ) ; then
	2500	echo "IGCM_sys_ncrename :" $@
	2501	fi
[714]	2502
	2503	typeset NB_ESSAI DELAI status i
	2504	# number of tentative
	2505	NB_ESSAI=3
	2506	# time delay between tentative
	2507	DELAI=2
	2508
	2509	i=0
	2510	while [ $i -lt $NB_ESSAI ] ; do
[978]	2511	ncrename $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2512	status=$?
	2513	if [ ${status} -gt 0 ] ; then
	2514	IGCM_debug_Print 2 "IGCM_sys_ncrename : error code ${status}"
[978]	2515	cat /tmp/out_command_${LOGIN}.$$
	2516	\rm /tmp/out_command_${LOGIN}.$$
[714]	2517	IGCM_debug_Print 2 "IGCM_sys_ncrename : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2518	sleep $DELAI
[714]	2519	else
[978]	2520	\rm /tmp/out_command_${LOGIN}.$$
[714]	2521	break
	2522	fi
	2523	(( i = i + 1 ))
	2524	done
	2525
	2526	if [ ${status} -gt 0 ] ; then
[811]	2527	echo "IGCM_sys_ncrename : ncrename error"
	2528	IGCM_debug_Exit "ncrename"
[657]	2529	fi
[16]	2530
[657]	2531	IGCM_debug_PopStack "IGCM_sys_ncrename"
[16]	2532	}
	2533
[811]	2534	#D-#==================================================
	2535	#D-function IGCM_sys_ncwa
	2536	#D-* Purpose: encapsulate ncwa call so as to manage error code and retry
	2537	#D-* Examples:
	2538	#D-
[16]	2539	function IGCM_sys_ncwa {
[1057]	2540	IGCM_debug_PushStack "IGCM_sys_ncwa" $@
[657]	2541	if ( $DEBUG_sys ) ; then
	2542	echo "IGCM_sys_ncwa :" $@
	2543	fi
[714]	2544
	2545	typeset NB_ESSAI DELAI status i
	2546	# number of tentative
	2547	NB_ESSAI=3
	2548	# time delay between tentative
	2549	DELAI=2
	2550
	2551	i=0
	2552	while [ $i -lt $NB_ESSAI ] ; do
[1091]	2553	ncwa -C $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2554	status=$?
	2555	if [ ${status} -gt 0 ] ; then
	2556	IGCM_debug_Print 2 "IGCM_sys_ncwa : error code ${status}"
[978]	2557	cat /tmp/out_command_${LOGIN}.$$
	2558	\rm /tmp/out_command_${LOGIN}.$$
[714]	2559	IGCM_debug_Print 2 "IGCM_sys_ncwa : ${i}/${NB_ESSAI} sleep ${DELAI} seconds and try again."
[800]	2560	sleep $DELAI
[714]	2561	else
[978]	2562	\rm /tmp/out_command_${LOGIN}.$$
[714]	2563	break
	2564	fi
	2565	(( i = i + 1 ))
	2566	done
	2567
	2568	if [ ${status} -gt 0 ] ; then
[811]	2569	echo "IGCM_sys_ncwa : ncwa error"
	2570	IGCM_debug_Exit "ncwa"
[657]	2571	fi
[16]	2572
[657]	2573	IGCM_debug_PopStack "IGCM_sys_ncwa"
[16]	2574	}
	2575
[284]	2576	##############################################################
	2577	# CDO OPERATOR
	2578
[811]	2579	#D-#==================================================
	2580	#D-function IGCM_sys_cdo
	2581	#D-* Purpose: encapsulate cdo call so as to manage error code and retry
	2582	#D-* Examples:
	2583	#D-
[284]	2584	function IGCM_sys_cdo {
[1057]	2585	IGCM_debug_PushStack "IGCM_sys_cdo" $@
[714]	2586	if ( $DEBUG_sys ) ; then
	2587	echo "IGCM_sys_cdo :" $@
	2588	fi
[284]	2589
[714]	2590	typeset status
	2591
[978]	2592	\cdo $@ > /tmp/out_command_${LOGIN}.$$ 2>&1
[714]	2593	status=$?
	2594	if [ ${status} -gt 0 ] ; then
	2595	echo "IGCM_sys_cdo : error code ${status}"
[978]	2596	cat /tmp/out_command_${LOGIN}.$$
	2597	\rm /tmp/out_command_${LOGIN}.$$
[284]	2598	IGCM_debug_PopStack "IGCM_sys_cdo"
[657]	2599	return 1
	2600	else
	2601	IGCM_debug_PopStack "IGCM_sys_cdo"
	2602	return 0
	2603	fi
[284]	2604
[657]	2605	IGCM_debug_PopStack "IGCM_sys_cdo"
[16]	2606	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: