Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_ada.ksh @ 1482

Last change on this file since 1482 was 1459, checked in by jgipsl, 6 years ago
Removed variable R_IN_ECMWF (never used in reference experiment setups)
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 54.1 KB

Rev	Line
[771]	1	#!/bin/ksh
	2
	3	#**************************************************************
	4	# Author: Sébastien Denvil
	5	# Contact: Sebastien.Denvil__at__ipsl.jussieu.fr
	6	# $Revision:: $ Revision of last commit
	7	# $Author:: $ Author of last commit
	8	# $Date:: $ Date of last commit
	9	# IPSL (2006)
	10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
	11	#
	12	#**************************************************************
	13
	14	#=========================================================
	15	# The documentation of this file can be automatically generated
	16	# if you use the prefix #D- for comments to be extracted.
	17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
	18	#=========================================================
	19
	20	#D-#==================================================
	21	#D-LibIGCM_sys for Ada
	22	#D-#==================================================
	23	#D-
	24	#D- This ksh library if a layer under some usefull
	25	#D-environment variables and shell commands.
	26	#D-All those definitions depend on host particularities.
	27	#D-It manages a stack mechanism and test validity of operations.
	28	#D-All function described bellow must be prefixed by IGCM_sys.
	29
	30	#====================================================
	31	# libIGCM_sys PARAMETERS
	32	#====================================================
	33
	34	#====================================================
	35	# set DEBUG_sys to true to output calls of function
	36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
	37
	38	#====================================================
	39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
	40	typeset -r DRYRUN=${DRYRUN:=0}
	41
	42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
	43	# -------------------------------------------------------------------------------------
	44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
[809]	45	# \| \| Cp/Exe/param/files \| \| \| \|
	46	# \| \| Chmod Qsub \| \| \| \|
[771]	47	# -------------------------------------------------------------------------------------
	48	# \| 0 \| yes \| yes \| yes \| yes \|
	49	# -------------------------------------------------------------------------------------
	50	# \| 1 \| yes \| yes \| yes \| no \|
	51	# -------------------------------------------------------------------------------------
	52	# \| 2 \| yes \| yes \| no \| no \|
	53	# -------------------------------------------------------------------------------------
	54	# \| 3 \| yes \| no \| no \| no \|
	55	# -------------------------------------------------------------------------------------
	56
	57	#=====================================================
	58	# Global Variables :
	59	#=====================================================
	60	# Language : "fr" or "en"
	61	typeset -r MYLANG="fr"
	62
	63	#=====================================================
	64	# Host and user names
	65	# $hostname ou hostname
[1179]	66	typeset HOST=${HOST:=$( hostname )}
[771]	67	# $username ou whoami
[1179]	68	typeset LOGIN=${LOGIN:=$( whoami )}
[771]	69	# $hostname of the MASTER job
	70	typeset MASTER=ada
[1201]	71	# project name
	72	typeset PROJECT=$(echo ${LOADL_STEP_GROUP:=NONE})
	73	# jobWarningDelay in seconds
[1236]	74	typeset jobWarningDelay=$( TZ=utc date -d '1970-01-01 '${wall_clock_limit} +%s )
[771]	75
	76	#D-
	77	#D-#==================================================
	78	#D-Program used in libIGCM
	79	#D-#==================================================
	80
	81	# Submit command
	82	typeset SUBMIT=${SUBMIT:=llsubmit}
	83	# rsync with path
	84	typeset -r RSYNC=/usr/bin/rsync
	85	# RSYNC_opt args to rsync
	86	typeset -r RSYNC_opt="-va"
	87	# ie storage filesystem
[1012]	88	typeset -r STOREHOST=ergon
[1180]	89	typeset -r REMOTE_RSYNC=/usr/bin/rsync
[771]	90
	91	#====================================================
[1180]	92	# Set environment tools (ferret, nco, cdo, rebuild, ...)
[771]	93	#====================================================
[993]	94	source /smplocal/pub/Modules/default/init/ksh
[1305]	95	if ( [ "X${LOADL_STEP_TYPE}" = "XPARALLEL" ] \|\| [ "X${LOADL_STEP_TYPE}" = "XSERIAL" ] ) ; then
[1358]	96	module load python/2.7.10 > /dev/null 2>&1
[1305]	97	. /smphome/rech/psl/rpsl035/.atlas_env_ada_intel_2013_0_bash > /dev/null 2>&1
	98	else
[1362]	99	module load python/2.7.10 > /dev/null 2>&1
	100	module unload intel > /dev/null 2>&1
[1305]	101	module load intel/2016.2 > /dev/null 2>&1
	102	. /smphome/rech/psl/rpsl035/.atlas_env_ada_bash > /dev/null 2>&1
[1358]	103	export PCMDI_MP=/workgpfs/rech/psl/rpsl035/PCMDI-MP
[1308]	104	export UVCDAT_ANONYMOUS_LOG=no
[1305]	105	fi
[1210]	106	[ ! X${TaskType} = Xchecking ] && IGCM_debug_Print 1 "List of loaded modules:"
	107	[ ! X${TaskType} = Xchecking ] && module list
[1208]	108
[993]	109	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddNoise/src_X64_ADA/bin
[1026]	110	export PATH=${PATH}:/smphome/rech/psl/rpsl035/AddPerturbation/src_X64_ADA/bin
[1179]	111	export PATH=${PATH}:/smphome/rech/psl/rpsl035/bin/
[771]	112
	113	#====================================================
	114	# Host specific DIRECTORIES
	115	#====================================================
	116
	117	#====================================================
	118	#- MirrorlibIGCM for frontend
	119	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
	120
	121	#====================================================
	122	#- libIGCM_POST for frontend
	123	typeset -r libIGCM_POST=${libIGCM}
	124
	125	#====================================================
	126	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
	127	typeset -r R_EXE="${MODIPSL}/bin"
	128
	129	#====================================================
	130	#- SUBMIT_DIR : submission dir
[1012]	131	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${LOADL_STEP_INITDIR}}
[771]	132
	133	#====================================================
	134	#- IN
	135	typeset -r R_IN=${R_IN:=/workgpfs/rech/psl/rpsl035/IGCM}
	136
	137	#====================================================
	138	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
	139	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=${TMPDIR}}
	140
	141	#====================================================
[1179]	142	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
[1186]	143	typeset -r OUTCOMMAND_PATH=${RUN_DIR_PATH:-/tmp}
[1179]	144
	145	#====================================================
[771]	146	#- HOST_MPIRUN_COMMAND
[1305]	147	if ( [ "X${LOADL_STEP_TYPE}" = "XPARALLEL" ] \|\| [ "X${LOADL_STEP_TYPE}" = "XSERIAL" ] ) ; then
	148	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time poe"}
	149	else
	150	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="/usr/bin/time mpirun"}
	151	fi
[771]	152	#====================================================
	153	#- Max number of arguments passed to nco operator or demigration command
	154	UNIX_MAX_LIMIT=360
	155
	156	#====================================================
[1180]	157	#- set PackDefault to true on ada
[771]	158	PackDefault=true
	159
	160	#====================================================
[1305]	161	#- Number of cores per node
	162	core_per_node=32
	163
	164	#====================================================
[771]	165	#- Default number of MPI task for IPSL coupled model
	166	#- required for backward compatibility
	167	#-
	168	DEFAULT_NUM_PROC_OCE=5
	169	DEFAULT_NUM_PROC_CPL=1
	170	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
	171
	172	#D-#==================================================
[1057]	173	#D-function IGCM_sys_defineArchives
	174	#D-* Purpose:
	175	#D-* Define ARCHIVE : Dedicated to large files
	176	#D-* Define STORAGE : Dedicated to small/medium files
	177	#D-* Define R_OUT : Output tree located on ARCHIVE
	178	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
	179	#D-* Define R_BUF : Output tree located on SCRATCHDIR hosting files waiting for rebuild or pack processes
	180	#D-* if SpaceName=TEST everything is stored on WORKDIR
[771]	181	#D-* Examples:
	182	#D-
[1057]	183	function IGCM_sys_defineArchives {
	184	IGCM_debug_PushStack "IGCM_sys_defineArchives"
[771]	185
[1060]	186	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
[1057]	187	#====================================================
	188	#- ARCHIVE (dedicated to large files)
[1063]	189	ARCHIVE=${config_UserChoices_ARCHIVE}
[1057]	190	else
	191	#====================================================
	192	#- ARCHIVE (dedicated to large files)
[1063]	193	ARCHIVE=$(echo ${HOME} \| sed -e "s,/.*home/,/arch/home/,")
[1057]	194	fi
[771]	195
[1060]	196	if [ ! X${config_UserChoices_STORAGE} = X ]; then
[1057]	197	#====================================================
	198	#- STORAGE (dedicated to small/medium files)
[1063]	199	STORAGE=${config_UserChoices_STORAGE}
[1057]	200	else
	201	#====================================================
	202	#- STORAGE (dedicated to small/medium files)
[1063]	203	STORAGE=${WORKDIR}
[1057]	204	fi
[771]	205
[1057]	206	if [ X${config_UserChoices_SpaceName} = XTEST ]; then
	207	#====================================================
	208	#- R_OUT
[1063]	209	R_OUT=${WORKDIR}/IGCM_OUT
[1057]	210
	211	#====================================================
	212	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
[1063]	213	R_FIG=${WORKDIR}/IGCM_OUT
[1057]	214
	215	IGCM_debug_Print 1 "SpaceName=TEST ==> OVERRULE destination path directories"
	216
	217	else
	218	#====================================================
	219	#- R_OUT
[1063]	220	R_OUT=${ARCHIVE}/IGCM_OUT
[1057]	221
	222	#====================================================
	223	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
[1063]	224	R_FIG=${ARCHIVE}/IGCM_OUT
[1057]	225	fi
	226
	227	#====================================================
[1431]	228	#- CMIP6 (hosting CMIP6 files produced by XIOS2 and configured by dr2xml)
	229	CMIP6_BUF=${STORAGE}/IGCM_OUT
	230
	231	#====================================================
[1057]	232	#- R_BUF (ONLY FOR double copy an scratch)
[1063]	233	R_BUF=${WORKDIR}/IGCM_OUT
[1057]	234
	235	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
	236	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
	237	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
	238
	239	IGCM_debug_PopStack "IGCM_sys_defineArchives"
[771]	240	}
	241
	242	#D-#==================================================
	243	#D-function IGCM_sys_RshArchive
	244	#D-* Purpose: Archive rsh command
	245	#D-* Examples:
	246	#D-
	247	function IGCM_sys_RshArchive {
	248	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
[798]	249	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
[1179]	250	if [ $? = 0 ] ; then
[798]	251	rsh ${STOREHOST} exec /bin/ksh <<-EOF
[771]	252	${@}
	253	EOF
[798]	254	status=$?
	255	else
	256	/bin/ksh <<-EOF
	257	${@}
	258	EOF
	259	status=$?
	260	fi
	261	if [ ${status} -gt 0 ] ; then
	262	IGCM_debug_Print 2 "IGCM_sys_RshArchive : rsh or command failed error code ${status}"
[771]	263	IGCM_debug_Exit "IGCM_sys_RshArchive"
	264	fi
	265	IGCM_debug_PopStack "IGCM_sys_RshArchive"
	266	}
	267
	268	#D-#==================================================
[845]	269	#D-function IGCM_sys_RshArchive_NoError
	270	#D-* Purpose: Archive rsh command, without error
	271	#D-* used only in monitoring.job
	272	#D-* Examples:
	273	#D-
	274	function IGCM_sys_RshArchive_NoError {
	275	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
	276	DEBUG_sys=false IGCM_sys_IsFileArchived "$@"
	277	if [ $? = 0 ] ; then
	278	rsh ${STOREHOST} exec /bin/ksh <<-EOF
	279	${@} 2> /dev/null
	280	EOF
	281	else
	282	/bin/ksh <<-EOF
	283	${@} 2> /dev/null
	284	EOF
	285	fi
	286	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
	287	}
	288
	289	#D-#==================================================
[771]	290	#D-function IGCM_sys_MkdirArchive
	291	#D-* Purpose: Mkdir on Archive
	292	#D-* Examples:
	293	#D-
	294	function IGCM_sys_MkdirArchive {
	295	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
	296	if ( $DEBUG_sys ) ; then
	297	echo "IGCM_sys_MkdirArchive :" $@
	298	fi
	299	#- creation de repertoire sur le serveur fichier
[798]	300	DEBUG_sys=false IGCM_sys_IsFileArchived $1
[793]	301	if [ $? = 0 ] ; then
	302	rsh ${STOREHOST} -n mkdir -p $1
[794]	303	status=$?
[793]	304	else
	305	mkdir -p $1
[794]	306	status=$?
[922]	307	fi
[771]	308
[794]	309	if [ ${status} -gt 0 ] ; then
	310	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : rsh or mkdir failed error code ${status}"
[771]	311	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
	312	fi
	313	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
	314	}
	315
	316	#D-#==================================================
	317	#D-function IGCM_sys_TestDirArchive
	318	#D-* Purpose: Test Directory that must exists on Archive
	319	#D-* Examples:
	320	#D-
	321	function IGCM_sys_TestDirArchive {
	322	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
	323	if ( $DEBUG_sys ) ; then
	324	echo "IGCM_sys_TestDirArchive :" $@
	325	fi
[1027]	326	typeset ExistFlag
[796]	327	#Command depends on targeted file system
[798]	328	DEBUG_sys=false IGCM_sys_IsFileArchived $1
[796]	329	if [ $? = 0 ] ; then
[1027]	330	ExistFlag=$( IGCM_sys_RshArchive "[ -d $1 ] && echo 0 \|\| echo 1" )
[796]	331	else
[1027]	332	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
[796]	333	fi
[771]	334	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
	335	return ${ExistFlag}
	336	}
	337
	338	#D-#==================================================
	339	#D-function IGCM_sys_IsFileArchived
[1012]	340	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
[771]	341	#D-* Examples:
	342	#D-
	343	function IGCM_sys_IsFileArchived {
	344	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
	345	if ( $DEBUG_sys ) ; then
	346	echo "IGCM_sys_IsFileArchived :" $@
	347	fi
	348	typeset IsArchivedFlag
[1171]	349	IsArchivedFlag=$( [ "X$( echo $@ \| grep \/arch\/home )" != "X" ] && echo 0 \|\| echo 1 )
[771]	350	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
	351
	352	return ${IsArchivedFlag}
	353	}
	354
	355	#D-#==================================================
	356	#D-function IGCM_sys_TestFileArchive
	357	#D-* Purpose: Test file that must NOT EXISTS on Archive
	358	#D-* Examples:
	359	#D-
[1148]	360	function IGCM_sys_TestFileArchive {
	361	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
[1179]	362	typeset ExistFlag
[1171]	363	ExistFlag=$( IGCM_sys_RshArchive "[ -f $1 ] && echo 0 \|\| echo 1" )
[1148]	364	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
	365
	366	return ${ExistFlag}
	367	}
	368
	369	#D-#==================================================
[1179]	370	#D-function IGCM_sys_CountFileArchive
	371	#D-* Purpose: Count files on Archive filesystem
	372	#D-* Examples:
	373	#D-
[1148]	374	function IGCM_sys_CountFileArchive {
	375	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
	376	#Command depends on targeted file system
	377	DEBUG_sys=false IGCM_sys_IsFileArchived $1
	378	if [ $? = 0 ] ; then
	379	IGCM_sys_RshArchive "ls ${@} 2>/dev/null \| wc -l"
	380	status=$?
	381	else
	382	ls ${@} 2>/dev/null \| wc -l
	383	status=$?
	384	fi
	385	if [ ${status} -gt 0 ] ; then
	386	echo "IGCM_sys_CountFileArchive : erreur."
	387	fi
	388	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
	389	}
	390
	391	#D-#==================================================
[771]	392	#D-function IGCM_sys_Tree
	393	#D-* Purpose: Tree directories with files on ${ARCHIVE}
	394	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
	395	#D-
	396	function IGCM_sys_Tree {
	397	IGCM_debug_PushStack "IGCM_sys_Tree" $@
	398	if ( $DEBUG_sys ) ; then
	399	echo "IGCM_sys_Tree :" $@
	400	fi
	401
[1032]	402	\mfls -R $@
[771]	403
	404	IGCM_debug_PopStack "IGCM_sys_Tree"
	405	}
	406
	407	#D-#==================================================
	408	#D-function IGCM_sys_Qsub
	409	#D-* Purpose: Qsub new job
	410	#D-* Examples:
	411	#D-
	412	function IGCM_sys_Qsub {
	413	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
	414	if ( $DEBUG_sys ) ; then
	415	echo "IGCM_sys_Qsub :" $@
	416	fi
[1179]	417	typeset status
	418
[771]	419	# We have to change output/error file
	420	[ ${#@} = 1 ] && REP_FOR_JOB=${SUBMIT_DIR}
	421	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
[774]	422	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Output}:" \
[1305]	423	-e "s:\# \@ error = .:\# \@ error = ${Script_Output}:" \
	424	$1 > ${REP_FOR_JOB}/JOB_FOR_IGCM
[1179]	425	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1 ; status=$? ; cd - ;
[771]	426
[1179]	427	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[811]	428	if [ ${status} -gt 0 ] ; then
	429	IGCM_debug_Print 2 "IGCM_sys_Qsub $1 : error code ${status}"
[771]	430	IGCM_debug_Exit "IGCM_sys_Qsub"
	431	else
[1179]	432	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ \| tail -1 \| sed -e s/\"//g )
[771]	433	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM
	434	fi
	435	IGCM_debug_PopStack "IGCM_sys_Qsub"
	436	}
	437
	438	#D-#==================================================
	439	#D-function IGCM_sys_QsubPost
	440	#D-* Purpose: Qsub new job on scalaire
	441	#D-* Examples:
	442	#D-
	443	function IGCM_sys_QsubPost {
	444	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
	445	if ( $DEBUG_sys ) ; then
	446	echo "IGCM_sys_QsubPost :" $@
	447	fi
[1179]	448	typeset status
	449
[771]	450	# We have to change output/error file
[784]	451	[ ${#@} = 1 ] && REP_FOR_JOB=${POST_DIR}
[771]	452	[ ${#@} = 2 ] && REP_FOR_JOB=${2}
	453
[784]	454	sed -e "s:\# \@ output = .:\# \@ output = ${Script_Post_Output}.out:" \
[1305]	455	-e "s:\# \@ error = .:\# \@ error = ${Script_Post_Output}.out:" \
	456	${libIGCM_POST}/$1.job > ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
[774]	457
[1179]	458	cd $REP_FOR_JOB ; /usr/bin/llsubmit JOB_FOR_IGCM_$$ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1 ; status=$? ; cd - ;
[948]	459
[1179]	460	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[811]	461	if [ ${status} -gt 0 ] ; then
	462	IGCM_debug_Print 2 "IGCM_sys_QsubPost $1 : error code ${status}"
[771]	463	IGCM_debug_Exit "IGCM_sys_QsubPost"
	464	else
[1179]	465	JobID=$( gawk {'print $4'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ \| tail -1 \| sed -e s/\"//g )
[887]	466	IGCM_sys_Rm ${REP_FOR_JOB}/JOB_FOR_IGCM_$$
[771]	467	fi
	468	IGCM_debug_PopStack "IGCM_sys_QsubPost"
	469	}
	470
	471	#D-*************************
	472	#D- File transfer functions
	473	#D-*************************
	474	#D-
	475
	476	#D-#==================================================
	477	#D-function IGCM_sys_RmRunDir
	478	#D-* Purpose: rm tmpdir (dummy function most of the time batch
	479	#D- scheduler will do the job)
	480	#D-* Examples:
	481	#D-
	482	function IGCM_sys_RmRunDir {
[1057]	483	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
[771]	484	if ( $DEBUG_sys ) ; then
	485	echo "IGCM_sys_RmRunDir :" $@
[1183]	486	IGCM_debug_Print 1 "Dummy call, let the scheduler do that."
[771]	487	fi
	488	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
	489	}
	490
	491	#D-#==================================================
	492	#D-function IGCM_sys_Put_Dir
	493	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
	494	#D-* Examples:
	495	#D-
	496	function IGCM_sys_Put_Dir {
	497	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
	498	if ( $DEBUG_sys ) ; then
	499	echo "IGCM_sys_Put_Dir :" $@
	500	fi
	501	if [ $DRYRUN = 0 ]; then
	502	if [ ! -d ${1} ] ; then
	503	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
	504	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
	505	return
	506	fi
	507
	508	typeset status
	509
	510	# Only if we use rsync
	511	#IGCM_sys_TestDirArchive $( dirname $2 )
	512	#
[792]	513	#Command depends on targeted file system
[798]	514	DEBUG_sys=false IGCM_sys_IsFileArchived $2
[792]	515	if [ $? = 0 ] ; then
[1179]	516	\rcp -r $1 ${STOREHOST}:$2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	517	status=$?
	518	else
[1179]	519	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	520	status=$?
	521	fi
[771]	522
	523	if [ ${status} -gt 0 ] ; then
[792]	524	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : rcp or cp failed error code ${status}"
[1179]	525	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	526	IGCM_debug_Exit "IGCM_sys_Put_Dir"
	527	else
[1179]	528	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	529	fi
	530	fi
	531	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
	532	}
	533
	534	#D-#==================================================
	535	#D-function IGCM_sys_Get_Dir
	536	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
	537	#D-* Examples:
	538	#D-
	539	function IGCM_sys_Get_Dir {
	540	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
	541	if ( $DEBUG_sys ) ; then
	542	echo "IGCM_sys_Get_Dir :" $@
	543	fi
	544	if [ $DRYRUN = 0 ]; then
	545	typeset NB_ESSAI DELAI status i
	546	# number of tentative
	547	NB_ESSAI=3
	548	# time delay between tentative
	549	DELAI=2
	550
	551	# Only if we use rsync
	552	#IGCM_sys_TestDirArchive $( dirname $2 )
	553	#
[793]	554	# Command depends on targeted filesystem
[798]	555	DEBUG_sys=false IGCM_sys_IsFileArchived $1
[793]	556	if [ $? = 0 ] ; then
	557	# add dmget (to demigrate all offline files) to reduce time of this command :
[1012]	558	#IGCM_sys_RshArchive "dmfind $1 -state MIG -o -state OFL -o -state PAR \| dmget -q -n"
[793]	559	i=0
	560	while [ $i -lt $NB_ESSAI ] ; do
[1179]	561	\rcp -rp ${STOREHOST}:$1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[793]	562	status=$?
	563	if [ ${status} -gt 0 ]; then
	564	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : rcp failed error code ${status} ${i}/${NB_ESSAI}"
	565	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
	566	sleep $DELAI
	567	else
	568	break
	569	fi
	570	(( i = i + 1 ))
	571	done
	572	else
[1179]	573	\cp -rp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	574	status=$?
[793]	575	if [ ${status} -gt 0 ] ; then
	576	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
[1179]	577	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[793]	578	IGCM_debug_Exit "IGCM_sys_Get_Dir"
[771]	579	else
[1179]	580	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	581	fi
[793]	582	fi
[771]	583	if [ ${status} -gt 0 ] ; then
[1179]	584	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : (r)cp failed error code ${status}"
	585	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	586	IGCM_debug_Exit "IGCM_sys_Get_Dir"
	587	else
[1179]	588	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	589	fi
	590	fi
	591	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
	592	}
	593
	594	#D-#==================================================
	595	#D-function IGCM_sys_Put_Rest
	596	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
	597	#D- File and target directory must exist.
	598	#D-* Examples:
	599	#D-
	600	function IGCM_sys_Put_Rest {
	601	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
	602	if ( $DEBUG_sys ) ; then
	603	echo "IGCM_sys_Put_Rest :" $@
	604	fi
	605	if [ $DRYRUN = 0 ]; then
	606	if [ ! -f ${1} ] ; then
	607	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
	608	IGCM_debug_Exit "IGCM_sys_Put_Rest"
	609	fi
	610
	611	typeset status
	612	#
	613	if [ X${JobType} = XRUN ] ; then
	614	IGCM_sys_Chmod 444 ${1}
	615	fi
	616	#
	617	#
[792]	618	#Command depends on targeted file system
[798]	619	DEBUG_sys=false IGCM_sys_IsFileArchived $2
[792]	620	if [ $? = 0 ] ; then
[1179]	621	mfput $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	622	status=$?
	623	else
[795]	624	IGCM_sys_MkdirArchive $( dirname $2 )
[1179]	625	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	626	status=$?
	627	fi
[771]	628
	629	# #RSYNC WITH NETWORK RSH CALL
[795]	630	# IGCM_sys_MkdirArchive $( dirname $2 )
[1179]	631	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	632	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	633
	634	# #RSYNC WITH NFS USE
[1179]	635	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	636	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	637
	638	# status=$?
	639	# IGCM_sys_Rsync_out $status
	640
[1179]	641	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	642	# (( status=status+$? ))
	643
	644	if [ ${status} -gt 0 ] ; then
[1179]	645	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : mfput or cp failed error code ${status}"
	646	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	647	IGCM_debug_Exit "IGCM_sys_Put_Rest"
	648	else
[1179]	649	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	650	fi
	651	fi
	652	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
	653	}
	654
	655	#D-#==================================================
	656	#D-function IGCM_sys_Put_Out
	657	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
	658	#D-* Examples:
	659	#D-
	660	function IGCM_sys_Put_Out {
	661	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
	662	if ( $DEBUG_sys ) ; then
	663	echo "IGCM_sys_Put_Out :" $@
	664	fi
	665
	666	typeset status
	667
	668	if [ $DRYRUN = 0 ]; then
	669	if [ ! -f ${1} ] ; then
	670	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
	671	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	672	return 1
	673	fi
	674	#
	675	if [ X${JobType} = XRUN ] ; then
	676	if [ X${3} = X ] ; then
	677	IGCM_sys_Chmod 444 ${1}
	678	fi
	679	fi
	680	#
	681	#
[792]	682	#Command depends on targeted file system
[798]	683	DEBUG_sys=false IGCM_sys_IsFileArchived $2
[792]	684	if [ $? = 0 ] ; then
[1179]	685	mfput $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	686	status=$?
	687	else
[795]	688	IGCM_sys_MkdirArchive $( dirname $2 )
[1179]	689	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	690	status=$?
	691	fi
[771]	692
	693	# #RSYNC WITH NETWORK RSH CALL
[795]	694	# IGCM_sys_MkdirArchive $( dirname $2 )
[1179]	695	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	696	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	697
	698	# #RSYNC WITH NFS USE
[1179]	699	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	700	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	701
	702	# status=$?
	703	# IGCM_sys_Rsync_out $status
	704
[1179]	705	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	706	# (( status=status+$? ))
	707
	708	if [ ${status} -gt 0 ] ; then
[792]	709	IGCM_debug_Print 2 "IGCM_sys_Put_Out : mfput or cp failed error code ${status}"
[1179]	710	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	711	IGCM_debug_Exit "IGCM_sys_Put_Out"
	712	else
[1179]	713	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	714	fi
	715	fi
	716	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	717	return 0
	718	}
	719
	720	#D-#==================================================
	721	#D-function IGCM_sys_Get
	722	#D-* Purpose: Get a file from ${ARCHIVE}
	723	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
	724	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
	725	function IGCM_sys_Get {
	726	IGCM_debug_PushStack "IGCM_sys_Get" $@
	727
	728	typeset DEST status dm_liste
	729
	730	if ( $DEBUG_sys ) ; then
	731	echo "IGCM_sys_Get :" $@
	732	fi
	733	if [ $DRYRUN -le 2 ]; then
	734	if [ X${1} = X'/l' ] ; then
	735	eval set +A dm_liste \${${2}}
	736	else
	737	dm_liste=${1}
	738	fi
	739	eval DEST=\${${#}}
	740
	741	# test if the (first) file is present in the old computation :
[798]	742	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
[796]	743	if [ $? = 0 ] ; then
	744	IGCM_sys_TestFileArchive ${dm_liste[0]}
	745	status=$?
	746	else
[798]	747	IGCM_sys_TestFileBuffer ${dm_liste[0]}
[796]	748	status=$?
	749	fi
	750
[771]	751	if [ ${status} -gt 0 ] ; then
	752	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
	753	IGCM_debug_Exit "IGCM_sys_Get"
	754	return
	755	fi
	756
[792]	757	#Command depends on targeted file system
[798]	758	DEBUG_sys=false IGCM_sys_IsFileArchived ${dm_liste[0]}
[792]	759	if [ $? = 0 ] ; then
[1179]	760	mfget ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	761	status=$?
	762	else
[1179]	763	\cp ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[792]	764	status=$?
	765	fi
[771]	766
	767	# #RSYNC WITH NETWORK RSH CALL
[1179]	768	# echo ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	769	# ${RSYNC} ${RSYNC_opt} --rsync-path=${REMOTE_RSYNC} -e rsh ${STOREHOST}:"${dm_liste}" ${STOREHOST}:${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	770
	771	# #RSYNC WITH NFS USE
[1179]	772	# echo ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	773	# ${RSYNC} ${RSYNC_opt} ${dm_liste} ${RUN_DIR}/${DEST} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[771]	774
	775	# status=$?
	776	# IGCM_sys_Rsync_out $status
	777
[1179]	778	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	779	# (( status=status+$? ))
	780
	781	if [ ${status} -gt 0 ] ; then
[792]	782	IGCM_debug_Print 2 "IGCM_sys_Get : mfget or cp failed error code ${status}"
[1179]	783	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	784	IGCM_debug_Exit "IGCM_sys_Get"
	785	else
[1179]	786	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[771]	787	fi
	788	fi
	789	IGCM_debug_PopStack "IGCM_sys_Get"
	790	}
	791
	792	#D-#==================================================
[835]	793	#D-function IGCM_sys_GetDate_Monitoring
	794	#D-* Purpose: get the last year for which the monitoring has been computed
	795	#D-* Examples:
	796	#D-
	797	function IGCM_sys_GetDate_Monitoring {
	798	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
	799	if ( $DEBUG_sys ) ; then
	800	echo "IGCM_sys_GetDate_Monitoring :" $@
	801	fi
	802
	803	IGCM_sys_Cd /tmp
[1129]	804	# NASTY HACK. We force /tmp here. This function is called only from the front-end and interactively.
	805	RUN_DIR_PATH=/tmp IGCM_sys_Get ${1} .
[835]	806	eval ${2}=$( cdo showyear $( basename ${1} ) 2> /dev/null \| gawk '{ print $NF }' )
[1131]	807	# Need to erase this file to avoid collision (permission denied getting file) between users on the front-end
	808	IGCM_sys_Rm -f $( basename ${1} )
[835]	809	IGCM_sys_Cd -
	810
	811	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
	812	}
	813
	814	#D-#==================================================
[1392]	815	#D-function IGCM_sys_Dods_Rm
	816	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
	817	#D-* Examples:
	818	#D-
	819	function IGCM_sys_Dods_Rm {
	820	if ( $DEBUG_sys ) ; then
	821	echo "IGCM_sys_Dods_Rm :" $@
	822	fi
	823
	824	typeset status
	825	if [ $DRYRUN = 0 ]; then
	826	if [ "$#" -eq 1 ]; then
	827	rsh ${STOREHOST} exec /bin/ksh <<EOF
	828	cd ${R_SAVE}
	829	/arch/local/bin/dods_rm DODS/pub/${LOGIN}/${R_DODS}/${1} > /dev/null 2>&1
	830	EOF
	831	status=$?
	832	else
	833	rsh ${STOREHOST} exec /bin/ksh <<EOF
	834	cd ${R_SAVE}
	835	/arch/local/bin/dods_rm DODS/pub/${LOGIN}/${R_DODS} > /dev/null 2>&1
	836	EOF
	837	status=$?
	838	fi
	839	fi
	840
	841	return ${status}
	842	}
	843
	844	#D-#==================================================
	845	#D-function IGCM_sys_Dods_Cp
	846	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
	847	#D-* Examples:
	848	#D-
	849	function IGCM_sys_Dods_Cp {
	850	if ( $DEBUG_sys ) ; then
	851	echo "IGCM_sys_Dods_Cp :" $@
	852	fi
	853	return 0
	854	}
	855
	856	#D-#==================================================
[771]	857	#D-function IGCM_sys_Put_Dods
[1179]	858	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole.
[771]	859	#D-* Examples:
	860	#D-
	861	function IGCM_sys_Put_Dods {
	862	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
	863	if ( $DEBUG_sys ) ; then
	864	echo "IGCM_sys_Put_Dods :" $@
	865	fi
[1179]	866	typeset status
[771]	867	if [ $DRYRUN = 0 ]; then
	868	# We take our time on that
	869	sleep 10
	870	IGCM_sys_TestDirArchive ${R_SAVE}/${1}
	871	if [ $? != 0 ] ; then
	872	echo "WARNING : IGCM_sys_Put_Dods ${R_SAVE}/${1} DOES NOT EXIST ."
	873	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
	874	return
	875	fi
	876
[1012]	877	rsh ${STOREHOST} exec /bin/ksh <<EOF
[771]	878	cd ${R_SAVE}
[1012]	879	/arch/local/bin/dods_rm DODS/pub/${LOGIN}/${R_DODS}/${1} > /dev/null 2>&1
[771]	880	/bin/chmod -R u+w ${R_SAVE}/${1}
[1012]	881	/arch/local/bin/dods_cp ${1} DODS/pub/${LOGIN}/${R_DODS} > /dev/null 2>&1
[771]	882	/bin/chmod -R +rX ${R_SAVE}/${1}
	883	/bin/chmod -R u+w ${R_SAVE}/${1}
	884	EOF
	885	status=$?
	886
	887	if [ ${status} -gt 0 ] ; then
	888	echo "IGCM_sys_Put_Dods : error."
	889	IGCM_debug_Exit "IGCM_sys_Put_Dods"
	890	fi
	891	fi
	892	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
	893	}
	894
	895	##############################################################
	896	# REBUILD OPERATOR
	897
[811]	898	#D-#==================================================
[1163]	899	#D-function IGCM_sys_sync
	900	#D-* Purpose: flush buffer on disk (dummy function on Ada)
	901	#D-* Examples:
	902	#D-
	903	function IGCM_sys_sync {
	904	IGCM_debug_PushStack "IGCM_sys_sync" $@
	905	if ( $DEBUG_sys ) ; then
	906	echo "IGCM_sys_sync :" $@
	907	echo "Dummy call, let the system do that."
	908	fi
	909	IGCM_debug_PopStack "IGCM_sys_sync"
	910	}
	911
[771]	912	############################################################
	913	# Activate Running Environnment Variables
	914
[811]	915	#D-#==================================================
[1239]	916	#D-function IGCM_sys_activ_variables
[811]	917	#D-* Purpose: set environement variables prior to execution
	918	#D-* Examples:
	919	#D-
[771]	920	function IGCM_sys_activ_variables {
	921	IGCM_debug_PushStack "IGCM_sys_activ_variables"
	922	if ( $DEBUG_sys ) ; then
[811]	923	echo "IGCM_sys_activ_variables"
[771]	924	fi
	925
[811]	926	# --------------------------------------------------------------------
	927	#D- MPI specifications
	928	# --------------------------------------------------------------------
	929
	930	# --------------------------------------------------------------------
	931	#D- Other specifications
	932	# --------------------------------------------------------------------
	933
[771]	934	ulimit -s unlimited
[1012]	935	## to be done only one time
	936	## export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
[1068]	937	## echo ${LD_LIBRARY_PATH} \| grep -i netcdf >/dev/null 2>&1 \|\| export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/smplocal/pub/NetCDF/4.1.3/lib:/smplocal/pub/HDF5/1.8.9/seq/lib
[1012]	938	IGCM_debug_Print 1 "set LD_LIBRARY_PATH=${LD_LIBRARY_PATH}"
[771]	939
[959]	940	export MP_STDOUTMODE=combined
	941	IGCM_debug_Print 1 "set MP_STDOUTMODE=${MP_STDOUTMODE}"
	942
[1195]	943	## fix to reduce memory usage. Required since 2014/22/04. On ada IDRIS.
	944	#export MP_EUILIBPATH=/smplocal/lib/ibmhpc/pe12012/ppe.pami/gnu/lib64/pami64
	945	#IGCM_debug_Print 1 "set MP_EUILIBPATH=${MP_EUILIBPATH}"
[1004]	946
[771]	947	IGCM_debug_PopStack "IGCM_sys_activ_variables"
	948	}
	949
	950	############################################################
	951	# Desactivate Running Environnment Variables
	952
[811]	953	#D-#==================================================
	954	#D-function IGCM_sys_desactiv_variables
	955	#D-* Purpose: unset environement variables after execution
	956	#D-* Examples:
	957	#D-
[771]	958	function IGCM_sys_desactiv_variables {
	959	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
	960	if ( $DEBUG_sys ) ; then
[811]	961	echo "IGCM_sys_desactiv_variables"
[771]	962	fi
	963	# --------------------------------------------------------------------
	964	#D- MPI specifications
	965	# --------------------------------------------------------------------
	966
	967	# --------------------------------------------------------------------
	968	#D- Other specifications
	969	# --------------------------------------------------------------------
	970
	971	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
	972	}
	973
	974	############################################################
[1238]	975	# Update job headers to be used by the scheduler
	976
	977	#D-#==================================================
	978	#D-function IGCM_sys_updateHeaders
	979	#D-* Purpose: Update job headers to be used by the scheduler
	980	#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
	981	#D-
	982	function IGCM_sys_updateHeaders {
	983	IGCM_debug_PushStack "IGCM_sys_updateHeaders"
	984	if ( $DEBUG_sys ) ; then
	985	echo "IGCM_sys_updateHeaders"
	986	fi
	987	typeset file
	988	file=$1
	989
	990	if [ ${executionType} -eq 1 ] ; then
	991	# MPMD + MPI
[1305]	992	if [ X${MPIEnvironment} = XIBM ] ; then
	993	sed -e "/::openMPthreads::/d" \
[1238]	994	-e "s/::JobNumProcTot::/${coreNumber}/" \
[1305]	995	${file} > ${file}.tmp
	996	else
	997	sed -e "/::openMPthreads::/d" \
	998	-e "s/@ job_type = parallel/@ job_type = mpich/" \
	999	-e "s/::JobNumProcTot::/${coreNumber}/" \
	1000	${file} > ${file}.tmp
	1001	fi
[1238]	1002
	1003	elif [ ${executionType} -eq 2 ] ; then
	1004	# MPMD + MPI + OMP
[1305]	1005	if [ X${MPIEnvironment} = XIBM ] ; then
	1006	sed -e "s/::openMPthreads::/${openMPthreads}/" \
[1242]	1007	-e "s/::JobNumProcTot::/${mpiTasks}/" \
[1305]	1008	${file} > ${file}.tmp
	1009	else
	1010	(( nodeNumber = coreNumber / core_per_node ))
	1011	[ $(( ${coreNumber} % ${core_per_node} )) -ne 0 ] && (( nodeNumber = nodeNumber + 1 ))
	1012	sed -e "/::openMPthreads::/d" \
	1013	-e "s/@ job_type = parallel/@ job_type = mpich/" \
	1014	-e "s/@ total_tasks = ::JobNumProcTot::/@ node = ${nodeNumber} /" \
	1015	-e "/@ as_limit = 3.5gb/d" \
	1016	-e "s/::JobNumProcTot::/${mpiTasks}/" \
	1017	${file} > ${file}.tmp
	1018	fi
[1238]	1019
	1020	elif [ ${executionType} -eq 3 ] ; then
	1021	# SPMD + MPI/OMP
[1305]	1022	if [ X${MPIEnvironment} = XIBM ] ; then
	1023	sed -e "s/::openMPthreads::/${openMPthreads}/" \
[1238]	1024	-e "s/::JobNumProcTot::/${mpiTasks}/" \
[1305]	1025	${file} > ${file}.tmp
	1026	else
	1027	sed -e "s/::openMPthreads::/${openMPthreads}/" \
	1028	-e "s/@ job_type = parallel/@ job_type = mpich/" \
	1029	-e "s/::JobNumProcTot::/${mpiTasks}/" \
	1030	${file} > ${file}.tmp
	1031	fi
[1238]	1032
	1033	elif [ ${executionType} -eq 4 ] ; then
	1034	# SPMD + MPI only
[1305]	1035	if [ X${MPIEnvironment} = XIBM ] ; then
	1036	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
[1238]	1037	-e "/::openMPthreads::/d" \
[1305]	1038	${file} > ${file}.tmp
	1039	else
	1040	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
	1041	-e "s/@ job_type = parallel/@ job_type = mpich/" \
	1042	-e "/::openMPthreads::/d" \
	1043	${file} > ${file}.tmp
	1044	fi
[1238]	1045
	1046	elif [ ${executionType} -eq 5 ] ; then
	1047	# SPMD + OMP only
	1048	sed -e "s/::openMPthreads::/${openMPthreads}/" \
[1240]	1049	-e "s/@ job_type = parallel/@ job_type = serial/" \
[1238]	1050	-e "/::JobNumProcTot::/d" \
	1051	${file} > ${file}.tmp
	1052
	1053	elif [ ${executionType} -eq 6 ] ; then
	1054	# SEQUENTIAL THEN
	1055	sed -e "s/::JobNumProcTot::/1/" \
[1240]	1056	-e "s/@ job_type = parallel/@ job_type = serial/" \
[1238]	1057	-e "/::openMPthreads::/d" \
	1058	${file} > ${file}.tmp
	1059
	1060	fi
	1061
	1062	IGCM_sys_Mv ${file}.tmp ${file}
	1063
	1064	IGCM_debug_PopStack "IGCM_sys_updateHeaders"
	1065	}
	1066
	1067	############################################################
[771]	1068	# Build MPI/OMP scripts run file (dummy function)
	1069
[811]	1070	#D-#==================================================
	1071	#D-function IGCM_sys_build_run_file
	1072	#D-* Purpose: build run file (deprecated)
	1073	#D-* Examples:
	1074	#D-
[771]	1075	function IGCM_sys_build_run_file {
	1076
[811]	1077	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
[771]	1078
	1079	}
	1080
	1081	############################################################
	1082	# Build MPI/OMP scripts
[811]	1083
	1084	#D-#==================================================
	1085	#D-function IGCM_sys_build_execution_scripts
	1086	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
	1087	#D-* Examples:
	1088	#D-
[771]	1089	function IGCM_sys_build_execution_scripts
	1090	{
	1091	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
	1092	if ( $DEBUG_sys ) ; then
	1093	echo "IGCM_sys_build_execution_scripts " $@
	1094	fi
	1095
	1096	EXECUTION=${HOST_MPIRUN_COMMAND}
	1097
[1179]	1098	# MPMD mode
[771]	1099	if ( ${OK_PARA_MPMD} ) ; then
	1100
[1305]	1101	# MPI IBM Environment
	1102	if ( [ "X${LOADL_STEP_TYPE}" = "XPARALLEL" ] \|\| [ "X${LOADL_STEP_TYPE}" = "XSERIAL" ] ) ; then
	1103	IGCM_debug_Print 1 "You use IBM MPI environment"
	1104	if [ -f run_file ] ; then
	1105	IGCM_sys_Rm -f run_file
	1106	fi
	1107	touch run_file
[771]	1108
[1305]	1109	# Build run_file
[771]	1110
[1305]	1111	# First loop on the components for the coupler ie oasis (only if oasis3)
	1112	# the coupler ie oasis3 must be the first one
	1113	for comp in ${config_ListOfComponents[*]} ; do
[771]	1114
[1305]	1115	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1116	eval ExeNameOut=\${config_Executable_${comp}[1]}
[771]	1117
[1305]	1118	# for CPL component only
	1119	if [ "X${comp}" = "XCPL" ] ; then
[771]	1120
[1305]	1121	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1122	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
[777]	1123
[1305]	1124	if ( ${OK_PARA_MPI} ) ; then
	1125	(( mpi_count = 1 ))
	1126	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
	1127	echo "./${ExeNameOut}" >> run_file
	1128	(( mpi_count = mpi_count + 1 ))
	1129	done
	1130	else
	1131	echo "./${ExeNameOut} " >> run_file
	1132	fi
[777]	1133	fi
[1305]	1134
	1135	done
	1136
	1137	# Then second loop on the components
	1138	for comp in ${config_ListOfComponents[*]} ; do
	1139
	1140	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1141	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1142
	1143	# Only if we really have an executable for the component and not the coupler ie oasis:
	1144	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	1145
	1146	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1147	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1148
	1149	if ( ${OK_PARA_OMP} ) ; then
	1150	# Check if the number of threads is correct
	1151	case ${comp_proc_omp_loc} in
	1152	1\|2\|4)
	1153	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads on IBM MPI Environment"
	1154	IGCM_debug_Print 1 "Beware : it may you use more CPU than needed."
	1155	;;
[1357]	1156	8)
	1157	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads on IBM MPI Environment"
	1158	IGCM_debug_Print 1 "Beware : it may you use more CPU than needed."
	1159	IGCM_debug_Print 1 "Beware : these settings should not be used for IPSLCM coupled model!"
	1160	;;
	1161	16)
[1305]	1162	IGCM_debug_Exit "ERROR with OMP parameters !"
	1163	IGCM_debug_Print 2 "Beware : ${comp_proc_omp_loc} is too much for MPMD mode"
	1164	IGCM_debug_Print 2 "You will use more CPU than needed : try to use Intel-MPI environment to do such a thing !"
	1165	IGCM_debug_Verif_Exit
	1166	;;
	1167	*)
	1168	IGCM_debug_Exit "ERROR with OMP parameters !"
	1169	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
	1170	IGCM_debug_Print 2 "Only 1,2,4,8,16 as number of OMP threads are possible "
	1171	IGCM_debug_Verif_Exit
	1172	;;
	1173	esac
	1174
	1175	fi
	1176
	1177	if ( ${OK_PARA_MPI} ) ; then
	1178	(( mpi_count = 1 ))
	1179	until [ ${mpi_count} -gt ${comp_proc_mpi_loc} ] ; do
	1180	if ( ${OK_PARA_OMP} ) ; then
	1181	echo "env OMP_NUM_THREADS=$comp_proc_omp_loc ./${ExeNameOut}" >> run_file
	1182	else
	1183	echo "./${ExeNameOut}" >> run_file
	1184	fi
	1185	(( mpi_count = mpi_count + 1 ))
	1186	done
	1187	else
	1188	echo "./${ExeNameOut} " >> run_file
	1189	fi
	1190	fi
	1191	done
	1192	if ( ${OK_PARA_OMP} ) ; then
	1193	export KMP_STACKSIZE=200m
[771]	1194	fi
	1195
[1305]	1196	EXECUTION="${HOST_MPIRUN_COMMAND} -pgmmodel mpmd -cmdfile ./run_file"
[771]	1197
[1305]	1198	IGCM_sys_Chmod u+x run_file
	1199	if ( $DEBUG_sys ) ; then
	1200	echo "run_file contains : "
	1201	cat run_file
	1202	fi
[771]	1203
	1204
[1305]	1205	# MPI Intel Environment
	1206	else
	1207	IGCM_debug_Print 1 "You use Intel MPI environment"
[777]	1208
[1305]	1209	# Only MPI (MPMD)
	1210	if ( ! ${OK_PARA_OMP} ) ; then
	1211	init_exec=n
	1212
	1213	# First loop on the components for the coupler ie oasis (only if oasis3)
	1214	# the coupler ie oasis3 must be the first one
	1215	for comp in ${config_ListOfComponents[*]} ; do
	1216
	1217	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1218	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1219
	1220	# for CPL component only
	1221	if [ "X${comp}" = "XCPL" ] && [ "X${ExeNameOut}" != X\"\" ] ; then
	1222
	1223	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1224	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1225
	1226	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1227	echo "" >> script_${ExeNameOut}.ksh
	1228	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1229	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1230	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
	1231	init_exec=y
	1232	fi
	1233	done
	1234	# Then second loop on the components
	1235	for comp in ${config_ListOfComponents[*]} ; do
	1236
	1237	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1238	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1239
	1240	# Only if we really have an executable for the component and not the coupler ie oasis:
	1241	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	1242
	1243	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1244	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1245
	1246	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1247	echo "" >> script_${ExeNameOut}.ksh
	1248	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1249	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1250
	1251	if [ ${init_exec} = y ] ; then
	1252	EXECUTION="${EXECUTION} : -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
[1093]	1253	else
[1305]	1254	EXECUTION="${EXECUTION} -np ${comp_proc_mpi_loc} ./script_${ExeNameOut}.ksh"
	1255	init_exec=y
[1093]	1256	fi
[1305]	1257	fi
	1258	done
	1259	# MPI-OpenMP (MPMD)
	1260	else
	1261
	1262	# Execution specifications
	1263	EXECUTION="${EXECUTION} -configfile run_file"
	1264	export KMP_STACKSIZE=200m
	1265	if [ -f run_file ] ; then
	1266	IGCM_sys_Rm -f run_file
[777]	1267	fi
[1305]	1268	touch run_file
	1269
	1270	# Initialisation of variables
	1271	string_final=""
	1272	string_final_hexa=""
	1273	current_core=0
	1274	executable_nb=1
	1275
	1276	# Hosts treatment
	1277	for nodes in `echo $LOADL_PROCESSOR_LIST`
	1278	do
	1279	host[$i]=$nodes
	1280	i=$((i+1))
	1281	done
	1282
	1283	# Loop on the components
	1284	for comp in ${config_ListOfComponents[*]} ; do
	1285
	1286	eval ExeNameIn=\${config_Executable_${comp}[0]}
	1287	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1288
	1289	# Not possible if oasis has an executable (i.e old version of oasis3)
	1290	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" = "XCPL" ] ) ; then
	1291	IGCM_debug_Exit "ERROR MPMD with hybrid MPI-OpenMP is not available with oasis3 version"
	1292	IGCM_debug_Print 2 "Only available with oasis3-MCT version coupler"
	1293	IGCM_debug_Verif_Exit
	1294	fi
	1295
	1296	# Only if we really have an executable for the component :
	1297	if [ "X${ExeNameOut}" != X\"\" ] ; then
	1298
	1299	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1300	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1301
	1302	# Check if the number of threads is correct
	1303	case ${comp_proc_omp_loc} in
	1304	1\|2\|4\|8\|16)
	1305	IGCM_debug_Print 1 "You run ${ExeNameOut} on ${comp_proc_omp_loc} OMP threads"
	1306	;;
	1307	*)
	1308	IGCM_debug_Exit "ERROR with OMP parameters !"
	1309	IGCM_debug_Print 2 "${comp_proc_omp_loc} is not possible as number of OMP threads"
	1310	IGCM_debug_Print 2 "Only 1,2,4,8,16 as number of OMP threads are possible "
	1311	IGCM_debug_Verif_Exit
	1312	;;
	1313	esac
	1314
	1315
	1316	# Build run_file for Ada Intel-MPI environment : method used to assign cores and nodes for the MPI process by using hexadecimal mask
	1317	# Example of run_file :
	1318	#-host host0 -n 4 -env I_MPI_PIN_DOMAIN=[ff,ff00,ff0000,ff000000] ./a.out
	1319	#-host host1 -n 4 -env I_MPI_PIN_DOMAIN=[ff,ff00,ff0000,ff000000] ./a.out
	1320	#-host host2 -n 10 -env I_MPI_PIN_DOMAIN=[3,c,30,c0,300,c00,3000,c000,30000,c0000,100000,200000,400000,800000,1000000,2000000,4000000,8000000,10000000,20000000] ./b.out
	1321	#-host host2 -n 10 ./c.out
	1322	# Example of final command :
	1323	# mpirun -configfile run_file
	1324
	1325	rank=0
	1326	# For one specific executable, loop on mpi process
	1327	for nb_proc_mpi in `seq 0 $(($comp_proc_mpi_loc-1))`; do
	1328	(( index_host = current_core / core_per_node ))
	1329	host_value=${host[${index_host}]}
	1330	(( slot = current_core % core_per_node ))
	1331	# loop on omp threads for each mpi process (set the appropriate bit to 1 and append it to previous one)
	1332	for index in `seq $slot $(($slot+$comp_proc_omp_loc-1))`; do
	1333	string_final="1"$string_final
	1334	done
	1335	# convert binary mask to hexadecimal mask
	1336	if [ $rank -ne 0 ] ; then
	1337	string_final_hexa=$string_final_hexa","$( printf '%x\n' "$((2#$string_final))" )
	1338	else
	1339	string_final_hexa=$( printf '%x\n' "$((2#$string_final))" )
	1340	fi
	1341	# replace bit 1 by bit 0 in order to append next one (next one wil be 1)
	1342	string_final=$( echo $string_final \| sed "s/1/0/g" )
	1343	# mpi rank = mpi_rank + 1
	1344	(( rank = rank + 1 ))
	1345	# current core takes into account the number of omp threads which was previously append
	1346	(( current_core = current_core + comp_proc_omp_loc ))
	1347	# We write to the configuration file either we switch to the next node or we switch to the next executable
	1348	if ( [ $(( current_core / core_per_node )) -ne $index_host ] \|\| [ $nb_proc_mpi -eq $(($comp_proc_mpi_loc-1)) ] ) ; then
	1349	# I_MPI_PIN_DOMAIN variable must be given once per node
	1350	if [ $executable_nb -eq 1 ] ; then
	1351	echo "-host $host_value -n $rank -env I_MPI_PIN_DOMAIN=[$string_final_hexa] ./$ExeNameOut" >> run_file
	1352	else
	1353	sed -i "/$host_value/s/\]/\,$string_final_hexa\]/g" run_file
	1354	echo "-host $host_value -n $rank ./$ExeNameOut" >> run_file
	1355	fi
	1356	# +1 for the number of executbale on the same node
	1357	if [ $nb_proc_mpi -eq $(($comp_proc_mpi_loc-1)) ] ; then
	1358	(( executable_nb = executable_nb + 1 ))
	1359	fi
	1360	# Some initializations if we switch to the next node
	1361	if [ $(( current_core / core_per_node )) -ne $index_host ] ; then
	1362	string_final=""
	1363	string_final_hexa=""
	1364	rank=0
	1365	executable_nb=1
	1366	fi
	1367	fi
	1368	done
	1369
	1370	fi
	1371	done
	1372	IGCM_sys_Chmod u+x run_file
	1373	if ( $DEBUG_sys ) ; then
	1374	echo "run_file contains : "
	1375	cat run_file
[1093]	1376	fi
[771]	1377	fi
	1378	fi
	1379
[1180]	1380	# Only one executable (SPMD mode).
	1381	else
[771]	1382
	1383	for comp in ${config_ListOfComponents[*]} ; do
	1384
	1385	# Only if we really have an executable for the component :
	1386	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1387	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
[847]	1388
	1389	if ( ${OK_PARA_OMP} ) ; then
	1390	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
[853]	1391	export KMP_STACKSIZE=200m
	1392	#export KMP_LIBRARY=turnaround
[847]	1393	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
	1394	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
	1395	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
	1396	fi
[1305]	1397	if ( ${OK_PARA_MPI} ) ; then
	1398	if ( [ "X${LOADL_STEP_TYPE}" = "XPARALLEL" ] \|\| [ "X${LOADL_STEP_TYPE}" = "XSERIAL" ] ) ; then
	1399	EXECUTION="${HOST_MPIRUN_COMMAND} ./${ExeNameOut}"
	1400	else
	1401	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1402	EXECUTION="${HOST_MPIRUN_COMMAND} -np ${comp_proc_mpi_loc} ./${ExeNameOut}"
	1403	fi
	1404	else
	1405	EXECUTION="/usr/bin/time ./${ExeNameOut}"
	1406	fi
[771]	1407	fi
	1408
	1409	done
	1410
	1411	fi
	1412
	1413	IGCM_debug_Print 1 "sys ada : execution command is "
	1414	IGCM_debug_Print 1 "$EXECUTION"
	1415
	1416	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
	1417	}
	1418
[933]	1419	#D-#==================================================
	1420	#D-function IGCM_sys_check_path
	1421	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
	1422	#D-* do not point to an important use directory. Stop immediately in that case.
	1423	#D-* Examples:
	1424	#D-
	1425	function IGCM_sys_check_path {
	1426	IGCM_debug_PushStack "IGCM_sys_check_path"
	1427	if ( $DEBUG_sys ) ; then
	1428	echo "IGCM_sys_check_path"
	1429	fi
[811]	1430
[1012]	1431	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${WORKDIR} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
[933]	1432	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
	1433	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
[936]	1434	IGCM_debug_Exit "This will stop the job"
[933]	1435	fi
	1436	IGCM_debug_PopStack "IGCM_sys_check_path"
	1437	}
	1438
[811]	1439	#D-#==================================================
	1440	#D-function IGCM_sys_check_quota
	1441	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
	1442	#D-* Examples:
	1443	#D-
[771]	1444	function IGCM_sys_check_quota {
	1445	IGCM_debug_PushStack "IGCM_sys_check_quota"
	1446	if ( $DEBUG_sys ) ; then
[811]	1447	echo "IGCM_sys_check_quota"
[771]	1448	fi
	1449	# Limit of quota (in %)
	1450	limit_quota=90
[811]	1451
[771]	1452	# Check of the volume
[945]	1453	volume_quota=$(quota_u -w 2>/dev/null\| grep 'Quota soft' \| gawk '{print $5}')
[771]	1454	if [ ! X${volume_quota} = X ] ; then
	1455	quota_volume=${volume_quota%%\%}
	1456	# echo $quota_volume
[946]	1457	if [ $(echo "${quota_volume} > ${limit_quota}" \| bc) -eq 1 ] ; then
[771]	1458	IGCM_debug_Print 1 "Please, check your quota of volume on workgpfs"
	1459	IGCM_debug_Print 1 "${quota_volume}% of your quota is used"
	1460	IGCM_debug_Print 1 "Use the quota_u -w command to check"
	1461	IGCM_debug_Print 1 "You must have more than 10% available to run"
	1462	IGCM_debug_Exit "Not enough space to run ! STOP HERE"
	1463	IGCM_debug_Verif_Exit
	1464	fi
	1465
	1466	fi
	1467	IGCM_debug_PopStack "IGCM_sys_check_quota"
	1468	}
	1469
	1470	#D-#==================================================
[1286]	1471	#D-function IGCM_sys_projectAccounting
	1472	#D-* Purpose: store project accounting information in a file
	1473	#D-* Examples:
	1474	#D-
	1475	function IGCM_sys_projectAccounting {
	1476	IGCM_debug_PushStack "IGCM_sys_projectAccounting"
	1477	if ( $DEBUG_sys ) ; then
	1478	echo "IGCM_sys_check_quota"
	1479	fi
	1480
	1481	cpt > $1
	1482
	1483	IGCM_debug_PopStack "IGCM_sys_projectAccounting"
	1484	}
	1485
	1486	#D-#==================================================
[1244]	1487	#D-function IGCM_sys_getJobSchedulerID
	1488	#D-* Purpose: Get the job ID during execution
	1489	#D-* Examples: IGCM_sys_getJobSchedulerID jobSchedulerID
	1490	#D-
	1491	function IGCM_sys_getJobSchedulerID {
	1492	IGCM_debug_PushStack "IGCM_sys_getJobSchedulerID"
	1493	if ( $DEBUG_sys ) ; then
	1494	echo "IGCM_sys_getJobSchedulerID"
	1495	fi
	1496
	1497	eval ${1}=$( echo $LOADL_STEP_ID \| awk -F. '{print $4}' )
	1498
	1499	IGCM_debug_PopStack "IGCM_sys_getJobSchedulerID"
	1500	}
	1501
	1502	#D-#==================================================
[958]	1503	#D-function IGCM_sys_GetJobID
[1244]	1504	#D-* Purpose: Get the job ID from the JobName
[1179]	1505	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
[958]	1506	#D-
	1507	function IGCM_sys_GetJobID {
	1508	IGCM_debug_PushStack "IGCM_sys_GetJobID"
	1509	if ( $DEBUG_sys ) ; then
	1510	echo "IGCM_sys_GetJobID"
	1511	fi
	1512
	1513	# Print only the full (-W) JobID (%id) and JobName (%jn)
	1514	ID=$( llq -u $2 -W -f %id %jn \| \
[1305]	1515	gawk -v JobName=$1 '( $NF ~ JobName ) { print $1 }' )
[958]	1516
	1517	eval ${3}=${ID}
	1518	IGCM_debug_PopStack "IGCM_sys_GetJobID"
	1519	}
	1520
	1521	#D-#==================================================
[771]	1522	#D-function IGCM_sys_CountJobInQueue
[1244]	1523	#D-* Purpose: Count number of users job
[771]	1524	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
	1525	#D-
	1526	function IGCM_sys_CountJobInQueue {
	1527	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
	1528	if ( $DEBUG_sys ) ; then
	1529	echo "IGCM_sys_CountJobInQueue"
	1530	fi
	1531
	1532	# Print only the full (-W) JobName (%jn)
	1533	NbRun=$( llq -W -f %jn \| grep -c "$1" )
	1534
	1535	eval ${2}=${NbRun}
	1536
	1537	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
	1538	}
	1539
[955]	1540	#D-#==================================================
	1541	#D-function IGCM_sys_ListJobInQueue
[1244]	1542	#D-* Purpose: Produce a list of users jobs
[955]	1543	#D-* Examples: IGCM_sys_ListJobInQueue ${User} JobNameList
	1544	#D-
	1545	function IGCM_sys_ListJobInQueue {
[1297]	1546	IGCM_debug_PushStack "IGCM_sys_ListJobInQueue"
[955]	1547	if ( $DEBUG_sys ) ; then
	1548	echo "IGCM_sys_ListJobInQueue"
	1549	fi
	1550
[960]	1551	# With -W option, column width is as large as necessary
	1552	set -A JobList $( llq -u $1 -W -f %jn \| head -n -2 \| tail -n +3 \| \
[1305]	1553	gawk '( $1 != /TS/ && \
[960]	1554	$1 !~ /PACK/ && \
	1555	$1 !~ /REBUILD/ && \
	1556	$1 !~ /pack/ ) \
	1557	{ print $1 }' \| sed -e "s/\(.\)\.[0-9]/\1/" )
[955]	1558
	1559	eval set -A ${2} ${JobList[*]}
	1560
	1561	IGCM_debug_PopStack "IGCM_sys_ListJobInQueue"
	1562	}
[1272]	1563
	1564	#D-#==================================================
	1565	#D-function IGCM_sys_atlas
	1566	#D-* Purpose: encapsulate atlas call so as to manage error code and curie specificity
	1567	#D-* Examples:
	1568	#D-
	1569	function IGCM_sys_atlas {
	1570	IGCM_debug_PushStack "IGCM_sys_atlas" $@
	1571	if ( $DEBUG_sys ) ; then
	1572	echo "IGCM_sys_atlas :" $@
	1573	fi
	1574
	1575	typeset status
	1576
	1577	\atlas $@ > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	1578	status=$?
	1579	if [ ${status} -gt 0 ] ; then
	1580	echo "IGCM_sys_atlas : error code ${status}"
[1305]	1581	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
	1582	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
	1583	IGCM_debug_PopStack "IGCM_sys_atlas"
	1584	return 1
	1585	else
	1586	IGCM_debug_PopStack "IGCM_sys_atlas"
	1587	return 0
	1588	fi
[1297]	1589
[1305]	1590	IGCM_debug_PopStack "IGCM_sys_atlas"
	1591	}
[1297]	1592
[1380]	1593	#D-#==================================================
	1594	#D-function IGCM_sys_rebuild_nemo
	1595	#D-* Purpose: rebuild nemo parallel files with general rebuild
	1596	#D-* Examples:
	1597	#D-
	1598
	1599	function IGCM_sys_rebuild_nemo {
	1600	IGCM_debug_PushStack "IGCM_sys_rebuild_nemo" $@
	1601	if ( $DEBUG_sys ) ; then
	1602	echo "IGCM_sys_rebuild_nemo :" $@
	1603	fi
	1604
	1605	nemo_generic_restart_file_name_out=${1}
	1606	nemo_extension_out=${3}
	1607	shift ; shift ; shift
	1608
	1609	IGCM_sys_rebuild -o ${nemo_generic_restart_file_name_out}.${nemo_extension_out} $@
	1610
	1611	IGCM_debug_PopStack "IGCM_sys_rebuild_nemo"
	1612	}
	1613
	1614

Note: See TracBrowser for help on using the repository browser.

Download in other formats: