Context Navigation

source: trunk/libIGCM/libIGCM_sys/libIGCM_sys_obelix.ksh @ 1239

Last change on this file since 1239 was 1239, checked in by sdipsl, 9 years ago
continuation of r1238 for ciclad, obelix, default and iitm. correct typo
Property licence set to The following licence information concerns ONLY the libIGCM tools ================================================================== Copyright © Centre National de la Recherche Scientifique CNRS Commissariat à l'Énergie Atomique CEA libIGCM : Library for Portable Models Computation of IGCM Group. IGCM Group is the french IPSL Global Climate Model Group. This library is a set of shell scripts and functions whose purpose is the management of the initialization, the launch, the transfer of output files, the post-processing and the monitoring of datas produce by any numerical program on any plateforme. This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/ or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. Property svn:keywords set to `Revision Author Date`
File size: 37.0 KB

Rev	Line
[657]	1	#!/bin/ksh
[16]	2
	3	#**************************************************************
	4	# Author: Sebastien Denvil, Martial Mancip, Christian Laguerre
[373]	5	# Contact: Martial.Mancip__at__ipsl.jussieu.fr
	6	# $Revision:: $ Revision of last commit
	7	# $Author:: $ Author of last commit
	8	# $Date:: $ Date of last commit
[16]	9	# IPSL (2006)
	10	# This software is governed by the CeCILL licence see libIGCM/libIGCM_CeCILL.LIC
[657]	11	#
[16]	12	#**************************************************************
	13
	14	#=========================================================
	15	# The documentation of this file can be automatically generated
[657]	16	# if you use the prefix #D- for comments to be extracted.
[16]	17	# Extract with command: cat lib* \| grep "^#D-" \| cut -c "4-"
	18	#=========================================================
	19
	20	#D-#==================================================
	21	#D-LibIGCM_sys for obelix
	22	#D-#==================================================
	23	#D-
[657]	24	#D- This ksh library if a layer under some usefull
[16]	25	#D-environment variables and shell commands.
	26	#D-All those definitions depend on host particularities.
	27	#D-It manages a stack mechanism and test validity of operations.
	28	#D-All function described bellow must be prefixed by IGCM_sys.
	29
	30	#====================================================
	31	# libIGCM_sys PARAMETERS
	32	#====================================================
	33
[343]	34	#====================================================
[16]	35	# set DEBUG_sys to true to output calls of function
	36	typeset -r DEBUG_sys=${DEBUG_sys:=true}
	37
[343]	38	#====================================================
[16]	39	# Turn in dry run mode ? (sys_Put_Rest, sys_Put_Out, sys_Get)
	40	typeset -r DRYRUN=${DRYRUN:=0}
	41
	42	# YOU MUST COMPILE YOUR EXE FILES FOR DRYRUN MODE !
	43	# -------------------------------------------------------------------------------------
	44	# \| DRYRUN= \| Date computations, \| sys_Get \| Exe \| sys_Put_Out; sys_Put_Rest \|
[809]	45	# \| \| Cp/Exe/param/files \| \| \| \|
	46	# \| \| Chmod Qsub \| \| \| \|
[16]	47	# -------------------------------------------------------------------------------------
	48	# \| 0 \| yes \| yes \| yes \| yes \|
	49	# -------------------------------------------------------------------------------------
	50	# \| 1 \| yes \| yes \| yes \| no \|
	51	# -------------------------------------------------------------------------------------
	52	# \| 2 \| yes \| yes \| no \| no \|
	53	# -------------------------------------------------------------------------------------
	54	# \| 3 \| yes \| no \| no \| no \|
	55	# -------------------------------------------------------------------------------------
	56
	57	#=====================================================
	58	# Global Variables :
	59	#=====================================================
	60	# Language : "fr" or "en"
	61	typeset -r MYLANG="fr"
	62
	63	#=====================================================
[657]	64	# Host and user names
[217]	65	# $hostname ou hostname
[1180]	66	typeset HOST=${HOST:=$( hostname )}
[16]	67	# $username ou whoami
[1180]	68	typeset LOGIN=${LOGIN:=$( whoami )}
[657]	69	# $hostname of the MASTER job
[742]	70	typeset MASTER=${MASTER:=$( hostname )}
[1201]	71	# project name
	72	typeset PROJECT=NONE
	73	# jobWarningDelay in seconds
	74	typeset jobWarningDelay=${PBS_WALLTIME}
[16]	75
	76	#D-
	77	#D-#==================================================
	78	#D-Program used in libIGCM
	79	#D-#==================================================
	80
[742]	81	# Submit command
	82	typeset SUBMIT=${SUBMIT:=qsub}
[16]	83	# rsync with path
	84	typeset -r RSYNC=/usr/bin/rsync
	85	# RSYNC_opt args to rsync
[282]	86	typeset -r RSYNC_opt="-va"
[657]	87	# ie storage filesystem
	88	typeset -r STOREHOST=${MASTER}
[16]	89
	90	#====================================================
[1180]	91	# Set environment tools (ferret, nco, cdo, rebuild, ...)
[305]	92	#====================================================
[1097]	93	. /home/users/igcmg/.atlas_env_asterix_ksh
[1180]	94	export PATH=${PATH}:/home/users/igcmg/rebuild/bin/
[16]	95
[343]	96	#====================================================
[421]	97	# Set lf95 environment
	98	#====================================================
	99	. /usr/local/install/lf6481/bash_laheyfort_setup
	100
	101	#====================================================
[16]	102	# Host specific DIRECTORIES
	103	#====================================================
	104
	105	#====================================================
[657]	106	#- MirrorlibIGCM for frontend
	107	typeset -r MirrorlibIGCM=${MirrorlibIGCM:=false}
	108
	109	#====================================================
	110	#- libIGCM_POST for frontend
	111	typeset -r libIGCM_POST=${libIGCM}
	112
	113	#====================================================
[16]	114	#- R_EXE (==> BIN_DIR = ${MODIPSL}/bin )
	115	typeset -r R_EXE="${MODIPSL}/bin"
	116
	117	#====================================================
	118	#- SUBMIT_DIR : submission dir
[256]	119	if [ X${PBS_O_WORKDIR} != X ] ; then
[1124]	120	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PBS_O_WORKDIR}}
[256]	121	else
[1124]	122	typeset -x SUBMIT_DIR=${SUBMIT_DIR:=${PWD}}
[256]	123	fi
[657]	124
[16]	125	#====================================================
[725]	126	#- IN
[978]	127	typeset -r R_IN=${R_IN:=/home/orchideeshare/igcmg/IGCM}
[1180]	128	typeset -r R_IN_ECMWF=${R_IN_ECMWF:=/home/orchideeshare/igcmg/IGCM}
[725]	129
	130	#====================================================
[16]	131	#- RUN_DIR_PATH : Temporary working directory (=> TMP)
[838]	132	if [ X${PBS_JOBID} != X ] ; then
	133	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/scratch/$PBS_O_LOGNAME.$PBS_JOBID}
	134	else
[839]	135	typeset -r RUN_DIR_PATH=${RUN_DIR_PATH:=/tmp/tmp$$}
[838]	136	fi
[16]	137
	138	#====================================================
[1179]	139	#- OUTCOMMAND_PATH : tmp place to store command lines standard error and outputs
[1186]	140	typeset -r OUTCOMMAND_PATH=/tmp
[1179]	141
	142	#====================================================
[16]	143	#- HOST_MPIRUN_COMMAND
	144	typeset -r HOST_MPIRUN_COMMAND=${HOST_MPIRUN_COMMAND:="time mpirun"}
	145
[61]	146	#====================================================
	147	#- Max number of arguments passed to nco operator or demigration command
	148	UNIX_MAX_LIMIT=360
	149
[657]	150	#====================================================
[1180]	151	#- set PackDefault to false on obelix
	152	PackDefault=false
	153
	154	#====================================================
[657]	155	#- Number of core per node (max number of OpenMP task)
	156	NUM_COREPERNODE=1
	157
	158	#====================================================
	159	#- Default number of MPI task for IPSL coupled model
	160	#- required for backward compatibility
	161	#-
	162	DEFAULT_NUM_PROC_OCE=1
	163	DEFAULT_NUM_PROC_CPL=1
	164	(( DEFAULT_NUM_PROC_ATM = BATCH_NUM_PROC_TOT - DEFAULT_NUM_PROC_OCE - DEFAULT_NUM_PROC_CPL ))
	165
[16]	166	#D-#==================================================
[1057]	167	#D-function IGCM_sys_defineArchives
	168	#D-* Purpose:
	169	#D-* Define ARCHIVE : Dedicated to large files
	170	#D-* Define STORAGE : Dedicated to small/medium files
	171	#D-* Define R_OUT : Output tree located on ARCHIVE
	172	#D-* Define R_FIG : Output tree located on STORAGE hosting figures (monitoring and atlas, and/or small files)
	173	#D-* Define R_BUF : Output tree located on STORAGE hosting files waiting for rebuild or pack processes
	174	#D-* if SpaceName=TEST nothing special will hapen
[725]	175	#D-* Examples:
	176	#D-
[1057]	177	function IGCM_sys_defineArchives {
	178	IGCM_debug_PushStack "IGCM_sys_defineArchives"
	179
[1060]	180	if [ ! X${config_UserChoices_ARCHIVE} = X ]; then
[1057]	181	#====================================================
	182	#- ARCHIVE (dedicated to large files)
[1063]	183	ARCHIVE=${config_UserChoices_ARCHIVE}
[1057]	184	else
	185	#====================================================
	186	#- ARCHIVE (dedicated to large files)
[1063]	187	ARCHIVE=${ARCHIVE:=/home/scratch01/${LOGIN}}
[1057]	188	fi
	189
[1060]	190	if [ ! X${config_UserChoices_STORAGE} = X ]; then
[1057]	191	#====================================================
	192	#- STORAGE (dedicated to small/medium files)
[1063]	193	STORAGE=${config_UserChoices_STORAGE}
[1057]	194	else
	195	#====================================================
	196	#- STORAGE (dedicated to small/medium files)
[1063]	197	STORAGE=${ARCHIVE}
[1057]	198	fi
	199
	200	# ON OBELIX NO SPECIAL CASE WHEN X${config_UserChoices_SpaceName} = XTEST
	201
	202	#====================================================
	203	#- R_OUT
[1063]	204	R_OUT=${ARCHIVE}/IGCM_OUT
[1057]	205
	206	#====================================================
	207	#- R_FIG (hosting figures : monitoring and atlas, and/or small files)
[1063]	208	R_FIG=${STORAGE}/IGCM_OUT
[1057]	209
	210	#====================================================
	211	#- R_BUF (ONLY FOR double copy an scratch)
[1063]	212	R_BUF=${STORAGE}/IGCM_OUT
[1057]	213
	214	IGCM_debug_Print 1 "R_OUT has been defined = ${R_OUT}"
	215	IGCM_debug_Print 1 "R_BUF has been defined = ${R_BUF}"
	216	IGCM_debug_Print 1 "R_FIG has been defined = ${R_FIG}"
	217
	218	IGCM_debug_PopStack "IGCM_sys_defineArchives"
[725]	219	}
	220
	221	#D-#==================================================
[37]	222	#D-function IGCM_sys_RshArchive
	223	#D-* Purpose: Archive rsh command
	224	#D-* Examples:
	225	#D-
	226	function IGCM_sys_RshArchive {
[657]	227	IGCM_debug_PushStack "IGCM_sys_RshArchive" $@
	228	/bin/ksh <<-EOF
[40]	229	${@}
	230	EOF
[811]	231	status=$?
	232	if [ ${status} -gt 0 ] ; then
	233	IGCM_debug_Print 2 "IGCM_sys_RshArchive : command failed error code ${status}"
[657]	234	IGCM_debug_Exit "IGCM_sys_RshArchive"
	235	fi
	236	IGCM_debug_PopStack "IGCM_sys_RshArchive"
[37]	237	}
	238
	239	#D-#==================================================
[1178]	240	#D-function IGCM_sys_RshArchive_NoError
	241	#D-* Purpose: Archive rsh command, without error
	242	#D-* used only in monitoring.job
	243	#D-* Examples:
	244	#D-
	245	function IGCM_sys_RshArchive_NoError {
	246	IGCM_debug_PushStack "IGCM_sys_RshArchive_NoError" $@
	247	/bin/ksh <<-EOF
[1181]	248	${@} 2> /dev/null
[1178]	249	EOF
	250	IGCM_debug_PopStack "IGCM_sys_RshArchive_NoError"
	251	}
	252
	253	#D-#==================================================
[16]	254	#D-function IGCM_sys_MkdirArchive
	255	#D-* Purpose: Mkdir on Archive
	256	#D-* Examples:
	257	#D-
	258	function IGCM_sys_MkdirArchive {
[657]	259	IGCM_debug_PushStack "IGCM_sys_MkdirArchive" $@
	260	if ( $DEBUG_sys ) ; then
	261	echo "IGCM_sys_MkdirArchive :" $@
	262	fi
[717]	263	#- creation de repertoire sur le serveur fichier
[657]	264	if [ ! -d ${1} ]; then
[1178]	265	\mkdir -p $1
[811]	266	status=$?
	267
	268	if [ ${status} -gt 0 ] ; then
	269	IGCM_debug_Print 2 "IGCM_sys_MkdirArchive : mkdir failed error code ${status}"
[657]	270	IGCM_debug_Exit "IGCM_sys_MkdirArchive"
[16]	271	fi
[657]	272	fi
	273	IGCM_debug_PopStack "IGCM_sys_MkdirArchive"
[16]	274	}
	275
	276	#D-#==================================================
	277	#D-function IGCM_sys_TestDirArchive
	278	#D-* Purpose: Test Directory that must exists on Archive
	279	#D-* Examples:
	280	#D-
	281	function IGCM_sys_TestDirArchive {
[657]	282	IGCM_debug_PushStack "IGCM_sys_TestDirArchive" $@
	283	if ( $DEBUG_sys ) ; then
	284	echo "IGCM_sys_TestDirArchive :" $@
	285	fi
	286	typeset ExistFlag
	287	ExistFlag=$( [ -d $1 ] && echo 0 \|\| echo 1 )
	288	IGCM_debug_PopStack "IGCM_sys_TestDirArchive"
	289	return ${ExistFlag}
[16]	290	}
	291
	292	#D-#==================================================
[800]	293	#D-function IGCM_sys_IsFileArchived
[1180]	294	#D-* Purpose: Test file that must NOT EXISTS on Archive based on filename only
[800]	295	#D-* Examples:
	296	#D-
	297	function IGCM_sys_IsFileArchived {
	298	IGCM_debug_PushStack "IGCM_sys_IsFileArchived" $@
	299	if ( $DEBUG_sys ) ; then
	300	echo "IGCM_sys_IsFileArchived :" $@
	301	fi
	302	typeset IsArchivedFlag
	303	# Never archived for this system
	304	IsArchivedFlag=1
	305	IGCM_debug_PopStack "IGCM_sys_IsFileArchived"
	306
	307	return ${IsArchivedFlag}
	308	}
	309
	310	#D-#==================================================
[16]	311	#D-function IGCM_sys_TestFileArchive
	312	#D-* Purpose: Test file that must NOT EXISTS on Archive
	313	#D-* Examples:
	314	#D-
	315	function IGCM_sys_TestFileArchive {
[657]	316	IGCM_debug_PushStack "IGCM_sys_TestFileArchive" $@
	317	typeset ExistFlag
	318	ExistFlag=$( [ -f $1 ] && echo 0 \|\| echo 1 )
	319	IGCM_debug_PopStack "IGCM_sys_TestFileArchive"
[67]	320
[657]	321	return ${ExistFlag}
[67]	322	}
	323
	324	#D-#==================================================
	325	#D-function IGCM_sys_CountFileArchive
	326	#D-* Purpose: Count files on Archive filesystem
	327	#D-* Examples:
	328	#D-
	329	function IGCM_sys_CountFileArchive {
[657]	330	IGCM_debug_PushStack "IGCM_sys_CountFileArchive" $@
	331	ls ${@} 2>/dev/null \| wc -l
	332	if [ $? -gt 0 ] ; then
	333	echo "IGCM_sys_CountFileArchive : erreur."
	334	fi
	335	IGCM_debug_PopStack "IGCM_sys_CountFileArchive"
[16]	336	}
	337
	338	#D-#==================================================
	339	#D-function IGCM_sys_Tree
	340	#D-* Purpose: Tree directories with files on ${ARCHIVE}
	341	#D-* Examples: IGCM_sys_Tree ${R_IN} ${R_OUT}
	342	#D-
	343	function IGCM_sys_Tree {
[657]	344	IGCM_debug_PushStack "IGCM_sys_Tree" $@
	345	if ( $DEBUG_sys ) ; then
	346	echo "IGCM_sys_Tree :" $@
	347	fi
[16]	348
[657]	349	\tree -f $@
[16]	350
[657]	351	IGCM_debug_PopStack "IGCM_sys_Tree"
[16]	352	}
	353
	354	#D-#==================================================
	355	#D-function IGCM_sys_Qsub
	356	#D-* Purpose: Qsub new job
	357	#D-* Examples:
	358	#D-
	359	function IGCM_sys_Qsub {
[657]	360	IGCM_debug_PushStack "IGCM_sys_Qsub" $@
	361	if ( $DEBUG_sys ) ; then
	362	echo "IGCM_sys_Qsub :" $@
	363	fi
[923]	364	typeset options status
	365	options="-o ${SUBMIT_DIR}/${Script_Output}"
[1179]	366	/usr/local/bin/qsub ${options} $1 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[811]	367	status=$?
[949]	368
[1179]	369	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[811]	370	if [ ${status} -gt 0 ] ; then
[1180]	371	IGCM_debug_Print 2 "IGCM_sys_Qsub ${options} $1 : error code ${status}"
[657]	372	IGCM_debug_Exit "IGCM_sys_Qsub"
[949]	373	else
[1179]	374	JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
[657]	375	fi
	376	IGCM_debug_PopStack "IGCM_sys_Qsub"
[16]	377	}
	378
	379	#D-#==================================================
	380	#D-function IGCM_sys_QsubPost
	381	#D-* Purpose: Qsub new job on scalaire
	382	#D-* Examples:
	383	#D-
	384	function IGCM_sys_QsubPost {
[657]	385	IGCM_debug_PushStack "IGCM_sys_QsubPost" $@
	386	if ( $DEBUG_sys ) ; then
	387	echo "IGCM_sys_QsubPost :" $@
	388	fi
[923]	389	typeset options status
	390	options="-o ${POST_DIR}/${Script_Post_Output}.out -v ${listVarEnv}"
[1179]	391	/usr/local/bin/qsub ${options} ${libIGCM_POST}/$1.job > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[811]	392	status=$?
[949]	393
[1179]	394	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[811]	395	if [ ${status} -gt 0 ] ; then
[923]	396	IGCM_debug_Print 2 "IGCM_sys_QsubPost ${options} ${libIGCM_POST}/$1.job : error code ${status}"
[657]	397	IGCM_debug_Exit "IGCM_sys_QsubPost"
[949]	398	else
[1179]	399	JobID=$( gawk {'print $1'} ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ )
[657]	400	fi
	401	IGCM_debug_PopStack "IGCM_sys_QsubPost"
[16]	402	}
	403
	404	#D-*************************
[657]	405	#D- File transfer functions
[16]	406	#D-*************************
	407	#D-
	408
	409	#D-#==================================================
[203]	410	#D-function IGCM_sys_RmRunDir
	411	#D-* Purpose: rm tmpdir (dummy function most of the time batch
	412	#D- scheduler will do the job)
	413	#D-* Examples:
	414	#D-
	415	function IGCM_sys_RmRunDir {
[1057]	416	IGCM_debug_PushStack "IGCM_sys_RmRunDir" $@
[657]	417	if ( $DEBUG_sys ) ; then
	418	echo "IGCM_sys_RmRunDir :" $@
[964]	419	echo "Dummy call, let the scheduler do that."
[657]	420	fi
	421	IGCM_debug_PopStack "IGCM_sys_RmRunDir"
[203]	422	}
	423
	424	#D-#==================================================
[16]	425	#D-function IGCM_sys_Put_Dir
	426	#D-* Purpose: Copy a complete directory on $(ARCHIVE)
	427	#D-* Examples:
	428	#D-
	429	function IGCM_sys_Put_Dir {
[657]	430	IGCM_debug_PushStack "IGCM_sys_Put_Dir" $@
	431	if ( $DEBUG_sys ) ; then
	432	echo "IGCM_sys_Put_Dir :" $@
	433	fi
	434	if [ $DRYRUN = 0 ]; then
	435	if [ ! -d ${1} ] ; then
	436	echo "WARNING : IGCM_sys_Put_Dir ${1} DOES NOT EXIST ."
	437	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
	438	return
[16]	439	fi
	440
[717]	441	typeset status
[16]	442
[717]	443	# Only if we use rsync
	444	#IGCM_sys_TestDirArchive $( dirname $2 )
	445	#
	446	#USUAL WAY
[1179]	447	\cp -r $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[717]	448	status=$?
[16]	449
[717]	450	if [ ${status} -gt 0 ] ; then
[811]	451	IGCM_debug_Print 2 "IGCM_sys_Put_Dir : cp failed error code ${status}"
[1179]	452	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	453	IGCM_debug_Exit "IGCM_sys_Put_Dir"
[16]	454	else
[1179]	455	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[16]	456	fi
[657]	457	fi
	458	IGCM_debug_PopStack "IGCM_sys_Put_Dir"
[16]	459	}
	460
	461	#D-#==================================================
	462	#D-function IGCM_sys_Get_Dir
[717]	463	#D-* Purpose: Copy a complete directory from ${ARCHIVE}
[16]	464	#D-* Examples:
	465	#D-
	466	function IGCM_sys_Get_Dir {
[657]	467	IGCM_debug_PushStack "IGCM_sys_Get_Dir" $@
	468	if ( $DEBUG_sys ) ; then
	469	echo "IGCM_sys_Get_Dir :" $@
	470	fi
	471	if [ $DRYRUN = 0 ]; then
[1180]	472	typeset NB_ESSAI DELAI status i
	473	# number of tentative
	474	NB_ESSAI=3
	475	# time delay between tentative
	476	DELAI=2
[16]	477
[1180]	478	#
	479	# USUAL WAY
	480	i=0
	481	while [ $i -lt $NB_ESSAI ] ; do
	482	\cp -ur $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	483	status=$?
	484	if [ ${status} -gt 0 ] ; then
	485	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status} ${i}/${NB_ESSAI}"
	486	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : sleep ${DELAI} seconds and try again."
	487	sleep $DELAI
	488	else
	489	break
	490	fi
	491	(( i = i + 1 ))
	492	done
[16]	493
[717]	494	if [ ${status} -gt 0 ] ; then
[811]	495	IGCM_debug_Print 2 "IGCM_sys_Get_Dir : cp failed error code ${status}"
[1179]	496	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	497	IGCM_debug_Exit "IGCM_sys_Get_Dir"
[16]	498	else
[1179]	499	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[16]	500	fi
[657]	501	fi
	502	IGCM_debug_PopStack "IGCM_sys_Get_Dir"
[16]	503	}
	504
	505	#D-#==================================================
	506	#D-function IGCM_sys_Put_Rest
[717]	507	#D-* Purpose: Put computied restarts on ${ARCHIVE}.
[282]	508	#D- File and target directory must exist.
[16]	509	#D-* Examples:
	510	#D-
	511	function IGCM_sys_Put_Rest {
[657]	512	IGCM_debug_PushStack "IGCM_sys_Put_Rest" $@
	513	if ( $DEBUG_sys ) ; then
	514	echo "IGCM_sys_Put_Rest :" $@
	515	fi
	516	if [ $DRYRUN = 0 ]; then
	517	if [ ! -f ${1} ] ; then
	518	echo "ERROR : IGCM_sys_Put_Rest ${1} DOES NOT EXIST ."
	519	IGCM_debug_Exit "IGCM_sys_Put_Rest"
[16]	520	fi
[717]	521
	522	typeset status
	523	#
[1178]	524	# USUAL WAY
[1179]	525	\cp $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[717]	526	status=$?
[16]	527
[717]	528	# #RSYNC WITH NETWORK SSH CALL
[1179]	529	# echo ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	530	# ${RSYNC} ${RSYNC_opt} -e ssh ${RUN_DIR}/$1 ${STOREHOST}:${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[16]	531
[717]	532	# #RSYNC WITH NFS USE
[1179]	533	# echo ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	534	# ${RSYNC} ${RSYNC_opt} ${RUN_DIR}/$1 ${2} >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[16]	535
[717]	536	# status=$?
	537	# IGCM_sys_Rsync_out $status
	538
[1179]	539	# ${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[717]	540	# (( status=status+$? ))
	541
	542	if [ ${status} -gt 0 ] ; then
[1180]	543	IGCM_debug_Print 2 "IGCM_sys_Put_Rest : cp failed error code ${status}"
[1178]	544	[ -f ${1} ] && ls -l ${1}
	545	[ -f ${2} ] && ls -l ${2}
	546	[ -f ${2}/${1} ] && ls -l ${2}/${1}
[1179]	547	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	548	IGCM_debug_Exit "IGCM_sys_Put_Rest"
	549	else
[1178]	550
	551	if [ X${JobType} = XRUN ] ; then
	552	[ -f ${2} ] && IGCM_sys_Chmod 444 ${2}
	553	[ -f ${2}/${1} ] && IGCM_sys_Chmod 444 ${2}/${1}
	554	fi
	555
[1179]	556	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	557	fi
	558	fi
	559	IGCM_debug_PopStack "IGCM_sys_Put_Rest"
	560	}
[16]	561
[657]	562	#D-#==================================================
[16]	563	#D-function IGCM_sys_Put_Out
[717]	564	#D-* Purpose: Copy a file on ${ARCHIVE} after having chmod it in readonly
[16]	565	#D-* Examples:
	566	#D-
	567	function IGCM_sys_Put_Out {
[657]	568	IGCM_debug_PushStack "IGCM_sys_Put_Out" $@
	569	if ( $DEBUG_sys ) ; then
	570	echo "IGCM_sys_Put_Out :" $@
	571	fi
[811]	572
	573	typeset status
	574
[657]	575	if [ $DRYRUN = 0 ]; then
[811]	576	if [ ! -f ${1} ] ; then
[657]	577	echo "WARNING : IGCM_sys_Put_Out ${1} DOES NOT EXIST ."
	578	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	579	return 1
[16]	580	fi
[719]	581	#
[811]	582	IGCM_sys_MkdirArchive $( dirname $2 )
	583	#
[657]	584	if [ X${JobType} = XRUN ] ; then
	585	if [ X${3} = X ] ; then
	586	IGCM_sys_Chmod 444 ${1}
	587	fi
	588	fi
[717]	589	#
[16]	590
[1179]	591	echo ${RSYNC} ${RSYNC_opt} $1 $2 > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
	592	${RSYNC} ${RSYNC_opt} $1 $2 >> ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[717]	593	status=$?
	594	IGCM_sys_Rsync_out $status
[16]	595
[1179]	596	${libIGCM}/libIGCM_sys/IGCM_analyse_rsync_out.awk ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[717]	597	(( status=status+$? ))
[16]	598
[717]	599	if [ ${status} -gt 0 ] ; then
[811]	600	IGCM_debug_Print 2 "IGCM_sys_Put_Out : rsync failed error code ${status}"
[1179]	601	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	602	IGCM_debug_Exit "IGCM_sys_Put_Out"
[16]	603	else
[1179]	604	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[16]	605	fi
[657]	606	fi
	607	IGCM_debug_PopStack "IGCM_sys_Put_Out"
	608	return 0
[16]	609	}
	610
	611	#D-#==================================================
	612	#D-function IGCM_sys_Get
	613	#D-* Purpose: Get a file from ${ARCHIVE}
	614	#D-* Examples: IGCM_sys_Get myfile /destpath/myfile_with_PREFIX
	615	#D- IGCM_sys_Get /l Array_contain_myfiles /destpath/
	616	function IGCM_sys_Get {
[657]	617	IGCM_debug_PushStack "IGCM_sys_Get" $@
[16]	618
[811]	619	typeset DEST status dm_liste
[16]	620
[657]	621	if ( $DEBUG_sys ) ; then
	622	echo "IGCM_sys_Get :" $@
	623	fi
	624	if [ $DRYRUN -le 2 ]; then
	625	if [ X${1} = X'/l' ] ; then
	626	eval set +A dm_liste \${${2}}
	627	else
	628	dm_liste=${1}
[16]	629	fi
[657]	630	eval DEST=\${${#}}
[16]	631
[717]	632	# test if the (first) file is present in the old computation :
[657]	633	IGCM_sys_TestFileArchive ${dm_liste[0]}
[717]	634	status=$?
	635	if [ ${status} -gt 0 ] ; then
[657]	636	echo "IGCM_sys_Get, ERROR : regular file ${dm_liste[0]} DOES NOT EXIST ."
	637	IGCM_debug_Exit "IGCM_sys_Get"
[811]	638	return
[657]	639	fi
[16]	640
[811]	641	#USUAL WAY
[1179]	642	\cp ${dm_liste[*]} ${DEST} > ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$ 2>&1
[811]	643	status=$?
[16]	644
[811]	645	if [ ${status} -gt 0 ] ; then
	646	IGCM_debug_Print 2 "IGCM_sys_Get : cp failed error code ${status}"
[1179]	647	cat ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[657]	648	IGCM_debug_Exit "IGCM_sys_Get"
[16]	649	else
[1179]	650	\rm ${OUTCOMMAND_PATH}/out_command_${LOGIN}.$$
[16]	651	fi
[657]	652	fi
	653	IGCM_debug_PopStack "IGCM_sys_Get"
[16]	654	}
	655
	656	#D-#==================================================
[1131]	657	#D-function IGCM_sys_GetDate_Monitoring
	658	#D-* Purpose: get the last year for which the monitoring has been computed
	659	#D-* Examples:
	660	#D-
	661	function IGCM_sys_GetDate_Monitoring {
	662	IGCM_debug_PushStack "IGCM_sys_GetDate_Monitoring" $@
	663	if ( $DEBUG_sys ) ; then
	664	echo "IGCM_sys_GetDate_Monitoring :" $@
	665	fi
	666
	667	eval ${2}=$( cdo showyear ${1} 2> /dev/null \| gawk '{ print $NF }' )
	668
	669	IGCM_debug_PopStack "IGCM_sys_GetDate_Monitoring"
	670	}
	671
	672	#D-#==================================================
[343]	673	#D-function IGCM_sys_Dods_Rm
[717]	674	#D-* Purpose: DO NOTHING ! Put ${ARCHIVE} files on DODS internet protocole.
[343]	675	#D-* Examples:
	676	#D-
	677	function IGCM_sys_Dods_Rm {
[657]	678	if ( $DEBUG_sys ) ; then
	679	echo "IGCM_sys_Dods_Rm :" $@
	680	fi
	681	return 0
[343]	682	}
	683
	684	#D-#==================================================
	685	#D-function IGCM_sys_Dods_Cp
[717]	686	#D-* Purpose: Copy $(ARCHIVE) files on DODS internet protocole.
[343]	687	#D-* Examples:
	688	#D-
	689	function IGCM_sys_Dods_Cp {
[657]	690	if ( $DEBUG_sys ) ; then
	691	echo "IGCM_sys_Dods_Cp :" $@
	692	fi
	693	return 0
[343]	694	}
	695
	696	#D-#==================================================
[16]	697	#D-function IGCM_sys_Put_Dods
[811]	698	#D-* Purpose: Put ${ARCHIVE} files on DODS internet protocole. Dummy function here
[16]	699	#D-* Examples:
	700	#D-
	701	function IGCM_sys_Put_Dods {
[657]	702	IGCM_debug_PushStack "IGCM_sys_Put_Dods" $@
	703	if ( $DEBUG_sys ) ; then
	704	echo "IGCM_sys_Put_Dods :" $@
	705	fi
	706	IGCM_debug_PopStack "IGCM_sys_Put_Dods"
[16]	707	}
	708
[657]	709	##############################################################
	710	# REBUILD OPERATOR
[536]	711
[811]	712	#D-#==================================================
[1163]	713	#D-function IGCM_sys_sync
	714	#D-* Purpose: flush buffer on disk (dummy function on Ada)
	715	#D-* Examples:
	716	#D-
	717	function IGCM_sys_sync {
	718	IGCM_debug_PushStack "IGCM_sys_sync" $@
	719	if ( $DEBUG_sys ) ; then
	720	echo "IGCM_sys_sync :" $@
	721	echo "Dummy call, let the system do that."
	722	fi
	723	IGCM_debug_PopStack "IGCM_sys_sync"
	724	}
	725
[657]	726	############################################################
	727	# Activate Running Environnment Variables
[536]	728
[811]	729	#D-#==================================================
[1239]	730	#D-function IGCM_sys_activ_variables
[811]	731	#D-* Purpose: set environement variables prior to execution
	732	#D-* Examples:
	733	#D-
[657]	734	function IGCM_sys_activ_variables {
	735	IGCM_debug_PushStack "IGCM_sys_activ_variables"
	736	if ( $DEBUG_sys ) ; then
	737	echo "IGCM_sys_activ_variables"
	738	fi
[811]	739
	740	# --------------------------------------------------------------------
	741	#D- MPI specifications
	742	# --------------------------------------------------------------------
	743
	744	# --------------------------------------------------------------------
	745	#D- Other specifications
	746	# --------------------------------------------------------------------
	747
[657]	748	IGCM_debug_PopStack "IGCM_sys_activ_variables"
[16]	749	}
	750
[657]	751	############################################################
	752	# Desactivate Running Environnment Variables
[16]	753
[811]	754	#D-#==================================================
	755	#D-function IGCM_sys_desactiv_variables
	756	#D-* Purpose: unset environement variables after execution
	757	#D-* Examples:
	758	#D-
[657]	759	function IGCM_sys_desactiv_variables {
	760	IGCM_debug_PushStack "IGCM_sys_desactiv_variables"
	761	if ( $DEBUG_sys ) ; then
	762	echo "IGCM_sys_desactiv_variables"
	763	fi
[811]	764	# --------------------------------------------------------------------
	765	#D- MPI specifications
	766	# --------------------------------------------------------------------
	767
	768	# --------------------------------------------------------------------
	769	#D- Other specifications
	770	# --------------------------------------------------------------------
	771
[657]	772	IGCM_debug_PopStack "IGCM_sys_desactiv_variables"
	773	}
	774
	775	############################################################
[1239]	776	# Update job headers to be used by the scheduler
	777
	778	#D-#==================================================
	779	#D-function IGCM_sys_updateHeaders
	780	#D-* Purpose: Update job headers to be used by the scheduler
	781	#D-* Examples: IGCM_sys_updateHeaders /path/to/Job_MYEXP
	782	#D-
	783	function IGCM_sys_updateHeaders {
	784	IGCM_debug_PushStack "IGCM_sys_updateHeaders"
	785	if ( $DEBUG_sys ) ; then
	786	echo "IGCM_sys_updateHeaders"
	787	fi
	788	typeset file
	789	file=$1
	790
	791	if [ ${executionType} -eq 1 ] ; then
	792	# MPMD + MPI
	793	sed -e "/::openMPthreads::/d" \
	794	-e "s/::JobNumProcTot::/${coreNumber}/" \
	795	${file} > ${file}.tmp
	796
	797	elif [ ${executionType} -eq 2 ] ; then
	798	# MPMD + MPI + OMP
	799	sed -e "s/::openMPthreads::/${openMPthreads}/" \
	800	-e "s/::JobNumProcTot::/${coreNumber}/" \
	801	${file} > ${file}.tmp
	802
	803	elif [ ${executionType} -eq 3 ] ; then
	804	# SPMD + MPI/OMP
	805	sed -e "s/::openMPthreads::/${openMPthreads}/" \
	806	-e "s/::JobNumProcTot::/${mpiTasks}/" \
	807	${file} > ${file}.tmp
	808
	809	elif [ ${executionType} -eq 4 ] ; then
	810	# SPMD + MPI only
	811	sed -e "s/::JobNumProcTot::/${mpiTasks}/" \
	812	-e "/::openMPthreads::/d" \
	813	${file} > ${file}.tmp
	814
	815	elif [ ${executionType} -eq 5 ] ; then
	816	# SPMD + OMP only
	817	sed -e "s/::openMPthreads::/${openMPthreads}/" \
	818	-e "/::JobNumProcTot::/d" \
	819	${file} > ${file}.tmp
	820
	821	elif [ ${executionType} -eq 6 ] ; then
	822	# SEQUENTIAL THEN
	823	sed -e "s/::JobNumProcTot::/1/" \
	824	-e "/::openMPthreads::/d" \
	825	${file} > ${file}.tmp
	826
	827	fi
	828
	829	IGCM_sys_Mv ${file}.tmp ${file}
	830
	831	IGCM_debug_PopStack "IGCM_sys_updateHeaders"
	832	}
	833
	834	############################################################
[800]	835	# Build MPI/OMP scripts run file (dummy function)
[657]	836
[811]	837	#D-#==================================================
	838	#D-function IGCM_sys_build_run_file
	839	#D-* Purpose: build run file (deprecated)
	840	#D-* Examples:
	841	#D-
[657]	842	function IGCM_sys_build_run_file {
	843
[664]	844	IGCM_debug_Print 3 " dummy function : IGCM_sys_build_run_file "
[657]	845
	846	}
	847
	848	############################################################
	849	# Build MPI/OMP scripts
[811]	850
	851	#D-#==================================================
	852	#D-function IGCM_sys_build_execution_scripts
	853	#D-* Purpose: build execution scripts to be launch by ${HOST_MPIRUN_COMMAND}
	854	#D-* Examples:
	855	#D-
[657]	856	function IGCM_sys_build_execution_scripts
	857	{
	858	IGCM_debug_PushStack "IGCM_sys_build_execution_scripts" $@
	859	if ( $DEBUG_sys ) ; then
	860	echo "IGCM_sys_build_execution_scripts " $@
	861	fi
[1180]	862
[657]	863	typeset nodes listnodes init_node start_num init_exec comp ExeNameIn ExeNameOut
	864	typeset node_num_current node_current comp_proc_mpi_loc comp_proc_omp_loc
	865	typeset num_corempi nombre_restant_node nombre_restant_comp
	866
	867	if ( ${OK_PARA_MPMD} ) ; then
	868
	869	if [ -f run_file ] ; then
	870	IGCM_sys_Rm -f run_file
[16]	871	fi
[657]	872	touch run_file
	873
	874	if ( ${OK_PARA_OMP} ) ; then
	875
	876	# Hosts treatment
	877
	878	${HOST_MPIRUN_COMMAND} hostname \| sort \| uniq > hosts.tmp
	879
	880	i=0
	881	rm -f hosts
	882	IGCM_debug_Print 1 "sys Obelix, Hosts avaible :"
	883	for nodes in `cat hosts.tmp` ; do
	884	host[$i]=$nodes
	885	echo "${host[$i]} slots=1 max_slots=1" >> hosts
	886	IGCM_debug_Print 1 ${host[$i]}
	887	i=$((i+1))
	888	done
	889	rm -f hosts.tmp
	890
	891	listnodes=${host[*]}
	892
	893	EXECUTION="${HOST_MPIRUN_COMMAND} -hostfile hosts"
	894
	895	# Initialisation
	896
	897	init_node=y
	898	node_num_current=0
	899	start_num=0
	900	init_exec=n
	901
[1180]	902	# Build run_file
[657]	903
[1180]	904	# First loop on the components for the coupler ie oasis (only if oasis3)
	905	# the coupler ie oasis3 must be the first one
[657]	906	for comp in ${config_ListOfComponents[*]} ; do
	907
	908	if [ "X${comp}" = "XCPL" ] ; then
	909
	910	eval ExeNameIn=\${config_Executable_${comp}[0]}
	911	eval ExeNameOut=\${config_Executable_${comp}[1]}
	912
	913	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	914	echo "" >> script_${ExeNameOut}.ksh
	915	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
	916	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
	917	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
	918	echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
	919	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	920
	921	init_node=n
	922
	923	(( nombre_restant_node = NUM_COREPERNODE - 1 ))
	924	node_num_current=0
	925	node_current=${host[${node_num_current}]}
	926
	927	EXECUTION="${EXECUTION} -H ${node_current} -np 1 ./script_${ExeNameOut}.ksh"
	928
	929	init_exec=y
	930	start_num=1
	931
	932	fi
	933
	934	done
	935
	936	# Then loop on the components (except for oasis)
	937
	938	for comp in ${config_ListOfComponents[*]} ; do
	939
	940	eval ExeNameIn=\${config_Executable_${comp}[0]}
	941	eval ExeNameOut=\${config_Executable_${comp}[1]}
	942
	943	# Only if we really have an executable for the component :
	944	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	945
	946	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	947	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	948
	949	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	950	# echo "set -vx" >> script_${ExeNameOut}.ksh
	951	echo "" >> script_${ExeNameOut}.ksh
	952	#echo "export KMP_STACKSIZE=3g" >> script_${ExeNameOut}.ksh
	953	#echo "export KMP_LIBRARY=turnaround" >> script_${ExeNameOut}.ksh
	954	#echo "export MKL_SERIAL=YES" >> script_${ExeNameOut}.ksh
	955	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
	956	echo "(( MYMPIRANK = OMPI_COMM_WORLD_RANK - ${start_num})) " >> script_${ExeNameOut}.ksh
	957	echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${MYMPIRANK} 2>out_${ExeNameOut}.err.\${MYMPIRANK}" >> script_${ExeNameOut}.ksh
	958	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	959
	960	node_num=0
	961
	962	# We define the number of MPI process to be assigned for the component
	963
	964	nombre_restant_comp=${comp_proc_mpi_loc}
	965
	966	# Loop on the allocated nodes
	967
	968	for node in ${listnodes} ; do
	969
	970	# We go to the current node
	971	if [ ${node_num} = ${node_num_current} ] ; then
	972
	973	node_current=${host[${node_num_current}]}
	974
	975	# If first time on the node : initialisation
	976
	977	if [ ${init_node} = y ] ; then
	978	nombre_restant_node=${NUM_COREPERNODE}
	979	fi
	980
	981	# Test on the number of OMP threads
	982
	983	if [ ${comp_proc_omp_loc} -gt ${nombre_restant_node} ] ; then
	984	(( node_num = node_num + 1 ))
	985	node_num_current=${node_num}
	986	init_node=y
	987	continue
	988	fi
	989
	990	# Number of MPI process to assign
	991
	992	(( num_corempi = nombre_restant_node / comp_proc_omp_loc ))
	993
	994	if [ ${num_corempi} -gt ${nombre_restant_comp} ] ; then
	995	num_corempi=${nombre_restant_comp}
	996	fi
	997
	998	(( nombre_restant_node = nombre_restant_node - num_corempi * comp_proc_omp_loc ))
	999	(( nombre_restant_comp = nombre_restant_comp - num_corempi ))
	1000
	1001	if [ ${init_exec} = y ] ; then
	1002	EXECUTION="${EXECUTION} : -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
	1003	else
	1004	EXECUTION="${EXECUTION} -H ${node_current} -np ${num_corempi} ./script_${ExeNameOut}.ksh"
	1005	init_exec=y
	1006	fi
	1007
	1008	(( start_num = num_corempi + start_num ))
	1009
	1010	else
	1011
	1012	(( node_num = node_num + 1 ))
	1013	continue
	1014	fi
	1015
	1016	# Test on the number of core/process remaining on the node/component
	1017
	1018	if [ ${nombre_restant_node} = 0 ] ; then
	1019	(( node_num = node_num + 1 ))
	1020	node_num_current=${node_num}
	1021	init_node=y
	1022
	1023	if [ ${nombre_restant_comp} = 0 ] ; then
	1024	break 1
	1025	fi
	1026	else
	1027
	1028	node_num_current=${node_num}
	1029	init_node=n
	1030
	1031	if [ ${nombre_restant_comp} = 0 ] ; then
	1032	break 1
	1033	fi
	1034	fi
	1035	done
	1036	fi
	1037	done
	1038
	1039	else
	1040
[664]	1041	# Then first loop on the components for the coupler ie oasis
[657]	1042
[664]	1043	## the coupler ie oasis must be the first one
	1044	for comp in ${config_ListOfComponents[*]} ; do
[657]	1045
[664]	1046	eval ExeNameOut=\${config_Executable_${comp}[1]}
[657]	1047
	1048	# for CPL component only
[664]	1049	if [ "X${comp}" = "XCPL" ] ; then
	1050	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1051	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut} " >> run_file
	1052	fi
	1053	done
[657]	1054
[811]	1055	# Then second loop on the components
[657]	1056
[664]	1057	for comp in ${config_ListOfComponents[*]} ; do
[657]	1058
[664]	1059	eval ExeNameOut=\${config_Executable_${comp}[1]}
[657]	1060
[664]	1061	# Only if we really have an executable for the component and not the coupler ie oasis:
	1062	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${comp}" != "XCPL" ] ) ; then
	1063	eval comp_proc_mpi_loc=\${${comp}_PROC_MPI}
	1064	echo "-np ${comp_proc_mpi_loc} ./${ExeNameOut}" >> run_file
	1065	fi
	1066	done
	1067	IGCM_sys_Chmod u+x run_file
[657]	1068
[664]	1069	EXECUTION="${HOST_MPIRUN_COMMAND} --app ./run_file"
[657]	1070
[16]	1071	fi
	1072
[657]	1073	else # Only one executable. launch it.
	1074
[664]	1075	for comp in ${config_ListOfComponents[*]} ; do
[657]	1076
[664]	1077	# Only if we really have an executable for the component :
	1078	eval ExeNameOut=\${config_Executable_${comp}[1]}
	1079	if ( [ "X${ExeNameOut}" != X\"\" ] && [ "X${ExeNameOut}" != "Xinca.dat" ] ) ; then
[657]	1080
[664]	1081	echo "#!/bin/ksh" > script_${ExeNameOut}.ksh
	1082	echo "" >> script_${ExeNameOut}.ksh
	1083	if ( ${OK_PARA_OMP} ) ; then
	1084	eval comp_proc_omp_loc=\${${comp}_PROC_OMP}
	1085	echo "OMP_NUM_THREADS=${comp_proc_omp_loc}" >> script_${ExeNameOut}.ksh
	1086	fi
	1087	if ( ${OK_PARA_MPI} ) ; then
	1088	# Default : mpirun used if nb_proc gt 1
	1089	# pour sortie out/err par process
	1090	# echo "./${ExeNameOut} > out_${ExeNameOut}.out.\${OMPI_COMM_WORLD_RANK} 2>out_${ExeNameOut}.err.\${OMPI_COMM_WORLD_RANK}" >> script_${ExeNameOut}.ksh
	1091	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1092	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1093	EXECUTION="${HOST_MPIRUN_COMMAND} ./script_${ExeNameOut}.ksh"
	1094	else
	1095	# Default : mpirun is NOT used if nb_proc eq 1
	1096	# pour sortie out/err par process
	1097	# echo "./${ExeNameOut} > out_${ExeNameOut}.out 2>out_${ExeNameOut}.err" >> script_${ExeNameOut}.ksh
	1098	echo "./${ExeNameOut}" >> script_${ExeNameOut}.ksh
	1099	IGCM_sys_Chmod u+x script_${ExeNameOut}.ksh
	1100	EXECUTION="time ./script_${ExeNameOut}.ksh"
	1101	fi
[1002]	1102
	1103	IGCM_debug_Print 1 "sys Obelix : script_${ExeNameOut}.ksh contains"
	1104	cat script_${ExeNameOut}.ksh
	1105
[664]	1106	fi
[1180]	1107
[664]	1108	done
[657]	1109
	1110	fi
	1111
[811]	1112	IGCM_debug_Print 1 "sys Obelix : execution command is"
	1113	IGCM_debug_Print 1 "$EXECUTION"
[657]	1114
	1115	IGCM_debug_PopStack "IGCM_sys_build_execution_scripts"
[16]	1116	}
	1117
[933]	1118	#D-#==================================================
	1119	#D-function IGCM_sys_check_path
	1120	#D-* Purpose: check that RUN_DIR_PATH that will be removed on some machine
	1121	#D-* do not point to an important use directory. Stop immediately in that case.
	1122	#D-* Examples:
	1123	#D-
	1124	function IGCM_sys_check_path {
	1125	IGCM_debug_PushStack "IGCM_sys_check_path"
	1126	if ( $DEBUG_sys ) ; then
	1127	echo "IGCM_sys_check_path"
	1128	fi
[811]	1129
[933]	1130	if ( [ X${RUN_DIR_PATH} = X${HOME} ] \|\| [ X${RUN_DIR_PATH} = X${ARCHIVE} ] ) ; then
	1131	IGCM_debug_Print 1 "Variable RUN_DIR_PATH is pointing to an important directory : ${RUN_DIR_PATH}"
	1132	IGCM_debug_Print 1 "Please check the RUN_DIR_PATH definition in your Job : Job_${config_UserChoices_JobName}"
[936]	1133	IGCM_debug_Exit "This will stop the job"
[933]	1134	fi
	1135	IGCM_debug_PopStack "IGCM_sys_check_path"
	1136	}
	1137
[811]	1138	#D-#==================================================
	1139	#D-function IGCM_sys_check_quota. Dummy call here
	1140	#D-* Purpose: check user quota. Stop the simulation if quota above 90%
	1141	#D-* Examples:
	1142	#D-
[657]	1143	function IGCM_sys_check_quota {
	1144	IGCM_debug_PushStack "IGCM_sys_check_quota"
	1145	if ( $DEBUG_sys ) ; then
	1146	echo "IGCM_sys_check_quota"
	1147	fi
	1148	IGCM_debug_PopStack "IGCM_sys_check_quota"
	1149	}
	1150
[811]	1151	#D-#==================================================
[1124]	1152	#D-function IGCM_sys_GetJobID
	1153	#D-* Purpose: Check if job_name is currently
	1154	#D- running or in queue
	1155	#D-* Examples: IGCM_sys_GetJobID ${JobName} ${TargetUsr} JobID
	1156	#D-
	1157	function IGCM_sys_GetJobID {
	1158	IGCM_debug_PushStack "IGCM_sys_GetJobID"
	1159	if ( $DEBUG_sys ) ; then
	1160	echo "IGCM_sys_GetJobID"
	1161	fi
	1162
	1163	# With -f option, the full job name is given in the last column
	1164	ID="$( qstat -u $2 \| grep -w $1 \| gawk '-F ' '{print $10}' )"
	1165
	1166	eval ${3}=${ID}
	1167	IGCM_debug_PopStack "IGCM_sys_GetJobID"
	1168	}
	1169
	1170	#D-#==================================================
[811]	1171	#D-function IGCM_sys_CountJobInQueue
	1172	#D-* Purpose: Check if job_name is currently
	1173	#D- running or in queue
	1174	#D-* Examples: IGCM_sys_CountJobInQueue ${JobName} NbRun
	1175	#D-
	1176	function IGCM_sys_CountJobInQueue {
	1177	IGCM_debug_PushStack "IGCM_sys_CountJobInQueue"
	1178	if ( $DEBUG_sys ) ; then
	1179	echo "IGCM_sys_CountJobInQueue"
	1180	fi
	1181	IGCM_debug_PopStack "IGCM_sys_CountJobInQueue"
	1182	}

Note: See TracBrowser for help on using the repository browser.

Download in other formats: