Changes between Version 28 and Version 29 of Doc/Setup


Ignore:
Timestamp:
09/24/15 16:59:44 (9 years ago)
Author:
nillod
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Doc/Setup

    v28 v29  
    2222 
    2323 
    24 Each of these subdirectories may contain a reference experiment (e.g. '''clim''' and '''amip''' for LMDZOR, NMHC_AER, AER and GES for LMDZORINCA, '''piControl''', '''historical''' and cie for IPSLCM5_v5) and the file ''config.card'' which will be your simulation's initial setup. 
     24Each of these subdirectories may contain a reference experiment (e.g. '''clim''' and '''amip''' for LMDZOR, NMHC_AER, AER and GES for LMDZORINCA, '''piControl''', '''historical''' and cie for IPSLCM5_v5) and the file ''config.card'' which will be your simulation's initial setup.  
    2525 
    2626Before preparing the working directory you must know which kind of simulation you want to perform. Then, you must copy the ''config.card'' file at the same level as the main ''Makefile''.  
    … …  
    3434[[Image(wiki:DocEsetup:creation_exp_v5.jpg, 50%)]]  
    3535 
    36 The header of ''config.card'' contains the '''!JobName''' field for which you must specify your simulation's name. Then run the '''`ins_job`''' script that will in first time ask you, if you are working on TGCC, your id group, and then create a directory for your experiment. If you are working on IDRIS it will directly create a directory for your experiment.  
     36The header of ''config.card'' contains the '''!JobName''' field for which you must specify your simulation's  name. Then run the '''`ins_job`''' script that will in first time ask you, if you are working on TGCC, your id group, and then create a directory for your experiment. If you are working on IDRIS it will directly create a directory for your experiment.  
    3737[[BR]] In the previous example, a simulation called ''DIADEME'' is created: 
    3838{{{ 
    … …  
    4242ls  
    4343  AA_Make Makefile EXPERIMENTS GENERAL config.card 
    44 vi config.card      # Change JobName=DIADEME 
    45                     # Change [Executable] section to specify your executionType. see config.card description 
     44vi config.card    # Change JobName=DIADEME 
    4645../../util/ins_job  # Check and complet job's header 
    4746ls  
    4847  AA_Make Makefile EXPERIMENTS GENERAL DIADEME 
    4948}}} 
    50 A directory called ''DIADEME'' is created and the config.card file is moved in. 
     49The ''config.card'' file is deleted and a directory called DIADEME is created.  
     50 
    5151 
    5252---- 
    … …  
    7777 * !DateEnd --> simulation end date. It must be the last day "included" in your simulation 
    7878 * !PeriodLength --> frequency of the executable run. This parameter can be 1M, 1Y or 10Y 
    79  * !JobNumProcTot --> number of processors required by your simulation. 
     79 * !JobNumProcTot --> number of processors required by your simulation. This parameter depends on the configurations and on the resolutions. For example, a configuration containing LMDZ in 96x95 can use 32 processors at the most. 
    8080 * ARCHIVE --> optional : path to base directory for output files. By default this is set in libIGCM depending on the machine. 
    8181 
    … …  
    8383[[BR]] [[BR]] 
    8484 
    85 '''Example 1: The output directory for the following case will be IGCM_OUT/LMDZOR/TEST/REINE/DIADEME''' 
     85Example 1: The output directory for the following case will be IGCM_OUT/LMDZOR/TEST/REINE/DIADEME 
    8686{{{ 
    8787#!sh  
    … …  
    9494 
    9595 
    96 '''Example 2: without !ExperimentName and !SpaceName''' 
     96Example 2: without !ExperimentName and !SpaceName 
    9797{{{ 
    9898#!sh  
    … …  
    102102The output directory will be IGCM_OUT/LMDZOR/DIADEME 
    103103 
     104 
    104105[[NoteBox(warn,The character "_" is not allowed in the variables !JobName\, !ExperimentName and !SpaceName, 600px)]] 
    105106 
    106 [[NoteBox(note,!PeriodLength allows you to determine the integration length of an execution for your configuration (restart files creation frequency), 600px)]] 
    107  
    108 [[NoteBox(warn,When using LMDz in your configuration the maximum possible !PeriodLength is 1M, 600px)]] 
    109  
    110 [[NoteBox(note,If !SpaceName=TEST all output will be store on scratchdir (on curie) or workdir (on ada), 600px)]] 
     107[[NoteBox(note,if !SpaceName=TEST all output will be store on scratchdir (on curie) or workdir (on ada), 600px)]] 
     108#### !PeriodLength #### 
     109The parameter !PeriodLength allows you to determine the continuous length of a simulation period for your configuration, i.e. the frequency at which restart files are created. '''The maximum !PeriodLength for any LMDZ model is 1M'''.  
     110 
     111 
     112### The [Restarts] section ### 
     113The Restarts section allow to start from an existing simulation. This simulation can be found at the archive machine or at local scratch- or workdir. Activate by setting '''!OverRule=y'''. All components (e.g. ATM, SRF, etc) will then use the same simulation as restart state.   
     114 
     115{{{ 
     116#!sh  
     117[Restarts] 
     118OverRule=y 
     119RestartDate=1999-12-31                                  # Last day of the experience used as restart for all components 
     120RestartJobName=EXP00                                    # Define restart simulation name for all components 
     121RestartPath=${ARCHIVE}/IGCM_OUT/IPSLCM5A/DEVT/pdControl # Path Server Group Login 
     122}}} 
     123 
     124The root path for the '''!RestartPath''' depend on the computing center. They are: 
     125{{{ 
     126#!sh  
     127${ARCHIVE}                           # The storage machine of the computing center 
     128                                     # (CCCSTOREDIR or ERGON). This space can contain   
     129                                     # tar of restarts or  
     130                                     # usual restarts files 
     131/ccc/store/cont003/dsm/login         # TGCC 
     132/u/rech/ces/login                    # IDRIS 
     133 
     134${SCRATCHDIR}                        # The large TGCC workspace (no backup)  
     135 
     136/ccc/scratch/cont003/dsm/login       # This kind of space can contain  
     137                                     # usual restarts files 
     138 
     139${WORKDIR}                           # The large IDRIS workspace (no backup) 
     140                                     
     141/workgpfs/rech/ces/login             # This kind of space can contain  
     142                                     # usual restarts files 
     143}}} 
     144 
     145[[NoteBox(note, libIGCM manages the difference in treatment between a path pointing to restart files that are directly accessible (without pack) and a path pointing to restart files that are in tar format (after pack). The management is made according to the path you provided. ,600px)]] 
     146 
     147  
     148### The [ATM], ..., sections of the model components ### 
     149 
     150This section for each of the model components allows you to:  
     151 * '''define the output frequency'''; 
     152 * '''define whether this component is installed''' which will only be considered if you specified '''!OverRule=n''' in the [Restarts] section. 
     153 
     154The possible settings for the '''!RestartPath''' options are the same as for the [Restarts] section. 
     155 
     156The possible settings for the '''!WriteFrequency''' options are: 
     157 * 1M    (monthly) 
     158 * 5D    (5-day) 
     159 * 1D    (daily) 
     160 * HF    (6-hour high frequency) 
     161 * HF3h  (real-time 3-hour frequency - specific to LMDZ) 
     162 * HF3hm (3-hour averaged high frequency - specific to LMDZ) 
     163 * STN   (instantaneous output only for the CFMIP stations - specific to LMDZ). 
     164 
     165 
     166{{{ 
     167#!sh  
     168[ATM] 
     169WriteFrequency="1M 1D"                                  # Activate the writing frequency of this component 
     170Restart=y                                               # If config_Restarts_OverRule == 'n' next 4 params are read 
     171RestartDate=1999-12-31                                  # Last day of the experience used as restart for this component if Restart=y 
     172RestartJobName=piControl25                              # Define restart simulation name for this component 
     173RestartPath=${ARCHIVE}/IGCM_OUT/IPSLCM5A/PROD/piControl # Path Server Group Login 
     174}}} 
     175 
     176 
     177'''!WriteFrequency specific to the model components'''[[BR]] 
     178 
     179 * '''LMDZ''' : ([ATM]) Each of the frequencies settings 1M, 1D, HF, HF3h, HF3hm, and STN correspond to a given output file. For example, if you specify ''1M'', a histmth.nc file will be created. If you want to change the output frequency in the histmth file you must change the corresponding lmdz parameter file. See [wiki:DocImodelBlmdz here]. 
     180 
     181 * '''ORCHIDEE'''  
     182   * '''[SRF]''' : The first frequency corresponds to the output frequency for the sechiba_history.nc file. The available frequencies are: xY, xM, 5D, 1D and xs, where x is an integer and s means seconds. This file is required. If you add HF, a second sechiba_out_2.nc file will be written with the 3H frequency.  
     183   * '''[SBG]''' : Only one frequency (xY, xM, 5D, 1D or xs) can be specified. It applies to the stomate_history.nc file. The stomate_history_ipcc.nc file always contains daily outputs.  
     184 
     185 * '''INCA''' : the section !WriteFrequency does not work. Click [wiki:DocImodelDinca here] to learn more about how to change the writing frequency. 
    111186 
    112187### The section [Executable] ### 
    113188This section contains one line for each model component giving the executable's name in the bin/ directory and the executable's name copied to the working directory. You should only change this section if your executable is running in parallel using MPI and OpenMP or if you have changed the executable's name. 
    114  
    115 Note : (",") indicates that this component has no executable. It is defined in a library linked to another executable (e.g. Orchidee in LMDZOR or Inca in LMDZINCA).  
    116  
    117 '''Example of an old syntax (configuration < v5). LMDz will use !JobNumProcTot MPI processes. Using IPSLCM5_v5, NEMO will use 5 MPI processes and LMDz will use (!JobNumProcTot-5) MPI processes''' 
    118189{{{ 
    119190#!sh 
    … …  
    124195... 
    125196}}} 
    126  
    127 '''Example for an MPMD MPI execution with NEMO and XIOS''' : Ocean on 127 MPI processes and IO Server on 1 MPI processes. 
     197Note : (",") indicates that this component has no executable. It is defined in a library linked to another executable (e.g. Orchidee in LMDZOR or Inca in LMDZINCA).  
     198 
     199'''Example for an hybrid MPI/OpenMP simulation with LMDZ''' : Atmosphere on 32 MPI process and 4 OMP threads per process. 
    128200{{{ 
    129201#!sh 
    130202[Executable] 
    131 #D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI processes, Number of OpenMP threads 
    132 OCE= (opa, opa.xx, 127MPI) 
    133 ICE= ("" ,"" ) 
    134 MBG= ("" ,"" ) 
    135 IOS= (xios_server.exe, xios.x, 1MPI) 
    136 }}} 
    137  
    138 '''Example for an MPMD hybrid MPI/OpenMP execution with IPSLCM coupled configuration''' : Atmosphere on 27 MPI processes and 4 OMP threads per processes, Ocean on 19 MPI processes, IO Server on 1 MPI processes. 
     203#D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI process, Number of OpenMP threads 
     204ATM= (gcm.e, lmdz.x, 32MPI, 4OMP) 
     205SRF= ("" ,"" ) 
     206SBG= ("" ,"" ) 
     207}}} 
     208 
     209'''Example for coupled configuration on curie''' : Atmosphere on 27 MPI process and 4 OMP threads per process, Ocean on 19 MPI process, IO Server on 1 MPI process. 
    139210{{{ 
    140211#!sh 
    141212[Executable] 
    142 #D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI processes, Number of OpenMP threads 
     213#D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI process, Number of OpenMP threads 
    143214ATM= (gcm.e, lmdz.x, 27MPI, 4OMP) 
    144215SRF= ("" ,"" ) 
    … …  
    150221IOS= (xios_server.exe, xios.x, 1MPI) 
    151222}}} 
    152  
    153 '''Another example for an MPMD hybrid MPI/OpenMP execution with LMDZ and XIOS''' : Atmosphere on 47 MPI processes, 8 OMP threads per processes and and IO server on 1 MPI processes. 
    154 {{{ 
    155 #!sh 
    156 [Executable] 
    157 #D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI processes, Number of OpenMP threads 
    158 ATM= (gcm.e, lmdz.x, 47MPI, 8OMP) 
    159 SRF= ("" ,"" ) 
    160 SBG= ("" ,"" ) 
    161 IOS= (xios_server.exe, xios.x, 1MPI) 
    162 }}} 
    163  
    164 '''Example for an SPMD hybrid MPI/OpenMP simulation with LMDZ''' : Atmosphere on 32 MPI processes and 4 OMP threads per processes. 
    165 {{{ 
    166 #!sh 
    167 [Executable] 
    168 #D- For each component, Real name of executable, Name of executable in RUN_DIR directory, Number of MPI processes, Number of OpenMP threads 
    169 ATM= (gcm.e, lmdz.x, 32MPI, 4OMP) 
    170 SRF= ("" ,"" ) 
    171 SBG= ("" ,"" ) 
    172 }}} 
    173  
    174 ### The [Restarts] section ### 
    175 The Restarts section allow to start from an existing simulation. This simulation can be found at the archive machine or at local scratch- or workdir. Activate by setting '''!OverRule=y'''. All components (e.g. ATM, SRF, etc) will then use the same simulation as restart state.   
    176  
    177 {{{ 
    178 #!sh  
    179 [Restarts] 
    180 OverRule=y 
    181 RestartDate=1999-12-31                                  # Last day of the experience used as restart for all components 
    182 RestartJobName=EXP00                                    # Define restart simulation name for all components 
    183 RestartPath=${ARCHIVE}/IGCM_OUT/IPSLCM5A/DEVT/pdControl # Path Server Group Login 
    184 }}} 
    185  
    186 The root path for the '''!RestartPath''' depend on the computing center. They are: 
    187 {{{ 
    188 #!sh  
    189 ${ARCHIVE}                           # The storage machine of the computing center 
    190                                      # (CCCSTOREDIR or ERGON). This space can contain   
    191                                      # tar of restarts or  
    192                                      # usual restarts files 
    193 /ccc/store/cont003/dsm/login         # TGCC 
    194 /u/rech/ces/login                    # IDRIS 
    195  
    196 ${SCRATCHDIR}                        # The large TGCC workspace (no backup)  
    197  
    198 /ccc/scratch/cont003/dsm/login       # This kind of space can contain  
    199                                      # usual restarts files 
    200  
    201 ${WORKDIR}                           # The large IDRIS workspace (no backup) 
    202                                      
    203 /workgpfs/rech/ces/login             # This kind of space can contain  
    204                                      # usual restarts files 
    205 }}} 
    206  
    207 [[NoteBox(note, libIGCM manages the difference in treatment between a path pointing to restart files that are directly accessible (without pack) and a path pointing to restart files that are in tar format (after pack). The management is made according to the path you provided. ,600px)]] 
    208  
    209   
    210 ### The [ATM], ..., sections of the model components ### 
    211  
    212 This section for each of the model components allows you to:  
    213  * '''define the output frequency'''; 
    214  * '''define whether this component is installed''' which will only be considered if you specified '''!OverRule=n''' in the [Restarts] section. 
    215  
    216 The possible settings for the '''!RestartPath''' options are the same as for the [Restarts] section. 
    217  
    218 The possible settings for the '''!WriteFrequency''' options are: 
    219  * 1M    (monthly) 
    220  * 5D    (5-day) 
    221  * 1D    (daily) 
    222  * HF    (6-hour high frequency) 
    223  * HF3h  (real-time 3-hour frequency - specific to LMDZ) 
    224  * HF3hm (3-hour averaged high frequency - specific to LMDZ) 
    225  * STN   (instantaneous output only for the CFMIP stations - specific to LMDZ). 
    226  
    227  
    228 {{{ 
    229 #!sh  
    230 [ATM] 
    231 WriteFrequency="1M 1D"                                  # Activate the writing frequency of this component 
    232 Restart=y                                               # If config_Restarts_OverRule == 'n' next 4 params are read 
    233 RestartDate=1999-12-31                                  # Last day of the experience used as restart for this component if Restart=y 
    234 RestartJobName=piControl25                              # Define restart simulation name for this component 
    235 RestartPath=${ARCHIVE}/IGCM_OUT/IPSLCM5A/PROD/piControl # Path Server Group Login 
    236 }}} 
    237  
    238  
    239 '''!WriteFrequency specific to the model components'''[[BR]] 
    240  
    241  * '''LMDZ''' : ([ATM]) Each of the frequencies settings 1M, 1D, HF, HF3h, HF3hm, and STN correspond to a given output file. For example, if you specify ''1M'', a histmth.nc file will be created. If you want to change the output frequency in the histmth file you must change the corresponding lmdz parameter file. See [wiki:DocImodelBlmdz here]. 
    242  
    243  * '''ORCHIDEE'''  
    244    * '''[SRF]''' : The first frequency corresponds to the output frequency for the sechiba_history.nc file. The available frequencies are: xY, xM, 5D, 1D and xs, where x is an integer and s means seconds. This file is required. If you add HF, a second sechiba_out_2.nc file will be written with the 3H frequency.  
    245    * '''[SBG]''' : Only one frequency (xY, xM, 5D, 1D or xs) can be specified. It applies to the stomate_history.nc file. The stomate_history_ipcc.nc file always contains daily outputs.  
    246  
    247  * '''INCA''' : the section !WriteFrequency does not work. Click [wiki:DocImodelDinca here] to learn more about how to change the writing frequency. 
    248223 
    249224### The [Post] section ### 
    … …  
    569544# Prepare ensembles with ins_job -e # 
    570545 
    571 {{{ 
    572 #!html 
    573 <span style="border: solid 1px #000000; padding: 5px; font-weight: bold;"><span style="color: #ff0000">Remark !</span> What follows applies to IPSLCM5_v5. </span> 
    574 }}} 
     546To create an ensemble configuration you need to create an ensemble.card file. 
     547 
     548'''NOTE''': If you want to do create ensemble with IPSLCM6 model, you need to create your own ensemble.card (no EXPERIMENT template is available yet). 
     549 
    575550 
    576551When IPSLCM5_v5 is downloaded with ./model IPSLCM5_v5 it will offer the possibility to launch experiments of the decadal type. 
    … …  
    589564## Usage ## 
    590565 
    591 Check that COMP, POST, PARAM and DRIVER (for IPSLCM5_v5) directories are present in the experiment folder. 
     566Check that COMP, POST, PARAM and DRIVER directories are present in the experiment folder. 
    592567Once [#Ensemble.card ensemble.card] and [#Config.card config.card] are correctly filled,  to create an ensemble simply type:  
    593        ins_job -e 
     568       ../../util/ins_job -e 
    594569This will create all the directories of the ensemble and Qsub.xxx.sh, a shell file containing all command to submit all jobs (!PeriodNb=60 for all simulations). 
    595570The Qclean.month/year.xxx.sh are bash files to use clean_month or clean_year script for all simulations.[[BR]][[BR]] 
    … …  
    600575 
    601576The file ''config.card'' is filled as a regular ''config.card'' ('''`ins_job`''' without the -e option). 
     577It will be used as a template for all simulations that will be created. 
    602578 
    603579The important lines for the ensemble set up are in the [!UserChoices] section. Make sure that !JobName and !ExperimentName are filled with proper values.[[BR]] 
    604 The variable !CalendarType should be consistent with the simulations from which you are initialising or else the restart file with the proper date may not exist.[[BR]] 
    605580The variables !DateBegin and !DateEnd will be overiden by variables present in [#Ensemble.card ensemble.card]. 
    606581{{{ 
    … …  
    658633There are 3 types of ensembles : 
    659634-       Parametric ensemble which is not implemented yet. 
    660 -       Date restart ensemble which allows to configure several simulations starting from different conditions. 
     635-       Date restart ensemble which allows to configure simulations starting from different restart dates. 
    661636-       Perturb ensemble which allows to generate members from an initial condition which is perturbed by different means. 
    662637 
    663638## Configure a Date Restart ensemble ## 
    664639 
    665 We cover here the section which allows to generate identical simulations excepted the initial restart file. Indeed, the « Date Restart ensemble » was implemented to configure a same simulation using several restart dates, generally chosen for a particular point (ex : randomly, particular climate oscillation phases, volcanic activity…).[[BR]] 
     640We cover here the section which allows to generate identical simulations excepted the initial restart file. Indeed, the « Date Restart ensemble » was implemented to configure a set of simulations using several restart dates, generally chosen for a particular point (ex : randomly, particular climate oscillation phases, volcanic activity…).[[BR]] 
    666641 
    667642In [#Ensemble.card ensemble.card] all configuration items of this ensemble are in `[Ens_DATE]` section.[[BR]] 
    … …  
    692667In [#Ensemble.card ensemble.card], it is possible to specify a periodic list of start dates. Restart files will be generated for each member at each date starting from `BEGIN_INIT` to `END_INIT` with a periodicity of `PERIODICITY`, using `BEGIN_RESTART` as first restart. Leave all ''NON_PERIODIC'' options empty (`NONPERIODIC`, `RESTART_NONPERIODIC`, `INITFROM_NONPERIODIC`,  `LENGTH_NONPERIODIC`).[[BR]] 
    693668 
    694 The following part of [#Ensemble.card ensemble.card] sets simulations from 1990-01-01 to 2000-01-01 every 2 years each lasting 10 years with a restarts starting from 1814-12-31 every 2 years: 
     669The following part of [#Ensemble.card ensemble.card] sets 10 years simulations from 1990-01-01 to 2000-01-01 every 2 years each with a restarts starting from 1814-12-31 every 2 years: 
    695670 
    696671{{{ 
    … …  
    723698/ccc/store/cont003/dsm/p86denv/dmf_import/IGCM_OUT/IPSLCM5A/PROD/historical 
    724699}}} 
     700 
     701'''CAUTION''': The variable !CalendarType from ''config.card'' will be used to determine the next restart date. It should be consistent with the simulations from which you are initialising. 
     702 
    725703 
    726704