1 | 04/06/2014 - N.LEBAS |
---|
2 | |
---|
3 | ********************************** |
---|
4 | ENSEMBLE JOB CREATION |
---|
5 | ********************************** |
---|
6 | |
---|
7 | Updates 04/06/2014 |
---|
8 | ------------------ |
---|
9 | 1) Updates of Ensemble DATE: |
---|
10 | Fix minor bug: periodic member names start to 0 as for non-periodic |
---|
11 | |
---|
12 | Create new files "Qclean.month.ENSNAME.sh" and "Qclean.year.ENSNAME.sh" which allow |
---|
13 | user to use clean_month and clean_year scripts for all members. |
---|
14 | |
---|
15 | New option "INITFROM_NONPERIODIC" is avalaible in ensemble.card to use different |
---|
16 | simulations for restart (use INITFROM option for all if it is not fill). |
---|
17 | |
---|
18 | 2) Updates of Ensemble PERTURB: |
---|
19 | Create new files "Qclean.month.ENSNAME.sh" and "Qclean.year.ENSNAME.sh" which allow |
---|
20 | user to use clean_month and clean_year scripts for all members. |
---|
21 | |
---|
22 | 3) Updates for ins_job: |
---|
23 | When a new directory is created from config.card, ensemble.card is copy inside if it |
---|
24 | exists. |
---|
25 | |
---|
26 | If -e option is used, ins_job also works when a new directory is created. But it is |
---|
27 | still only consider the ensemble.card present in the directory where the command was |
---|
28 | used (it doesn't look for all ensemble.card present in configuration). |
---|
29 | |
---|
30 | Update variable name for "ensemble.card" from $FileName to $F_CFG_ENS to be coherent |
---|
31 | with $F_CFG for 'config.card'. |
---|
32 | |
---|
33 | |
---|
34 | Updates 11/04/2014 |
---|
35 | ------------------ |
---|
36 | 1) Updates of libIGCM_ensemble.ksh: |
---|
37 | New function "IGCM_ensemble_Init" to read the "active" parameter for each |
---|
38 | ensemble type. Ins_job could then call only functions link to active ensemble. |
---|
39 | |
---|
40 | Function "IGCM_ensemble_SetAlpha" fill Alpha and AlphaMonth vectors used to create |
---|
41 | member name suffix (A,B...). Actually return Alpha= A to Z vector and AlphaMonth=a to l |
---|
42 | but it could be adapted later to return n suffixes (n = members number). |
---|
43 | |
---|
44 | 11/04/2014: QsubXXX.txt file become QsubXXX.sh (seems more coherent to it content). |
---|
45 | |
---|
46 | 2) Updates of Ensemble PERTURB |
---|
47 | In "ensemble.card" all parameters are in english, so as comments over each param. |
---|
48 | Read "JobName" in "config.card" directly, so ensemble.card NAME could be different. |
---|
49 | Creation of symbolic link to DRIVER directory for each member (mandatory with IPSLCM5_v5). |
---|
50 | Check if directories for each member exist. If this is the case, there is no modification |
---|
51 | of it (don't touch config.car and Jobxxx inside). Create it else and add command line to |
---|
52 | Qsub.xxx.txt file. |
---|
53 | |
---|
54 | 3) Ensemble DATE |
---|
55 | Create automatically a set of experiments with defined length and restart file. The |
---|
56 | descritpion could be defined in a periodic or non-periodic way (or a mix of both). |
---|
57 | 11/04/2014: Update subdir names as ${ENSNAME}${restartnb}A/ and add "_per" suffix for |
---|
58 | periodic directories. |
---|
59 | |
---|
60 | 4) Ensemble PARAMETRIC |
---|
61 | Test active parameter in ins_job and print "not implemented yet". |
---|
62 | |
---|
63 | |
---|
64 | Usage |
---|
65 | ----- |
---|
66 | First of all you need to add a new section called "Ensemble" in config.card - ignored if |
---|
67 | ins_job script is not using for creating an ensemble - as below: |
---|
68 | [Ensemble] |
---|
69 | #D- Ensemble run ? 'y' or 'n' |
---|
70 | #D- If 'y', fill in ensemble.card !! |
---|
71 | EnsembleRun= y |
---|
72 | EnsembleName= |
---|
73 | EnsembleDate= |
---|
74 | EnsembleType= |
---|
75 | |
---|
76 | Note that all empty fields will be filled automatically with the ensemble. All of them |
---|
77 | are only to help you to remind the conditions of the run, they don't change the run of |
---|
78 | the Job. |
---|
79 | |
---|
80 | All parameters for ensemble description are in "ensemble.card" and global template of |
---|
81 | simulations are in the "config.card". |
---|
82 | |
---|
83 | Check that you get COMP, POST, PARAM and DRIVER (for IPSLCM5_v5) directories in experiment |
---|
84 | folder. |
---|
85 | Once "ensemble.card" and "config.card" are correctly written, to create an ensemble |
---|
86 | simply type: |
---|
87 | |
---|
88 | ins_job -e |
---|
89 | |
---|
90 | This will create all the directories of the ensemble and Qsub.xxx.sh, a shell file |
---|
91 | containing all command to submit all jobs (PeriodNb=60 for all simulations). |
---|
92 | |
---|
93 | The Qclean.month/year.xxx.sh are bash files to use clean_month or clean_year script for |
---|
94 | all simulations. |
---|
95 | |
---|
96 | NOTE: If a directory exists, ins_job won't modify it. If only some directories of |
---|
97 | the ensemble are presents, it will create the missing one and complete Qsub.xxx.sh |
---|
98 | shell file. |
---|
99 | |
---|
100 | NOTE: If you put ensemble.card and config.card directly in [...]/modipsl/config/IPSLCM5_v5 |
---|
101 | folder, it will create a new directory from config.card "JobName" value. Then copy the 2 |
---|
102 | card files and generate the ensemble structure. |
---|
103 | |
---|
104 | Configuration |
---|
105 | ------------- |
---|
106 | |
---|
107 | 1) Peturb ensembles |
---|
108 | To be done by S. Nguyen. |
---|
109 | |
---|
110 | 2) Date restart ensembles |
---|
111 | |
---|
112 | In ensemble.card all configuration of this ensemble are in [Ens_DATE] section. |
---|
113 | Available options for this section are: |
---|
114 | active*: 'y' for using date ensemble, 'n' else. |
---|
115 | NAME*: name of the ensemble (used to create root directory) |
---|
116 | LENGTH*: default length of the simulation for non periodic and duration |
---|
117 | for all periodic |
---|
118 | INITFROM*: Restart name to find all restart files (also use as prefix for |
---|
119 | all simulations) |
---|
120 | INITPATH*: Restart global directory |
---|
121 | |
---|
122 | BEGIN_INIT(1): start date of the first periodic simulation |
---|
123 | END_INIT(1): start date of the last periodic simulation |
---|
124 | PERIODICITY(1): duration between the start of 2 periodic simulations |
---|
125 | BEGIN_RESTART(1): date of the first restart file (next files will be |
---|
126 | determined adding periodicity length conform to |
---|
127 | config.card calendar). |
---|
128 | |
---|
129 | NONPERIODIC(2): list of start dates for all simulations |
---|
130 | RESTART_NONPERIODIC(2): list of corresponding restart dates |
---|
131 | INITFROM_NONPERIODIC(2): simulation name to restart for each simulation. |
---|
132 | IF empty all simulations will use INITFROM one. |
---|
133 | LENGTH_NONPERIODIC: lenght of each simulation. IF empty all simulations |
---|
134 | duration will be the default LENGTH option. |
---|
135 | |
---|
136 | All parameters with * must be filled. |
---|
137 | All parameters with (1) must be filled for periodic description are empty else. |
---|
138 | All parameters with (2) must be filled for non periodic description are empty else. |
---|
139 | |
---|
140 | WARNING: For list variables, use space between values (no coma). |
---|
141 | |
---|
142 | ins_job -e will create a "NAME" root ensemble directory containing all periodic and |
---|
143 | non periodics simulations subdir. Subdir names, such as Jobs, are build by concatening |
---|
144 | <NAME><RESTART_NB>A (ex: ensNino0A, ensNino1A...) for non periodic and add suffix |
---|
145 | "_per" for periodic (ex: ensNino0A_per). |
---|
146 | |
---|
147 | 3) Parametric ensembles |
---|
148 | Not implemented yet. |
---|
149 | |
---|
150 | |
---|
151 | |
---|