Changes between Version 8 and Version 9 of Documentation/UserGuide/ParallelCoding
- Timestamp:
- 2013-11-28T12:44:46+01:00 (11 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Documentation/UserGuide/ParallelCoding
v8 v9 1 1 {{{ 2 2 #!html 3 <h1> How to parallelize my code ?</h1>3 <h1> Parallelization in ORCHIDEE </h1> 4 4 }}} 5 5 6 6 [[PageOutline]] 7 7 8 ORCHIDEE is parallelized with MPI and since soon, with OpenMP (only for LMDZ). 9 If you develop some code, think of the parallelization. It is not so complicated as it seems in ORCHIDEE. 8 ORCHIDEE is parallelized in hybrid mode, using both MPI and OpenMP. Running the offline ORCHIDEE driver it is only possible to activate the pur MPI parallelization. Using ORCHIDEE coupled to LMDZ, it is possible to run in hybrid mode using reccent versions of both models. For OpenMP in LMDZ, use LMDZ5/trunk rev 1729 with some modifications (see below) or rev 1907 or later and for ORCHIDEE use revision 1615 on the trunk or later. 9 [[BR]] 10 11 Read also this documentation in frensh version [attachment:"parallelisation_orchidee.pdf"]. 12 13 ---- 10 14 11 15 = MPI = 12 13 ORCHIDEE has to be runned in parallel with MPI. The parallelization is transparent as far as there are no 16 The parallelization is transparent as far as there are no 14 17 interactions between the pixels.[[BR]] 15 18 '''Except if you develop the routing module''', you don't need to worry about 16 19 the parallelization. However, you have some rules to respect : 17 * 1. Use the subroutines called restget_p, restput_p and getin_p instead of restget, restput and getin. 20 * 1. Use the subroutines called restget_p, restput_p and getin_p instead of restget, restput and getin. Note that all processes should call these subroutines _p and not only the master. 18 21 * 2. Exception : if you '''restart a scalar''', you have to use restget followed by a bcast command : 19 20 22 {{{ 21 23 IF (is_root_prc) THEN … … 27 29 ENDIF 28 30 CALL bcast(day_counter) 29 30 31 }}} 31 for restput, follow the example :32 for restput, follow the example : 32 33 {{{ 33 34 IF (is_root_prc) THEN … … 37 38 ENDIF 38 39 }}} 40 Note that only the master does the call to restget and restput. 41 42 43 ---- 39 44 40 45 = OpenMP = 41 46 42 Since december 2013, OpenMP has been included into ORCHIDEE coupledwith LMDZ (configuration LMDZOR_v5.2)47 Since december 2013, rev 1615 in trunk/ORCHIDEE, OpenMP has been included into ORCHIDEE used in coupled mode with LMDZ (configuration LMDZOR_v5.2) 43 48 44 49 == How to launch a simulation on curie (or ada) with MPI-OMP parallelisation ? == 45 50 46 === Update LMDZ if you are working with a version older than rev 1905 === 51 === Update LMDZ === 52 You can work with LMDZ5/trunk rev 1729 but you need to update 2 files, as below, or use LMDZ5/trunk rev 1907 or later. Revisions older than 1729 has not been tested.[[BR]] 53 54 55 If you use rev 1729, update as follow: 56 47 57 {{{ 48 58 cd modeles/LMDZ/libf/phylmd … … 69 79 70 80 __LMDZ__[[BR]] 71 It's '''mandatory''' to have '''the same compilation''' for LMDZ than forORCHIDEE and IOIPSL. So you need to check the file ''modeles/LMDZ/arch/arch***.fcm''.81 It's '''mandatory''' to have '''the same compilation''' for LMDZ, ORCHIDEE and IOIPSL. So you need to check the file ''modeles/LMDZ/arch/arch***.fcm''. 72 82 * On Curie 73 83 {{{ … … 88 98 }}} 89 99 90 And then modify it by adding omp compilation for LMDZ. (in ''config/LMDZOR/Makefile'')100 Then modify it by activating hybrid mpi-omp compilation for LMDZ. In ''config/LMDZOR_v5.2/Makefile'', change into -parallel mpi_omp : 91 101 {{{ 92 102 (cd ../../modeles/LMDZ; ./makelmdz_fcm -d $(RESOL_LMDZ) -cosp true -v true -parallel mpi_omp -arch $(FCM_ARCH) gcm ; cp bin/gcm_$(RESOL_LMDZ)_phylmd_para_orch.e ../../bin/gcm.e ; ) … … 94 104 95 105 === Setup your first simulation === 96 Follow [http://forge.ipsl.jussieu.fr/igcmg/wiki/platform/en/documentation/E_miseenplace th is documentation] to know how setup your simulation. There is few change to indicate to libIGCM that you want work with omp.97 * change the parts "Executable" in ''config.card'' by indicate the number of omp threads (2, 4, 8)106 Follow [http://forge.ipsl.jussieu.fr/igcmg/wiki/platform/en/documentation/E_miseenplace the platform documentation] to know how setup your simulation using libIGCM. Some changes are needed to indicate to libIGCM that you work in hybrid MPI-OpenMP mode. 107 * Change "Executable" section in ''config.card'' by indicating the number of MPI processes and the nubmer of omp threads (2, 4 or 8). For exemple: 98 108 {{{ 99 109 ATM= (gcm.e, lmdz.x, 24MPI, 2OMP) 100 101 110 }}} 102 * modify the script header111 * Modify the main job header 103 112 * On Curie 104 113 {{{ … … 115 124 116 125 126 === How to launch a simple job in MPI_OMP === 127 See here how to launch a simple test job [https://forge.ipsl.jussieu.fr/orchidee/wiki/HowTo/TestCaseBatch]. Use also ''curie.info'' documentation (write curie.info in a Curie terminal) for more information. 128 129 ---- 130 117 131 == Coding Rules == 118 132 119 133 === Forbidden === 120 It's forbidden to use '''!$''' at the beginning of a line, except for OMP directive . Note that you can use '''!!$''' if you want for comments.134 It's forbidden to use '''!$''' at the beginning of a line, except for OMP directives. Note that you can use '''!!$''' if you want for comments. 121 135 122 136 === Basics Rules === … … 129 143 !$OMP THREADPRIVATE(indexlai) 130 144 }}} 131 * Except if you are working with the process 0 (master) all call to IOIPSL routines need to be suffix by '''_p''' (these routines are declared in ''src_parallel/ioipsl_para.f90'')145 * All call to IOIPSL routines need to be suffix by '''_p''' except if working with master process. These routines are declared in ''src_parallel/ioipsl_para.f90''. All processes and threads do the call. For exemple: 132 146 {{{ 133 147 CALL histwrite_p(hist_id, 'beta', kjit, vbeta, kjpindex, index) … … 137 151 * If you need to add an OMP barrier, use the routine '''barrier2_omp''' 138 152 {{{ 139 callbarrier2_omp()153 CALL barrier2_omp() 140 154 }}} 141 155 142 === variables useful for parallelisation ===156 === Useful variables for parallelization === 143 157 * '''is_root_prc''' declared in ''src_parallel/mod_orchidee_para.F90''---> mpi and omp process master (=omp master thread on mpi process master) 144 158 * '''is_omp_root''' declared in ''src_parallel/mod_orchidee_omp_data.F90'' --> omp thread master 145 159 * '''is_mpi_root''' declared in ''src_parallel/mod_orchidee_mpi_data.F90'' --> mpi process master 146 160 147 148 == More information on OMP parallelisation ==149 === Documentation ===150 Read this documentation (french version) [attachment:"parallelisation_orchidee.pdf"].151 152 === How to launch a simple job in MPI_OMP ===153 Read this [https://forge.ipsl.jussieu.fr/orchidee/wiki/HowTo/TestCaseBatch#ExempleatCurieTGCC documentation] and read ''curie.info'' documentation (write curie.info in a Curie terminal)
