= ORCHIDEE trunk rev 2247 with LMDZ5 trunk rev 2076 = This page is dedicated to techincal validation of the model. For scientific validation, see more simulations here: [wiki:testing/rev2247/simulations] [[PageOutline]] == Parallelism == * ORCHIDEE gives the identical results for simulations with different number of cores MPI and/or OMP. It does not matter if the executable is compiled for pur MPI mode or for mixt MPI/OMP mode. * At curie, compile options must be updated, see below * Simulations coupled to LMDZ do not succeed in debug mode (and not dev at ada). This seems be due to following change on LMDZ : [http://web.lmd.jussieu.fr/trac/changeset/2011/LMDZ5/trunk/libf/phylmd/coefcdrag.F90]. The problem seems to be only in the first time-step. * The executable compiled in pur MPI and in mixt MPI_OMP gives the same results if the same optimization mode is used (prod/dev/debug) * Sequential mode is tested with gfortran : simulations are reproductible ==== ADA/IDRIS ==== * Compilation en mode prod, dev and debug are successful for pur mpi and mixt mpi_omp. * '''Coupled mode in dev and debug mode: simulations stops''' * Changing the number of cores MPI and/or OMP does not change the results. For example running on 23MPI=32MPI=16MPI.2OMP * The executable compiled in pur MPI and in mixt MPI_OMP gives the same results if the same optimization mode is used (prod/dev/debug) * Using optimization mode prod does not give the same results as optimization mode dev or debug * Using optimization mode dev or debug gives the same results ==== CURIE/TGCC ==== * Compilation en mode prod, dev and debug are successful for pur mpi and mixt mpi_omp. * Offline mode: all simulations are identical : compile mode, different number of cores MPI, pur mpi or mixt mpi_omp (but only running on 1 omp) * Coupled mode: * '''Simulations are not identical while changing the number of cores MPI and or OMP''' also without ORCHIDEE using LMDZ bucket * In debug mode: simulations stops (stops when ORCHIDEE is activated, ok with bucket) * Coupled mode with modifications in compile options for LMDZ as in commit [http://web.lmd.jussieu.fr/trac/changeset/2108]: * '''Simulations are identical while changing the number of cores MPI and or OMP''' * '''Simulations are identical between prod, dev and debug compile options''' * Simulations still stops in debug mode for LMDZOR simulations ==== local PC using gfortran ==== * Compiling with gfortran for sequential mode : prod, dev and debug is ok. * Simulations are reproductive * Tests are done with ORCHIDEE offline Choisnel and CWRR, for prod, dev and debug mode * Tests are done in coupled mode for prod mode == Restartablilty == * Same simulation set up is reproductible (see above) * '''1+1 not = 2''' in coupled and in offline mode == Performance == === LMDZOR at curie === Following performances are observed using '''LMDZ r2076 with added -fp-model precise''' in compile options. AP physics. * Using Choisnel or CWRR do not change the run time in coupled mode. * No difference in time using executable MPI or MPI_OMP on the same number of cores MPI and 1OMP. * 32MPI*1OMP - 740hCpu/10y : Elapsed time : 23h * 32MPI*2OMP - 850hCpu/10y : Elapsed time : 13h * 32MPI*4OMP - 1100hCpu/10y : Elapsed time : 8h30min ---- ---- == Details of simulations == === Default set up === LMDZ is always run with it's old physics package (called AP). ORCHIDEE is always run activating Stomate. Tests are done with both Choisnel and CWRR hydrology. === ADA/IDRIS === The model is installed by the trusting here : /workgpfs/rech/lmd/rlmd979/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140717T1820/modipsl {{{ ****** LMDZORChoi en prod OK (les test bigtest=n) *********************************************************************************************** results.out.cross:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORCWRR en prod tout OK *********************************************************************************************** results.out.cross:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORChoi en dev not OK *********************************************************************************************** results.out.dev_mpi:Following 0 DIRECTORIES finished: results.out.dev_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi ****** ORChoi prod/dev/debug OK ****** Compilation debug=dev mais pas debug=prod ou dev=prod ****** *********************************************************************************************** results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp ****** ORCWRR OK *********************************************************************************************** results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp }}} === CURIE/TGCC === The model is installed by the trusting here : /ccc/scratch/cont003/dsm/p86ghatt/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140717T1824/modipsl {{{ ****** LMDZORCWRR ****** debug : simulations did not finish ****** prod and dev : all simulations finished, results differs when changing the number of cores MPI/OMP. ****** Simulations are reproductable at same number of cores. ************************************************************************************************************* results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 0 DIRECTORIES finished: results.out.debug_mpi_omp:Following 3 DIRECTORIES DID NOT FINISH: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.dev_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:Following 3 DIRECTORIES finished: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:Following 6 DIRECTORIES finished: 16mpi_4omp.1 16mpi_4omp.2 16mpi_4omp.3 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.prod_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/32mpi_2omp ****** LMDZORChoi ****** debug: No simulations finished ****** prod and dev: simulations finished, results differs when changing the number of cores MPI/OMP. ****** Simulations are reprodusable at same number of cores. ************************************************************************************************************* results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi_omp/16mpi_8omp results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = prod_mpi_omp/16mpi_1omp results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 0 DIRECTORIES finished: results.out.debug_mpi_omp:Following 3 DIRECTORIES DID NOT FINISH: 16mpi_8omp 32mpi_1omp 32mpi_2omp results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_8omp NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_1omp = prod_mpi_omp/16mpi_2omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.1 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/1mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_2omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_4omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/32mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/4mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_1omp NOT = prod_mpi_omp/8mpi_1omp ****** ORChoi ****** All simulations finished. All simulations are identic. ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi/16mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = prod_mpi/16mpi results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/18mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/1mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/20mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/23mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/32mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/4mpi results.out.dev_mpi:DIRECTORIES OK : dev_mpi/16mpi = dev_mpi/8mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/16mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/1mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/2mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/4mpi_1omp results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/8mpi_1omp results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/18mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/1mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/20mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/23mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/32mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/4mpi results.out.prod_mpi:DIRECTORIES OK : prod_mpi/16mpi = prod_mpi/8mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/16mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/1mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/2mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/4mpi_1omp results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/8mpi_1omp ****** ORCWRR ****** All simulation that finished gives the same results : compile mode, number of cores MPI and or OMP ****** debug: Not all simulations finished. This is probably due to a cpu time exit. The option CHECK_CWRR is very time consuming. ****** prod: OK. All simulations finished. Same results changing the number of cores MPI/OMP. Pur mpi gives the same results as mixt mpi_omp ****** Simulations are reprodusable at same number of cores. ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = dev_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.1 results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = prod_mpi/23mpi results.out.cross:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.1 results.out.debug_mpi:Following 0 DIRECTORIES finished: results.out.debug_mpi:Following 2 DIRECTORIES DID NOT FINISH: 23mpi 32mpi results.out.debug_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.debug_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/10mpi_1omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/10mpi_1omp.1 = debug_mpi_omp/32mpi_1omp results.out.dev_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.dev_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi:DIRECTORIES OK : dev_mpi/23mpi = dev_mpi/32mpi results.out.dev_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.dev_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/10mpi_1omp.3 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/10mpi_1omp.1 = dev_mpi_omp/32mpi_1omp results.out.prod_mpi:Following 2 DIRECTORIES finished: 23mpi 32mpi results.out.prod_mpi:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi:DIRECTORIES OK : prod_mpi/23mpi = prod_mpi/32mpi results.out.prod_mpi_omp:Following 4 DIRECTORIES finished: 10mpi_1omp.1 10mpi_1omp.2 10mpi_1omp.3 32mpi_1omp results.out.prod_mpi_omp:Following 0 DIRECTORIES DID NOT FINISH: results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/10mpi_1omp.3 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/10mpi_1omp.1 = prod_mpi_omp/32mpi_1omp ****** LMDZ with bucket (no ORCHIDEE) ****** All simulation finished ****** Simulations with same number of cores are identical ****** Simulations are not the same while changing the number of cores MPI and/or OMP ****** Debug simulations seems to be identical while changing the number of cores MPI and/or OMP ************************************************************************************************************* results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = dev_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = dev_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi/23mpi results.out.cross:DIRECTORIES DIFFER : debug_mpi/23mpi NOT = prod_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi_omp/16mpi_4omp.1 results.out.cross:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi_omp/16mpi_4omp.1 results.out.debug_mpi:DIRECTORIES OK : debug_mpi/23mpi = debug_mpi/32mpi results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_4omp.2 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_4omp.3 results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/16mpi_8omp results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/32mpi_1omp results.out.debug_mpi_omp:DIRECTORIES OK : debug_mpi_omp/16mpi_4omp.1 = debug_mpi_omp/32mpi_2omp results.out.dev_mpi:DIRECTORIES DIFFER : dev_mpi/23mpi NOT = dev_mpi/32mpi results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/16mpi_4omp.1 = dev_mpi_omp/16mpi_4omp.2 results.out.dev_mpi_omp:DIRECTORIES OK : dev_mpi_omp/16mpi_4omp.1 = dev_mpi_omp/16mpi_4omp.3 results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/16mpi_8omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/32mpi_1omp results.out.dev_mpi_omp:DIRECTORIES DIFFER : dev_mpi_omp/16mpi_4omp.1 NOT = dev_mpi_omp/32mpi_2omp results.out.prod_mpi:DIRECTORIES DIFFER : prod_mpi/23mpi NOT = prod_mpi/32mpi results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.2 results.out.prod_mpi_omp:DIRECTORIES OK : prod_mpi_omp/16mpi_4omp.1 = prod_mpi_omp/16mpi_4omp.3 results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/16mpi_8omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_1omp results.out.prod_mpi_omp:DIRECTORIES DIFFER : prod_mpi_omp/16mpi_4omp.1 NOT = prod_mpi_omp/32mpi_2omp }}} == obelix/LSCE == ... == gfortran/ private pclinux laptop == The model is installed here /home/jgipsl/TRUSTING/ORCHIDEE/RUNDIR/trunk_rev2247_20140812T1536/modipsl