Changeset 2654 for XIOS3/dev/XIOS_KOKKOS/arch/arch-X64_JEANZAY_KOKKOS.env

Timestamp:

09/19/24 12:05:23 (9 days ago)

Author:

jderouillat

Message:

Implement Kokkos kernels compatible with GPU : removing all C++ object from kernels (access to class members).

The behavior of NumTraits?<double>::isNan is not correct on GPU, a solution must be found (temporary solution consists in checking that values are not too large).

Revert some CArray_WF introduction in the initialisation (mask management in CGrid).
For a given process, the CArray_WF class can't address host and device. Clients should be compile with a GPU backend, while servers with a CPU backend (serial, openMP).
Some transfer methods of connectors have been duplicated to manage CArray (initialisation) and CArray_WF (workflow).

A gpu_computing parameter (gpu_computing, 0 by default) has been introduced in the generic_testcase to upload field3D management on GPU.
Other xios_send_field have been temporarily disabled.

File:

• XIOS3/dev/XIOS_KOKKOS/arch/arch-X64_JEANZAY_KOKKOS.env (modified) (1 diff)

XIOS3/dev/XIOS_KOKKOS/arch/arch-X64_JEANZAY_KOKKOS.env

@@ -8 +8 @@
 export CPATH=${KOKKOS_HOME}/include:${CPATH}
 
+module load openmpi/4.1.1-cuda
+export OMPI_CXX=nvcc_wrapper
+module load hdf5/1.12.0-mpi-cuda
+module load netcdf-c/4.7.4-mpi-cuda
 
-#module load gcc/9.1.0
-module load intel-all/2020.4
-module load hdf5/1.12.0-mpi 
-module load netcdf-c/4.7.4-mpi
-module load netcdf-fortran/4.5.3-mpi
-module load nco
-module load cdo
-
-#source $I_MPI_ROOT/intel64/bin/mpivars.sh release_mt
-#export I_MPI_OFI_PROVIDER=verbs # workaround for a MPI_Win_flush bug appearing with IntelMPI in a multi-context (XIOS) environment
 export TMP=/dev/shm    # to compile on node with enough memory
 export TMPDIR=/dev/shm    # to compile on node with enough memory