cuda_mpi_sendrecv

Requirement

• CUDA-Aware MPI

Check OpenMPI version

$ ompi_info --parsable --all | grep mpi_built_with_cuda_support:value

It should return true

Check OpenMPI, infiniband and CUDA for multi-node

$ ompi_info --all | grep btl_openib_have_cuda_gdr
$ ompi_info --all | grep btl_openib_have_driver_gdr

It should return true

Run

$ mpirun -n 2 --allow-run-as-root ./cuda_mpi_sendrecv
OR:
$ mpirun -n 2 --mca btl_openib_want_cuda_gdr 1 ./cuda_mpi_sendrecv

Reference

• An Introduction to CUDA-Aware MPI
• Benchmarking CUDA-Aware MPI
• Building CUDA-aware Open MPI
• Running CUDA-aware Open MPI