Gromacs OpenCL incompatible GPUs

User discussions
,
1

GROMACS version: 2022
GROMACS modification: No

I am attempting to run Gromacs with OpenCL, but it seems that GPUs are considered as incompatible.

My hardware is: AMD EPYC 7643 48-Core + 8 NVIDIA RTX A6000 GPUs.

I have built Gromacs with:

cmake … -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_BUILD_OWN_FFTW=ON -DGMX_GPU=OpenCL

clinfo:

Number of platforms 1
Platform Name NVIDIA CUDA
Platform Vendor NVIDIA Corporation
Platform Version OpenCL 3.0 CUDA 12.1.68
Platform Profile FULL_PROFILE
…
Platform Name NVIDIA CUDA
Number of devices 8
…

./gmx --version

Command line:
gmx --version

GROMACS version: 2022
Precision: mixed
Memory model: 64 bit
MPI library: thread_mpi
OpenMP support: enabled (GMX_OPENMP_MAX_THREADS = 128)
GPU support: OpenCL
SIMD instructions: AVX2_256
CPU FFT library: fftw-3.3.8-sse2-avx-avx2-avx2_128
GPU FFT library: clFFT
RDTSCP usage: enabled
TNG support: enabled
Hwloc support: disabled
Tracing support: disabled
C compiler: /usr/bin/cc GNU 11.3.0
C compiler flags: -mavx2 -mfma -Wno-missing-field-initializers -fexcess-precision=fast -funroll-all-loops -O3 -DNDEBUG
C++ compiler: /usr/bin/c++ GNU 11.3.0
C++ compiler flags: -mavx2 -mfma -Wno-missing-field-initializers -fexcess-precision=fast -funroll-all-loops -fopenmp -O3 -DNDEBUG
OpenCL include dir: /usr/include
OpenCL library: /usr/lib/x86_64-linux-gnu/libOpenCL.so
OpenCL version: 2.2

Command line:
gmx mdrun -v -deffnm em

Reading file em.tpr, VERSION 2022 (single precision)
Using 96 MPI threads
Using 1 OpenMP thread per tMPI thread

gmx mdrun log:

Running on 1 node with total 96 cores, 96 processing units, 0 compatible GPUs
Hardware detected:
CPU info:
Vendor: AMD
Brand: AMD EPYC 7643 48-Core Processor
Family: 25 Model: 1 Stepping: 1
Features: aes amd apic avx avx2 clfsh cmov cx8 cx16 f16c fma htt lahf misalignsse mmx msr nonstop_tsc pcid pclmuldq pdpe1gb popcnt pse rdrnd rdtscp sha sse2 sse3 sse4a sse4.1 sse4.2 ssse3
Hardware topology: Basic
Packages, cores, and logical processors:
[indices refer to OS logical processors]
Package 0: [ 0] [ 1] [ 2] [ 3] [ 4] [ 5] [ 6] [ 7] [ 8] [ 9] [ 10] [ 11] [ 12] [ 13] [ 14] [ 15] [ 16] [ 17] [ 18] [ 19] [ 20] [ 21] [ 22] [ 23] [ 24] [ 25] [ 26] [ 27] [ 28] [ 29] [ 30] [ 31] [ 32] [ 33] [ 34] [ 35] [ 36] [ 37] [ 38] [ 39] [ 40] [ 41] [ 42] [ 43] [ 44] [ 45] [ 46] [ 47]
Package 1: [ 48] [ 49] [ 50] [ 51] [ 52] [ 53] [ 54] [ 55] [ 56] [ 57] [ 58] [ 59] [ 60] [ 61] [ 62] [ 63] [ 64] [ 65] [ 66] [ 67] [ 68] [ 69] [ 70] [ 71] [ 72] [ 73] [ 74] [ 75] [ 76] [ 77] [ 78] [ 79] [ 80] [ 81] [ 82] [ 83] [ 84] [ 85] [ 86] [ 87] [ 88] [ 89] [ 90] [ 91] [ 92] [ 93] [ 94] [ 95]
CPU limit set by OS: -1 Recommended max number of threads: 96
GPU info:
Number of GPUs detected: 8
#0: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#1: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#2: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#3: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#4: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#5: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#6: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)
#7: name: NVIDIA RTX A6000, vendor: NVIDIA Corporation, device version: OpenCL 3.0 CUDA, status: incompatible (please use CUDA build for NVIDIA Volta GPUs or newer)

My CUDA version according to nvidia-smi is 12.1.
Could it be due to OpenCL version mismatch: 2.2 vs 3.0?

Any ideas what to try?

Thanks,

T

2

Hi!

The OpenCL backend of GROMACS only supports older GPUs: https://manual.gromacs.org/2023.2/install-guide/index.html#gpu-support. Please use CUDA build (-DGMX_GPU=CUDA). It has better performance and supports more features.

You can set GMX_GPU_DISABLE_COMPATIBILITY_CHECK=1 environment variable to override the check, but the computation results will almost certainly be incorrect.