Using GROMACS on the DGX B200

User discussions
1

GROMACS version: 2024.5
GROMACS modification: No

Dear Gromacs users,

Is GROMACS currently unavailable on the DGX B200 with CUDA 12.8 (Ubuntu 24.04)?

I am encountering the following error:

WARNING: An error occurred while sanity checking device #0. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #1. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #2. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #3. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #4. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #5. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #6. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

WARNING: An error occurred while sanity checking device #7. An unhandled error from a previous CUDA operation was detected. CUDA error #209 (cudaErrorNoKernelImageForDevice): no kernel image is available for execution on the device.

I’d be grateful for any feedback or suggestions.
Thank you in advance.

2

Hi!

GROMACS 2024 should be compatible with newer hardware but was not tested on NVIDIA B200. Could you share the output of gmx -version and nvidia-smi?

Also, keep in mind that this Friday, we’re planning to make the GROMACS 2025.0 release, which was tested on Blackwell devices and has some nice performance tweaks for this hardware. You can try the 2025 release candidate in the mean time; feedback would be appreciated :)

3

Thanks for your reply.

GROMACS version: 2024.5
CUDA version: 12.8
Ubuntu 24.04

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.36                 Driver Version: 570.36         CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA B200                    On  |   00000000:1B:00.0 Off |                    0 |
| N/A   32C    P0            192W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA B200                    On  |   00000000:43:00.0 Off |                    0 |
| N/A   33C    P0            193W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA B200                    On  |   00000000:52:00.0 Off |                    0 |
| N/A   39C    P0            200W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA B200                    On  |   00000000:61:00.0 Off |                    0 |
| N/A   38C    P0            196W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   4  NVIDIA B200                    On  |   00000000:9D:00.0 Off |                    0 |
| N/A   33C    P0            200W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   5  NVIDIA B200                    On  |   00000000:C3:00.0 Off |                    0 |
| N/A   33C    P0            211W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   6  NVIDIA B200                    On  |   00000000:D1:00.0 Off |                    0 |
| N/A   38C    P0            235W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   7  NVIDIA B200                    On  |   00000000:DF:00.0 Off |                    0 |
| N/A   40C    P0            249W / 1000W |       4MiB / 183359MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
4

And gmx -version?

5 
                     :-) GROMACS - gmx, 2024.5 (-:

Executable: /home/chip/local/gromacs_2024.5/bin/gmx
Data prefix: /home/chip/local/gromacs_2024.5
Working dir: /home/chip
Command line:
gmx --version

GROMACS version: 2024.5
Precision: mixed
Memory model: 64 bit
MPI library: thread_mpi
OpenMP support: enabled (GMX_OPENMP_MAX_THREADS = 128)
GPU support: CUDA
NBNxM GPU setup: super-cluster 2x2x2 / cluster 8
SIMD instructions: AVX2_256
CPU FFT library: fftw-3.3.10-sse2-avx-avx2-avx2_128-avx512
GPU FFT library: cuFFT
Multi-GPU FFT: none
RDTSCP usage: enabled
TNG support: enabled
Hwloc support: hwloc-2.11.1
Tracing support: disabled
C compiler: /usr/bin/gcc GNU 13.3.0
C compiler flags: -fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -O3 -DNDEBUG
C++ compiler: /usr/bin/g++ GNU 13.3.0
C++ compiler flags: -fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -Wno-cast-function-type-strict -fopenmp -O3 -DNDEBUG
BLAS library: External - detected on the system
LAPACK library: External - detected on the system
CUDA compiler: /usr/local/cuda-12.8/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2025 NVIDIA Corporation;Built on Wed_Jan_15_19:20:09_PST_2025;Cuda compilation tools, release 12.8, V12.8.61;Build cuda_12.8.r12.8/compiler.35404655_0
CUDA compiler flags:-std=c++17;–generate-code=arch=compute_50,code=sm_50;–generate-code=arch=compute_52,code=sm_52;–generate-code=arch=compute_60,code=sm_60;–generate-code=arch=compute_61,code=sm_61;–generate-code=arch=compute_70,code=sm_70;–generate-code=arch=compute_75,code=sm_75;–generate-code=arch=compute_80,code=sm_80;–generate-code=arch=compute_86,code=sm_86;–generate-code=arch=compute_89,code=sm_89;–generate-code=arch=compute_90,code=sm_90;-Wno-deprecated-gpu-targets;–generate-code=arch=compute_53,code=sm_53;–generate-code=arch=compute_80,code=sm_80;-use_fast_math;-Xptxas;-warn-double-usage;-Xptxas;-Werror;-D_FORCE_INLINES;-Xcompiler;-fopenmp;-fexcess-precision=fast -funroll-all-loops -mavx2 -mfma -Wno-missing-field-initializers -Wno-cast-function-type-strict -fopenmp -O3 -DNDEBUG
CUDA driver: 12.80
CUDA runtime: 12.80

6

Ok, thanks. Our “forward-compatibility” setting is broken, as it turned out. Long time without new hardware :)

You should be able to work around it by adding -DGMX_CUDA_TARGET_SM='100;120' to your cmake invocation and rebuilding GROMACS.

7

It’s working! Thanks for your help in solving the problem :)

1 Like
8

Hi! :)

Building on this, I have a 5090 which should also be a Blackwell, and I got the same error as the original post, so I arrived here. I used the suggested flags but in the make I get this error

CMake Error at libgromacs_generated_nbnxm_cuda_kernel_pruneonly.cu.o.Release.cmake:280

both with CUDA12.8 and 12.9. Any idea what it might be? Because I would probably need to install both GROMACS version 23.0 and 24.5. On a side note, I installed GROMACS 25.1 and it worked out of the box, no additional flags!

9

Hi!

Yes, RTX 5090 is also Blackwell, but it’s CC 12.0, while B200 is CC 10.0. See CUDA GPU Compute Capability | NVIDIA Developer.

Currently, CC 12.0 is not supported by GROMACS 2024-series. The patch is trivial (src/gromacs/gpu_utils/cuda_arch_utils.cuh part of MR !4933); it will be in GROMACS 2024.6 (just like the fix for the issues above), which we plan to release eventually.

For now, on RTX 50xx, please use GROMACS 2025-series. Applying the patch above is at your own risk :)

1 Like