Does gromacs not support Tesla V100 GPU?

User discussions
1

GROMACS version: 2019.3.
GROMACS modification: Yes/No
Here post your question

Dear all,

I am trying to do gromacs MD simulation on a computer equipping with 8 Tesla V100 GPUs. I give the following command:

gmx mdrun -v -deffnm md -gpu_id 0,1,2,3

However it returns the following error:
Fatal error:
You limited the set of compatible GPUs to a set that included ID #0, but that
ID is not for a compatible GPU. List only compatible GPUs.

The error tells that GPU ID #0 is not compatible, but actually changing the GPU ID from 0 to 7 will all produce the same error.

Do it mean that gromacs do not support Tesla V100 GPU, or is there anything wrong with the mdrun command?

Thank you!

2

We use V100s all the time. More likely something is wrong with the hardware or driver on those GPUs that leads to a detection failure.

3

Justin’s correct, V100’s are fine. We may be able to help a bit if you post the log file.

4

Hello, Kevin,

The complete log file is:
:-) GROMACS - gmx mdrun, 2019.2 (-:

                        GROMACS is written by:
 Emile Apol      Rossen Apostolov      Paul Bauer     Herman J.C. Berendsen
Par Bjelkmar      Christian Blau   Viacheslav Bolnykh     Kevin Boyd

Aldert van Buuren Rudi van Drunen Anton Feenstra Alan Gray
Gerrit Groenhof Anca Hamuraru Vincent Hindriksen M. Eric Irrgang
Aleksei Iupinov Christoph Junghans Joe Jordan Dimitrios Karkoulis
Peter Kasson Jiri Kraus Carsten Kutzner Per Larsson
Justin A. Lemkul Viveca Lindahl Magnus Lundborg Erik Marklund
Pascal Merz Pieter Meulenhoff Teemu Murtola Szilard Pall
Sander Pronk Roland Schulz Michael Shirts Alexey Shvetsov
Alfons Sijbers Peter Tieleman Jon Vincent Teemu Virolainen
Christian Wennberg Maarten Wolf
and the project leaders:
Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright © 1991-2000, University of Groningen, The Netherlands.
Copyright © 2001-2018, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.

GROMACS is free software; you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License
as published by the Free Software Foundation; either version 2.1
of the License, or (at your option) any later version.

GROMACS: gmx mdrun, version 2019.2
Executable: /home/sxcryoem/Softwares/gromacs-2019.2/build/bin/gmx
Data prefix: /usr/local/gromacs
Working dir: /home/sxcryoem/sunyp/projects/NA/1myN2-native/5_md_1
Process ID: 215612
Command line:
gmx mdrun -v -deffnm md_1 -gpu_id 7

GROMACS version: 2019.2
Precision: single
Memory model: 64 bit
MPI library: thread_mpi
OpenMP support: enabled (GMX_OPENMP_MAX_THREADS = 64)
GPU support: CUDA
SIMD instructions: NONE
FFT library: fftw-3.3.8
RDTSCP usage: disabled
TNG support: enabled
Hwloc support: disabled
Tracing support: disabled
C compiler: /usr/bin/cc GNU 4.8.5
C compiler flags: -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
C++ compiler: /usr/bin/c++ GNU 4.8.5
C++ compiler flags: -std=c++11 -O3 -DNDEBUG -funroll-all-loops -fexcess-precision=fast
CUDA compiler: /usr/local/cuda/bin/nvcc nvcc: NVIDIA ® Cuda compiler driver;Copyright © 2005-2019 NVIDIA Corporation;Built on Wed_Apr_24_19:10:27_PDT_2019;Cuda compilation tools, release 10.1, V10.1.168
CUDA compiler flags:-gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=compute_75;-use_fast_math;;; ;-std=c++11;-O3;-DNDEBUG;-funroll-all-loops;-fexcess-precision=fast;
CUDA driver: 10.0
CUDA runtime: N/A

NOTE: Detection of GPUs failed. The API reported:
CUDA driver version is insufficient for CUDA runtime version
GROMACS cannot run tasks on a GPU.

Running on 1 node with total 48 cores, 96 logical cores, 0 compatible GPUs
Hardware detected:
CPU info:
Vendor: Intel
Brand: Intel® Xeon® Platinum 8160M CPU @ 2.10GHz
Family: 6 Model: 85 Stepping: 4
Features: aes apic avx avx2 avx512f avx512cd avx512bw avx512vl clfsh cmov cx8 cx16 f16c fma hle htt intel lahf mmx msr nonstop_tsc pcid pclmuldq pdcm pdpe1gb popcnt pse rdrnd rdtscp rtm sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
Number of AVX-512 FMA units: Cannot run AVX-512 detection - assuming 2
Hardware topology: Basic
Sockets, cores, and logical processors:
Socket 0: [ 0 48] [ 1 49] [ 2 50] [ 3 51] [ 4 52] [ 5 53] [ 6 54] [ 7 55] [ 8 56] [ 9 57] [ 10 58] [ 11 59] [ 12 60] [ 13 61] [ 14 62] [ 15 63] [ 16 64] [ 17 65] [ 18 66] [ 19 67] [ 20 68] [ 21 69] [ 22 70] [ 23 71]
Socket 1: [ 24 72] [ 25 73] [ 26 74] [ 27 75] [ 28 76] [ 29 77] [ 30 78] [ 31 79] [ 32 80] [ 33 81] [ 34 82] [ 35 83] [ 36 84] [ 37 85] [ 38 86] [ 39 87] [ 40 88] [ 41 89] [ 42 90] [ 43 91] [ 44 92] [ 45 93] [ 46 94] [ 47 95]

Highest SIMD level requested by all nodes in run: AVX_512
SIMD instructions selected at compile time: None
This program was compiled for different hardware than you are running on,
which could influence performance.

The current CPU can measure timings more accurately than the code in
gmx mdrun was configured to use. This might affect your simulation
speed as accurate timings are needed for load-balancing.
Please consider rebuilding gmx mdrun with the GMX_USE_RDTSCP=ON CMake option.

Program: gmx mdrun, version 2019.2
Source file: src/gromacs/mdrun/runner.cpp (line 531)

Fatal error:
You limited the set of compatible GPUs to a set that included ID #7, but that
ID is not for a compatible GPU. List only compatible GPUs.

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors

5

You have a CUDA runtime issue, so mdrun basically concludes that the GPUs aren’t working.