GROMACS version: 2022.3, 2022.5, 2023.1
GROMACS modification: No
Hello, I’ve been having issues, trying all three of these GROMACS distributions. I’m running on an HPC, and I have included the basic error logs for the outputs of cmake and make check for the blank run after logging into the HPC. After gcc-7.5.0 fails, I install gcc-10/g+±10 and try the installation once more with the updated GNU against CUDA 11.1.
I am wondering if the nvcc flags for sm_89 and sm_90 in the initial cmake run are indicative of the error I’m receiving in make. With gcc-7.5.0, as per the release notes, cmake assumes its OK, but we see the actual failed test when I’mrunning on gcc-10. I’ve even re-installed cmake to custom directories independently, one with the base gnu and one with gcc-10 and run the gcc-10 cmake towards a gcc-10 GROMACS installation. None of this has seemed to alleviate these failed tests.
Interestingly, I was able to install GROMACS 2022.3 earlier in the week, but the installation stopped working (Illegal Instruction). Now, I’ve been unable to produce another successful installation.
All previous installations/build attempts have been removed.
UPDATE: I performed the same installation with both MPI and CUDA off, and the MdrunIOTest, MdrunTestsOneRank, and MdrunTestsTwoRanks fail during make check. Thus, it seems the nvcc compilation flags are unrelated to the build error.
settings upon login:
Ubuntu 18.04
gcc 7.5.0, g++ 7.5.0
CMake 3.26.4 - installed to separate directories with cmake using CMAKE_C_COMPILER=gcc, CMAKE_CXX_COMPILER=g++
and then gcc-10 and g+±10. I source these only before I want to use them in a new terminal.
±----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02 Driver Version: 450.80.02 CUDA Version: 11.1 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100-SXM2… Off | 00000000:0B:00.0 Off | 0 |
| N/A 26C P0 43W / 163W | 0MiB / 32510MiB | 0% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+
±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+
source /userhome/user/software/cmake-g7/bin
untar clean gromacs tarball, make and cd into build dir
GROMACS 2022.5 installation with cmake 3.26.4, installed with gcc/g++ 7.5.0
cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=OFF \
-DGMX_MPI=off -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ \
-DCMAKE_INSTALL_PREFIX=/userhome/user/software/gmx-22 \
-DREGRESSIONTEST_PATH=/userhome/user/software/regressiontests-2022.5/ \
-DPython3_EXECUTABLE=/userhome/user/software/miniconda3/envs/img/bin/python \
-DGMX_GPU=CUDA -DGMX_THREAD_MPI=ON
– Checking if nvcc accepts flags --generate-code=arch=compute_89,code=sm_89 - Assumes success because gcc-7
– Checking if nvcc accepts flags --generate-code=arch=compute_90,code=sm_90
– Checking if nvcc accepts flags --generate-code=arch=compute_90,code=sm_90 - Assumes success because gcc-7
make check -j8
Note that mdrun will redetermine rlist based on the actual pair-list setup
This run will generate roughly 0 Mb of data
Writing final coordinates.
NOTE: 14 % of the run time was spent in pair search,
you might want to increase nstlist (this has no effect on accuracy)
Core t (s) Wall t (s) (%)
Time: 0.224 0.112 199.5
(ns/day) (hour/ns)
Performance: 1.540 15.588
Opened /userhome/user/software/gromacs-2022.5/build/src/programs/mdrun/tests/Testing/Temporary/Checking_InitialConstraintsTest_Works_2_spc-and-methanol.edr as single precision energy file
Reading energy frame 1 time 0.001 [ OK ] Checking/InitialConstraintsTest.Works/2 (707 ms)
[----------] 3 tests from Checking/InitialConstraintsTest (1986 ms total)
[----------] Global test environment tear-down
[==========] 66 tests from 13 test suites ran. (111441 ms total)
[ PASSED ] 64 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] MdrunTerminationTest.CheckpointRestartAppendsByDefault
[ FAILED ] MdrunTerminationTest.WritesCheckpointAfterMaxhTerminationAndThenRestarts
2 FAILED TESTS
Start 63: MdrunTestsOneRank
[ OK ] AccelerationWorks/AccelerationGroupTest.WithinTolerances/3 (641 ms)
[----------] 4 tests from AccelerationWorks/AccelerationGroupTest (2920 ms total)
[----------] Global test environment tear-down
[==========] 22 tests from 6 test suites ran. (21159 ms total)
[ PASSED ] 20 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] FreezeWorks/FreezeGroupTest.WithinTolerances/3
[ FAILED ] 1 test, listed below:
[ FAILED ] CompelTest.SwapCanRun
1 FAILED TEST
97% tests passed, 3 tests failed out of 87
Label Time Summary:
GTest = 714.62 secproc (81 tests)
IntegrationTest = 341.05 secproc (25 tests)
MpiTest = 402.34 secproc (19 tests)
QuickGpuTest = 89.35 secproc (17 tests)
SlowTest = 323.97 secproc (13 tests)
UnitTest = 49.59 secproc (43 tests)
Total Test time (real) = 1058.26 sec
The following tests FAILED:
62 - MdrunIOTests (Failed)
63 - MdrunTestsOneRank (Failed)
64 - MdrunTestsTwoRanks (Failed)
Errors while running CTest
CMakeFiles/run-ctest-nophys.dir/build.make:70: recipe for target ‘CMakeFiles/run-ctest-nophys’ failed
make[3]: *** [CMakeFiles/run-ctest-nophys] Error 8
CMakeFiles/Makefile2:3269: recipe for target ‘CMakeFiles/run-ctest-nophys.dir/all’ failed
make[2]: *** [CMakeFiles/run-ctest-nophys.dir/all] Error 2
CMakeFiles/Makefile2:3302: recipe for target ‘CMakeFiles/check.dir/rule’ failed
make[1]: *** [CMakeFiles/check.dir/rule] Error 2
Makefile:641: recipe for target ‘check’ failed
make: *** [check] Error 2
cmake installation with toolchain gcc-10/g+±10, cmake also installed with gcc-10/g+±10:
cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=OFF \
-DGMX_MPI=off -DCMAKE_C_COMPILER=gcc-10 -DCMAKE_CXX_COMPILER=g++-10 \
-DCMAKE_INSTALL_PREFIX=/userhome/user/software/gmx-22 \
-DREGRESSIONTEST_PATH=/userhome/user/software/regressiontests-2022.5/ \
-DPython3_EXECUTABLE=/userhome/user/software/miniconda3/envs/img/bin/python \
-DGMX_GPU=CUDA -DGMX_THREAD_MPI=ON
– Checking if nvcc accepts flags --generate-code=arch=compute_89,code=sm_89 - Failed
– Checking if nvcc accepts flags --generate-code=arch=compute_90,code=sm_90
– Checking if nvcc accepts flags --generate-code=arch=compute_90,code=sm_90 - Failed
make check -j8 error output:
The following tests FAILED:
62 - MdrunIOTests (Failed)
63 - MdrunTestsOneRank (Failed)
64 - MdrunTestsTwoRanks (Failed)
Errors while running CTest
CMakeFiles/run-ctest-nophys.dir/build.make:70: recipe for target ‘CMakeFiles/run-ctest-nophys’ failed
make[3]: *** [CMakeFiles/run-ctest-nophys] Error 8
CMakeFiles/Makefile2:3269: recipe for target ‘CMakeFiles/run-ctest-nophys.dir/all’ failed
make[2]: *** [CMakeFiles/run-ctest-nophys.dir/all] Error 2
CMakeFiles/Makefile2:3302: recipe for target ‘CMakeFiles/check.dir/rule’ failed
make[1]: *** [CMakeFiles/check.dir/rule] Error 2
Makefile:641: recipe for target ‘check’ failed
make: *** [check] Error 2
-DGMX_CUDA_TARGET_SM=90 -DGMX_CUDA_TARGET_COMPUTE=90 ## example output
CMake Error at cmake/gmxManageNvccConfig.cmake:194 (message):
Your choice of 90 in GMX_CUDA_TARGET_SM was not accepted by nvcc, please
choose a target that it accepts
Call Stack (most recent call first):
cmake/gmxManageCuda.cmake:101 (include)
CMakeLists.txt:610 (include)