GROMACS 2023.5 installation error when implementing PME decomposition（-DGMX_USE_CUFFTMP=ON)

User discussions
1

GROMACS version:2023.5
GROMACS modification: No
Hello gmx developers, I encountered issues while compiling GROMACS 2023.5 with support for PME decomposition on HPC clusters(CenOS7) . The following are the specific commands and error messages."
Since the HPC SDK seemed not to be installed in the standard location, I specified the header file directory to prevent situations where the *.h files could not be found (this issue had occurred previously：
export C_INCLUDE_PATH=/software/nvidia/hpcsdk/2023/Linux_x86_64/23.3/math_libs/12.0/include

export CPLUS_INCLUDE_PATH=/software/nvidia/hpcsdk/2023/Linux_x86_64/23.3/math_libs/12.0/include
Then:
module load cmake/3 scl/gcc9.3
then Cmake configuration:
cmake …
-DCMAKE_INSTALL_PREFIX=/project2/elapsebos/gmx2023.5
-DCMAKE_PREFIX_PATH=/project2/elapsebos/fftw3_mpi410_gcc
-DGMX_BUILD_OWN_FFTW=off
-DGMX_FFT_LIBRARY=fftw3
-DFFTWF_LIBRARY=/project2/elapsebos/fftw3_mpi410_gcc9/lib/libfftw3f.a
-DGMX_DOUBLE=off -DGMX_MPI=off -DGMX_SIMD=AVX_256
-DGMX_GPU=CUDA -DCUDA_TOOLKIT_ROOT_DIR=/software/nvidia/hpcsdk/2023/Linux_x86_64/23.3/cuda/12.0
-DGMX_USE_CUFFTMP=ON -DcuFFTMp_ROOT=/software/nvidia/hpcsdk/2023/Linux_x86_64/23.3/math_libs/12.0
make -j && make -j install
and cmake configuration error occured:
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cufft_LIBRARY (ADVANCED)
linked by target “testutils” in directory /home/elapsebos/scratch/gromacs-2023.5/src/testutils
linked by target “libgromacs” in directory /home/elapsebos/scratch/gromacs-2023.5/src/gromacs

– Generating done
CMake Warning:
Manually-specified variables were not used by the project:

cuFFTMp_ROOT

CMake Generate step failed. Build files cannot be regenerated correctly.
According to the error message above, I thus checked the CMakeLists.txt in the directories ./src/gromacs and testutils, and found the following information related to cufftmp and cuFFT :

  1. testutils (in line 52):
    if(GMX_USE_cuFFTMp)
    target_link_libraries(testutils PRIVATE {cuFFTMp_LIBRARY}) else() target_link_libraries(testutils PRIVATE {CUDA_CUFFT_LIBRARIES})
    endif()
  2. gromacs(in line 186):
    if(GMX_USE_cuFFTMp)
    target_link_libraries(libgromacs PRIVATE {cuFFTMp_LIBRARY}) else() target_link_libraries(libgromacs PRIVATE {CUDA_CUFFT_LIBRARIES})
    endif()
    The two pieces of similar code above seem to indicate that cuFFT and cufftmp are incompatible, I’m not clear on the reason, so I still proceeded to execute make -j and encountered the expected error：
    [ 93%] Linking CXX shared library …/…/lib/libgromacs.so
    /software/rh/devtoolset-9/root/usr/bin/…/libexec/gcc/x86_64-redhat-linux/9/ld: cannot find -lCUDA_cufft_LIBRARY-NOTFOUND
    collect2: error: ld returned 1 exit status
    make[2]: *** [src/gromacs/CMakeFiles/libgromacs.dir/build.make:13052: lib/libgromacs.so.8.0.0] Error 1
    make[1]: *** [CMakeFiles/Makefile2:4251: src/gromacs/CMakeFiles/libgromacs.dir/all] Error 2
    make: *** [Makefile:166: all] Error 2

Then I attempted to delete the incompatible else()，and this time cmake configuration passed with no error related to CUDA_cufft_library, but another error occured when linking libgromacs.so:

[ 95%] Building CXX object src/testutils/CMakeFiles/testutils.dir/filematchers.cpp.o
/software/rh/devtoolset-9/root/usr/bin/…/libexec/gcc/x86_64-redhat-linux/9/ld: …/…/lib/libgromacs.so.8.0.0: undefined reference to cufftPlanMany' /software/rh/devtoolset-9/root/usr/bin/../libexec/gcc/x86_64-redhat-linux/9/ld: ../../lib/libgromacs.so.8.0.0: undefined reference to cufftDestroy’
/software/rh/devtoolset-9/root/usr/bin/…/libexec/gcc/x86_64-redhat-linux/9/ld: …/…/lib/libgromacs.so.8.0.0: undefined reference to cufftExecC2R' /software/rh/devtoolset-9/root/usr/bin/../libexec/gcc/x86_64-redhat-linux/9/ld: ../../lib/libgromacs.so.8.0.0: undefined reference to cufftExecR2C’
/software/rh/devtoolset-9/root/usr/bin/…/libexec/gcc/x86_64-redhat-linux/9/ld: …/…/lib/libgromacs.so.8.0.0: undefined reference to `cufftSetStream’
collect2: error: ld returned 1 exit status
make[2]: *** [src/programs/CMakeFiles/gmx.dir/build.make:92: bin/gmx] Error 1
make[1]: *** [CMakeFiles/Makefile2:6128: src/programs/CMakeFiles/gmx.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs…

I have tried GCC 11, GCC 12, GCC 13, as well as NVIDIA HPC SDK 2022, but the same problem persists. I’m confused about the imcompatition between cufft and cufftmp, and wondering how should I do to configure correctly, I would be very grateful if someone could help.

2

updated: solved by simply setting -DGMX_MPI=on , its just a huge mistake :)