GROMACS version: 2023.2
GROMACS modification: No
Hi,
I apologize if this is not the right forum to ask for help regarding installation issues.
I am trying to build a docker image for our users on a HPC system. Here is my recipe:
FROM nvcr.io/nvidia/cuda:11.8.0-devel-ubuntu22.04
WORKDIR /opt
ENV DEBIAN_FRONTEND noninteractive
RUN \
apt-get update && \
apt-get install --yes \
build-essential autoconf cmake flex bison zlib1g-dev apbs \
libicu-dev libbz2-dev libboost-all-dev libgmp-dev bc \
libblas-dev liblapack-dev automake lsb-core libxc-dev git \
unzip clang llvm gcc g++ libgsl-dev libhdf5-serial-dev \
cmake intel-mkl-full vim mlocate libtool indent netpbm \
python3-sphinx libudev-dev librust-libudev-sys-dev fig2dev \
ocaml ocamlbuild ocaml-findlib libfindlib-ocaml-dev camlp4 \
ghostscript wget netpbm xfig texinfo && \
apt-get clean all
ARG mpich=4.1.1
ARG mpich_prefix=mpich-$mpich
RUN \
wget https://www.mpich.org/static/downloads/$mpich/$mpich_prefix.tar.gz && \
tar xvzf $mpich_prefix.tar.gz && \
cd $mpich_prefix && \
./configure FFLAGS=-fallow-argument-mismatch FCFLAGS=-fallow-argument-mismatch && \
make -j 16 && \
make install && \
make clean && \
cd .. && \
rm -rf $mpich_prefix.tar.gz
RUN /sbin/ldconfig
ENV PATH=$PATH:/usr/local/cuda/lib64/stubs
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64/stubs
ENV PATH=$PATH:/usr/local/cuda-12.3/targets/x86_64-linux/lib/stubs
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-12.3/targets/x86_64-linux/lib/stubs
RUN ln -s /usr/local/cuda-11.8/targets/x86_64-linux/lib/stubs/libnvidia-ml.so /usr/local/cuda-11.8/targets/x86_64-linux/lib/stubs/libnvidia-ml.so.1
#Install HWLOC
WORKDIR /opt
RUN git clone -b v2.10 https://github.com/open-mpi/hwloc.git hwloc && \
cd hwloc && \
./autogen.sh && \
./configure && \
make -j 16 && \
make install
#Install OCAML NUM
WORKDIR /opt
RUN git clone https://github.com/ocaml/num.git num
RUN cd num && \
make all PROFILE=release && \
make test && \
make install && \
make clean
#Install FFTW
WORKDIR /opt
RUN git clone https://github.com/FFTW/fftw3.git fftw3 && \
cd fftw3 && \
sed -i '25d' bootstrap.sh && \
sed -i '25i ./configure --enable-shared --enable-maintainer-mode \
--enable-openmp --enable-mpi --enable-avx2 \
--enable-threads --prefix=/opt/fftw3/install $*' bootstrap.sh && \
sh bootstrap.sh && \
make -j 20 && \
make install && \
make clean && \
sed -i '25d' bootstrap.sh && \
sed -i '25i ./configure --enable-shared --enable-maintainer-mode \
--enable-openmp --enable-mpi --enable-avx2 --enable-float \
--enable-threads --prefix=/opt/fftw3/install $*' bootstrap.sh && \
sh bootstrap.sh && \
make -j 4 && \
make install
ENV PATH=$PATH:/opt/fftw3/install/bin
ENV PATH=$PATH:/opt/fftw3/install/lib
ENV PATH=$PATH:/opt/fftw3/install/include
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/fftw3/install/lib
#Install HeFFTe
WORKDIR /opt
RUN git clone -b v2.4.0 https://github.com/icl-utk-edu/heffte.git heffte
RUN cd heffte && \
mkdir build && cd build && \
cmake \
-D CMAKE_BUILD_TYPE=Release -D BUILD_SHARED_LIBS=ON -D CMAKE_INSTALL_PREFIX=/opt/heffte/install \
-D Heffte_ENABLE_AVX=ON -D Heffte_ENABLE_FFTW=ON -D FFTW_ROOT=/opt/fftw3/install \
-D Heffte_ENABLE_CUDA=ON .. && \
make -j 20 && \
make install
ENV PATH=/opt/heffte/install/bin:$PATH
ENV PATH=/opt/heffte/install/lib:$PATH
ENV LD_LIBRARY_PATH=/opt/heffte/install/lib:$LD_LIBRARY_PATH
# Install GROMACS
WORKDIR /opt
ARG gromacs=2023.2
ARG gmx_prefix=gromacs-$gromacs
RUN git clone -b v2023.2 https://github.com/gromacs/gromacs.git gromacs
WORKDIR /opt/gromacs/src/gromacs/utility
RUN sed -i 's\MPIX_Query_cuda_support()\1\g' mpiinfo.cpp
WORKDIR /opt
RUN cd gromacs && \
mkdir build && cd build && \
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/gromacs/install \
-DCMAKE_CXX_STANDARD=17 \
-DGMX_FFT_LIBRARY=fftw3 -DREGRESSIONTEST_DOWNLOAD=ON \
-DGMX_MPI=on -DGMX_SIMD=AVX2_256 -DGMX_GPU=CUDA \
-DGMX_CUDA_TARGET_COMPUTE=80 -DBUILD_SHARED_LIBS=OFF \
-DGMX_USE_HEFFTE=ON -DHeffte_ROOT=/opt/heffte/install \
-DGMX_PREFER_STATIC_LIBS=ON ../ && \
make -j 20 && \
make install
ENV PATH=/opt/gromacs/install/bin:$PATH
ENV PATH=/opt/gromacs/install/lib:$PATH
ENV LD_LIBRARY_PATH=/opt/gromacs/install/lib:$LD_LIBRARY_PATH
ENV PKG_CONFIG_PATH=/opt/gromacs/install/lib/pkgconfig:$PKG_CONFIG_PATH
ENV MANPATH=/opt/gromacs/install/share/man/:$MANPATH
I want to use the heffte libraries to enable gromacs pme gpu support. During the build, I am hitting the following compilation error:
#25 99.95 /usr/include/c++/11/initializer_list(62): error: "void (const heffte::stock::Complex<float, 8> *, std::size_t)" contains a vector, which is not supported in device code
#25 99.95 detected during:
#25 99.95 instantiation of class "std::initializer_list<_E> [with _E=heffte::stock::Complex<float, 8>]"
#25 99.95 /opt/heffte/install/include/stock_fft/heffte_stock_algos.h(86): here
#25 99.95 instantiation of "heffte::stock::biFuncNode<F, L>::biFuncNode(size_t, size_t, size_t) [with F=float, L=8]"
#25 99.95 /opt/heffte/install/include/stock_fft/heffte_stock_tree.h(215): here
#25 99.95 instantiation of "size_t heffte::stock::init_fft_tree(heffte::stock::biFuncNode<F, L> *, size_t) [with F=float, L=8]"
#25 99.95 /opt/heffte/install/include/heffte_backend_stock.h(248): here
#25 99.95 instantiation of "heffte::plan_stock_fft<std::complex<F>, dir>::plan_stock_fft(int, int, int, int) [with F=float, dir=heffte::direction::forward]"
#25 99.95 /opt/heffte/install/include/heffte_backend_stock.h(529): here
#25 99.95 instantiation of "void heffte::stock_fft_executor::make_plan(std::unique_ptr<heffte::plan_stock_fft<scalar_type, dir>, std::default_delete<heffte::plan_stock_fft<scalar_type, dir>>> &) const [with scalar_type=std::complex<float>, dir=heffte::direction::forward]"
#25 99.95 /opt/heffte/install/include/heffte_backend_stock.h(471): here
#25 99.95
May I please request your help with this? Am I missing any libraries during the installation process?
Thank you!
Neil Mehta