Hello, Dear all!
I am a person only intermediately experienced with Linux installs and also fairly easily frustrated :D. After a few days of struggle, I managed to install Gromacs with AMD GPU support on my Laptop and thought others might benefit from what I’ve learnt.
Stuff:
Computer: Framework 16 Laptop (2024)
CPU: AMD Ryzen 7 7840HS
GPU: AMD Radeon RX7700S
OS: clean install of Ubuntu 24.04
GROMACS version: 2024.4
GROMACS modification: No
Install overview:
cmake, gromacs and for AMD GPU support: rocm, rocm-llvm-dev, acpp
…and dependencies
Workflow:
I recommend using Timeshift to set up system recovery points every now and then.
Adjust package versions in the code as needed.
As I messed up in the first try, I did a clean install of Ubuntu 24.04 and made sure the essentials are there by running the system updates and:
sudo apt update
sudo apt upgrade -y
sudo apt install -y build-essential
Now, install rocm using the convenience service amdgpu-install of AMD (which is just a wrapper to installing with the system package manager).
This also installs some amdgpu drivers - if secure boot is enabled in your system, you will be prompted to set a one-time password for enrolling a new “MOK” at next startup. Choose something easy like “12345678” and on next reboot select “Enroll MOK” > “Continue” > “Yes” > Enter the Password > “Reboot”. This only has to be done once.
#taken from rocm documentation
sudo apt update
sudo apt install -y "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"
sudo apt install -y python3-setuptools python3-wheel
cd ~/Downloads
wget https://repo.radeon.com/amdgpu-install/6.3/ubuntu/noble/amdgpu-install_6.3.60300-1_all.deb
sudo apt install -y ./amdgpu-install_6.3.60300-1_all.deb
sudo apt update
sudo amdgpu-install -y --usecase=rocm
# Add the current user to the render and video groups
sudo usermod -a -G render,video $LOGNAME
Reboot.
Check the user groups (should belong to render and video now) and the rocm install:
groups
rocminfo
rocminfo also informs about the rocm “Name” of the GPU, in my case “gfx1102”, which is needed later on.
rocm comes “bundled” with it’s own version of llvm. The current gromacs documentation recommends doing the installation of acpp and gromacs with that very same llvm of the rocm installation. So I tried. However, it took me quite some time to realize that another package is needed to be able to do that: rocm-llvm-dev. This will add llvm components essential for the further installation to the existing rocm directory.
sudo apt install -y rocm-llvm-dev
For installing acpp (AdaptiveCpp, formerly named hipSYCL/Open SYCL), we also need libboost-all-dev (for building acpp), git (to get acpp) and cmake (for building acpp and gromacs).
sudo apt install -y libboost-all-dev git cmake
Now to get, build and install AdaptiveCpp.
Building with cmake (see code) will get a lot of options such as:
-DCMAKE_INSTALL_PREFIX=/usr/local
for providing the install destination. According to acpp documentation, this is actually the default path, but I chose to explicitly set it.
As mentioned, the version of llvm coming with rocm and rocm-llvm-dev is used. Also, clang and clang++ which are shipped with rocm will be used instead of generic versions. cmake is told to look for those by providing the paths with:
-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++
-DLLVM_DIR=/opt/rocm/llvm/lib/cmake/llvm/
AdaptiveCpp offers a variety of backends for different systems. When building gromacs it is explicitly stated it does not support the “generic SSCP compiler” and aborts building. Therefore, this compiler is switched off. Actually, for AMD GPUs the HIP/ROCm backend is the only other option, and therefore only this remains switched on:
-DWITH_ROCM_BACKEND=ON
-DWITH_SSCP_COMPILER=OFF
-DWITH_OPENCL_BACKEND=OFF
-DWITH_LEVEL_ZERO_BACKEND=OFF
-DWITH_CUDA_BACKEND=OFF
The build process states that these options are deprecated, but right now they still do work.
The last option sets the default target device, which might help in avoiding mixing up with the on board graphics. This is where the rocm “Name” of the GPU is used:
-DDEFAULT_TARGETS=‘hip:gfx1102’
cd ~/Downloads
git clone https://github.com/AdaptiveCpp/AdaptiveCpp
cd AdaptiveCpp
mkdir build && cd build
sudo cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local \
-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang \
-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ \
-DLLVM_DIR=/opt/rocm/llvm/lib/cmake/llvm/ \
-DWITH_ROCM_BACKEND=ON \
-DWITH_SSCP_COMPILER=OFF \
-DWITH_OPENCL_BACKEND=OFF \
-DWITH_LEVEL_ZERO_BACKEND=OFF \
-DWITH_CUDA_BACKEND=OFF -DDEFAULT_TARGETS='hip:gfx1102'
sudo make install
Check acpp:
acpp-info
Finally, get, build and install gromacs.
The cmake options:
-DGMX_BUILD_OWN_FFTW=ON
-DREGRESSIONTEST_DOWNLOAD=ON
are the default recommendation according to gromacs documentation.
The paths to clang and clang++ in the rocm directory are again provided with:
-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang
-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++
cmake is instructed to set acpp for gpu support, which is an implementation of SYCL (hence the two options). Also, the rocm “Name” of the GPU is provided again:
-DGMX_GPU=SYCL
-DGMX_SYCL=ACPP
-DHIPSYCL_TARGETS=‘hip:gfx1102’
#taken from gromacs documentation
cd ~/Downloads
wget ftp://ftp.gromacs.org/gromacs/gromacs-2024.4.tar.gz
tar xfz gromacs-2024.4.tar.gz
cd gromacs-2024.4
mkdir build
cd build
sudo cmake .. -DGMX_BUILD_OWN_FFTW=ON \
-DREGRESSIONTEST_DOWNLOAD=ON \
-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang \
-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ \
-DGMX_GPU=SYCL \
-DGMX_SYCL=ACPP \
-DHIPSYCL_TARGETS='hip:gfx1102'
The make commands took quite a while.
sudo make
sudo make check
With make check, one test failed, MDRunIOTests. I chose to continue with the installation.
sudo make install
Now, all that’s left is to delete the downloaded content from ~/Downloads and get started with gromacs.
Remember to source gromacs before each use:
source /usr/local/gromacs/bin/GMXRC
Hope this can help fellow AMD GPU users :)
l
Questions:
Gromacs seems to be running well, but I would like to know if the following may cause problems:
-
make check: test #63 MDRunIOTests failed. This is very similar to Stuck at 'make': libgromacs.so.7: undefined reference to `__kmpc_...' - #9 by al42and. Therefore, I’m not too worried, but would still like to share the log: Dropbox
-
make check, gmx mdrun: produce a warning
WARNING: While sanity checking device #1, Dummy kernel produced invalid values
It’s also in the log file. What to make of this?
Many thanks to the developers, contributors and the community :)
Greetings!