PME on GPUs with Version 2021.6

User discussions
1

GROMACS version: 2021.6
GROMACS modification: No

In the release notes from version 2022.1 there is a fix where mdrun could behave incorrectly (Only allow 1D PME GPU decomposition). We have a cluster that due to OS limitations can only accept up to CUDA 10. This restricts us to GROMACS version 2021.6. I did my due diligence trying to understand whether the 2021 versions can accurately use PME with GPUs but couldn’t find a definitive answer.

Specifically:

Can anyone explain what needs to be done on 2021 versions with GMX_PMEONEDD? Set to zero?

Is this a fix for the bug that was addressed in version 2022.1 or should PME on GPUs be completely avoided in version 2021.x?

I appreciate your time in advance.

2

Hi @mrson which OS limitation prevents using CUDA 11?

The oldest major Linux OS still maintained is RHEL/CentOS 7, for which NVIDIA still releases suitable RPMs (latest is 11.8).

Giacomo

3

Thanks for your answer. Our system is on CentOS 6, but my query is independent of OS.

My questions specifically are the following:

  1. Is the default calculation of PME in gpus in GROMACS 2021.x broken?

  2. If so can it be fixed with an environmental variable as hinted in
    “Only allow 1D PME GPU decomposition
    Due to correctness issues in PME grid reduction with 1D decomposition, this feature could produce incorrect results. This would however in most real-world cases be masked by an overly large halo size. 0D decomposition cases are unaffected and only such setups will be allowed in the current release (0D PME decomposition can be forced using the GMX_PMEONEDD env var).”

  3. If GROMACS 2021.x can indeed be used correctly with PME on GPUs, what is the exact value that the environmental variable GMX_PMEONEDD needs to be set to?

4

No.

The bug was only present in the initial release of 2022 and it is related to PME decomposition (not supported in the 2021 release).

In the initial release (before the fix), the GMX_PMEONEDD variable could be could be used to avoid the bug.

5

Dear pszilard,

Thank you for your response. That is exactly the answer I was looking for.