Scaling problem for very large systems

Gusten
Mon, 20 May 2024 09:50:05 +0000
User discussions

GROMACS version:2024.2
GROMACS modification: No
I’m currently benchmarking gromacs against my own code (that operates in a significantly different way) for large scale particle simulation. I’m comparing an atomistic representation to a coarse-grained representation, and to my method, for different sizes systems of small carbon nanotubes that are randomly distributed at the same concentration. Gromacs appears to scale well, approximately linearly until I get to systems with several million atoms, where I see a drastic slowdown. For example when I increase the number of atoms from 480000 to 4800000 in the atomistic model, the system takes ~250x longer, rather than the expected ~10x, and I’m not sure why.

I’m running the systems with nonbonded interaction on an RX6800XT, and while the utilization shows 100%, from what I can see in the power use goes down a lot for these bigger systems. I initially suspected the neighbor finding to be the problem, but looking at CPU activity this does not appear to be the case, as the bursts of activity on the CPU are relatively short.
Currently my only hypothesis is related to the 128MB ‘infinity cache’ on this GPU, and that the larger systems somehow don’t work well with it. However, I would not expect such a huge performance degradation, as the cache system is only marketed as an ‘up to 3.25x’ bandwidth improvement.

Is this a known issue? What could be the cause and are there any solutions, except for running on different hardware?

15 posts - 3 participants