Multiple energy groups is not implemented for GPUs, falling back to the CPU. For better performance,

User discussions
1

Hi all,
I have a system containing around 1 million atoms. With energygrps specified in the .mdp the jobs take about 7 days as the PME calculations can not be done on GPUs. I ran a simulation without energygrps in the .mdp file on GPU and then reran the trajectory with .tpr with energygrps to get the best performance possible. The first job took around 1.5 days without energygrps. Then I specified energygrps in the .mdp and made a new .tpr and used the following command: mpirun -bind-to none gmx_mpi mdrun -rerun mdprevious.trr -s mdnew.tpr -pin on -pme gpu -npme 1 -deffnm md-re. Once again I got this warning ‘Multiple energy groups is not implemented for GPUs, falling back to the CPU. For better performance, run on the GPU without energy groups and then do gmx mdrun -rerun option on the trajectory with an energy group .tpr file’ and the job stopped. I wonder if there is something wrong with the command or the procedure. Also, would it be necessary to have the simulations with energygrps? If I run the jobs without energygrps within 1.5 days would it be enough to get results?
Thank you.

2

Add -nb cpu to your mdrun -rerun command. Re-evaluating energies is very fast and can be done in a short time, even on CPU.

3

Thank you so much Dr Lemkul.
So, when I add -nb cpu to the command having -pme gpu -npme (mx_mpi mdrun -rerun mdprevious.trr -s mdnew.tpr -pin on -pme gpu -npme 1 -deffnm md-re -nb cpu), it complains about the inconsistency between nb and pme as nb is on cpu and pme on gpu (Nonbonded interactions on the GPU were required, but not supported for these). Should I omit pme flag or do pme on cpu as well (something like -pme cpu -npme 1)? mpirun -bind-to none gmx_mpi mdrun -rerun md.trr -s md.tpr -pin on -nb cpu -deffnm md-re is working fine but I am not sure if pme calculations would be redone or not.
Thank you.

4

PME isn’t decomposed using energygrps anyway, but if you’re doing this kind of calculation, nothing can use GPU.

5

Thank you very much Dr Lemkul.
So, if I got you right, the slow simulations were due to the energy groups, not PME. I don’t think I need calculations related to PME. I might need to get mmpbsa of the system, but not sure if it is related.
Thank you.