GROMACS version:2023
GROMACS modification: Yes PLUMED MPI CUDA
I am running HREMD on a protein-ligand system. I have determined that I need 32 replicas with an effective temperature schedule from 300K to 425K. This gives me an acceptance rate of about 25%.
Each simulation is using HMR and a timestep of 4fs.
I am running it on nodes containing 8 GPUs and 16-cores. It is not using the GPUs very effectively (they are mostly idle). The md.log gives reasons why it does not offload fully to the GPU:
Update groups can not be used for this system because there are three or more consecutively coupled constraints
Update task can not run on the GPU, because the following condition(s) were not satisfied:
PLUMED is not supported.
The number of coupled constraints is higher than supported in the GPU LINCS code.
8 GPUs selected for this run.
Mapping of GPU IDs to the 64 GPU tasks in the 32 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:1,PME:1,PP:1,PME:1,PP:1,PME:1,PP:1,PME:1,PP:2,PME:2,PP:2,PME:2,PP:2,PME:2,PP:2,PME:2,PP:3,PME:3,PP:3,PME:3,PP:3,PME:3,PP:3,PME
:3,PP:4,PME:4,PP:4,PME:4,PP:4,PME:4,PP:4,PME:4,PP:5,PME:5,PP:5,PME:5,PP:5,PME:5,PP:5,PME:5,PP:6,PME:6,PP:6,PME:6,PP:6,PME:6,PP:6,PME:6,PP:7,PME:7,PP:7,PME:7,PP:7,PME:7,PP:7,PME:7
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
PME tasks will do all aspects on the GPU
This is simulation 0 out of 32 running as a composite GROMACS
multi-simulation job. Setup for this simulation:
Using 1 MPI process
Using 1 OpenMP thread
As far as I can tell, all I need from plumed is the partial tempering tool to create the modified Hamiltonians (I am tempering the non-bonded interactions). Could I run HREMD with gromacs without plumed?
For reference, here is the command I use for executing:
touch plumed.dat
mpirun -np 32 -mca btl vader,self,tcp --oversubscribe gmx_mpi mdrun -cpi -cpt 5 -maxh 330 -plumed plumed.dat -replex 250 -hrex -nsteps 5000000 -multidir run0 run1 run2 run3 run4 run5 run6 run7 run8 run9 run10 run11 run12 run13 run14 run15 run16 run17 run18 run19 run20 run21 run22 run23 run24 run25 run26 run27 run28 run29 run30 run31