Issue with segmentation fault

GROMACS version: 2021.2, 2019.6
GROMACS modification: Yes
Here post your question

I am facing a segmentation fault issue with GROMACS patched with plumed and I was wondering if you could help me. I am trying to run a replica averaged MD simulation by calculating the backbone chemical shifts for a protein (using PLUMED: CS2BACKBONE). I have tried this with gromacs-2021.2 patched with plumed-2.7.3 to end with the segmentation fault.

I did follow through a couple of discussions that have dealt with similar issues and according to HREX bug patched with Gromacs-2020.2 · Issue #605 · plumed/plumed2 · GitHub, I have tried to run the multi simulations with gromacs-2019.6 patched with plumed-2.6.3 as well. However, the segmentation fault occurs with both the versions of gromacs and plumed (with and without the use of GPU). Please note that the multi simulations run fine when the plumed instructions are not supplied, which if I am not wrong could mean that my system is not blowing up.

The command used and error message are as follows
mpiexec -np 4 gmx_mpi mdrun -v -deffnm md -plumed plumed -multidir a b c d (I have also tried mpiexec -np 16 gmx_mpi mdrun -v -deffnm md -multidir a b c d -plumed plumed)
Reading file md.tpr, VERSION 2019.6 (single precision)
Reading file md.tpr, VERSION 2019.6 (single precision)
Reading file md.tpr, VERSION 2019.6 (single precision)
Reading file md.tpr, VERSION 2019.6 (single precision)
Changing nstlist from 5 to 80, rlist from 1 to 1.151

This is simulation 3 out of 4 running as a composite GROMACS
multi-simulation job. Setup for this simulation:
Using 1 MPI process
Using 8 OpenMP threads
Changing nstlist from 5 to 80, rlist from 1 to 1.151
This is simulation 0 out of 4 running as a composite GROMACS
multi-simulation job. Setup for this simulation:
Using 1 MPI process
Using 8 OpenMP threads
Changing nstlist from 5 to 80, rlist from 1 to 1.151
This is simulation 2 out of 4 running as a composite GROMACS
multi-simulation job. Setup for this simulation:
Using 1 MPI process
Using 8 OpenMP threads
Changing nstlist from 5 to 80, rlist from 1 to 1.151
This is simulation 1 out of 4 running as a composite GROMACS
multi-simulation job. Setup for this simulation:
Using 1 MPI process
Using 8 OpenMP threads
1 GPU selected for this run.
Mapping of GPU IDs to the 8 GPU tasks in the 4 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PME tasks will do all aspects on the GPU
1 GPU selected for this run.
Mapping of GPU IDs to the 8 GPU tasks in the 4 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PME tasks will do all aspects on the GPU
1 GPU selected for this run.
Mapping of GPU IDs to the 8 GPU tasks in the 4 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PME tasks will do all aspects on the GPU
1 GPU selected for this run.
Mapping of GPU IDs to the 8 GPU tasks in the 4 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PME tasks will do all aspects on the GPU

starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.
starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.
starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.
starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.

Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.


mpiexec noticed that process rank 3 with PID 0 on node exited on signal 11 (Segmentation fault).

The mdp options used are

; Run parameters
integrator = md ; leap-frog integrator
nsteps = 500000 ; 0.002 * 500000 = 1000 ps (1 ns)
dt = 0.002 ; 2 fs
comm_mode = Linear ; remove center of mass translation
nstcomm = 1 ; [steps] frequency of mass motion removal
comm_grps = Protein Non-Protein ; group(s) for center of mass motion removal
; Output control
nstxout = 0 ; suppress .trr output
nstvout = 0 ; suppress .trr output
nstenergy = 1000 ; save energies every 2 ps
nstlog = 1000 ; update log file every 2 ps
nstxtcout = 1000 ; write .xtc trajectory every 2 ps
; energygrps = Protein Non-Protein
; Bond parameters
continuation = no ; first dynamics run
constraint_algorithm = lincs ; holonomic constraints
constraints = h-bonds ; all bonds (even heavy atom-H bonds) constrained
lincs_iter = 2 ; accuracy of LINCS
lincs_order = 6 ; also related to accuracy
; Neighborsearching
ns_type = grid ; search neighboring grid cells
nstlist = 5 ; 10 fs
rlist = 1.0 ; short-range neighborlist cutoff (in nm)
rcoulomb = 1.0 ; short-range electrostatic cutoff (in nm)
rvdw = 1.0 ; short-range van der Waals cutoff (in nm)
; Electrostatics
coulombtype = PME ; Particle Mesh Ewald for long-range electrostatics
coulomb-modifier = Potential-shift-Verlet
vdw-type = Cut-off
vdw-modifier = Potential-shift-Verlet
pme_order = 4 ; cubic interpolation
fourierspacing = 0.16 ; grid spacing for FFT
; Temperature coupling
tcoupl = nose-hoover ; modified Berendsen thermostat
nsttcouple = -1
nh-chain-length = 1
tc-grps = Protein Non-protein ; two coupling groups - more accurate
tau_t = 0.5 0.5 ; time constant, in ps
ref_t = 300 300 ; reference temperature, one for each group, in K
; Pressure coupling
pcoupl = no ; pressure coupling is on for NPT
pcoupltype = isotropic ; uniform scaling of box vectors
tau_p = 2.0 ; time constant, in ps
ref_p = 1.0 ; reference pressure, in bar
compressibility = 4.5e-5 ; isothermal compressibility of water, bar^-1
; Periodic boundary conditions
pbc = xyz ; 3-D PBC
; Dispersion correction
DispCorr = EnerPres ; account for cut-off vdW scheme
; Velocity generation
gen_vel = yes ; assign velocities from Maxwell distribution
gen_temp=300
gen_seed= -1

; SIMULATED ANNEALING
; Type of annealing for each temperature group (no/single/periodic)
annealing = periodic periodic
; Number of time points to use for specifying annealing in each group
annealing_npoints = 5 5
; List of times at the annealing points for each group
annealing_time = 0 100 200 300 600 0 100 200 300 600
; Temp. at each annealing point, for each group.
annealing_temp = 300 300 454 454 300 300 300 454 454 300

Kindly help in resolving the issue.

Thanks and best regards,
Vidya