Question on domain decomposition grid error

User discussions
1

GROMACS version:2019 doublé precision
GROMACS modification: Yes/No

Hello, I’m somewhat new to gromacs and found and issue i need some help with.

I´ve been running some NPT simulations on ionic liquids with success, recently ran in to the error “The domain decomposition grid has shifted too much in the Y-direction around cell 3 1 0. This should not have happened. Running with fewer ranks might avoid this issue.”

I tried looking into the manuals and online but I don’t think I’m understanding the issue.

Could someone explain what this error is about and what could cause it?

2

Hi,

The error would indicate that your particles shift around a lot between the logical compartmentalisatoin that is used to speed up the simulations on multiple nodes.

Some guess what this error might indicacte:

  • that you simulate very few particles on many nodes
  • you might have some sorts of cavity developing in your system
  • you might have extreme temperatures emerging in your system
3

Thank you.

If I understand correctly the particles move through the zone where they were originally placed at the start of the simulation, changing which processor is responsible for calculations on that particle. The problem would be that a processor has little or much to do triggering the error.

Certainly a difference between the job that failed and some of the successful ones is temperature, which i increased to match experimental data. Also checking the trajectories for cavities seems like a reasonable practice.

4

Hello,
I’m also getting a similar error while performing NPT equilibration.
Herewith I have attached command prompt.
Kindly suggest how this error can be fixed and any changes needs to be made.

Step=20778, Dmax= 1.2e-04 nm, Epot= -2.25233e+05 Fmax= 1.33392e+02, atom= 80155
Step=20786, Dmax= 1.1e-06 nm, Epot= -2.25233e+05 Fmax= 1.32090e+02, atom= 80155
Energy minimization has stopped, but the forces have not converged to the
requested precision Fmax < 1 (which may not be possible for your system). It
stopped because the algorithm tried to make a new step whose size was too
small, or there was no change in the energy since last step. Either way, we
regard the minimization as converged to within the available machine
precision, given your starting configuration and EM parameters.

Double precision normally gives you higher accuracy, but this is often not
needed for preparing to run molecular dynamics.
You might need to increase your constraint accuracy, or turn
off constraints altogether (set constraints = none in mdp file)

writing lowest energy coordinates.

Back Off! I just backed up em.gro to ./#em.gro.1#

Steepest Descents converged to machine precision in 20787 steps,
but did not reach the requested Fmax < 1.
Potential Energy = -2.2523256e+05
Maximum force = 1.3339163e+02 on atom 801
Norm of force = 2.6295207e+00

Command line:
gmx_mpi grompp -f nvt.mdp -c em.gro -r em.gro -p topol.top -o nvt.tpr -maxwarn 1

Ignoring obsolete mdp entry ‘title’
Ignoring obsolete mdp entry ‘ns_type’
Setting the LD random seed to 1235768857
Generated 380 of the 1326 non-bonded parameter combinations
Excluding 3 bonded neighbours molecule type ‘Protein_chain_A’
Excluding 2 bonded neighbours molecule type ‘SOL’
Excluding 1 bonded neighbours molecule type ‘NA’

WARNING 1 [file topol.top, line 8192]:
The GROMOS force fields have been parametrized with a physically
incorrect multiple-time-stepping scheme for a twin-range cut-off. When
used with a single-range cut-off (or a correct Trotter
multiple-time-stepping scheme), physical properties, such as the density,
might differ from the intended values. Since there are researchers
actively working on validating GROMOS with modern integrators we have not
yet removed the GROMOS force fields, but you should be aware of these
issues and check if molecules in your system are affected before
proceeding. Further information is available at
and a longer explanation of our
decision to remove physically incorrect algorithms can be found at

Setting gen_seed to 192348891
Velocities were taken from a Maxwell distribution at 300 K
Cleaning up constraints and constant bonded interactions with virtual sites
Number of degrees of freedom in T-Coupling group Protein is 3588.61
Number of degrees of freedom in T-Coupling group non-Protein is 24318.39

NOTE 1 [file nvt.mdp]:
Removing center of mass motion in the presence of position restraints
might cause artifacts. When you are using position restraints to
equilibrate a macro-molecule, the artifacts are usually negligible.

Estimate for the relative computational load of the PME mesh part: 0.82

NOTE 2 [file nvt.mdp]:
The optimal PME mesh load for parallel simulations is below 0.5
and for highly parallel simulations between 0.25 and 0.33,
for higher performance, increase the cut-off and the PME grid spacing.

There were 2 notes

There was 1 warning

Back Off! I just backed up nvt.tpr to ./#nvt.tpr.1#

Command line:
gmx_mpi mdrun -deffnm nvt

Back Off! I just backed up nvt.log to ./#nvt.log.1#
Reading file nvt.tpr, VERSION 2020.3 (single precision)
Changing nstlist from 10 to 100, rlist from 1 to 1

Using 64 MPI processes
Using 1 OpenMP thread per MPI process

Back Off! I just backed up nvt.trr to ./#nvt.trr.1#

Back Off! I just backed up nvt.edr to ./#nvt.edr.1#
starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.

Writing final coordinates.

Back Off! I just backed up nvt.gro to ./#nvt.gro.1#

Dynamic load balancing report:
DLB was turned on during the run due to measured imbalance.
Average load imbalance: 289.7%.
The balanceable part of the MD step is 16%, load imbalance is computed from this.
Part of the total run time spent waiting due to load imbalance: 45.4%.
Steps where the load balancing was limited by -rdd, -rcon and/or -dds: X 0 % Y 0 %

NOTE: 45.4 % of the available CPU time was lost due to load imbalance
in the domain decomposition.
You can consider manually changing the decomposition (option -dd);
e.g. by using fewer domains along the box dimension in which there is
considerable inhomogeneity in the simulated system.

           Core t (s)   Wall t (s)        (%)
   Time:    45032.538      703.634     6400.0
             (ns/day)    (hour/ns)

Performance: 122.791 0.195

Command line:
gmx_mpi grompp -f npt.mdp -c nvt.gro -r nvt.gro -t nvt.cpt -p topol.top -o npt.tpr -maxwarn 1

Ignoring obsolete mdp entry ‘title’
Ignoring obsolete mdp entry ‘ns_type’
Setting the LD random seed to 71025073
Generated 380 of the 1326 non-bonded parameter combinations
Excluding 3 bonded neighbours molecule type ‘Protein_chain_A’
Excluding 2 bonded neighbours molecule type ‘SOL’
Excluding 1 bonded neighbours molecule type ‘NA’

WARNING 1 [file topol.top, line 8192]:
The GROMOS force fields have been parametrized with a physically
incorrect multiple-time-stepping scheme for a twin-range cut-off. When
used with a single-range cut-off (or a correct Trotter
multiple-time-stepping scheme), physical properties, such as the density,
might differ from the intended values. Since there are researchers
actively working on validating GROMOS with modern integrators we have not
yet removed the GROMOS force fields, but you should be aware of these
issues and check if molecules in your system are affected before
proceeding. Further information is available at
and a longer explanation of our
decision to remove physically incorrect algorithms can be found at

Cleaning up constraints and constant bonded interactions with virtual sites

NOTE 1 [file topol.top, line 8192]:
You are combining position restraints with Parrinello-Rahman pressure
coupling, which can lead to instabilities. If you really want to combine
position restraints with pressure coupling, we suggest to use Berendsen
pressure coupling instead.

The center of mass of the position restraint coord’s is 6.698 6.706 6.703
The center of mass of the position restraint coord’s is 6.698 6.706 6.703
Number of degrees of freedom in T-Coupling group Protein is 3588.61
Number of degrees of freedom in T-Coupling group non-Protein is 24318.39

NOTE 2 [file npt.mdp]:
Removing center of mass motion in the presence of position restraints
might cause artifacts. When you are using position restraints to
equilibrate a macro-molecule, the artifacts are usually negligible.

Reading Coordinates, Velocities and Box size from old trajectory
Will read whole trajectory
Last frame -1 time 1000.000
Using frame at t = 1000 ps
Starting time for run is 0 ps
Estimate for the relative computational load of the PME mesh part: 0.82

NOTE 3 [file npt.mdp]:
The optimal PME mesh load for parallel simulations is below 0.5
and for highly parallel simulations between 0.25 and 0.33,
for higher performance, increase the cut-off and the PME grid spacing.

There were 3 notes

There was 1 warning

Back Off! I just backed up npt.tpr to ./#npt.tpr.1#

Command line:
gmx_mpi mdrun -deffnm npt

Back Off! I just backed up npt.log to ./#npt.log.1#
Reading file npt.tpr, VERSION 2020.3 (single precision)
Changing nstlist from 10 to 100, rlist from 1 to 1

Using 64 MPI processes
Using 1 OpenMP thread per MPI process

Back Off! I just backed up npt.trr to ./#npt.trr.1#

Back Off! I just backed up npt.edr to ./#npt.edr.1#
starting mdrun ‘Protein in water’
500000 steps, 1000.0 ps.

Program: gmx mdrun, version 2020.3
Source file: src/gromacs/domdec/partition.cpp (line 654)
MPI rank: 2 (out of 64)

Fatal error:
step 10100: The domain decomposition grid has shifted too much in the
Y-direction around cell 0 2 0. This should not have happened. Running with
fewer ranks might avoid this issue.

For more information and tips for troubleshooting, please check the GROMACS
website at

application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2

Program: gmx mdrun, version 2020.3
Source file: src/gromacs/domdec/partition.cpp (line 654)
MPI rank: 58 (out of 64)

Fatal error:
step 10100: The domain decomposition grid has shifted too much in the
Y-direction around cell 7 2 0. This should not have happened. Running with
fewer ranks might avoid this issue.