RMSD is higher in dodecahedron then in triclinic box for the same protein

GROMACS version: 2023.3
GROMACS modification: No

I have done several MD simulations for the same protein-protein complex (two chains), using different boxes: dodecahedron and triclinic (basic procedure for proteins with identical parameters in .mdp files). Then I centered my trajectories on the interface residues of the one chain in order to keep chains together in the trajectory and prevent jumps (the only difference was -ur compact option for dodecahedron box).

  1. for cubic box:
    gmx trjconv -s md.tpr -f md.xtc -o md_1.xtc -pbc mol -center -n index.ndx
  2. for dodecahedron box:
    gmx trjconv -s md.tpr -f md.xtc -o md_1.xtc -pbc mol -center -ur compact -n index.ndx

Then used command for RMSD:
gmx rms -s md.tpr -f md_1.xtc -o rmsd.xvg -tu ns

But after calculation of the RMSD I get very differents values:

triclinic box: near 0,5 nm
dodecahedron near 3-5 nm (beginning form the first frame and till the end of simulation)

What may be the reason of the difference and what may be necessary to change in order to get a comparable values for the dodecahedron box?

P. S. When I use em.tpr through the -s flag for the dodecahedron box RMSD is much lower and is near 0,37 nm:
gmx rms -s em.tpr -f md_1.xtc -o rmsd.xvg -tu ns

What box shape did you use with “triclinic”?

One simulation doesn’t say much. RMSD can vary wildly between simulations. You would need at least 3 simulations per box shape to say something with some certainty.

I used triclinic, cubic and dodecahedron boxes when defined box shape (gmx editconf command). After processing my trajectories RMSD was high for dodecahedron box and rather smaller for triclinic and cubic boxes.
I tried to center not only my trajectory, but also the reference structure under the -s flag in gmx rms command and this solved my problem.
It seems that it is necessary to center both trajectory and reference structure using the same index group before rmsd calculations.