Automatic reduction of energy terms with DD

Developers discussions
,
1

I’m implementing a new algorithm in GROMACS (v2024-dev-20240126-b427519bec-dirty-unknown) called MD in trajectory space or simply Path MD. This is made possible using the multisim feature.

For the logging, I’ve introduced four new terms F_EPOTPMD, F_EKINPMD, F_ETOTPMD and F_TEMPPMD in legacy/include/gromacs/topology/ifunc.h. Then, I’ve activated their output in gromacs/mdlib/energyoutput.cpp.

Now, I’m implementing domain decomposition support for the algorithm. In an example setup with 2 cells per simulation, I’m noticing that the term F_EPOTPMD is being reduced across the cells as shown:

Path MD progress is at 0% (Step 0).
Path MD progress is at 1% (Step 1).
[    2] World rank 0 reports PE PMD  8030.653320
Path MD progress is at 2% (Step 2).
[    3] World rank 1 reports PE PMD  16051.753906
[    3] World rank 0 reports PE PMD  8102.214355
Path MD progress is at 3% (Step 3).
[    4] World rank 1 reports PE PMD  24153.968750
[    4] World rank 0 reports PE PMD  8225.242188
Path MD progress is at 4% (Step 4).
[    5] World rank 1 reports PE PMD  32379.210938
[    5] World rank 0 reports PE PMD  8378.788086
Path MD progress is at 5% (Step 5).
[    6] World rank 1 reports PE PMD  40758.000000
[    6] World rank 0 reports PE PMD  8535.273438
Path MD progress is at 6% (Step 6).

Similarly, the term F_EKINPMD is also being treated similarly:

Path MD progress is at 0% (Step 0).
Path MD progress is at 1% (Step 1).
[    2] World rank 0 reports KE PMD  73354.546875
Path MD progress is at 2% (Step 2).
[    3] World rank 1 reports KE PMD  73354.546875
[    3] World rank 0 reports KE PMD  73294.109375
Path MD progress is at 3% (Step 3).
[    4] World rank 1 reports KE PMD  146648.656250
[    4] World rank 0 reports KE PMD  73223.125000
Path MD progress is at 4% (Step 4).
[    5] World rank 1 reports KE PMD  219871.781250
[    5] World rank 0 reports KE PMD  73101.343750

However, in the energy report, we see:

Energy                      Average   Err.Est.       RMSD  Tot-Drift
-------------------------------------------------------------------------------
PE (PMD)                     410272     110000     228139     782145  (kJ/mol)
KE (PMD)                    73535.7        250         545.52    1508.51  (kJ/mol)
TE (PMD)                     483808     110000     228577     783654  (kJ/mol)

F_EKINPMD is being averaged only using the value in world rank 0. I’m not sure what’s being done with F_EPOTPMD.

Could anyone please explain to me what’s happening here – how and where are the energy terms (automatically) being reduced?

Thanks in advance.

2

I found it, it’s being done in gromacs/mdlib/stat.cpp. This question can be considered closed.