Extend option in gmx convert-tpr

GROMACS version:2020.3
GROMACS modification: Yes/No

In order to extend (or reinitiate) an MD simulation (say from 25 ns to 100ns), one of the options in gromax is to use the ‘-extend’ option as in gmx convert-tpr, wherein, the velocities are coordinates are retrieved from the earlier .trr (trajectory) and the energy (.edr) files.

The following script used to run for an earlier version of GROMACS (in my collaborator’s system) with -f for the existing trajectory and -e for the energy input files.

Production

set cnt = 1
set cntmax = 10

while ( {cnt} <= {cntmax} )
if ( {cnt} == 1 ) then gmx grompp -f step5_production.mdp -o step5_{cnt}.tpr -c step4.1_equilibration.gro -n index.ndx -p topol.top
gmx mdrun -v -deffnm step5_{cnt} else @ pcnt = {cnt} - 1
gmx convert-tpr -s step5_{pcnt}.tpr -f step5_{pcnt}.trr -e step5_{pcnt}.edr -o step5_{cnt}.tpr -extend 1000
gmx mdrun -v -deffnm step5_${cnt}
endif
@ cnt += 1
end

However, in the current version (version 2020.3) of gromacs that I am using in my server, the -e and -f options seem to be valid no more.

Command line:
gmx convert-tpr -s 6vw1_AE-orig_md_25ns.tpr -f 6vw1_AE-orig_md_25ns.trr -e 6vw1_AE-orig_md_25ns.edr -o 6vw1_AE-orig_md_100ns.tpr -extend 75000


Program: gmx convert-tpr, version 2020.3
Source file: src/gromacs/commandline/cmdlineparser.cpp (line 275)
Function: void gmx::CommandLineParser::parse(int*, char**)

Error in user input:
Invalid command-line options
Unknown command-line option -f
Unknown command-line option -e

The command however runs fine without returning an error in a reduced form.
gmx convert-tpr -s 6vw1_AE-orig_md_25ns.tpr -o 6vw1_AE-orig_md_100ns.tpr -extend 75000

We are not sure opting this would be correct as we are unsure of how the latest velocities (or for that matter coordinates) are passed on to the next MD run without the .trr and .edr inputs.

Any help would be deeply appreciated.

Hi,

gmx convert-tpr is a tools that allow you just to edit some part the tpr file.
Checkpoint files (cpt) contain the complete state of the system. A simulation can be continued by reading the full state from checkpoint file using the option -cpi in gmx mdrun
(see http://manual.gromacs.org/current/onlinehelp/gmx-mdrun.html?highlight=mdrun)

Best regards
Alessandra

Hi Alessandra,
Many thanks. I shall check this out thoroughly and get back to you if I need more help.
I hope you won’t mind.
Thanks again.

kind regards,
Sankar

Hi again,

Just to make sure that I understand your point correctly - you possibly mean to say that the first of the two commands (gmx convert-tpr with the extend option) may run as it already have (as the following).

gmx convert-tpr -s old.tpr -o new.tpr -extend N

N : run-time in ps

And then the next production-run (‘mdrun’) command using the built new.tpr and the -cpi option can be made to run in the desired way (as an extension of the erlier MD run).

Could you kindly confirm whether I get it correctly.

Thanks again and Kind regards,
Sankar

Yes, that was what I meant
Alessandra

Thanks a lot for confirming.

Regards,
Sankar

Hi Alessandra,

I followed the tutorial page you sent. Checkpoint files (.cpt) are binary files. So, there doesn’t seem to be a question of manual edit. In that case, it appeared that just adding -cpi latest .cpt to the ‘mdrun’ command would serve the purpose of the desired extension. However, there was some renaming done to the output files and hence mismatches were found with respect to information obtained from the latest .cpt file when I ran mdrun with the -cpi option. I tried editing the latest (binary) .cpi file from a vi editor to match the new names (.log, .edr, .trr, .xtc) using the :1,$s/old/new substituion, it resulted in the desired changes in the file but in the process, the file was said to be corrupted. I shall avoid doing this in the future.

Then I found in the tutorial page it says:

A simulation can be continued by reading the full state from file with option -cpi . This option is intelligent in the way that if no checkpoint file is found, GROMACS just assumes a normal run and starts from the first step of the [.tpr](http://manual.gromacs.org/current/reference-manual/file-formats.html#tpr) file.

So, I opted to add (to the mdrun command) the option -cpi without specifying any (.cpt) filename.

nohup gmx mdrun -pin on -pinoffset 0 -reprod -v -deffnm 6vw1_AE-orig_md_30ns -cpi &

It is running at the moment (5 ns in a 24-core server, said to be taking ~16 hrs). I assume that it will produce only but an extension from where I ended last and that latest the .tpr is read (according to the above-quoted part of the tutorial), though, I am not sure whether information related to the latest velocities were recorded in the latest .tpr (from where originally the discussion started).

In the nohup.out I found that the desired .tpr was indeed read.

Back Off! I just backed up 6vw1_AE-orig_md_30ns.log to ./#6vw1_AE-orig_md_30ns.log.1# Reading file 6vw1_AE-orig_md_30ns.tpr, VERSION 2020.3 (single precision) Changing nstlist from 10 to 100, rlist from 1.2 to 1.343

Could you possibly suggest if it should produce an extension only (in this way)?

Thanks again and kind regards,
Sankar

Hi,

Kind regards
Alessandra

Hi!

Many thanks again. Far more clear now.
To check I ran:
gmx check -s1 6vw1_AE-orig_md_25ns.tpr -s2 6vw1_AE-orig_md_30ns.tpr

comparing inputrec

inputrec->nsteps (12500000 - 15000000)

comparing mtop topology
comparing force field parameters
comparing molecule types
comparing atoms
comparing t_resinfo
comparing t_resinfo
comparing t_resinfo
… …
comparing t_resinfo
comparing InteractionLists
comparing blocka excls[0]
comparing atoms
comparing t_resinfo
comparing t_resinfo
… …
comparing t_resinfo
comparing t_resinfo
comparing InteractionLists
comparing blocka excls[1]
comparing atoms
comparing t_resinfo
comparing InteractionLists
comparing blocka excls[2]
comparing atoms
comparing t_resinfo
comparing InteractionLists
comparing blocka excls[3]
comparing atoms
comparing t_resinfo
comparing InteractionLists
comparing blocka excls[4]
comparing molecule blocks
comparing InteractionLists
comparing atomtypes
comparing groups
comparing intermolecular exclusions
comparing moleculeBlockIndices
comparing flags
comparing box
comparing box_rel
comparing boxv
comparing x
comparing v

The only difference seems to be in the number of states (as it should be). At least there are no other comments prompted on any other difference.

kind regards,
Sankar

Hi,

Now that this question is out - could you maybe comment on why the -f option was dropped? I do get corrupted .cpt files due to file system failures once in a while (not something I can control, unfortunately, and often the backup checkpoint won’t be functional either), and these days I have to resort to older gmx versions to get an exact continuation of my run. Not to mention that with no backward compatibility, I first need to recreate the .tpr with an older version, and then use the older convert-tpr. I guess this will only become less sustainable in the long run. Is anyone considering bringing this capability back in future versions? Or is there another ‘official’ way to recover your run from corrupted .cpts?

Best regards,
Miłosz

GROMACS relies on checkpoint files being correct. We removed the -f option from convert-tpr after adding checkpoint files because convert-tpr does not guarantee exact continuation. Things like thermo- and barostat states might not be set (even it can read some from energy files). For non-exact continuation, grompp can read coordinates and velocities from a trr file.

There have been almost no issues with checkpoint files. mdrun computes a checksum, writes the file to disk, reads it again and checks the checksum. The only way the file can be unusable is when the filesystem buffers and the buffer is not flushed in time before the actual filesystem fills up or crashes.

Thanks for the comment. I can’t elaborate on filesystem details, but as I frequently run dozens of synchronized multi-simulation runs with Plumed on top, every two or three months I will come across a corrupted .cpt here and there, usually coinciding with some cluster-wide hardware issues. I guess that could be solved for good with mdrun’s -cpnum, if only you could keep every n-th .cpt instead of storing every single one (just a loose idea).

Best,
Miłosz