Streaming Interface (as update of IMD)
During the meeting, we (Matthias Heyden, Amruthesh Thirumalaiswamy, Heekun Cho, Arizona State University) presented a streaming interface to enable on-the-fly analysis of MD simulation trajectories, which we implemented (fork of gromacs-2024.4: Files · imd-v3 · ljwoods2 / GROMACS · GitLab) on the basis of the existing interactive molecular dynamics (IMD) protocol that is already implemented in GROMACS.
Our motivation is that, especially in microsecond simulations, a lot information on fast processes (solvent dynamics, intermolecular vibrations) is lost due to infrequent output of trajectory information, e.g. every 10-100 ps (higher output frequencies would lead to VERY large files and excessive I/O traffic). Using a streaming interface, which transmits data during the simulation to analysis software (here MDAnalysis), information from fast fluctuations can be extracted from a running simulation, without requiring high output frequencies.
More details are given below. As the next step, we will draft an issue in the GitLab repository to propose the implementation of our updated IMD protocol (IMD v2 → IMD v3).
Starting Point: Interactive Molecular Dynamics (IMD v2)
The existing IMD protocol allows for the communication of atomic coordinates and energies from the server (GROMACS) to a client (i.e., VMD) via a TCP/IP socket interface. However, the purpose of IMD is limited to on-the-fly visualization of trajectories and receiving forces from the client (e.g., generated by user interactions such as pulling atoms with a haptic device or mouse). However, any distance-based trajectory analysis would require additional information, e.g., box dimensions.
Project: Updated Streaming Interface (IMD v3)
In collaboration with the Beckstein group at Arizona State University, we developed an updated IMD protocol (IMD v3, defined in: IMDv3 Protocol — IMDClient documentation). This protocol allows for the (optional) communication of all pieces of information relelvant for the analysis of a trajectory: time, coordinates, velocities, forces, box dimensions, energies. In addition, the communication frequency can be defined, and various control commands can be sent to the GROMACS server (some existing in IMD v2, some added). Further, our implementation protocol is fully backwards compatible with IMD v2 and only requires minimal changes in the source code outside of the IMD module.
We added similar implementations of IMD v3 already in next release of LAMMPS and are currently finalized the same for the next release of NAMD3, all of which featured implementations of IMD v2.
Client Implementation
On the client side, e.g., to receive information from GROMACS via IMD v3, we developed a standalone python IMDClient interface (GitHub - Becksteinlab/imdclient: Streaming analysis from running MD simulations.), which receives data from the TCP/IP socket, handles buffering etc. This IMDClient acts as a simple interface to other programs. Specifically for MDAnalysis, we developed the IMDReader class (part of IMDClient), which reads data from the received stream. The usage within MDAnalysis effectively mimics reading from a file. The main differences are that instead of a filename, the user provides a host address for the server and the TCP/IP port used for communication. Further, random access to trajectory frames is naturally not possible and processing can only occur in a feed-forward manner.
We recently held a virtual workshop for members of the MDAnalysis developer community (Workshop Material: GitHub - MDAnalysis/imd-workshop-2024: Materials for the virtual 2024 MDAnalysis IMD streaming workshop.) in which we demonstrated the functionality of the interface (in GROMACS, NAMD3 and LAMMPS), including demos for simple live analysis tools enabled by streaming:
- customizable trajectory output control
- vibrational density of states
- water transport
- distance monitoring