# Multiple node simulation using gromacs

**URL:** <https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916>\
**Category:** User discussions\
**Created:** [October 6, 2020, 1:47pm UTC](https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916 "2020-10-06T13:47:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![scinikhil](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/scinikhil/32/3483_2.png) [@scinikhil](https://gromacs.bioexcel.eu/u/scinikhil)\
**Post date:** [October 6, 2020, 1:47pm UTC](https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916/1 "2020-10-06T13:47:47Z")

</div>

GROMACS version:  
GROMACS modification: Yes/No  
Here post your question Dear all,  
We are having two cluster nodes with two cards each. I wanted to run GROMACS using the s four cards. Anyone here having experience of doing multiple node GROMACS simulation? If yes, kindly let me know how to do that.

---

<div class="post-metadata">

**Author:** ![lindahl](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/lindahl/32/415_2.png) [@lindahl](https://gromacs.bioexcel.eu/u/lindahl)\
**Post date:** [October 13, 2020, 9:19am UTC](https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916/2 "2020-10-13T09:19:50Z")

</div>

Just follow the instructions to create an MPI build.

Having said that, paralleling over multiple GPUs is difficult unless the system is large enough, and you have a very low-latency network (e.g. Infiniband). If you try to use gigabit ethernet with cheap cards/switches, you might have round-trip times of 300 microseconds. Since we need to send a handful of messages in a typical simulation step, you can easily use 1-2 milliseconds just waiting for the network.

… and since a single GPU will typically get you more than 500 steps/second, you will be bottlenecked by your network, rather than GPUs.

---

<div class="post-metadata">

**Author:** ![alexmathew](https://avatars.discourse-cdn.com/v4/letter/a/f4b2a3/32.png) [@alexmathew](https://gromacs.bioexcel.eu/u/alexmathew)\
**Post date:** [October 13, 2020, 11:05am UTC](https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916/3 "2020-10-13T11:05:17Z")

</div>

so it is better to add more cards on a single node than more nodes with cards?

---

<div class="post-metadata">

**Author:** ![lindahl](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/lindahl/32/415_2.png) [@lindahl](https://gromacs.bioexcel.eu/u/lindahl)\
**Post date:** [October 13, 2020, 11:23am UTC](https://gromacs.bioexcel.eu/t/multiple-node-simulation-using-gromacs/916/4 "2020-10-13T11:23:17Z")

</div>

Yes, with some caveats:

1. If you are looking for highest absolut performance, start with the fastest GPUs you can afford.

2. You need a recent version of GROMACS

3. You still need a decent number number of CPU cores per GPU (although much less than what it used to be)

4. The connectivity CPU-to-GPU matters, so you can’t just plug four GPUs in a cheapo consumer motherboard. Dual CPU sockets also means it will be slower to communicate between some cards.

5. There is a reason some people shell out lots of $$$ for professional cards/nodes with direct GPU-to-GPU interconnects (such as DGX-1).

You can find some multi-GPU benchmarks in the recent paper by @pszilard :

[https://aip.scitation.org/doi/full/10.1063/5.0018516](https://aip.scitation.org/doi/full/10.1063/5.0018516)

Cheers,

Erik