# Parallelization over several GPU nodes

**URL:** <https://gromacs.bioexcel.eu/t/parallelization-over-several-gpu-nodes/1825>\
**Category:** User discussions\
**Tags:** mdrun, gpu\
**Created:** [March 25, 2021, 3:27pm UTC](https://gromacs.bioexcel.eu/t/parallelization-over-several-gpu-nodes/1825 "2021-03-25T15:27:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sperezconesa](https://avatars.discourse-cdn.com/v4/letter/s/4af34b/32.png) [@sperezconesa](https://gromacs.bioexcel.eu/u/sperezconesa)\
**Post date:** [March 25, 2021, 3:27pm UTC](https://gromacs.bioexcel.eu/t/parallelization-over-several-gpu-nodes/1825/1 "2021-03-25T15:27:57Z")

</div>

GROMACS version: 2020 or 2021  
GROMACS modification: No  
Hello,  
I am trying to run 4 different gromacs simulations simultaneously on two nodes each with 2 gpus. All of this should be done in a single slurm job because this calculation is a part of a bigger algorithm. I would like to assing each of the four simulations a single GPU. How should I run this? Like this:  
gmx mdrun -gpu\_id 0  
gmx mdrun -gpu\_id 1  
gmx mdrun -gpu\_id 2  
gmx mdrun -gpu\_id 3  
Or like this?  
gmx mdrun -gpu\_id 0  
gmx mdrun -gpu\_id 1  
gmx mdrun -gpu\_id 0  
gmx mdrun -gpu\_id 1  
Thank you very much for your help!  
Best,  
Sergio

---

<div class="post-metadata">

**Author:** ![Carsten](https://avatars.discourse-cdn.com/v4/letter/c/f14d63/32.png) [@Carsten](https://gromacs.bioexcel.eu/u/Carsten)\
**Post date:** [March 30, 2021, 11:03am UTC](https://gromacs.bioexcel.eu/t/parallelization-over-several-gpu-nodes/1825/2 "2021-03-30T11:03:00Z")

</div>

You could use the multidir functionality for that. For that you would make 4 directories, say, dirA, dirB, dirC, dirD, put a different topol.tpr in each of them, and then start the multi-simulation with something like

`srun -N 4 --tasks-per-node=2 gmx_mpi mdrun -s topol -multidir dirA dirB dirC dirD`

Then, mdrun would automatically assign one simulation per GPU and also do the pinning of the CPU threads correctly for you.

See  
[https://manual.gromacs.org/current/user-guide/mdrun-features.html](https://manual.gromacs.org/current/user-guide/mdrun-features.html)

Best,  
Carsten

---

<div class="post-metadata">

**Author:** ![sperezconesa](https://avatars.discourse-cdn.com/v4/letter/s/4af34b/32.png) [@sperezconesa](https://gromacs.bioexcel.eu/u/sperezconesa)\
**Post date:** [March 30, 2021, 4:18pm UTC](https://gromacs.bioexcel.eu/t/parallelization-over-several-gpu-nodes/1825/3 "2021-03-30T16:18:10Z")

</div>

Thank you very much for the suggestion Karsten. Unfortunatelly the simulation method I am implementing is not programmed for multidir, although maybe it is a good option for the future. I have sort figured out that doing:  
gmx mdrun -gpu\_id 0  
gmx mdrun -gpu\_id 1  
gmx mdrun -gpu\_id 0  
gmx mdrun -gpu\_id 1  
seems to work out alright.  
Thanks!  
Sergio