# Running Simulations in Parallel Across Multiple Nodes/GPUs

**URL:** <https://gromacs.bioexcel.eu/t/running-simulations-in-parallel-across-multiple-nodes-gpus/1339>\
**Category:** User discussions\
**Created:** [December 26, 2020, 2:41pm UTC](https://gromacs.bioexcel.eu/t/running-simulations-in-parallel-across-multiple-nodes-gpus/1339 "2020-12-26T14:41:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Carmen](https://avatars.discourse-cdn.com/v4/letter/c/a9adbd/32.png) [@Carmen](https://gromacs.bioexcel.eu/u/Carmen)\
**Post date:** [December 26, 2020, 2:41pm UTC](https://gromacs.bioexcel.eu/t/running-simulations-in-parallel-across-multiple-nodes-gpus/1339/1 "2020-12-26T14:41:39Z")

</div>

GROMACS version: 2020.4  
GROMACS modification: No

I’m having trouble running simulations in parallel, and I’m not sure if it’s something wrong with the flags I’m using or if it’s because of the way my versions of GROMACS are compiled.

If I want to run using 2 GPUS, my submission file (the relevant part) looks like this:

_#SBATCH --nodes=1_  
_#SBATCH --gres=gpu:V100:2_  
_#SBATCH --ntasks-per-node=2_  
_#SBATCH --cpus-per-task=20_

_mpirun -np 2 mdrun\_mpi -ntomp 20 -deffnm pull_

What I’m thinking that I’m doing is running using 2 ranks (one for each GPU), and each rank uses 20 OMP threads (so 20 CPUs/GPU ). Did I understand this correctly?  
Either way, this is giving me segmentation faults (output file “out-gpu.log” attached). [out-gpu.log](https://gromacs.bioexcel.eu/uploads/short-url/qpsxNn5jEUiVZqprkwNJ2xXymFW.log) (5.2 KB)

On the other hand, if I want to run a simulation using CPUs only across multiple nodes, this is what my submission file looks like:

_#SBATCH --nodes=2_  
_#SBATCH --ntasks-per-node=1_  
_#SBATCH --cpus-per-task=20_

_mpirun -np 2 mdrun\_mpi -ntomp 20 -deffnm pull -pf pullf.xvg -px pullx.xvg_

So here I think I’m using 1 MPI process for each node, and 20 CPUs per node. I also got a segmentation fault (output file “out-cpu.log” attached). [out-cpu.log](https://gromacs.bioexcel.eu/uploads/short-url/3rjYUkzgKBKG5KHN06Bpdel1eCA.log) (5.8 KB)

Any help would be highly appreciated.

---

<div class="post-metadata">

**Author:** ![cblau](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/cblau/32/5_2.png) [@cblau](https://gromacs.bioexcel.eu/u/cblau)\
**Post date:** [January 7, 2021, 5:41pm UTC](https://gromacs.bioexcel.eu/t/running-simulations-in-parallel-across-multiple-nodes-gpus/1339/2 "2021-01-07T17:41:02Z")

</div>

Hi Carmen,

in your case, the error indicates that there is an issue with the simulation box rather than the way you use CPUs and GPUs. If you have a look at the final line of the `.gro` file you use for starting the simnulation (or the `cryst` record if you use a `pdb` file), you should see how your box is defined and should find some extreme angle there.