# Offloading NB and PME to GPUs in multi-sim run

**URL:** <https://gromacs.bioexcel.eu/t/offloading-nb-and-pme-to-gpus-in-multi-sim-run/8085>\
**Category:** User discussions\
**Tags:** mdrun, gpu, simulation-setup\
**Created:** [January 19, 2024, 12:41pm UTC](https://gromacs.bioexcel.eu/t/offloading-nb-and-pme-to-gpus-in-multi-sim-run/8085 "2024-01-19T12:41:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nitinmalapally](https://avatars.discourse-cdn.com/v4/letter/n/a9adbd/32.png) [@nitinmalapally](https://gromacs.bioexcel.eu/u/nitinmalapally)\
**Post date:** [January 19, 2024, 12:41pm UTC](https://gromacs.bioexcel.eu/t/offloading-nb-and-pme-to-gpus-in-multi-sim-run/8085/1 "2024-01-19T12:41:53Z")

</div>

GROMACS version: 2024-dev-20231116-b967ab0c74-dirty-unknown  
GROMACS modification: Yes  
I’m trying to offload NB and PME in a 2 directory multi-sim setup with 4 GPUs, running `mdrun` using a command which looks like this:

```auto
export CUDA_VISIBLE_DEVICES=0,1,2,3
mpirun -n 2 gmx_mpi mdrun -deffnm bench -ntomp 24 -nb gpu -pme gpu -update cpu -multidir ms0 ms1

```

and I get the following (excerpt from .err):

_Error in user input:_  
_The GPU task assignment requested mdrun to use more than one GPU device on a_  
_rank, which is not supported. Request only one GPU device per rank._

How should I call `mdrun` such that each simulation gets 2 GPUs, 1 for NB PP and 1 for PME? I cannot use thread-MPI because the multi-dir simulation requires MPI.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![hess](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/hess/32/416_2.png) [@hess](https://gromacs.bioexcel.eu/u/hess)\
**Post date:** [January 22, 2024, 4:20pm UTC](https://gromacs.bioexcel.eu/t/offloading-nb-and-pme-to-gpus-in-multi-sim-run/8085/2 "2024-01-22T16:20:09Z")

</div>

You need to use mpirun -n 4 (as you have 2 simulations each using 2 ranks) and add the -npme 1 option.