# Optimizing number of cpu cores in a gpu node run

**URL:** <https://gromacs.bioexcel.eu/t/optimizing-number-of-cpu-cores-in-a-gpu-node-run/8031>\
**Category:** User discussions\
**Tags:** mdrun\
**Created:** [January 13, 2024, 10:53am UTC](https://gromacs.bioexcel.eu/t/optimizing-number-of-cpu-cores-in-a-gpu-node-run/8031 "2024-01-13T10:53:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![GuhLelouch](https://avatars.discourse-cdn.com/v4/letter/g/53a042/32.png) [@GuhLelouch](https://gromacs.bioexcel.eu/u/GuhLelouch)\
**Post date:** [January 13, 2024, 10:53am UTC](https://gromacs.bioexcel.eu/t/optimizing-number-of-cpu-cores-in-a-gpu-node-run/8031/1 "2024-01-13T10:53:32Z")

</div>

GROMACS version: 2023.2  
GROMACS modification: No

Dear GMX users,

I’ve tested the performance of mdrun on a gpu node using different numbers of cpu cores. I’ve noticed that the simulations do not benefit from an increased number of cores (see plot below). Is this an expected behavior when all bonded and non-bonded calculations are offloaded to the gpu? Is there a way to take advantage of the larger number of cores available?

Here are the commands that I’ve used:

```auto
export OMP_NUM_THREADS=40 #Vary this number

export GMX_ENABLE_DIRECT_GPU_COMM=1

export GMX_FORCE_UPDATE_DEFAULT_GPU=true

mpirun -np 1 gmx_mpi mdrun -deffnm md_0_1 -nb gpu -bonded gpu -pme gpu -resetstep 90000 -noconfout

```

 ![image](https://europe1.discourse-cdn.com/flex017/uploads/bioexcel1/original/2X/3/364fc0c9b68e196c948be82133967178b8fe2c93.png)

Best regards,

Gustavo

---

<div class="post-metadata">

**Author:** ![al42and](https://dub1.discourse-cdn.com/flex017/user_avatar/gromacs.bioexcel.eu/al42and/32/1393_2.png) [@al42and](https://gromacs.bioexcel.eu/u/al42and)\
**Post date:** [January 13, 2024, 10:12pm UTC](https://gromacs.bioexcel.eu/t/optimizing-number-of-cpu-cores-in-a-gpu-node-run/8031/2 "2024-01-13T22:12:05Z")

</div>

Hi!

> Is this an expected behavior when all bonded and non-bonded calculations are offloaded to the gpu?

You are already offloading most of the work to the GPU. CPU still has things to do, but, unless you have some CPU-only forces in your simulations, your observations are to be expected.

> Is there a way to take advantage of the larger number of cores available?

You can try using `-bonded cpu` to move some load back from GPU to CPU; since you have plenty CPU resources, that can speed things up.

Note that it’s quite possible that the best performance would be achieved without fully loading all CPU cores. There is granularity and overheads when distributing tasks between CPU and GPU, and we cannot achieve perfectly uniform utilization of all hardware resources in all cases.

---

<div class="post-metadata">

**Author:** ![GuhLelouch](https://avatars.discourse-cdn.com/v4/letter/g/53a042/32.png) [@GuhLelouch](https://gromacs.bioexcel.eu/u/GuhLelouch)\
**Post date:** [January 14, 2024, 12:55am UTC](https://gromacs.bioexcel.eu/t/optimizing-number-of-cpu-cores-in-a-gpu-node-run/8031/3 "2024-01-14T00:55:17Z")

</div>

Dear al42and,

Thanks for the reply.

I’ve tested using -bonded cpu, but the performance using the gpu for all calculations is way superior (more than 2x faster than with -bonded cpu). Maybe the gpu (NVIDIA Tesla V100 GPU card) is just very good at the job?

Best regards,

Gustavo