Hi,
There is no special setting for vacuum simulation in GROMACS. You can perform a vacuum simulation by setting your protein in a very large box and running using an NVT ensemble.
The option ‘coulombtype’ in the mdp file allows you to define the setting for the electrostatic interaction (PME vs cut-off). Note that the non-bonded interaction setting usually is link to the force field parameterization, and usually biomolecule force fields are not parameterized for vacuum simulations.
Here a post on similar topic How to simulate protein in vacuum in Gromacs 2021.4
I hope it helps
\Alessandra