Why is mpi thinking gromacs args are for mpi?

User discussions
1

Brief description of tools/files:
In my docker file compile OMPI with

RUN cd /opt && git clone https://github.com/NVIDIA/gdrcopy.git && cd gdrcopy/packages && CUDA=/usr/local/cuda ./build-deb-packages.sh
RUN cd /opt && git clone https://github.com/openucx/ucx.git && cd ucx && ./autogen.sh && ./contrib/configure-release --prefix=/usr/local && make -j8 && make install
RUN cd /opt/ompi && ./autogen.pl && ./configure --prefix=/usr/local --with-cuda=/usr/local/cuda --with-gdrcopy=/usr

And compile gromacs with:

wget https://ftp.gromacs.org/gromacs/gromacs-2021.5.tar.gz && tar xvzf gromacs-2021.5.tar.gz && cd gromacs-2021.5 && mkdir build && cd build && cmake .. -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DGMX_BUILD_OWN_FFTW=OFF -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_FFT_LIBRARY=fftw3 -DCMAKE_INSTALL_PREFIX=/usr/local/gromacs -DGMX_GPU=CUDA -DGMX_PYTHON_PACKAGE=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda -DGMX_THREAD_MPI=ON -DGMX_MPI=on -DMPI_C_COMPILER=mpicc

But when I try to pass parameters other than -n* params to gmx_mpi I get the silliest mpirun/prterun errors:

This “runs/works” but it doesn’t know my input and output files from -deffnm:

mpirun -np 12 /usr/local/gromacs/bin/gmx_mpi mdrun -ntomp 2 -npme 4 -ntomp_pme 1

Then when I add -deffnm, mpirun thinks the last commandline arg before -deffnm is for prterun, i.e. for this is thinks the 1 after -ntomp_pme is a prterun param

$ mpirun -np 12 /usr/local/gromacs/bin/gmx_mpi mdrun -ntomp 2 -npme 4 -ntomp_pme 1 -deffnm md_0_1
--------------------------------------------------------------------------
An unrecognized option was included on the prterun command line:

  Option: 1

Please use the "prterun --help" command to obtain a list of all
supported options.
--------------------------------------------------------------------------

And for this is just thinks mdrun is an argument to mpirun. So WEIRD!!!

$ /usr/local/bin/mpirun -np 12 /usr/local/gromacs/bin/gmx_mpi mdrun -deffnm md_0_1
--------------------------------------------------------------------------
An unrecognized option was included on the prterun command line:

  Option: mdrun

Please use the "prterun --help" command to obtain a list of all
supported options.
--------------------------------------------------------------------------
2

I compiled with the lastest open-MPI stable release and now it’s “working” in that I can run it. However I am running on Tesla V100 on LSF cluster and seem to be getting read/write issues such as:

Submission:

mpirun -np 32 /usr/local/gromacs/bin/gmx mdrun -ntmpi 12 -ntomp 3 -npme 4 -ntomp_pme 1 -deffnm md_0_1 -nb gpu

File input/output error:
Cannot rename checkpoint file; maybe you are out of disk space?

Maybe it could just be my cluster but running standard /usr/local/gromacs/bin/gmx mdrun -ntmpi 12 -ntomp 3 -npme 4 -ntomp_pme 1 -deffnm md_0_1 -nb gpu without mpirun works fine.

Here is my call to cmake:

mkdir build && cd build && cmake .. -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DGMX_BUILD_OWN_FFTW=OFF -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_FFT_LIBRARY=fftw3 -DCMAKE_INSTALL_PREFIX=/usr/local/gromacs -DGMX_GPU=CUDA -DGMX_PYTHON_PACKAGE=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda -DGMX_THREAD_MPI=ON -DMPI_C_COMPILER=mpicc

I can attach entire Dockerfile if needed.

3

Just noticed the error also gives reference to mdoutf.cpp

Program:     gmx mdrun, version 2021.5
Source file: src/gromacs/mdlib/mdoutf.cpp (line 463)
MPI rank:    0 (out of 12)

File input/output error:
Cannot rename checkpoint file; maybe you are out of disk space?

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
4

It sounds like you resolved the original question and now you have a new question. It would be most helpful for other members of the forum if you start a new thread.

As I recall, this error message may occur for insufficient write access as well as for low disk space.
I suspect your current problem is that the user that you entered the the Docker container with does not have write access to the working directory from which you launch the simulation. If you open a new thread, please include the docker command line you used to launch the container, check the user id before the mpirun command, and provide the ls -ld output for the full path of the working directory.

In other words, I think this is a Docker usage error rather than a GROMACS error.