Error in the REMD md run

User discussions
1

GROMACS version: 2021.5
GROMACS modification: No

Dear users,

I am trying to run a REMD simulation, with the replica files in the working directory having names like testrun_0.tpr, testrun_1.tpr, etc.

However, upon running the command as below:

gmx_mpi mdrun -v -deffnm remd_.tpr -multidir remd_0.tpr remd_1.tpr remd_2.tpr -replex 500
or
gmx_mpi mdrun -v -deffnm remd_ -multidir 0 1 2 -replex 500

I get errors like:
Program: gmx mdrun, version 2021.5
Program: gmx mdrun, version 2021.5
Program: gmx mdrun, version 2021.5
Source file: src/gromacs/utility/futil.cpp (line 739)
Source file: src/gromacs/utility/futil.cpp (line 739)
Function: void gmx_chdir(const char *)
Source file: src/gromacs/utility/futil.cpp (line 739)
Function: void gmx_chdir(const char *)
Function: void gmx_chdir(const char *)
MPI rank: 0 (out of 3)
MPI rank: 1 (out of 3)
MPI rank: 2 (out of 3)

System I/O error:

System I/O error:
System I/O error:
While changing directory for multi-simulation to remd_0.tpr
Cannot change directory to ‘remd_0.tpr’. Reason: Not a directory

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/ErrorsWhile changing directory for multi-simulation to remd_2.tpr
While changing directory for multi-simulation to remd_1.tpr

Cannot change directory to ‘remd_2.tpr’. Reason: Not a directory
Cannot change directory to ‘remd_1.tpr’. Reason: Not a directory

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/ErrorsFor more information and tips for troubleshooting, please check the GROMACS

Rank 1 [Thu Sep 28 12:13:22 2023] [c0-0c1s0n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
Rank 2 [Thu Sep 28 12:13:22 2023] [c0-0c1s0n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
Rank 0 [Thu Sep 28 12:13:22 2023] [c0-0c1s0n1] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
_pmiu_daemon(SIGCHLD): [NID 00065] [c0-0c1s0n1] [Thu Sep 28 12:13:22 2023] PE RANK 1 exit signal Aborted
[NID 00065] 2023-09-28 12:13:22 Apid 21544482: initiated application termination

Please suggest on how to proceed further regarding this. Thanks a lot.

2

The -multidir option implies that you have one directory for each simulation, and you specify the names of the directories with this option. That is why it says that the directories don’t exist. I would suggest creating directories called remd_0, remd_1 etc. Then just put your tpr files in there - one in tpr in each directory. Easiest is if you use the default name topol.tpr. Then gmx_mpi mdrun -multidir remd_0 remd_1 [...] -replex 500 should work. Easiest is of course to avoid writing all the directory names and just use wildcards, such as gmx_mpi mdrun -multidir remd_* -replex 500 or gmx_mpi mdrun -multidir remd_{0..49} -replex 500 or whatever fits your situation.

3

Thank you for your suggestions.
I was applied the and now getting the following error after running the “gmx_mpi mdrun -multidir remd_0 remd_1 remd_2 -replex 500”

GROMACS: gmx mdrun, version 2021.5
Executable: /scratch/apps/gromacs_cray-2021.5/bin/gmx_mpi
Data prefix: /scratch/apps/gromacs_cray-2021.5
Working dir: /home/PrecisionOncologyGrp/20003317/AKT/L1
Command line:
gmx_mpi mdrun -multidir remd_0 remd_1 remd_2 -replex 500

Program: gmx mdrun, version 2021.5
Program: gmx mdrun, version 2021.5
Source file: src/gromacs/utility/futil.cpp (line 481)
Source file: src/gromacs/utility/futil.cpp (line 481)
MPI rank: 1 (out of 3)
MPI rank: 2 (out of 3)

File input/output error:
File input/output error:
topol.tpr

topol.tpr

Program: gmx mdrun, version 2021.5
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/ErrorsFor more information and tips for troubleshooting, please check the GROMACS
website at Common Errors — GROMACS webpage https://www.gromacs.org documentation

Source file: src/gromacs/utility/futil.cpp (line 481)

MPI rank: 0 (out of 3)

File input/output error:
topol.tpr

Rank 1 [Sun Oct 8 17:31:43 2023] [c0-0c0s14n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
Rank 2 [Sun Oct 8 17:31:43 2023] [c0-0c0s14n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
Rank 0 [Sun Oct 8 17:31:43 2023] [c0-0c0s14n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
_pmiu_daemon(SIGCHLD): [NID 00059] [c0-0c0s14n3] [Sun Oct 8 17:31:43 2023] PE RANK 2 exit signal Aborted
[NID 00059] 2023-10-08 17:31:43 Apid 21599487: initiated application termination

Please suggest on how to proceed further regarding this.

Thanks a lot.

4

gmx mdrun cannot find your input files (topol.tpr) Have you checked that you’ve got one topol.tpr file in each of the directories remd_0, remd_1 and remd_2?

5

Yes,
My each folder named with remd_0 , remd_1 and remd_2 have remd_0.tpr’ , remd_1.tpr and remd_2.tpr. with their mdp file
Is any thing ealse I have to put in the folder.

6

I think things get significantly easier if you stick to the default names (topol.tpr), at least when using -multidir. At least they should not have different names in the different directories.

7

Hi,
I am agetting again same error while i have changed the .tpr file name with topol.tpr

gmx_mpi mdrun -multidir remd_0 remd_1 remd_2 -replex 500

Back Off! I just backed up md.log to ./#md.log.1#

Back Off! I just backed up md.log to ./#md.log.1#

Back Off! I just backed up md.log to ./#md.log.1#

Program: gmx mdrun, version 2021.5
Source file: src/gromacs/utility/futil.cpp (line 481)
MPI rank: 2 (out of 3)

File input/output error:
topol.tpr

…
This is my md.mdp file
please check it and suggest me
; Run control
integrator = md
dt = 0.002 ; ps !
nsteps = 50000000; total 10 ns.

; Output control
nstxout = 100
nstvout = 100
nstfout = 100
nstlist = 10
nstlog = 100
nstenergy = 100

; Neighbor searching
cutoff-scheme = group
rlist = 5
pbc = no
ns_type = simple

; Electrostatics and VdW
rvdw = 5
rcoulomb = 5
coulombtype = cut-off
vdwtype = cut-off

;Langevin dynamics
bd_fric = 0
ld_seed = -1

; Bonds
constraints = all-bonds
lincs_order = 4
lincs_iter = 1
lincs-warnangle = 30

; Temperature and Pressure coupling
Tcoupl = v-rescale
tau_t = 1.0
tc-grps = Protein
ref_t = 300
Pcoupl = no

; Velocity generation
gen_vel = yes
gen_temp = 300
gen_seed = 173529
;[gmx-users] Temperature in Brownian Dynamics simulation

;mine addition
comm_mode = angular
nstcomm = 10

; IMPLICIT SOLVENT ALGORITHM
implicit_solvent = gbsa

; GENERALIZED BORN ELECTROSTATICS
; Algorithm for calculating Born radii
gb_algorithm = Still
; Frequency of calculating the Born radii inside rlist
nstgbradii = 1
; Cutoff for Born radii calculation; the contribution from atoms
; between rlist and rgbradii is updated every nstlist steps
rgbradii = 5
; Dielectric coefficient of the implicit solvent
gb_epsilon_solvent = 80
; Salt concentration in M for Generalized Born models
gb_saltconc = 0
; Scaling factors used in the OBC GB model. Default values are OBC(II)
gb_obc_alpha = 1
gb_obc_beta = 0.8
gb_obc_gamma = 4.85
gb_dielectric_offset = 0.009
sa_algorithm = Ace-approximation
; Surface tension (kJ/mol/nm^2) for the SA (nonpolar surface) part of GBSA
; The value -1 will set default value for Still/HCT/OBC GB-models.
sa_surface_tension = -1

8

That’s strange. Could you do ls */ in the directory from where you are starting the simulations? So that we can see that the topol.tpr files are present in all the subdirectories. I’m getting exactly the same error as you get when there is no topol.tpr file in each of the remd_0/, remd_1/ and remd_2/ directories.

9

yes sure
image

this is snapshot of directories.
all folder have the topol.tpr file.

10

Your image doesn’t show the remd_2 directory, but I trust the topol.tpr file is there.

I’m fairly sure the problem is not caused by the mdp file settings. In the output you posted two hours ago, did you remove anything? Were there any other error messages? Because before, it complained that it could not find tpr files in any of the run directories. In your last output it only complained on MPI rank 2 (indicating a problem with the remd_2 directory). You could start a simulation (just gmx_mpi mdrun) only from that directory and see if that works. If that works you could try the same from the other two directories.

Finally, if nothing else helps, it would be good to test with a more recent version of GROMACS (2023.2) to see if it works better with that.

11

Having a closer look at your mdp options I see that you are running with implicit solvent. That is not possible since GROMACS 2018. You should also have a closer look at your Electrostatics and VdW settings and make sure that they match the settings to be used with whatever forcefield you are using.

12

Thank you for suggestions.
But now am getting other type of error as …

Command line:
gmx_mpi mdrun -multidir remd_0 remd_1 remd_2 -replex 500

Reading file topol.tpr, VERSION 2021.4-Ubuntu-2021.4-2 (single precision)
Reading file topol.tpr, VERSION 2021.4-Ubuntu-2021.4-2 (single precision)
Reading file topol.tpr, VERSION 2021.4-Ubuntu-2021.4-2 (single precision)
Changing nstlist from 20 to 100, rlist from 1.222 to 1.343

Changing nstlist from 20 to 100, rlist from 1.227 to 1.353

Changing nstlist from 20 to 100, rlist from 1.23 to 1.359

1 GPU selected for this run.
Mapping of GPU IDs to the 6 GPU tasks in the 3 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
PME tasks will do all aspects on the GPU
1 GPU selected for this run.
Mapping of GPU IDs to the 6 GPU tasks in the 3 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
PME tasks will do all aspects on the GPU
1 GPU selected for this run.
Mapping of GPU IDs to the 6 GPU tasks in the 3 ranks on this node:
PP:0,PME:0,PP:0,PME:0,PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
PME tasks will do all aspects on the GPU

Program: gmx mdrun, version 2021.5
Program: gmx mdrun, version 2021.5
Source file: src/gromacs/gpu_utils/cudautils.cuh (line 81)
Source file: src/gromacs/gpu_utils/cudautils.cuh (line 81)
Function: void gmx::_GLOBAL__N__66ca6d5f_16_device_stream_cu_cfe55bbe::checkDeviceError(const cudaError_t, const std::__cxx11::string &)
Function: void gmx::_GLOBAL__N__66ca6d5f_16_device_stream_cu_cfe55bbe::checkDeviceError(const cudaError_t, const std::__cxx11::string &)
MPI rank: 2 (out of 3)
MPI rank: 1 (out of 3)

Internal error (bug):
Internal error (bug):
Could not create CUDA stream. CUDA error #46 (cudaErrorDevicesUnavailable):
Could not create CUDA stream. CUDA error #46 (cudaErrorDevicesUnavailable):
CUDA-capable device(s) is/are busy or unavailable.
CUDA-capable device(s) is/are busy or unavailable.

Rank 1 [Tue Oct 10 18:46:22 2023] [c0-0c0s6n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
Rank 2 [Tue Oct 10 18:46:22 2023] [c0-0c0s6n3] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
_pmiu_daemon(SIGCHLD): [NID 00027] [c0-0c0s6n3] [Tue Oct 10 18:46:22 2023] PE RANK 2 exit signal Aborted
[NID 00027] 2023-10-10 18:46:22 Apid 21604357: initiated application termination
…
Please suggest on how to proceed further regarding this.

Thanks a lot.

13

It seems like there are problems with running on your GPU - or at least that the multidir run can only use the GPU for the first run (flagged as busy for two processes).

This might be due to a CUDA or Gromacs issue, but I cannot say for sure. If so, it’s hopefully fixed in recent versions. There’s not much point in digging deep into it - before upgrading to a supported Gromacs version. If you cannot upgrade to a supported version you can run with the options -nb cpu -pme cpu.

14

Hi,
I am getting an other error as below

GROMACS: gmx mdrun, version 2021.5
Executable: /scratch/apps/gromacs_cray-2021.5/bin/gmx_mpi
Data prefix: /scratch/apps/gromacs_cray-2021.5
Working dir: /home/PrecisionOncologyGrp/sushil.kumar/AKM/L2/L2
Command line:
gmx_mpi mdrun -deffnm -pin on -cpi md_0_10

Program: gmx mdrun, version 2021.5
Source file: src/gromacs/commandline/cmdlineparser.cpp (line 275)
Function: void gmx::CommandLineParser::parse(int *, char **)

Error in user input:
Invalid command-line options
In command-line option -deffnm
Too few (valid) values

Rank 0 [Sat Nov 4 21:46:53 2023] [c0-0c0s14n0] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
_pmiu_daemon(SIGCHLD): [NID 00056] [c0-0c0s14n0] [Sat Nov 4 21:46:53 2023] PE RANK 0 exit signal Aborted

…
Please suggest on how to proceed further regarding this

Regards

15

The -deffnm option requires that you specify what name you want to use for your simulation input/output. You don’t have to use -deffnm at all, as long as you separate your simulations in different directories.