My simulation crashes when the output file grows to 2.4 GB. The tail content of the nohup.out log is attached below. Could you advise on solutions to fix this problem?
nohup.out
/opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_Z19dd_partition_systemP8_IO_FILElP9t_commreciiP7t_statePK10gmx_mtop_tPK10t_inputrecS4_PSt6vectorIN3gmx11BasicVectorIfEENSC_9AllocatorISE_NSC_23AlignedAllocationPolicyEEEEPNSC_7MDAtomsEP14gmx_localtop_tP10t_forcerecP11gmx_vsite_tP10gmx_constrP6t_nrnbP13gmx_wallcyclei+0x3e0d)[0x7fcad248a15d]
[customerr:05225] [ 6] /opt/software/gromacs-2018/bin/gmx_mpi(+0x10f5b)[0x561998345f5b]
[customerr:05225] [ 7] /opt/software/gromacs-2018/bin/gmx_mpi(+0x31e8e)[0x561998366e8e]
[customerr:05225] [ 8] /opt/software/gromacs-2018/bin/gmx_mpi(+0x19857)[0x56199834e857]
[customerr:05225] [ 9] /opt/software/gromacs-2018/bin/gmx_mpi(+0x1a355)[0x56199834f355]
[customerr:05225] [10] /opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_ZN3gmx24CommandLineModuleManager3runEiPPc+0x288)[0x7fcad2449328]
[customerr:05225] [11] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe39c)[0x56199834339c]
[customerr:05225] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fcad12b7c87]
[customerr:05225] [13] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x56199834347a]
[customerr:05225] *** End of error message ***
/opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_ZN3gmx24CommandLineModuleManager3runEiPPc+0x288)[0x7fedda90d328]
[customerr:05238] [11] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe39c)[0x56274937b39c]
[customerr:05238] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7fedd977bc87]
[customerr:05238] [13] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x56274937b47a]
[customerr:05238] *** End of error message ***
/usr/lib/x86_64-linux-gnu/libgomp.so.1(GOMP_parallel+0x3f)[0x7f94f70e5edf]
[customerr:05272] [ 4] /opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_Z17nbnxn_put_on_gridP12nbnxn_searchiPA3_fiPfS3_iifPKiS2_iPiiP16nbnxn_atomdata_t+0x12ee)[0x7f94f9a1598e]
[customerr:05272] [ 5] /opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_Z19dd_partition_systemP8_IO_FILElP9t_commreciiP7t_statePK10gmx_mtop_tPK10t_inputrecS4_PSt6vectorIN3gmx11BasicVectorIfEENSC_9AllocatorISE_NSC_23AlignedAllocationPolicyEEEEPNSC_7MDAtomsEP14gmx_localtop_tP10t_forcerecP11gmx_vsite_tP10gmx_constrP6t_nrnbP13gmx_wallcyclei+0x3e0d)[0x7f94f8d6315d]
[customerr:05272] [ 6] /opt/software/gromacs-2018/bin/gmx_mpi(+0x10f5b)[0x55668eaaef5b]
[customerr:05272] [ 7] /opt/software/gromacs-2018/bin/gmx_mpi(+0x31e8e)[0x55668eacfe8e]
[customerr:05272] [ 8] /opt/software/gromacs-2018/bin/gmx_mpi(+0x19857)[0x55668eab7857]
[customerr:05272] [ 9] /opt/software/gromacs-2018/bin/gmx_mpi(+0x1a355)[0x55668eab8355]
[customerr:05272] [10] /opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_ZN3gmx24CommandLineModuleManager3runEiPPc+0x288)[0x7f94f8d22328]
[customerr:05272] [11] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe39c)[0x55668eaac39c]
)[0x7fc62b19cc87]
[customerr:05298] [13] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x5591c4b3947a]
[customerr:05298] *** End of error message ***
/opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_ZN3gmx24CommandLineModuleManager3runEiPPc+0x288)[0x7f8819247328]
[customerr:04958] [11] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe39c)[0x56380922339c]
[customerr:04958] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f88180b5c87]
[customerr:04958] [13] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x56380922347a]
[customerr:04958] *** End of error message ***
[customerr:05272] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f94f7b90c87]
[customerr:05272] [13] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x55668eaac47a]
[customerr:05272] *** End of error message ***
+0xe47a)[0x5607e2a0c47a]
[customerr:05041] *** End of error message ***
/opt/software/gromacs-2018/bin/../lib/libgromacs_mpi.so.3(_ZN3gmx24CommandLineModuleManager3runEiPPc+0x288)[0x7f2bc2a25328]
[customerr:05284] [10] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe39c)[0x55dc1f39739c]
[customerr:05284] [11] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f2bc1893c87]
[customerr:05284] [12] /opt/software/gromacs-2018/bin/gmx_mpi(+0xe47a)[0x55dc1f39747a]
[customerr:05284] *** End of error message ***
mpirun noticed that process rank 29 with PID 0 on node customerr exited on signal 11 (Segmentation fault).