Fatal error: Internal error in LAPACK diagonalization

User discussions
1

GROMACS version: 2023
GROMACS modification: Yes/No
I used gmx covar and got the following error

Note: the fit and analysis group are identical,
while the fit is mass weighted and the analysis is not.
Making the fit non mass weighted.

Calculating the average structure …
Reading frame 160 time 661.000

Back Off! I just backed up average.pdb to ./#average.pdb.3#
Constructing covariance matrix (828x828) …
Reading frame 160 time 661.000
Read 162 frames

Trace of the covariance matrix: 113.256 (nm^2)

Diagonalizing …

Intel MKL INTERNAL ERROR: Condition 52500 detected in function SSYEVR.
MKL_VERBOSE oneMKL 2021.0 Update 4 Product build 20210904 for Intel(R) 64 architecture Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) with support of Intel(R) Deep Learning Boost (Intel(R) DL Boost), Lnx 3.00GHz ilp64 sequential
MKL_VERBOSE SSYEVR(V,I,L,22548578304828,0x85ace90,22548578304828,0x7ffe0975253c,0x7ffe09752538,1,828,0x7ffe09752548,6344003425922978946,0x7512070,0x943a160,22548578304828,0x7514320,0x7ffe09752550,-1,0x7ffe09752554,4294967295,52500) 131.06us CNR:OFF Dyn:1 FastMM:1

Program: gmx covar, version 2023
Source file: src/gromacs/linearalgebra/eigensolver.cpp (line 139)

Fatal error:
Internal error in LAPACK diagonalization.

For more information and tips for troubleshooting, please check the GROMACS

website at Common Errors — GROMACS webpage https://www.gromacs.org documentation

Abort(1) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

Please provide your valuable suggestion.

2

Hi!

There appears to be an error in your GROMACS build. Could you please provide gmx -version output and how you build it?

Details: The ilp64 in the error message hints that a wrong flavor of MKL library is linked: GROMACS expects lp64 libraries, so GROMACS provides matrix size as a 4 bytes integer (828 = 0x33c), but ILP64 MKL expects that the size is 8 bytes and reads some adjacent garbage (22548578304828 = 0x0x14820000033c). However, it is not clear why ilp64 gets used; GROMACS build system should prevent that.

3

Here is the output of gmx_mpi -version

image
image1673×463 33.7 KB

4

Thanks. And please provide details on how you built GROMACS. CMake flags, environment modules loaded, etc.

5

I didn’t build GROMACS myself, so I don’t have those details at the moment. However, if you can guide me on how to check the CMake flags and the environment modules loaded, I can look into it and share the information.

Thankyou.

6

CMake settings are part of the GROMACS build tree and not part of the installation.

What I suspect happened is that GROMACS was built with GMX_BLAS_USER and GMX_LAPACK_USER pointing to the ilp64 versions of the MKL libraries. Alternatively, it could be a bug in GROMACS build system leading to wrong library variant to be picked up by it. Since you didn’t build GROMACS yourself, tracking down the root cause isn’t straightforward.

If you’re on an HPC cluster, ask your sysadmin to check whether a newer module is available or to rebuild with the correct MKL (lp64) variant. It could be a good opportunity to also upgrade to a newer, supported version of GROMACS (2025 or 2026); the version you’re using (2023) is no longer supported.

7

Okay, Thankyou.