ake_s
February 23, 2021, 2:03pm
#1
GROMACS version: 2021
GROMACS modification: No
I’m getting failures in the FourCenter tests when building 2021.
Hardware: Intel® Xeon® CPU E5-2690 v4
Compiler: GCC/9.3.0
Libraries: OpenBlas/0.3.9, FFTW/3.3.8
No cmake options set.
[ RUN ] FourCenter/ProperDihedralTest.CheckListed/0
/scratch/ake/gromacs/gromacs-2021.orig/src/testutils/refdata.cpp:934: Failure
In item: /forces/[1]/Z
Actual: 2.9845130443572998
Reference: -14.707831382751465
Difference: 17.6923 (2175423882 single-prec. ULPs, rel. 1.2), signs differ
Tolerance: abs. 1e-12, rel. 1e-12, 1000 ULPs
/scratch/ake/gromacs/gromacs-2021.orig/src/testutils/refdata.cpp:934: Failure
In item: /forces/[2]/Z
Actual: -5.9854903221130371
Reference: 29.496799468994141
Difference: 35.4823 (2192278165 single-prec. ULPs, rel. 1.2), signs differ
Tolerance: abs. 1e-12, rel. 1e-12, 1000 ULPs
/scratch/ake/gromacs/gromacs-2021.orig/src/testutils/refdata.cpp:934: Failure
In item: /forces/[3]/Z
Actual: 3.0009772777557373
Reference: -14.788968086242676
Difference: 17.7899 (2175578016 single-prec. ULPs, rel. 1.2), signs differ
Tolerance: abs. 1e-12, rel. 1e-12, 1000 ULPs
[ FAILED ] FourCenter/ProperDihedralTest.CheckListed/0, where GetParam() = ({ 12-byte object <7F-92 EA-BF 00-00 70-41 02-00 00-00> }, { 12-byte object <00-00 00-00 00-00 00-00 00-00 00-00>, 12-byte object <00-00 00-00 00-00 00-00 CD-CC 4C-3E>, 12-byte object <0A-D7 A3-3B 00-00 00-00 CD-CC CC-3D>, 12-byte object <6F-12 83-BA CD-CC CC-3D 00-00 00-00> }) (0 ms)
and more.
Are they expected or?
Doing a -DCMAKE_BUILD_TYPE=Debug build there are no problems.
ake_s
February 23, 2021, 3:11pm
#2
Doing the build on our Skylakes (Intel® Xeon® Gold 6132 CPU) and otherwise with the same setup on compilers/libs there are no errors in the tests.
This does not reproduce with gcc 9.1 nor 10.2 so I assume it is quite gcc version-specific. Can you check another gcc version? Could you try -O2 and -O1 optimization levels just so we know at which level does it appear?
The numbers seem quite off, so if this does reproduce, I suggest filing an issue on gitlab.
Cheers,
Szilárd
ake_s
February 24, 2021, 9:14pm
#4
I don’t have problems using GCC/10.2.0 either.
Some more data:
env CFLAGS="-O2 -march=native -fno-math-errno" CXXFLAGS="-O2 -march=native -fno-math-errno" cmake -DCMAKE_INSTALL_PREFIX=/scratch/ake/gromacs/inst -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None …/gromacs-2021
No errors
env CFLAGS="-O3 -march=native -fno-math-errno" CXXFLAGS="-O3 -march=native -fno-math-errno" cmake -DCMAKE_INSTALL_PREFIX=/scratch/ake/gromacs/inst -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None …/gromacs-2021
Errors.
env CFLAGS="-O3 " CXXFLAGS="-O3 " cmake -DCMAKE_INSTALL_PREFIX=/scratch/ake/gromacs/inst -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None …/gromacs-2021
No errors
env CFLAGS="-O3 -march=native" CXXFLAGS="-O3 -march=native" cmake -DCMAKE_INSTALL_PREFIX=/scratch/ake/gromacs/inst -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None …/gromacs-2021
No errors
env CFLAGS="-O3 -fno-math-errno" CXXFLAGS="-O3 -fno-math-errno" cmake -DCMAKE_INSTALL_PREFIX=/scratch/ake/gromacs/inst -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None …/gromacs-2021
Errors
So, it’s the -O3 -fno-math-errno combo that is causing problems with GCC/9.3.0
I do not see why disabling setting errno would cause such failures, so I suspect it may be indirectly causing the error.
Can you please check whether this reproduces with other SIMD targets as well (e.g. AVX or SSE4)?
ake_s
February 26, 2021, 4:26pm
#6
Don’t have any such old systems available, but I guess there is a CMAKE flag in GROMACS to target that?
ake_s
February 26, 2021, 5:41pm
#8
GMX_SIMD:
SSE4.1 works
AVX2_128 fails
AVX_256 works
AVX2_256 fails
So i guess it’s fma related.
I’ll try the same + more on our skylakes later.
Can you please file a gitlab issue and record your finding there?