Code instrumentation for performance analysis

User discussions
1

For a check of applicability of performance tools I have chosen gromacs for scorep instrumentation. A successful attempt has been describeb in a prace paper “Profiling and Tracing Tools for Performance Analysis of Large Scale Applications”
Things in practice are not this easy…

Without instrumentation a call to cmake like:

cmake .. -DGMX_BUILD_OWN_FFTW=ON -DGMX_MPI=on -DCMAKE_INSTALL_PREFIX=destination_folder> -DCMAKE_CXX_COMPILER=mpic++ -DCMAKE_C_COMPILER=mpicc

works for gromacs-2024 with gcc-11.4.0 and openmpi-4.1.6. It becomes problematic when switching to the scorep wrappers in combination with the aforementioned version triple (scorep itself is 8.3). The call to cmake then is changed to

cmake .. -DGMX_BUILD_OWN_FFTW=ON -DGMX_MPI=on -DCMAKE_INSTALL_PREFIX=destination_folder> -DCMAKE_CXX_COMPILER=scorep-mpic++ -DCMAKE_C_COMPILER=scorep-mpicc

This causes the compilation to fail for muparser:

muParserBase.cpp:1880:77: error: expected '#pragma omp' clause before ')' token
 1880 | #pragma omp parallel for schedule(static, std::max(nBulkSize/nMaxThreads, 1)) private(nThreadID)
muParserBase.cpp:1880:77: error: expected ')' before 'nowait'
 1880 | #pragma omp parallel for schedule(static, std::max(nBulkSize/nMaxThreads, 1)) private(nThreadID)

In order to cross check the cause I have tried different combinations of gromacs, scorep and compilers. The result is the same.

By adding

-DGMX_OPENMP=OFF

to the cmake call gromacs does compile for the price of having openmp being disabled completely. When run gromacs causes in this case scorep to gather data.

Since the use of an external muparser build is possible I have used the code provided as part of the gromacs source download to be compiled seperatly. Then adding the following to the cmake call:

-DCMAKE_PREFIX_PATH=<path_to_muparser_dir> -DGMX_USE_MUPARSER=EXTERNAL

Compilation then stops with a no such file or directory for muparser.h.

It looks like for some reason cmake neither by -DCMAKE_... nor by -DGMX_... is able to recognize what it is desired to do.

Any ideas how to convince cmake to act properly?

2

Hi!

Curiously, the same problem was observed for NVIDIA’s NVC++ compiler.

You can safely replace the offending line in src/external/muparser/src/muParserBase.cpp with these two:

		const int chunkSize = std::max(nBulkSize/nMaxThreads, 1);
#pragma omp parallel for schedule(static, chunkSize) private(nThreadID)

Then you should be able to use the internal muparser just fine.

In general, -DGMX_USE_MUPARSER=EXTERNAL works well (that’s how, e.g., Debian packages GROMACS), so I suspect something is wrong either with CMAKE_PREFIX_PATH or with your muParser installation.

3

The isssu with muparser indeed is resolved by the code change. Comoilation then goes on until

[ 54%] Building CXX object src/gromacs/CMakeFiles/libgromacs.di/math/utilities.cpp.o
[Score-P] ERROR: Execution failed: .../bin/opari2 --omp-task-untied=keep --preprocessed --disable=pomp optimization_1710153935_790989.input.prep.cpp optimization_1710153935_790989.input.prep.opari.cpp

Errors point to

src/gromacs/CMakeFiles/libgromacs.dir/build.make:4339: src/gromacs/CMakeFiles/libgromacs.dir/math/optimization.cpp.o

and

CMakeFiles/Makefile2:4260: src/gromacs/CMakeFiles/libgromacs.dir/all

without any direct reference to a line contained in

gromacs-2023.4/src/gromacs/math/optimization.cpp

Cross checked with gromacs-2023.4

4

Well, unlike the pragma omp problem, we don’t have a workaround at hand for that.

Luckily, the functionality from this file is not used currently (the function there is in preparation for a new feature), so you can try removing the mention of it from src/gromacs/math/CMakeLists.txt. A dirty solution, but the only alternative I see is debugging Score-P compiler/parser/whatever it is that fails on this file.

5

For which combination of versiosns (gromacs, compilers, scorep, etc.) do you get a working setup?

6

We don’t use ScoreP internally. As for compilers, GCC 9-13 and Clang 9-17 are regularly tested.