Problem building gmxapi Python package

User discussions
,
1

@ericirrgang
Hello everyone,

I installed Gromacs with GMXAPI=ON (please see the command below) but when I try to import gmx in a python script (import gmxapi as gmx) it shows Module Error: No module name gmxapi

cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=OFF -DREGRESSIONTEST_PATH=/home/biomedical/gromacs/gromacs-2022.2/regressiontests-2022 -DGMX_MPI=OFF -DGMX_GPU=ON -DCMAKE_BUILD_TYPE=Debug -DGMX_GPU=CUDA -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda -DGMXAPI=ON

Gromacs Installation details:

GROMACS version:    2022.2
Precision:          mixed
Memory model:       64 bit
MPI library:        thread_mpi
OpenMP support:     enabled (GMX_OPENMP_MAX_THREADS = 128)
GPU support:        CUDA
SIMD instructions:  AVX_512
CPU FFT library:    fftw-3.3.8-sse2-avx-avx2-avx2_128-avx512
GPU FFT library:    cuFFT
RDTSCP usage:       enabled
TNG support:        enabled
Hwloc support:      disabled
Tracing support:    disabled
C compiler:         /bin/cc GNU 8.3.1
C compiler flags:   -mavx512f -mfma -mavx512vl -mavx512dq -mavx512bw -Wno-missing-field-initializers -fno-inline -g
C++ compiler:       /bin/c++ GNU 8.3.1
C++ compiler flags: -mavx512f -mfma -mavx512vl -mavx512dq -mavx512bw -Wno-missing-field-initializers -fno-inline -fopenmp -g
CUDA compiler:      /usr/local/cuda/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver;Copyright (c) 2005-2022 NVIDIA Corporation;Built on Tue_May__3_18:49:52_PDT_2022;Cuda compilation tools, release 11.7, V11.7.64;Build cuda_11.7.r11.7/compiler.31294372_0
CUDA compiler flags:-std=c++17;;-D_FORCE_INLINES;-mavx512f -mfma -mavx512vl -mavx512dq -mavx512bw -Wno-missing-field-initializers -fno-inline -fopenmp -g
CUDA driver:        11.70
CUDA runtime:       11.70

I also tried to install gmxapi by following Full installation instructions — GROMACS 2022.2 documentation

python3 -m ensurepip --default-pip

Requirement already satisfied: setuptools in ./myvenv/lib/python3.6/site-packages
Requirement already satisfied: pip in ./myvenv/lib/python3.6/site-packages



pip install --upgrade pip setuptools wheel

Requirement already satisfied: pip in /usr/local/lib/python3.6/site-packages (21.3.1)
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (59.6.0)
Requirement already satisfied: wheel in /usr/local/lib/python3.6/site-packages (0.37.1)

sudo pip3 install --no-cache-dir gmxapi

ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.6 /usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpb41pares
       cwd: /tmp/pip-install-nac_bfaf/gmxapi_6874d278d79843fd843f1b66aa4f7879
  Complete output (50 lines):
  
  The `gmxapi` package requires an existing GROMACS installation, version 2020 or higher.
  To specify the GROMACS installation to use, provide a GMXTOOLCHAINDIR
  environment variable when running setup.py or `pip`.
  
  Example:
      GMXTOOLCHAINDIR=/path/to/gromacs/share/cmake/gromacs pip install gmxapi
  
  If you have multiple builds of GROMACS, distinguished by a suffix `$SUFFIX`, the
  tool chain directory will use that suffix.
  
  Example:
      GMXTOOLCHAINDIR=/path/to/gromacs/share/cmake/gromacs$SUFFIX pip install gmxapi

Traceback (most recent call last):
    File "/usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-f9j_55w2/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 163, in get_requires_for_build_wheel
      config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-f9j_55w2/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 143, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-f9j_55w2/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 268, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-f9j_55w2/overlay/lib/python3.6/site-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 127, in <module>
      raise GmxapiInstallError('Could not configure for GROMACS installation. Provide GMXTOOLCHAINDIR.')
  __main__.GmxapiInstallError: Could not configure for GROMACS installation. Provide GMXTOOLCHAINDIR.

I also tried installing gmxapi by giving the path of gromacs installation
sudo GMXTOOLCHAINDIR=/usr/local/gromacs/share/cmake/gromacs/ pip3 install --no-cache-dir gmxapi

but shows the following error

ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.6 /usr/local/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmphg05fl_6
       cwd: /tmp/pip-install-kiayp7g7/gmxapi_dd620b39d47646f4aa139f12a15b94a9
  Complete output (78 lines):
  Not searching for unused variables given on the command line.
  CMake Error at /tmp/pip-build-env-rykyc8mt/overlay/lib64/python3.6/site-packages/cmake/data/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:130 (message):
    Could not find toolchain file:
    /usr/local/gromacs/share/cmake/gromacs/gromacs-toolchain.cmake
  Call Stack (most recent call first):
    CMakeLists.txt:2 (PROJECT)
  
  
  CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
  -- Configuring incomplete, errors occurred!
  Not searching for unused variables given on the command line.
  CMake Error at /tmp/pip-build-env-rykyc8mt/overlay/lib64/python3.6/site-packages/cmake/data/share/cmake-3.22/Modules/CMakeDetermineSystem.cmake:130 (message):
    Could not find toolchain file:
    /usr/local/gromacs/share/cmake/gromacs/gromacs-toolchain.cmake
  Call Stack (most recent call first):
    CMakeLists.txt:2 (PROJECT)
  
  
  CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
  -- Configuring incomplete, errors occurred!
  
  
  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator

  -- Trying "Ninja" generator - failure
  --------------------------------------------------------------------------------

  -- Trying "Unix Makefiles" generator

  --------------------------------
  -- Trying "Unix Makefiles" generator - failure
  --------------------------------------------------------------------------------
  
  ********************************************************************************
  scikit-build could not get a working generator for your system. Aborting build.
  
  Building Linux wheels for Python 3.6 requires a compiler (e.g gcc).
  But scikit-build does *NOT* know how to install it on rhel
  
  To build compliant wheels, consider using the manylinux system described in PEP-513.
  Get it with "dockcross/manylinux-x64" docker image:
  
    https://github.com/dockcross/dockcross#readme
  
  For more details, please refer to scikit-build documentation:
  
    http://scikit-build.readthedocs.io/en/latest/generators.html#linux
  
  ********************************************************************************
  ----------------------------------------
  ERROR: Failed building wheel for gmxapi
Failed to build gmxapi
ERROR: Could not build wheels for gmxapi, which is required to install pyproject.toml-based projects

Please help me in troubleshooting and installing python package for Gromacs.

2

The error during import is because the Python package is not installed.

The Python package is not installed because of the CMake failure during pip install.

Please note that gmxapi 0.3+ requires Python 3.7. Your Python 3.6 environment is causing pip to try to install an older version of gmxapi that does not know how to build against GROMACS 2022.

Older versions of gmxapi are still available (and will be considered by pip when no version is specified with pip install gmxapi), but will not be patched to maintain compatibility with newer GROMACS releases. GROMACS 2022 no longer provides the CMake “toolchains” file that was required by older gmxapi Python packages.

Also, please reconsider installing Python packages as root / with sudo. For one thing, it can cause temporary files that you don’t own to be placed in your home directory. It looks like you are using a venv called myvenv, though, so I don’t know why you are using sudo.

I’m surprised that ensurepip notes pip in ./myvenv/lib/python3.6/site-packages while pip install --upgrade pip notes pip in /usr/local/lib/python3.6/site-packages. I wonder if pip resolved to something other than ./myvenv/bin/pip (e.g. outside of the venv). Do you find that you still need to use python3 and pip3 (rather than just python and pip) even after activating your venv?

3

@ericirrgang
Thank you for your kind consideration.

I have two versions of python, python 3.6 and python 3.9

pydoc3 -> /etc/alternatives/pydoc3
pydoc3.6
pyinotify
python3 -> /etc/alternatives/python3
python3.6 -> /usr/libexec/platform-python3.6
python3.6-config -> python3.6m-config
python3.6m -> /usr/libexec/platform-python3.6m
python3.6m-config -> /usr/libexec/platform-python3.6m-config
python3.6m-x86_64-config -> /usr/libexec/platform-python3.6m-x86_64-config
python3-config -> /etc/alternatives/python3-config
pyvenv-3 -> /etc/alternatives/pyvenv-3
pyvenv-3.6

where python or python3 is python 3.9


$ python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58) 
[GCC 7.5.0] :: Anaconda, Inc. on linux

So can this problem be fixed by deleting python 3.6 libraries?

if I am using simple python or pip it says permission denied and if I use sudo python /pip it says command not found so I have to use python3 and pip3 with sudo in order to give permission to write/place files. and sudo python3 is invoking python3.6 .

Since my academic background is entirely different so I have no clue at all why is this happening.
Please help me out by guiding me over how to fix this problem.

Thanks…

4

It is very important to use a pip that is consistent with the python you intend to use.

I realize that it can be confusing to manage multiple Python installations on a single computer, but it is far too complex a topic to discuss in much detail here. I cannot advise you on what to do with your Python 3.6 installation.

If your Python 3.9 installation is from Anaconda, then you should follow the Conda/Anaconda instructions for creating a conda virtual environment. Once that environment is installed and activated, use conda install to install pip in that environment. Then you should be able to use that particular version of pip to install gmxapi, according to the instructions.

Note that activating and deactivating a Python virtual environment (conda or otherwise) manipulates your PATH environment variable, which affects which executable is used when you use pip or python on the command line without an absolute path.

If any pip command gives you a permission error, it is either because

  • you have not activated a virtual environment
  • you are accidentally using a different pip than the one associated with your virtual environment,
  • you have broken your virtual environment through a previous use of sudo, or
  • you have broken pip’s temporary directories through a previous use of sudo

For the first two cases, make sure that you create a fresh virtual environment, activate it, and make sure that you have pip installed in the virtual environment. which pip should help you figure that out.

For the latter two cases, it should be sufficient to remove the virtual environment directory (which you may need sudo for), your $HOME/.local directory, or whereever the files are that pip is unable to manipulate.

Note that you don’t absolutely have to use a virtual environment. When not using a virtual environment, you can provide the --user option to pip install to install a package into your home directory instead of to a system location (which would give a permission error). However, when multiple Python installations or virtual environments are present, the --user option installs the package to a place visible to all python invocations, which can be so confusing that I hesitate to mention it. (Ref: “It is very important to use a pip that is consistent with the python you intend to use.”)

5

@ericirrgang

Thanks for your kind support.

Regarding this, If I install older version of Gromacs ie Gromacs 2021 or Gromacs 2020, will that be okay with the python3.6 configuration?

Thanks

6

Yes. But I still encourage you to create a virtual environment using the python 3.6 interpreter. For example:

/usr/bin/python3.6 -m venv $HOME/myvenv
. $HOME/myvenv/bin/activate
python3 -m pip install --upgrade pip setuptools wheel

Please see the corresponding documentation. (Full installation instructions — GROMACS 2021.5 documentation) Installation instructions and other requirements can be different across releases.