Running a pytest test from CMake where the test and sources are in different folders

Question

I have a CMake based project that uses both C++ and python sources. Tests are added via add_test for all C++ tests in a folder structure like:

src
  mynamespace/foo.cpp
  mypyspace/mypkg/__init__.py
  mypyspace/mypkg/bar.py
test
  mynamespace/testFoo.cpp
  mypyspace/testBar.py

In testBar.py I have a import mypkg and want to run this with add_test(... COMMAND pytest WORKING_DIRECTORY .../test)

Of course it does not find my package, but I also do not want to install it (it is required to generate some stuff during build and is actively developed as part of the other sources).

What is the correct/preferred way of executing those tests?

Notes:

I don't want to move the sources out of src or tests out of test
set_tests_properties may set environment variables.

I think setting PYTHONPATH=.../src/mypyspace should work, but that would also ignore changes to the users PYTHONPATH.

Hm, you ask how to run these tests from CMake, but how e.g. `test/mypyspace/testBar.py` script is intended to be run **from a command line**? How environment variables should be set for properly run it? — Tsyvarev, Sep 25 '18 at 09:47
Don't know. I assume there exists something I can set (like `PYTHONPATH`) and this may ore may not be easier in CMake. — Flamefire, Sep 25 '18 at 10:40
You don't know how the tests are expected to be run, but ask us how to do that in CMake, really? — Tsyvarev, Sep 25 '18 at 10:51
Yes. The approach of separating the src and test in folders is so common (especially in CMake projects) that I assume, someone is doing this already. See related questions: https://stackoverflow.com/questions/47859986/usual-pytest-workflow-cant-run-tests-in-a-file (but answer was like: "pytest requires installation of src") — Flamefire, Sep 25 '18 at 10:56
In `set_tests_properties` you may assign value for `PYTHONPATH` variable equal to its current value plus additional paths, like: `$ENV{PYTHONPATH}:${CMAKE_CURRENT_SOURCE_DIR}/../src`. Note, that current value of the `PYTHONPATH` variable is taken at the time when `cmake` is run, but it is unlikely that a user will change that path when run the tests. — Tsyvarev, Sep 25 '18 at 11:02
Yes, but wanted to avoid this. In HPC modules may change this. But based on your criticism I was able to find something in the docu of pytest and posted that as an answer. — Flamefire, Sep 25 '18 at 11:16

score 15 · Accepted Answer · edited May 11 '22 at 14:46

There are 2 options:

Use PYTHONPATH

add_test (NAME python-tests
  COMMAND ${PYTHON_EXECUTABLE} -m pytest # Or just COMMAND pytest
  WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)
set_tests_properties(python-tests
    PROPERTIES ENVIRONMENT "PYTHONPATH=${PROJECT_SOURCE_DIR}/src/mypyspace:$ENV{PYTHONPATH}")

Invoke inside of the src directory

add_test (NAME python-tests
  COMMAND ${PYTHON_EXECUTABLE} -m pytest ${CMAKE_CURRENT_SOURCE_DIR}
  WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}/src/mypyspace
)

The first approach has the disadvantage, that the user might configure and later change its environment so that PYTHONPATH is different (e.g. loading modules in HPC clusters) and find that the tests incorrectly succeed.

The 2nd is the most flexible one. It relies on information from https://docs.pytest.org/en/latest/pythonpath.html. Most important part:

the [python -m pytest] call will add the current directory to sys.path.

So we only have to set the current directory when running the tests to the source directory.

Note: For other alternatives specific to unittest (the module) you may also find something in Running unittest with typical test directory structure

Running a pytest test from CMake where the test and sources are in different folders

1 Answers1