How to run Batsim tests?

Note

Feel free to give a look at our Continuous Integration script to see how our robots run the tests.

Unit tests

Batsim unit tests are integrated into the Meson build system. Unit tests are also integrated in default.nix. Batsim can be built with unit tests: nix-build -A batsim --arg doUnitTests true (assuming that your current working directory is the root of Batsim’s git repository).

Unit tests can also be run manually. In this case you can enter a dedicated environment to build batsim thanks to nix-shell -A batsim — or define your own environment. This enables incremental builds and can be convenient while your are editing Batsim’s source code.

  1. Tell Meson to compile unit tests by setting the -Ddo_unit_tests option when configuring your Meson build: meson build -Ddo_unit_tests=true.

  2. Compile as usual (with Ninja): ninja -C build. This should generate an executable file batunittest in charge of running unit tests.

  3. Run batunittest manually (./build/batunittest) or via Meson (meson test -C build).

Integration tests

Most Batsim tests are integration tests that do the following.

  1. Execute a simulation instance on well-defined input and with a well-defined scheduler (including scheduler parameters).

  2. Make sure that the execution went well (no infinite loop, no Batsim or scheduler crash…).

  3. (Read simulation output files and check that they match some properties.)

Integration tests are written in Python with pytest in the test directory (from the root directory of Batsim’s git repository). As in Running your first simulation, most of these tests use batsched and robin.

The simplest way to run the integration tests on the current local sources of Batsim is to run nix-build -A integration_tests from the root directory of Batsim’s git repository. This will do the following.

  1. (Recompile Batsim from local files if needed.)

  2. Get into an environment with the local Batsim and the various test dependencies.

  3. Run all integration tests.

  4. Generate an HTML test report.

  5. Put results in the nix store — accessible from a result symbolic link.

Results and logs should be convenient to explore from the HTML report. In other words: firefox ./result/pytest_report.html.

Alternatively, you can enter a shell where all the dependencies are available (nix-shell -A integration_tests) and call pytest manually from there. Please note that with this approach, you must reenter the shell whenever you modify Batsim’s source code (as it needs to be compiled again). When you are working on a specific test, it can be useful to only run this test while compiling Batsim if needed before running it. More advanced nix-shell commands such as nix-shell -A integration_tests --command "pytest test/test_nosched.py" can be very useful in this case.

Other tests

The generation of Batsim’s Doxygen documentation should not issue any warning. This can be checked manually or just by running nix-build -A doxydoc from Batsim’s repository root directory.