Testing Practices

Testing Practices#

Edgegraph follows a strict rule of unit testing and code coverage. *All code* is to be covered by unit testing, with exceptions only made for cases which are well and truly needed (and justification is documented). The brilliant PyTest framework is used to drive unit testing, and Coverage.py provides coverage analysis. pytest-cov links the two together, and pytest-randomly is also employed to ensure order of unit tests does not impact the tests.

Tests are written in the tests/ directory of the project, and are not bundled with the module to end users. Nonetheless, standards for documentation and code quality are enforced on all unit tests. The entire directory is scanned with PyLint on all GitHub pull requests.

Running tests#

From the command line, you can run all tests with a simple pytest command issued from the project root. Ideally, all tests will be executed and pass. PyTest can be integrated with numerous IDEs probably including yours, though configuring that is outside the scope of this documentation (probably found in your IDE docs).

By default, some slower stress tests will be executed as well. These tests are marked as slow, and can be excluded by passing -m "not slow" as an argument to PyTest, for example pytest -m "not slow". This flag is set in the GitHub actions invocation of tests, and therefore, 100% code coverage must be achieved without slow testing.

Coverage analysis#

The included scripts/ut_cov.sh shell script executes all unit tests and performs coverage analysis. The invocation used here is identical to that in the GitHub actions pipeline; therefore, this script accurately matches the behavior of the pipeline. This is useful for testing changes locally before pushing to GitHub.

The script places an HTML report in the docs/_build/htmlcov folder; after running it, open docs/_build/htmlcov/index.html in your browser to see results in an interactive format showing exactly what lines / branches were missed.

Writing tests#

Writing unit tests is an essential part of adding, changing, or removing functionality from EdgeGraph. Organization of the tests closely mirrors organization of the sourcecode they test, and all tests relevant to any given module should be in a test subdirectory easily apparent to match the code which it tests. An extra test subdirectory, tests.integration, is used for testing activities which combine functionality of multiple sourcecode modules.

The following testing practices are kept:

  • Unit tests should be kept relatively small where possible (i.e., they should test “a single unit of code.”)

  • All assertions are provided a message to indicate in plain English what failed, should the assertion fail

  • All unit tests must provide at least a short docstring to summarize what the test tests.

  • All test modules must provide at least a short docstring to summarize the overall topic of the tests contained.

  • PyLint messages may be ignored at the module level, if needed. Justification must be provided in a code comment near to the pylint disable flag. (This is NOT a freebie to write bad code – only ignore messages that are truly unavoidable. W0212 is a common offender.)

For convenience and code cleanliness, test authors may wish to familiarize themselves with the tests.conftest module. This module contains PyTest fixtures available to all test modules (automatically, no import needed!). Commonly-used graph constructors can be found here. See the module documentation for more info.

Test List#

You can find an API-like summary & description of all unit tests here:

tests

Contains unit, integration, and interactive tests for EdgeGraph.