dawnpy-tests

Dawn’s QA and test orchestration package. Exposes the standalone dawnpy-tests CLI - the single entry point for the QA pipeline that must pass before any code change is committed.

This is also where the QA runner lives that other documentation pages (notably QA) link back to.

Source: github.com/railab/dawnpy-tests

Installation

Requires the core dawnpy package plus all transport extensions - dawnpy-tests depends on the full communication feature surface because the integration suites exercise every transport:

pip install -e tools/dawnpy
pip install -e tools/dawnpy-serial \
               tools/dawnpy-can \
               tools/dawnpy-udp \
               tools/dawnpy-modbus
pip install -e tools/dawnpy-tests

Direct dependencies (installed automatically): dawnpy, dawnpy-serial, dawnpy-can, dawnpy-udp, dawnpy-modbus, click >= 8.1, ntfc >= 0.0.2.

Command

dawnpy-tests
dawnpy-tests -j 8
dawnpy-tests --test-timeout 120
dawnpy-tests --ntfc-only
dawnpy-tests --skip-ntfc
dawnpy-tests --size-only
dawnpy-tests --help

Common options:

  • -c / --config-file PATH: batch build configuration file. Defaults to tools/config-build-all.txt.

  • --build-root PATH: root directory for build directories. Defaults to build.

  • --test-build-dir NAME: simulator test build directory under --build-root. Defaults to build-sim-sim-tests.

  • --test-timeout SECONDS: simulator test timeout. Defaults to 60.

  • -j / --jobs N: number of parallel build jobs.

  • -v / --verbose: enable verbose runner output.

  • --batch-only: run only batch build and build size analysis.

  • --skip-ntfc: skip NTFC tox and integration test steps.

  • --ntfc-only: run only NTFC tox and integration test steps.

  • --ntfc-list PATH: NTFC manifest YAML file. Defaults to ntfc/manifest-host.yaml.

  • --size-only: run only build size analysis, building missing configurations first.

  • --debug / --no-debug: enable verbose logging.

Pipeline

The runner executes the following steps in order:

  1. Code format check - runs tools/scripts/check-format.sh fix, auto-fixing C/C++ formatting in-place.

  2. cppcheck - runs tools/scripts/cppcheck.sh static analysis.

  3. NTFC tox checks - runs tox -c ntfc/tox.ini unless NTFC is skipped.

  4. Build all configurations - compiles every board/config listed in tools/config-build-all.txt. Supports -j N for parallel builds.

  5. Simulator unit tests - runs the nuttx binary from the simulator test build directory. See Unit Tests.

  6. NTFC integration tests - runs each entry from the active NTFC manifest. The default manifest is ntfc/manifest-host.yaml. See NTFC Integration Tests.

  7. Build size analysis - reports ARM binary memory usage for tracked configurations. Regression visibility only, not a pass/fail gate.

Required Outcome

All steps must pass. No exceptions.

  • Formatting must be clean after auto-fix.

  • cppcheck must pass.

  • NTFC tox checks must pass when NTFC is enabled.

  • Every configuration in tools/config-build-all.txt must compile.

  • All unit tests must pass with no failures.

  • All active NTFC manifest entries must pass.

If any step fails, fix the issue and re-run the full suite before committing. The runner depends on host facilities prepared by testenv_init.sh; see QA for prerequisites.

Tests (the runner’s own tox suite)

The package itself is also tested with tox:

cd tools/dawnpy-tests && tox
tox -e py        # tests + coverage
tox -e format    # formatting check
tox -e flake8    # linting
tox -e type      # type checking