Skip to content

Testing

make test          # cargo test
make lint          # clippy
make fmt           # rustfmt --check

The whole suite runs offline in under a second — no network, no compiler toolchain, no writes outside a temp dir. Anything that would need a real source build belongs in the Docker e2e matrix instead (see below).

Layout

File Covers
tests/cli_help.rs --help/--version and every subcommand's help
tests/cli_scenarios.rs End-to-end binary runs against a throwaway --prefix
tests/database.rs Installed-package database: persistence, reverse deps, corruption
tests/registry.rs Loading definitions, version selection, malformed files, search
tests/resolver.rs Dependency resolution and topological build order
tests/package_parse.rs Package JSON parsing and per-OS/arch overrides
tests/platform.rs Prefix resolution and platform naming
src/**/mod tests Unit tests next to the code (see below)

Unit tests inside src/ cover the parts with no public API to drive from a test binary:

  • src/ops/fetch.rs — archive extraction for every supported compression, magic-byte fallback for misnamed downloads, and the zip loop's refusal to write outside the destination. Archives are built in-memory, so nothing is downloaded.
  • src/cli/ui/ — filter/navigation/batch logic, plus render smoke tests that draw every tab and overlay into a TestBackend at sizes down to 1×1. Layout math that underflows panics inside draw, so a successful render is the assertion.
  • src/util/sha256.rs, src/core/registry.rs — hashing and version comparison.

tests/fixtures/packages/ holds a tiny three-package registry (curl → zlib, openssl) used where a realistic definition is needed.

Writing a test

Pure logic (resolver, registry, parsing) — build definitions inline as JSON strings in a tempfile::tempdir() and load a Registry from it. registry_of() in tests/resolver.rs and registry_from() in tests/registry.rs do exactly that; copy the shape rather than reaching for a fixture file, so the test's input is readable next to its assertion.

A whole command — add to tests/cli_scenarios.rs. It runs the real binary via env!("CARGO_BIN_EXE_tsi") with --prefix <tempdir>, so every filesystem effect lands in the temp dir. seed_db() writes a db/installed.json directly, which is how you get to a "packages already installed" state without building anything.

Two things to know when asserting on output:

  • All human-facing output goes to stderr; only --json output goes to stdout. Use the combined() helper unless you are specifically testing machine-readable output.
  • Commands that refuse to act exit non-zero (e.g. a blocked uninstall). Assert on the exit status as well as the message.

System scenarios

tests/cli_scenarios.rs deliberately exercises the states a real machine lands in:

  • fresh, empty prefix
  • prefix with no package definitions (must point the user at tsi update)
  • prefix populated from a local definitions directory (tsi update --local)
  • database with dependency relationships (uninstall guard)
  • a concurrently held install lock

When you fix a bug that only shows up in one of those states, add the state here.

Docker e2e

Real source builds across distros live in docker/ and run in the e2e.yml workflow:

cd docker && ./run-tests.sh

That matrix (Alpine/Ubuntu, minimal and no-tools images, x86_64 and arm64) is where toolchain-dependent behavior is verified. Keep it for things that genuinely need a compiler; everything else should be a cargo test.

Cross-architecture package validation

Package builds are CI's job. test-build-packages.yml builds every changed package on Linux-x86_64, Linux-aarch64, macOS-aarch64 and macOS-x86_64 per PR, and validate-all-packages.yml rebuilds the whole catalogue weekly on the same four -- every platform GitHub has hosted hardware for -- and regenerates PACKAGES_STATUS.md. Each slow package (gcc, llvm, ...) gets a job of its own, so one long build cannot time out the rest. Where a package cannot build on a platform, the reason is kept in scripts/status-notes.tsv and shown in the table's Notes column. Those results are the ones that count; a laptop's are not recorded anywhere.

Locally, smoke-test the definition you are editing and let CI do the rest:

make validate PKGS="zlib bzip2"                  # linux/arm64 + linux/amd64
./docker/validate-packages.sh --platform linux/amd64 rsync

make validate with no PKGS refuses rather than starting a full run, and --all needs an explicit TSI_VALIDATE_ALL_I_MEAN_IT=1: a full catalogue run needs tens of gigabytes per architecture and hours of build time, and it filled this project's development machine once.

The script builds tsi inside a container per platform and runs tsi install for each package. On an Apple Silicon host linux/amd64 runs under emulation — correct, just slow. Results land in .validate-logs/<arch>/results.tsv in the same format CI produces, and failing build logs stay next to them.

A package that genuinely cannot build somewhere declares it with platforms (see OS-specific configuration) instead of quietly failing there.

Nothing here has to be run locally to be trusted — every check below runs in CI. The local runner exists so you get the answer in minutes instead of after a push.

In this repository

Workflow When What it proves
ci.yml push / PR (release on main) lint stage: clippy -D warnings and fmt --check on Linux, macOS and Windows; actionlint; shellcheck over tsi-bootstrap.sh and docker/*.sh. Then the build and e2e stages below. On a push to main it finally tags and publishes the next minor release; a major release is a manual run with bump = major
e2e.yml called by ci.yml a real source build of bzip2 inside bare containers on every Linux release architecture: Alpine/Debian/Ubuntu/Fedora natively on x86_64 and aarch64, Alpine and Debian under qemu on i686, armv7, armv6 (Alpine only), riscv64 and ppc64le; plus a zero-dependency proof on each of them that the static binary works with no toolchain at all
build-binaries.yml called by ci.yml the release binary for every shipped platform builds: Linux x86_64/aarch64 natively and i686/armv7/armv6/riscv64/ppc64le with cargo-zigbuild (static, smoke-tested under qemu-user), macOS aarch64 and x86_64, Windows x86_64 and aarch64. the release publishes exactly these artifacts, and runs cargo test on the native platforms

In tsi-packages

Workflow When What it proves
package-validation.yml push / PR catalogue schema and invariants, version ordering, the scripts' self-checks, shellcheck, and that changed packages' sources are pinned and reachable
test-build-packages.yml push / PR changed packages really build on Linux-x86_64, Linux-aarch64, macOS-aarch64 and macOS-x86_64, and their installed binaries can actually load; a failing leg ends with why each package failed
verify-sources.yml weekly every package's default source still downloads and matches its recorded sha256
validate-all-packages.yml weekly + manual the whole catalogue on all four platforms, slow packages included, regenerating PACKAGES_STATUS.md with the reason for every package that cannot build somewhere

The behaviour that a full-catalogue run would otherwise be the only witness to is pinned by ordinary tests instead, so it is checked on every commit rather than every week: the archive-cache keying (src/ops/fetch.rs), make_args expansion (src/ops/build.rs), and the platform gate and update --local self-copy guard (tests/cli_scenarios.rs).

CI

ci.yml runs lint, then the build and test matrix, then the e2e matrix on push and PR. A change is expected to be green before merge.