Testing
The whole suite runs offline in under a second — no network, no compiler toolchain, no writes outside a temp dir. Anything that would need a real source build belongs in the Docker e2e matrix instead (see below).
Layout
| File | Covers |
|---|---|
tests/cli_help.rs |
--help/--version and every subcommand's help |
tests/cli_scenarios.rs |
End-to-end binary runs against a throwaway --prefix |
tests/database.rs |
Installed-package database: persistence, reverse deps, corruption |
tests/registry.rs |
Loading definitions, version selection, malformed files, search |
tests/resolver.rs |
Dependency resolution and topological build order |
tests/package_parse.rs |
Package JSON parsing and per-OS/arch overrides |
tests/platform.rs |
Prefix resolution and platform naming |
src/**/mod tests |
Unit tests next to the code (see below) |
Unit tests inside src/ cover the parts with no public API to drive from a test binary:
src/ops/fetch.rs— archive extraction for every supported compression, magic-byte fallback for misnamed downloads, and the zip loop's refusal to write outside the destination. Archives are built in-memory, so nothing is downloaded.src/cli/ui/— filter/navigation/batch logic, plus render smoke tests that draw every tab and overlay into aTestBackendat sizes down to 1×1. Layout math that underflows panics insidedraw, so a successful render is the assertion.src/util/sha256.rs,src/core/registry.rs— hashing and version comparison.
tests/fixtures/packages/ holds a tiny three-package registry (curl → zlib,
openssl) used where a realistic definition is needed.
Writing a test
Pure logic (resolver, registry, parsing) — build definitions inline as JSON strings in
a tempfile::tempdir() and load a Registry from it. registry_of() in
tests/resolver.rs and registry_from() in tests/registry.rs do exactly that; copy the
shape rather than reaching for a fixture file, so the test's input is readable next to its
assertion.
A whole command — add to tests/cli_scenarios.rs. It runs the real binary via
env!("CARGO_BIN_EXE_tsi") with --prefix <tempdir>, so every filesystem effect lands in
the temp dir. seed_db() writes a db/installed.json directly, which is how you get to a
"packages already installed" state without building anything.
Two things to know when asserting on output:
- All human-facing output goes to stderr; only
--jsonoutput goes to stdout. Use thecombined()helper unless you are specifically testing machine-readable output. - Commands that refuse to act exit non-zero (e.g. a blocked
uninstall). Assert on the exit status as well as the message.
System scenarios
tests/cli_scenarios.rs deliberately exercises the states a real machine lands in:
- fresh, empty prefix
- prefix with no package definitions (must point the user at
tsi update) - prefix populated from a local definitions directory (
tsi update --local) - database with dependency relationships (uninstall guard)
- a concurrently held install lock
When you fix a bug that only shows up in one of those states, add the state here.
Docker e2e
Real source builds across distros live in docker/ and run in the e2e.yml
workflow:
That matrix (Alpine/Ubuntu, minimal and no-tools images, x86_64 and arm64) is where
toolchain-dependent behavior is verified. Keep it for things that genuinely need a
compiler; everything else should be a cargo test.
Cross-architecture package validation
Package builds are CI's job. test-build-packages.yml builds every changed package on
Linux-x86_64, Linux-aarch64, macOS-aarch64 and macOS-x86_64 per PR, and
validate-all-packages.yml rebuilds the whole catalogue weekly on the same four -- every
platform GitHub has hosted hardware for -- and regenerates PACKAGES_STATUS.md. Each slow
package (gcc, llvm, ...) gets a job of its own, so one long build cannot time out the rest.
Where a package cannot build on a platform, the reason is kept in
scripts/status-notes.tsv and shown in the table's Notes column. Those results are
the ones that count; a laptop's are not recorded anywhere.
Locally, smoke-test the definition you are editing and let CI do the rest:
make validate PKGS="zlib bzip2" # linux/arm64 + linux/amd64
./docker/validate-packages.sh --platform linux/amd64 rsync
make validate with no PKGS refuses rather than starting a full run, and --all needs an
explicit TSI_VALIDATE_ALL_I_MEAN_IT=1: a full catalogue run needs tens of gigabytes per
architecture and hours of build time, and it filled this project's development machine once.
The script builds tsi inside a container per platform and runs tsi install for each
package. On an Apple Silicon host linux/amd64 runs under emulation — correct, just slow.
Results land in .validate-logs/<arch>/results.tsv in the same format CI produces, and
failing build logs stay next to them.
A package that genuinely cannot build somewhere declares it with platforms (see
OS-specific configuration)
instead of quietly failing there.
Nothing here has to be run locally to be trusted — every check below runs in CI. The local runner exists so you get the answer in minutes instead of after a push.
In this repository
| Workflow | When | What it proves |
|---|---|---|
ci.yml |
push / PR (release on main) |
lint stage: clippy -D warnings and fmt --check on Linux, macOS and Windows; actionlint; shellcheck over tsi-bootstrap.sh and docker/*.sh. Then the build and e2e stages below. On a push to main it finally tags and publishes the next minor release; a major release is a manual run with bump = major |
e2e.yml |
called by ci.yml |
a real source build of bzip2 inside bare containers on every Linux release architecture: Alpine/Debian/Ubuntu/Fedora natively on x86_64 and aarch64, Alpine and Debian under qemu on i686, armv7, armv6 (Alpine only), riscv64 and ppc64le; plus a zero-dependency proof on each of them that the static binary works with no toolchain at all |
build-binaries.yml |
called by ci.yml |
the release binary for every shipped platform builds: Linux x86_64/aarch64 natively and i686/armv7/armv6/riscv64/ppc64le with cargo-zigbuild (static, smoke-tested under qemu-user), macOS aarch64 and x86_64, Windows x86_64 and aarch64. the release publishes exactly these artifacts, and runs cargo test on the native platforms |
In tsi-packages
| Workflow | When | What it proves |
|---|---|---|
package-validation.yml |
push / PR | catalogue schema and invariants, version ordering, the scripts' self-checks, shellcheck, and that changed packages' sources are pinned and reachable |
test-build-packages.yml |
push / PR | changed packages really build on Linux-x86_64, Linux-aarch64, macOS-aarch64 and macOS-x86_64, and their installed binaries can actually load; a failing leg ends with why each package failed |
verify-sources.yml |
weekly | every package's default source still downloads and matches its recorded sha256 |
validate-all-packages.yml |
weekly + manual | the whole catalogue on all four platforms, slow packages included, regenerating PACKAGES_STATUS.md with the reason for every package that cannot build somewhere |
The behaviour that a full-catalogue run would otherwise be the only witness to is pinned by
ordinary tests instead, so it is checked on every commit rather than every week: the
archive-cache keying (src/ops/fetch.rs), make_args expansion (src/ops/build.rs), and
the platform gate and update --local self-copy guard (tests/cli_scenarios.rs).
CI
ci.yml runs lint, then the build and test matrix, then the e2e matrix on push and PR.
A change is expected to be green before merge.