incremental-and-llvm-parallelism

Goal

Stop re-running unchanged spec tests in test-all.sh by wiring the runner’s already-shipped --incremental flag into both ori test invocations, gated so a full forced run stays available. The runner supports --incremental end-to-end on both backends today (main.rs:157-158 sets config.incremental; the eval leg skips via run_sequential/run_parallel + incremental::plan_file_skips, and the LLVM leg skips via run_llvm_isolated + incremental::plan_file_skips reporting SkippedUnchanged); test-all.sh does not pass the flag, so every run re-executes every unchanged target.

Implementation Sketch

Wire test-all.sh to pass --incremental on the two ori test invocations — the interpreter leg at test-all.sh:280 and the LLVM leg at test-all.sh:310 — guarded by an env toggle (e.g. ORI_TEST_FORCE_FULL=1 omits --incremental) so a full forced run stays available for CI and clean-state verification. The on-disk per-function body-hash cache is parent-owned via ORI_TEST_INCREMENTAL_CACHE (per compiler-ops.md), so a missing cache file degrades to a full run automatically. No runner-internal change is required: --incremental already drives incremental::plan_file_skips on both legs.

The LLVM-backend leg already runs with per-file subprocess isolation (run_llvm_isolated, runner/mod.rs:271-306), one worker subprocess per test file. Lifting LLVM-backend parallelism by splitting Context::create from test execution is SUPERSEDED — see ## Superseded below. The standing measured decision is that the LLVM backend runs sequentially because LLVM Context::create() has global-lock contention; sequential is faster (1-2s vs 57s) and matches Roc/rustc patterns (runner/mod.rs:260-264). This section does not relitigate that decision.

Spec References

compiler_repo/compiler/oric/src/test/change_detection/ — per-function body-hash change detection.
compiler_repo/compiler/oric/src/test/runner/incremental.rs — plan_file_skips + SkipPlan (skip planning consumed by both legs).
compiler_repo/compiler/oric/src/test/runner/mod.rs:260-264 — documented sequential-LLVM Chesterton’s-Fence decision (Context::create global-lock contention; sequential faster).
compiler_repo/compiler/oric/src/test/runner/mod.rs:271-306 — run_llvm_isolated (per-file subprocess isolation + incremental skip via plan_file_skips).
compiler_repo/compiler/oric/src/main.rs:157-158 — user-facing --incremental flag sets config.incremental.
compiler_repo/compiler/oric/src/main.rs:170-177 — internal --__skip-unchanged= worker-protocol flag (parent→worker forwarding; NOT the surface test-all.sh wires).
compiler_repo/test-all.sh:280 — interpreter-leg ori test invocation (run_ori_interpreter).
compiler_repo/test-all.sh:310 — LLVM-leg ori test --backend=llvm invocation (run_ori_llvm).

Work Items

Wire test-all.sh to pass --incremental on both ori test invocations (interpreter leg test-all.sh:280, LLVM leg test-all.sh:310), guarded by ORI_TEST_FORCE_FULL=1 so a full forced run stays available. The runner already supports --incremental on both legs; this work item only changes the harness call sites, not the runner.

Superseded

SUPERSEDED — split LLVM Context::create (serialized) from test execution (parallel). The premise (LLVM tests serialize on a global Context::create lock so splitting it lifts parallelism) contradicts the standing measured decision at runner/mod.rs:260-264: the LLVM backend runs sequentially BECAUSE Context::create global-lock contention makes rayon parallelism serialize anyway, and sequential is faster (1-2s vs 57s, matches Roc/rustc). The LLVM leg already uses per-file subprocess isolation (run_llvm_isolated, runner/mod.rs:271-306), not a throttled in-process rayon pool, so the work item’s throttle target does not exist. Splitting the lock to “lift parallelism” is architecturally regressive. If a future LLVM release lifts the global Context::create lock, that is the gating precondition to revisit LLVM-leg parallelism; this work item is not re-proposed now.

Success Criteria

#	Criterion	Probe (deterministic)	Layer
sc-1 (positive)	`test-all.sh` passes `--incremental` to both `ori test` invocations via the `$INCREMENTAL_FLAG` variable (default `--incremental`, emptied by `ORI_TEST_FORCE_FULL=1`)	`grep -c 'INCREMENTAL_FLAG="--incremental"' compiler_repo/test-all.sh` ≥ 1 (the default) AND `grep -cE 'ori test .*\$INCREMENTAL_FLAG' compiler_repo/test-all.sh` == 2 (interp leg + LLVM leg both pass the flag)	L12 production entry point
sc-2 (positive)	A warm second run skips unchanged targets end-to-end	Two consecutive `./test-all.sh` runs with no source change between them: the second run’s `compiler_repo/build/test-all-summary.json` reports `skipped_unchanged > 0` for at least one Ori spec leg (`ori_interp` and/or `ori_llvm` suite)	L12 production entry point
sc-3 (negative)	A forced-full run skips nothing	`ORI_TEST_FORCE_FULL=1 ./test-all.sh` reports `skipped_unchanged == 0` for both Ori spec legs in `test-all-summary.json` (the toggle omits `--incremental`, so every target runs)	L12 production entry point
sc-4 (negative)	A changed target is NOT skipped	After sc-2’s warm run, edit one function body in a spec test file under `compiler_repo/tests/spec/`, then re-run `./test-all.sh`: that target appears in the executed set (not in `skipped_unchanged`) for its backend leg	L11 spec / L12 production entry point
sc-5 (parity / non-regression)	Pass/fail VERDICTS are identical with and without `--incremental`	A warm incremental run skips unchanged targets into the `skipped_unchanged` bucket, so its `passed`+`failed` is LOWER than a forced-full run by exactly the skipped count — raw totals do NOT match. The parity pin folds skips: `warm.passed + warm.failed + warm.skipped_unchanged == forced_full.passed + forced_full.failed` in `test-all-summary.json` (every target the forced-full run EXECUTES is either re-executed or skipped-because-unchanged in the warm run, never dropped), AND `warm.failed == forced_full.failed` (incremental never flips a verdict). Equivalently, compare the per-target verdict set (executed ∪ skipped-unchanged) for set-equality across the two runs	L9 dual-execution parity (interp ≡ LLVM) holds independently per leg

Layer coverage (per layer-coverage.md §2 — addressed-or-justified)

This is a tooling-tier section (harness call-site wiring; no compiler .rs/.ori semantics change). Per compiler-ops.md, runtime tooling carries SRP/SSOT + runtime pytest, not the full compiler matrix. Layer dispositions:

L1 parse / L2 typeck / L3 canon / L4 ARC — N/A: no compiler source changed; the change is entirely in test-all.sh invocation flags. The runner’s --incremental path is already shipped and covered by its own crate tests.
L5 eval / L6 LLVM-debug / L7 LLVM-release — Covered: sc-2/sc-3/sc-5 exercise both the interpreter leg (./target/debug/ori test) and the LLVM-release leg (./target/release/ori test --backend=llvm) through test-all.sh; the LLVM leg’s per-file subprocess isolation already runs each file as a release binary.
L8 AOT — N/A: no AOT artifact is produced by this change; test-all.sh’s AOT leg is unaffected.
L9 dual-execution parity — Covered (sc-5): incremental skipping is a re-execution optimization that must not alter pass/fail verdicts on either backend; the parity-of-totals probe pins this.
L10 leak — N/A: no RC/memory-touching code changes; the change is shell-level flag passing.
L11 spec conformance — Covered (sc-4): the changed-target probe asserts spec tests still execute and pin behavior when their source changes.
L12 production execution entry point — Covered (sc-1/sc-2/sc-3): the real entry point is a live ./test-all.sh run consuming compiler_repo/build/test-all-summary.json; sc-2 (warm skip), sc-3 (forced full), and sc-1 (call-site wiring) drive it end-to-end. This mirrors the sibling test-all-json-consumption section’s fresh-summary L12 evidence.