0%

s16 — Conformance: Full Spec Corpus Under the Native Backend

Goal

ori test --backend=native tests/ passes the FULL spec corpus with triple-parity (interpreter == LLVM == native observable behavior), valgrind + leak checks clean — the fast tier is semantically DONE.

Implementation Sketch

  • Ratchet loop: run the corpus under —backend=native; every failure classifies as (a) backend bug -> fix via /fix-bug with full TDD rigor, (b) genuine GAP vs LLVM lowering -> implement, (c) pre-existing spec-corpus disposition (carries BUG-id per test-disposition.md) -> untouched. Failure count ratchets monotonically to ZERO; the ratchet table lives in this section and updates per round.
  • tri-exec sweeps (s04 harness) run per ratchet round; MISMATCH_NATIVE_* classes drive the fix queue.
  • Suites: tests/spec + tests/run-pass + tests/compile-fail (driver diagnostics identical — backend selection must not perturb error paths) + phase tests where applicable; valgrind corpus (tests/valgrind incl. cow/, fat_matrix/, iter_rc/, trmc/) executed under native AOT; ORI_CHECK_LEAKS=1 across the run-pass corpus.
  • AOT integration parity: the ori_llvm/tests/aot suites (match_alias, generics, closure_drop, higher_order, aims_burden_alias) get native-backend twins or a shared parameterized harness (decision: prefer parameterizing over copying — LEAK discipline).
  • test-all.sh: run_ori_native flips fully active (all targets it can reach on host; qemu targets stay in the s09 CI matrix).
  • Disposition discipline: any native-specific #skip carries a BUG-XX-NNN; Dispositions counter stays untracked-zero.

Test Strategy

  • This section IS test execution; its own deliverable-tests are the ratchet table, the tri-exec green sweep, and the parameterized AOT harness.
  • Semantic pin: the tri-exec sweep wired as a CI job (regression gate from here forward).
  • Negative pin: a known-divergent fixture (test-only) is DETECTED by tri-exec (harness sensitivity proof).

Work Items

  • Ratchet rounds to zero failures on tests/spec + run-pass + compile-fail under —backend=native (every fix via /fix-bug rigor; ratchet table maintained).
  • tri-exec sweep green across the corpus; CI job wired; harness-sensitivity negative pin.
  • Valgrind corpus + ORI_CHECK_LEAKS green under native AOT.
  • AOT integration suites parameterized across backends (shared harness, no copy-paste suite).
  • test-all.sh native suite fully active on host; dispositions untracked-zero.