63%

s-01 Summary attribution: per-.ori-file -> section LLVM dual-exec verdict

Goal

Give compiler_repo/build/test-all-summary.json per-section LLVM dual-execution attribution so the completion gate (s-02) can ask “is THIS section’s .ori spec corpus green on the LLVM backend, matching the interpreter?” Today the summary’s failures[] carry a test_id but no .ori path (0 of 285), and there is no per_section_llvm_parity block — section-granular gating is impossible without this foundation.

Implementation Sketch

  • Extend the ori_interp + ori_llvm spec-test legs of compiler_repo/test-all.sh (and the summary writer) so each failure records its source .ori path.
  • Build a .ori path -> roadmap section index (section frontmatter touches / owns_crates / content references); unattributable paths are recorded as parity: unknown (fail-closed downstream), never dropped.
  • Emit per_section_llvm_parity: {<section_id>: {interp_pass, llvm_pass, parity}} into the summary JSON alongside per_suite.

Spec References

  • v7-plan-system.md §8 (verifiable success criteria are the gate’s teeth).
  • CLAUDE.md §Test & Doc Disciplines (test-all = compiler-state SSOT).

Work Items

  • test-all spec-test legs (ori_interp + ori_llvm) emit each failing test’s source .ori path into test-all-summary.json failures[] (attribution field per failure)
  • map each .ori path to its owning roadmap section via a path->section index (section frontmatter touches / owns_crates / content reference)
  • emit a per_section_llvm_parity verdict block (section_id -> {interp_pass, llvm_pass, parity}) into test-all-summary.json
  • tests: a known LLVM-red .ori failure maps to its section; summary carries per_section_llvm_parity; an unattributable failure degrades fail-closed (parity=unknown), never silently green

Items

  • map each .ori path to its owning plan-node OR bug via a path->node index that scans plans/ AND bug-tracker/plans/ (section frontmatter touches / owns_crates / content reference, and bug-plan .ori TDD tests), not the roadmap alone; unattributable paths are fail-closed (unknown), never dropped
  • emit a per-node verdict block (node_id -> {interp_pass, llvm_pass, parity, leak_free}) into test-all-summary.json, keyed by owning plan-node OR bug across plans/ + bug-tracker/plans/, where leak_free is derived from the ORI_CHECK_LEAKS=1 leg
  • tests (split): (a) a known LLVM-red .ori maps to its owning node and the per-node block carries parity=false; (b) a leaking-but-parity-green .ori (ORI_CHECK_LEAKS=1 positive) surfaces leak_free=false in its per-node verdict, never silently green; (c) a non-roadmap plan’s .ori and a bug-plan TDD .ori each attribute to their owning node (plans/ + bug-tracker/plans/), not to the roadmap; (d) an unattributable failure degrades fail-closed (parity=unknown AND leak_free=unknown), never silently green
  • extend the test-all spec-test legs to record each .ori file’s ORI_CHECK_LEAKS=1 result (leak-positive / leak-free) per failure, so the per-node verdict’s leak_free leg has a source signal alongside the parity legs

Fresh intel (regenerated)

{ “schema_version”: 1, “target”: { “kind”: “plan-section”, “ref”: “roadmap-llvm-verification/s-58d06676” }, “generated_at”: “2026-06-21T02:31:09.593074+00:00”, “graph_state”: { “head_sha”: “b0bd51a8”, “last_code_import_at”: “2026-06-21T02:11:33.945Z”, “embedding_stale”: false, “insights_stale”: true, “cpg_stale”: false }, “surfaces”: {}, “summary”: “intel-package[plan-section:roadmap-llvm-verification/s-58d06676] agent-authored dossier”, “degraded”: false, “dossier”: { “objective_symbols”: [], “tiers”: {}, “agent_authored”: true, “difficulty”: { “class”: “routine”, “research_online”: false, “research_mode”: “auto”, “signals”: [], “plan_dir”: “/home/eric/projects/ori_lang/plans/roadmap-llvm-verification” } } }

(full dossier: summary-attribution—s-58d06676.intel.json)