100%

walking-skeleton

Goal

Prove the JSON spine end-to-end on ONE metric across three legs: the Ori runner emits a single structured field (runner-emit), test-all.sh consumes it via json.load rather than a bash scrape (harness-consume), and one number reaches a recorded disc-backed artifact (record). The thinnest vertical slice that exercises runner-emit -> harness-consume -> record. All three legs are in scope for THIS section; none is deferred to a sibling.

Scope Boundary

This skeleton owns the THIN end-to-end thread, not the full JSON surface:

  • The runner-emit leg here emits ONLY the minimal {passed, failed, skipped} object. The full --format json field set (per-test records, durations, error payloads, etc.) is owned by the sibling section runner-json-output (s-4aa2d17b), which extends the same --format json branch this section introduces. The minimal-vs-full overlap is intentional: this section establishes the thread, the sibling fills the surface.
  • The harness-consume leg here adds a minimal json.load parse for ONE suite directly into test-all.sh. That parser integration is a throwaway scaffold (see Handoff below), replaced by the sibling section test-all-json-consumption (s-bec662fc), which owns the durable consumption path across all suites.
  • This section is the root of the JSON-spine dependency chain; siblings that build on its scaffold are its logical successors.

Implementation Sketch

Add a minimal --format json path emitting at least {passed, failed, skipped} from compiler_repo/compiler/oric/src/commands/test.rs print_test_summary’s TestSummary; have compiler_repo/test-all.sh parse that JSON with python3 -c json.load for one suite; assert the parsed count matches the text leg. The default (no --format) text-summary path stays unchanged. Skeleton only — the full field set lands in sibling runner-json-output (s-4aa2d17b).

Acceptance

Each leg below is a concrete, artifact-anchored acceptance condition the implementation and its test MUST assert. These are the per-leg verification gates for this section; the runner-emit gate maps to the existing section success_criterion, the harness-consume / record / preservation gates require new success_criteria the orchestrator/script must write into plan.json (this body states the intent the criteria pin).

  1. Runner-emit (probed today): ori test --format json on one suite emits a JSON object carrying passed, failed, and skipped integer fields, parseable by python3 -c "import json,sys; d=json.load(sys.stdin); print(d['passed'],d['failed'],d['skipped'])".
  2. Harness-consume (needs criterion): test-all.sh consumes the runner JSON via python3 -c json.load (NOT a bash/grep/awk text scrape) for the one wired suite, and the parsed passed/failed/skipped counts EQUAL the counts the text-format leg reports for the same suite (parity assertion). The probe drives test-all.sh on that suite and compares the JSON-derived counts against the text-derived counts; pass iff equal.
  3. Record (needs criterion): the parsed count reaches a disc-backed artifact — test-all.sh writes the parsed passed count (or the full {passed,failed,skipped} triple) to a named file. Verifiable via test -f <artifact> && grep -q "<expected-count>" <artifact>; pass iff the file exists and contains the recorded number. This is the leg the metrics-ledger sibling (s-5494ed74) currently maps to an echo ok no-op probe; THIS section’s acceptance demands a real file-existence + content assertion so the record leg is not vacuously green.
  4. Default-text preservation (needs criterion): ori test with NO --format flag still emits the human-readable text summary unchanged (the same summary print_test_summary produces today), and the JSON-leg count for a suite matches that suite’s text-leg count. The probe runs the runner with and without --format json on one suite and asserts (a) the no-flag run still prints the text summary and (b) the two count sources agree.

Handoff

The test-all.sh json.load integration this section adds is a THROWAWAY scaffold, not the durable consumption path. The sibling section test-all-json-consumption (s-bec662fc) owns the permanent, all-suites consumption and REPLACES this section’s minimal one-suite parse. When that sibling lands, this section’s scaffold parse is removed, not retained. This handoff makes the temporary integration tracked rather than silent permanent debt: a reviewer of s-bec662fc treats the scaffold here as the seam to supersede.

Spec References

compiler_repo/compiler/oric/src/commands/test.rs (print_test_summary L99-114; next fn print_file_errors at L117); compiler_repo/compiler/oric/src/test/result/mod.rs (TestSummary at L180, carries passed/failed/skipped); compiler_repo/test-all.sh (parse_ori_results at L308).

Work Items

  • Emit a minimal {passed,failed,skipped} JSON object from the ori test runner behind —format json and consume it in test-all.sh for one suite, asserting parity with the text leg.