100%

Goal

Controlled best-of-3 A/B measurement + record the ev14 ratify/refute outcome to the benchmark-grind-lexer ledger

Implementation Sketch

See the work items below; each is the bounded deliverable.

Spec References

  • ledger evolution ev14 (eligible) in plans/benchmark-grind-lexer (the hypothesis + kill_criterion).

  • compiler_repo/compiler/ori_ir/src/interner/mod.rs (the InternShard surface).

Work Items

  • Controlled same-load best-of-3 A/B of the fused interner vs the pristine Box::leak/Vec<&‘static str> HEAD via compiler_repo/scripts/bench_probes/measure.sh lexer (the L12 production probe), plus callgrind Ir-delta on cook_ident->try_intern->lookup as the sensitivity-tightened secondary signal. FULL layer matrix: build debug+release, ./test-all.sh no new failure over the 1803 floor, interp<->LLVM parity, ORI_CHECK_LEAKS zero.
  • Record the measured outcome as a benchmark-grind-lexer ledger cycle (verbatim before->after teeth) and flip ev14 via the §11.6 one-writer: ratified iff measure.sh lexer exceeds 212.762 beyond the +-5% noise band with no new test-all failure AND no Name-identity regression; refuted iff it reads <= 212.762 / within noise / CI crosses zero (closing the interner-lever search + confirming the cycles-10/12/13 mission-mis-fit verdict).

Fresh intel (regenerated)

VERDICT: The section’s own text describes a measurement that can no longer be run as literally written, and the object it names (“the fused interner”) was never built. Three independent facts converge on one decisive finding:

  1. The “fused interner” does not exist. w-fb942b5b — the work item in sibling section s-d483addc that specified the fused design (bytes: Vec<u8> arena + spans: Vec<(u32,u32)>) — is now status: abandoned in plan.json with the reason “rewritten toward mission: … the literal technical prescription … is stale and unsound.” What actually shipped (commit faf5ae1d8, confirmed live at HEAD via direct read of compiler/ori_ir/src/interner/shard.rs) is the chunk-pool-store half ONLY — map: FxHashMap<&'static str,u32> + spans: Vec<&'static str> (still a fat-pointer vec, NOT (u32,u32) offsets) + a leaked-chunk key pool. The flat-arena-resolve half (the piece that would make this genuinely “fused,” distinct from cycle 10) was never implemented — ledger cycle 2 on s-d483addc tried it via a lock-returning API and was REVERTED (WORSE, real parking_lot::RwLock deadlocks in ori_arc/ori_eval).
  2. The shipped structure is architecturally near-identical to benchmark-grind-lexer ledger cycle 10 (“Implement evolution 3’s eligible safe chunk-pool interner: replace the per-unique Box::leak … with a per-shard append-only arena … predicted 202.70 -> 215.40”), which was ALREADY measured: chunk-pool 198.72 vs Box::leak 197.13 (+0.8%, inside the ±5% noise floor), status INERT. evolution_n 14’s own constraint_audit (attempt 10, relation: challenges) explicitly acknowledges this — it frames the “new evidence” as testing the JOINT removal of allocation-count AND resolve-indirection, but the resolve-indirection half never landed. [JOIN] this section is poised to re-run cycle 10’s exact experiment under a slightly different code shape (per-shard leaked chunks vs the described per-shard append-only arena) and, absent the flat-arena half, has no NEW axis to attribute a different outcome to.
  3. The “pristine Box::leak HEAD” comparison baseline no longer exists in the working tree. git log --diff-filter=A -- compiler/ori_ir/src/interner/shard.rs shows shard.rs was ADDED in commit faf5ae1d8 — the SAME commit as the currently-committed HEAD. The parent commit b6e428db0 (2026-07-01T08:32:54-04:00) is the last commit carrying the original single-file Box::leak-per-ident mod.rs with no shard.rs. A controlled A/B against “pristine HEAD” requires reconstructing arm A from b6e428db0 (git worktree / checkout into a scratch build), NOT from the current tree — there is no live pristine arm to bench against today.

Decisions this package forces before running the measurement:

  1. Reconstruct arm A explicitly: git -C compiler_repo worktree add /tmp/interner-baseline b6e428db0 (or an equivalent scratch checkout), then run measure.sh lexer against THAT build for the “before” reading — never assume the live tree still contains a Box::leak arm.
  2. Name what is actually being measured: the criterion-throughput measure.sh lexer A/B compares HEAD’s chunk-pool-only InternShard against the b6e428db0 pristine Box::leak interner — this is cycle-10’s experiment re-run under a slightly restructured (per-shard-leaked-chunks vs per-shard-append-only-arena) but STILL allocation-axis-only design. It is NOT the “fused chunk-pool + flat-arena-resolve” ev14 was written to test. Record this explicitly in the ledger cycle’s hypothesis field so a future reader does not conflate it with a genuine joint-axis test.
  3. Expect (per §7 external prior art + cycle 10’s own precedent) a result inside the ±5% noise band — the shipped structure removes the SAME allocation-count axis cycle 10 already measured INERT, via a structurally similar (not identical) mechanism. A refuted outcome that matches cycle 10’s magnitude (+0.8%, or anywhere inside ±5%) is the EXPECTED result per every internal + external signal gathered — it is not evidence of a bad measurement.
  4. best-of-3 is an outer-loop repeat of measure.sh lexer, not a raw timing min-of-3. Precedent (cycle 10’s own evidence: 188.98/198.38/198.72 chunk-pool vs 193.48/192.16/197.13 Box::leak) shows “best-of-3” = 3 SEPARATE measure.sh lexer invocations (each already a full cargo bench criterion run with its own internal bootstrap statistics), NOT 3 raw wall-clock samples of a single run. Per §7’s external prior art, reporting only the max of the 3 (a naive “best”) risks an optimism bias favoring whichever arm had higher measurement variance — cite the full 3-reading spread (as cycle 10 did) and the criterion thrpt: median from each run, not a single cherry-picked number.
  5. The callgrind Ir-delta on cook_ident -> try_intern -> lookup is diagnosis-only, never verdict-bearing — per cycle 10’s own generalizes_to (“the ~9% _int_malloc is a CALLGRIND INSTRUCTION-count figure … removing those instructions does not transfer to wall-clock”) and independently corroborated by rustc-perf’s own documented instruction-count/wall-clock divergence (§7). The GATE is measure.sh lexer wall-clock; callgrind explains WHY, never decides ratify/refute.
  6. The ledger flip is a write-API call, not a hand-edit. sc-9398a1a5 (kind: functional, the plan-wide criterion) requires BOTH the recorded A/B reading AND evolution_n 14 flipped ratified/refuted in plans/benchmark-grind-lexer’s ledger via the append-only one-writer (scripts/plan_corpus/schemas.py validate_evolution_entry / validate_evolution_append_only) — never a direct edit to plan.json:evolutions[].

DIG DEEPER

python -m scripts.plan_corpus.ledger show plans/fused-interner-ev14 --rank --all
python -m scripts.plan_corpus.ledger show-evolutions plans/benchmark-grind-lexer
cd compiler_repo && git log --diff-filter=A --oneline -- compiler/ori_ir/src/interner/shard.rs
cd compiler_repo && git log -1 --format='%H %ci' faf5ae1d8^

(full dossier: measure-and-record-ev14—s-24059b3f.intel.md)