50%

§09 derived layers

Goal

Derived intelligence over the completed substrate: sibling-divergence detection (the class of the original burden-seam escape), co-change mining with full-scale calibration, and test topology (TESTS/COVERS + production-entry-point reachability + test-only-reachable dead-code classification).

Implementation Sketch

  • Sibling divergence: within a MISSING_ABSTRACTION/clone family sharing a writer target, diff callee-set signatures; nominate divergent members (the register_error_type vs register_type_decl class).
  • Co-change: ROSE-style association mining over :Commit TOUCHED sets; support/confidence/last_seen props; max-changeset filter (CodeScene lesson); FULL-scale calibration per the SZZ subset-illusion risk.
  • Test topology: Rust test fns resolved via §02/§03 call edges + #[cfg(test)] module conventions; spec corpus mapped via harness manifests; production-entry-point reachability = path from oric main without test-root traversal (TEST-14/L12 class).
  • Dead-code classification rule: production-source code whose ONLY inbound reachability is from test roots (no path from a production entry point) IS dead code — test callers never count as liveness; feeds the WASTE:dead-code detector surface. Carve-outs: test infrastructure itself (#[cfg(test)] modules, tests/ dirs, harness crates — legitimately test-only, never flagged) and deliberately-exported public API with no in-repo production caller (routes to EXPOSURE:over-exposed-pub adjudication, not dead-code).

Spec References

  • intel_repo/neo4j/szz_blame.py + :Commit TOUCHED edges; intel_repo/neo4j/metrics.py (derived-metric writer precedent); .claude/rules/impl-hygiene.md DESIGN-21 (names the co-change gap), SSOT-09/COVER-12 (sibling drift), TEST-13/14 + layer-coverage L12 (test topology); .claude/rules/finding-categories.md WASTE:dead-code + EXPOSURE:over-exposed-pub (dead-code classification consumers)

Work Items

  • Sibling-divergence detection: group candidate functions PRIMARILY by shared writer-target (same type-registry/burden-writer callee membership, e.g. type_registry_mut / register_struct / register_user_burden) — MISSING_ABSTRACTION/clone-family linkage is a secondary refinement signal, NEVER a required gate, since the register_error_type / register_type_decl acceptance oracle is not itself a MISSING_ABSTRACTION pair and would be wrongly excluded by an AND-style family rule; diff callee SETS (Jaccard on the callee signature), never callee counts, since the register_* family’s raw callee counts (7-16) are heterogeneous and count-based grouping false-positives across unrelated siblings; detector candidate surface.
  • Co-change mining over TOUCHED edges: CO_CHANGED_WITH{support, confidence, last_seen} derived edges with the max-changeset filter; full-scale calibration (never a subset sample)
  • Co-change mining over TOUCHED edges: CO_CHANGED_WITH{support, confidence, last_seen} derived edges with the max-changeset filter; full-scale calibration (never a subset sample — the live 130-commit graph window returns 0 co-change rows even for the section’s own register_error_type exemplar, empirically confirming the SZZ subset-illusion risk); EXTEND szz_blame.py’s existing TOUCHED producer (_upsert_attributions) and its noise filters (_is_format_only / _is_rename_only / _is_comment_only / _is_plan_doc_only) rather than forking a second commit-walk; follow metrics.py’s declarative REMOVE-then-SET writer contract (no additive-MERGE-without-prune, which leaves ghost edges across refreshes); register CO_CHANGED_WITH’s identity/producer/prune/freshness/confidence row in the §01 edge-class-contract registry before first ingestion.
  • Test topology: TESTS/COVERS edges from resolved test-fn call paths + module conventions + spec-corpus mapping; production-entry-point reachability query (CLI root vs test-only reachability)
  • Test topology: TESTS/COVERS edges from resolved test-fn call paths + module conventions + spec-corpus mapping; production-entry-point reachability query (CLI root vs test-only reachability); follow metrics.py’s declarative REMOVE-then-SET writer contract and register TESTS + COVERS identity/producer/prune/freshness/confidence rows in the §01 edge-class-contract registry before first ingestion; depends on §02/§03 call-resolution completeness — an unresolved method-receiver test call is a test-topology false negative (a genuinely-pinned function looks unpinned), so build against CURRENT resolution coverage and surface the gap rather than assuming resolution-complete input.
  • Test-only-reachable dead-code classification: query + classification marking production-source symbols whose only inbound reachability is from test roots (no production-entry-point path) as dead code, feeding the WASTE:dead-code detector surface; carve-outs enforced with negative pins — test infrastructure (#[cfg(test)] modules, tests/ dirs, harness crates) never flagged, exported public API with zero in-repo production callers routed to EXPOSURE:over-exposed-pub instead of dead-code
  • Test-only-reachable dead-code classification: query + classification marking production-source symbols whose only inbound reachability is from test roots (no production-entry-point path) as dead code, feeding the WASTE:dead-code detector surface; carve-outs enforced with negative pins — test infrastructure (#[cfg(test)] modules, tests/ dirs, harness crates) never flagged, exported public API with zero in-repo production callers routed to EXPOSURE:over-exposed-pub instead of dead-code; the EXPOSURE carve-out route depends on visibility-field completeness for the candidate set — verify the extractor actually emits visibility for test-only-reachable candidates (distinguish ‘empty = genuinely private, flaggable’ from ‘empty = extractor-missed, unknown’) before trusting the carve-out, since the current 500-symbol candidate set is uniformly visibility-empty and this must be confirmed as extractor-complete rather than assumed.
  • Probes + tests for all three derived layers; readings recorded; derived-layer staleness riding their producing facets
  • Probes + tests for all three derived layers; readings recorded; derived-layer staleness riding their producing facets; each probe MUST be a genuine negative pin that reads RED when its layer is absent or stale (never a ghost probe that passes vacuously regardless of layer state); record as baseline readings: co-change support/confidence recall, the test-topology resolved-vs-total test-edge ratio (so partial call-resolution coverage stays visible rather than silently hidden), and the dead-code candidate count after carve-out application.

HISTORY

  • 2026-07-11 — Scope add (user-directed): test-only-reachable code = dead code: Test-topology layer amended — production-source code whose only inbound callers are test-reachable is classified dead code (feeds WASTE:dead-code), with test-infrastructure and public-API (EXPOSURE:over-exposed-pub) carve-outs. New work item w-462f0a32; Implementation Sketch + Spec References updated.

Fresh intel (regenerated)

§09 is NOT a graph-consumer section — it is a graph-PRODUCER section that writes THREE new derived layers (CO_CHANGED_WITH, TESTS/COVERS, sibling-divergence + test-only-reachable dead-code classification) on top of substrate that is ALREADY LIVE. The decisive recon fact is a clean split: every input substrate exists; every output derived edge is absent (count 0). That makes this a low-discovery, high-idempotency-risk section — the danger is not “can the query be written” but “does the new writer honor the existing declarative-replace contract and the FULL-scale calibration the substrate demands.”

The single most important [JOIN]: co-change over the section’s own exemplar (register_error_type) returns ZERO rows at the current 130-commit window, while 19,502 TOUCHED edges exist globally — the substrate is present but the commit sample is too shallow to surface per-symbol co-change. That empirically PROVES the section’s own stated risk (“FULL-scale calibration per the SZZ subset-illusion; never a subset sample”). The 130-commit window IS the subset illusion, visible in the graph right now.

Decisions this package forces before any code is written:

  1. The three layers are ADDITIVE writers over live substrate, not new extractors. Reuse metrics.py’s declarative REMOVE-then-SET writer precedent (§3.5) and szz_blame.py’s existing TOUCHED producer (§3.2); do NOT fork a second commit-walk or a second additive-MERGE path — that is the LEAK:scattered-knowledge / LEAK:algorithmic-duplication the section is exposed to (§3H).
  2. Co-change calibration must run at FULL git scale, not the 130-commit graph window. The subset illusion is demonstrated in §3.2; a support/confidence mined over 130 commits is a calibration artifact, not the signal.
  3. The dead-code classifier’s carve-outs are the whole risk. 500 production symbols are test-only-reachable RIGHT NOW (§3.4); most are legitimate (test infra, private helpers). The negative pins (test-infra never flagged; exported public API → EXPOSURE:over-exposed-pub, not dead-code) are load-bearing — and the visibility signal they depend on is only PARTIALLY populated (10,579 symbols carry it; all 500 candidates carry it EMPTY — §3.4 / §9).
  4. Sibling-divergence has a demonstrable, live exemplar. register_error_type (6 distinct callees) vs register_type_decl (16 distinct callees, 5 shared) is the exact “builtin-vs-user burden registration split” the section names, and the callee-set diff separates them cleanly today (§3.1). Build the detector against this pair as the acceptance oracle.
["enrich_metrics", "write_missing_abstractions", "write_hotspot_scores", "_upsert_attributions"]

(The four are the EXISTING intel_repo derived-fact writers whose idempotent declarative-replace + TOUCHED-upsert contracts the three new layers must EXTEND / reuse the substrate of, and MUST NOT fork or perturb. register_error_type / register_type_decl are the section’s demonstrated detector TARGETS — subject symbols, deliberately EXCLUDED per the §6 output-contract rule.)

DIG DEEPER

python -m scripts.plan_corpus.read plans/intel-graph-deepening --section s-6a840bd1 --criteria
scripts/intel-query.sh status --human

(full dossier: derived-layers—s-6a840bd1.intel.md)