95%

Section 02 — Corrected Converter

Cites §01 north star INV-3 (JSON routing-only) + INV-12 (content keyed by id). Consumed by §04 self-convert + §10 corpus migration.

Goal

See frontmatter. Bootstrap foundation — format conversion ONLY; MUST NOT depend on §05 cross-plan merge engine.

Intelligence Reconnaissance

  • 2026-05-27 — graph queries run for the converter-design grounding:
    • scripts/intel-query.sh symbol-plans build_plan_json_from_dir --repo ori — confirms the 006 parser SSOT ([ori:scripts/plan_corpus/migrations/006_md_to_json.py]:549) is referenced by §01 + §02 (single md→dict converter; grounds INV-3 owning-section mapping for §02 reuse). §04 self-convert + §10 corpus migration call this converter at execution time per §02’s plan-body Goal (line 41), not via direct symbol citation.
    • scripts/intel-query.sh symbol-plans 010_edges_to_route --repo ori — confirms 010’s identity/slug/status helpers ([ori:scripts/plan_corpus/migrations/010_edges_to_route.py]:47/:92/:97/:120) are consumed by §02 + downstream route sections; identifies the §02.1 extraction target ([ori:scripts/plan_corpus/node_identity.py]) so §11B’s strip carve-out has a declared lifecycle.
    • scripts/intel-query.sh dag-ascii scripts-first-restructure --repo ori — confirms §02 → §03 → §05 ordering + §10 → §11A → §11B ([ori:plans/scripts-first-restructure/section-11B-strip-parsers.md] parser-strip ordering retains 006 as archive-restore-only).
  • Summary (≤500 chars): §02 is a bootstrap converter (format conversion only) consuming the 006 parser SSOT + 010 identity helpers (extracted to node_identity.py in §02.1 for §11B strip-lifecycle clarity); emits per-plan routing-only plan.json validated against the §03-delivered routing-node JSON schema; renders flat id-prefixed content with INV-17 attestation on done-nodes; round-trip-tested against the §02.2-updated 006_json_to_md. NO §05 dependency; NO aliases/JSONL/items-on-node.

02.1 — Per-plan emitter (replaces 010._run) reusing the parser SSOT

  • Author scripts/plan_corpus/convert_plan_dir.py (or extend 010): consume 006.build_plan_json_from_dir(plan_dir) -> ParsedPlanDir{plan_json, sidecar_plan.files} as the sole parser (006_md_to_json.py:549). Never write a second md parser.
  • Extract node_id / slugify / migrate_plan / _STATUS_FWD from scripts/plan_corpus/migrations/010_edges_to_route.py (:47/:92/:97/:120 — currently in a migration CLI script, not a library) into a non-migration library module scripts/plan_corpus/node_identity.py BEFORE the converter imports them; the converter + §04 self-convert + §10 corpus migration consume from node_identity so §11B’s strip carve-out (currently retains only 006) has a single declared-lifecycle target.
  • Reuse node_identity.node_id / slugify / migrate_plan / _STATUS_FWD from the §02.1-extracted library for id/slug/key/status/hint_needs generation.
  • Replace 010._run (010_edges_to_route.py:129) emission: write ONE plan.json (indent=2, json.dump(..., sort_keys=False)) with top-level {schema, plan_id, conceptual_integrity_owner, mission_ref, node_count, nodes:[...]}; each node carries EXACTLY {id, slug, key, status, hint_needs, body_ref}.
  • Drop from the routing node: items, plan, plan_review, tpr, content_checkpoint, relocation_log (POC drops these in _build_routing_node at convert_plan_dir.py:241; emission tested in test_convert_plan_dir.py).
  • Subsection close (02.1) — all [x]; status: complete.

02.2 — Items → id-keyed markdown checkboxes; flat id-prefixed content

  • For each node, resolve body text from parsed.sidecar_plan.files (direct body_ref-keyed lookup, not the POC’s fragile migrated_from indirection); write to content/<id>--<slug>.md.
  • Render items[] back into the content body as id-keyed checkboxes: - [ ] [<item-id>] <text> / - [x] [<item-id>] <text> (the form §03’s scanner reads). Strip (empty body) placeholder text (Pass 3 POC finding).
  • Set body_ref = content/<id>--<slug>.md; ALWAYS flat + id-prefixed (override any nested body_ref from _sidecar_emit); refs resolve by id-prefix glob content/<id>--*.md (rename-safe).
  • Emit NO aliases.json, NO graph.jsonl, NO nested content/section-NN/ tree.
  • Compute + write the §03 INV-17 completion attestation field on every done-node WITH source attestation: hash per the §03 schema (scripts/plan_corpus/schemas/v6/plan-routing.schema.json routing-node completion shape, authored at §03:49). Policy: reset legacy done-nodes to in-progress (DD-01 cure per §02.R TPR-02-006 + AR-01 INV-19 cure): legacy done-nodes lacking a source attestation are RESET to status: in-progress (no fabricated attestation; §06.4 owns re-verification per decisions/01-completion-integrity-invariants.md §3 Class A).
  • Update scripts/plan_corpus/migrations/006_json_to_md.py to render the new flat content/<id>--<slug>.md + id-keyed-checkbox layout (the inverse of §02.2’s emission shape) so the §02.3 reverse-round-trip-via-006_json_to_md parity test is achievable; the existing 006 reverse path predates the flat-content/id-keyed-checkbox layout and would round-trip-fail without this update. (v6-schema detection routes to _v6_reverse_regenerate; --legacy flag preserves the v5 render-based path for archive-restore back-compat.)
  • Subsection close (02.2) — all [x]; status: complete.

02.3 — Pre-write validation + tests

  • Pre-write validate (raise on failure): id ~ ^n-[0-9a-z]{4,}$; slug ~ ^[a-z][a-z0-9-]*$; every body_ref resolves on disk after content write; node_count == len(nodes); keys distinct + sorted; emitted routing nodes pass jsonschema.validate(nodes, routing_node_schema) against the §03-delivered routing-node shape at scripts/plan_corpus/schemas/v6/plan-routing.schema.json (mechanical schema target — replaces prose-only definition so §02 emission cannot drift from §03 consumption). (bootstrap-stub schema landed in same commit; §03.1 owns the formal authoritative version.)
  • Tests in scripts/plan_corpus/tests/test_convert_plan_dir.py: (a) per-plan emission shape; (b) no aliases.json/graph.jsonl emitted; (c) content flat + id-prefixed; (d) every body_ref resolves; (e) items rendered as id-keyed checkboxes (none left on nodes); (f) idempotency (re-run = byte-identical); (g) reverse round-trip parity with the §02.2-updated 006_json_to_md (flat-content + id-keyed-checkbox round-trip); (h) NEGATIVE: importing the §05 merge engine fails the boundary test (converter must not depend on it); (i) every emitted done-node carries a completion attestation OR legacy-done was reset to in-progress (NEVER done-without-attestation per INV-17).
  • Subsection close (02.3) — all [x]; status: complete.

02.R — Third Party Review Findings

  • Minor — references citation precision (resolved 2026-05-28): §02 References block refreshed with current :line anchors against HEAD — _sidecar_emit.py:50 (slugify def, 60-char cap), :25 (300-char items threshold), :475/:573/:587/:643/:666 (slugify call sites). Replaces the prior _sidecar_emit.py (layout/slugify/thresholds) bare reference. Concurrent fixes: 010_edges_to_route.py anchors migrated to node_identity.py post-§02.1 extraction (:45/:51/:90/:30 for node_id/slugify/migrate_plan/_STATUS_FWD); 010_edges_to_route.py retains _run at :129. The dangling /tmp/sfwa-poc/poc_convert.py reference replaced with convert_plan_dir.py:241 (_build_routing_node — production location of the formerly-POC clean_nodes field drop).

  • [TPR-02-004-audit][Medium] GROUNDEDNESS:stale-line-anchors (resolved 2026-05-28): refreshed in concurrent References-block + §02.1-body edit; all 010_edges_to_route.py:47/:92/:97/:120/:216 anchors replaced with current HEAD positions (node_identity.py:45/51/90/30 + 010_edges_to_route.py:129 for _run).

  • [TPR-02-005-audit][Medium] GROUNDEDNESS:unanchored-symbol-reference (resolved 2026-05-28): clean_nodes reference at §02.1 line 72 replaced with concrete _build_routing_node at convert_plan_dir.py:241; the dangling /tmp/sfwa-poc/poc_convert.py reference at References line 119 replaced with the production convert_plan_dir.py:241 location.

  • [TPR-02-010-audit][Low] INTEGRITY:depends-on-vs-citation-drift (resolved 2026-05-28): confirmed informational per drift_cure_synthesis Agent verdict 2026-05-28 — body mentions §[‘03’, ‘04’, ‘05’, ‘06’, ‘10’, ‘11’] as work-subjects (not predecessor claims); citations correctly absent from depends_on: per topological-audit invariant. No edit required; acknowledged.

  • [TPR-02-001-audit][Critical] MISSION:inv20-binary-completion-violation (resolved 2026-05-28): verified post-reversal state holds — §02.N body lines 97-101 all [ ]; frontmatter §02.N status: not-started; INV-20 binary-completion invariant honored (no §02.N item flips while §02.R in-progress). Plan-cleanup commit a3f67f0e is the cure; this checkbox is the close-out confirmation.

  • [TPR-02-002-audit][High] COHESION:status-checkbox-divergence (resolved 2026-05-28): §02.N body checkboxes (lines 97-101) match frontmatter sections[].status: not-started declaration; auto-reversal preserved + verified.

  • [TPR-02-003-audit][High] COHESION:overview-index-status-desync (resolved 2026-05-28): 00-overview.md:124 + index.md:31 §02 row resynced from not-started → in-progress to match §02 frontmatter declaration. SSOT per state-discipline.md §1 honored.

  • [TPR-02-008-blind-spots-agy+claude-ds][Major] BS-05+BS-07+BS-08:test-coverage-gaps (resolved 2026-05-28): (a) already covered by test_k_validate_raises_on_every_invariant_path + test_j_inv20_gates_verified_positive_negative_pins — negative pins for id pattern, slug pattern, node_count mismatch, duplicate, sorted/distinct, body_ref resolution; (b) already covered by test_h_negative_boundary_no_section_05_import (subprocess.run on the import-boundary check) — re-using subprocess.run(sys.executable, '-m', ...) pattern across the suite; (c) NEW: convert_plan_dir.py:_self_test() + --self-test CLI flag + test_l_self_test_cli_subcommand end-to-end pin asserting python -m scripts.plan_corpus.convert_plan_dir --self-test materializes minimal plan → convert → validate → idempotency check → exit 0. 13 convert_plan_dir tests pass.

  • [TPR-02-007-blind-spots-claude-ds+agy][Major] AR-02:silent-state-drift-convergence (resolved 2026-05-28): all three convergent gaps cured: (a) SSOT import EMPTY_BODY_PLACEHOLDER extracted as module-level constant in _sidecar_emit.py:50 + imported in convert_plan_dir.py:53-55; (b) item-id pre-commit lint authored at scripts/plan_corpus/item_id_lint.py — scans flat content/<id>--*.md files for un-keyed checkboxes (lines matching - [ ] / - [x] without [<item-id>] prefix), exits 1 on violation, exempts fenced code blocks; --self-test passes positive (keyed-only file) + negative (un-keyed flagged) + fenced-exempt pins. Pre-commit wire-up at next user touchpoint (lefthook OR git pre-commit hook against staged content/—*.md paths); (c) write_plan_dir now raises PlanJsonValidationError on glob-collision (≥2 files matching content/<id>--*.md for any node) — fail-loud per CLAUDE.md §The One Rule (no silent cleanup); test_o_glob_collision_raises pin asserts the ambiguity surfaces.

  • [TPR-02-009-blind-spots-claude-ds+codex][Major] BS-09+BS-10+BS-11:invariant-pinning (resolved 2026-05-28): all three pins landed: (a) _resolve_mission_ref INV-19 invariant docstring + test_m_resolve_mission_ref_lexorank_first_overview positive + negative + empty-list pins; (b) 006_md_to_json.py:362-368 propagates conceptual_integrity_owner from 00-overview frontmatter through to v6 plan.json; test_n_conceptual_integrity_owner_emission pins positive + negative; (c) test_p_scripts_first_workflow_architecture_dry_run_parity runs convert_plan_dir against the actual 11+ dependent subject plan + asserts node_count parity + body_ref resolution + non-empty node set; companion test_p_scripts_first_workflow_architecture_id_uniqueness_xfail is an XFAIL pin documenting a converter regression (duplicate node_id n-750483 at idx 4 fails validate_plan_json’s id-uniqueness check — surfaced by the parity gate; root cause TBD: node_identity.node_id hash collision OR duplicate sub_id in source 00-overview). 17 convert_plan_dir tests pass + 1 xfail.

  • [TPR-02-006-blind-spots-codex+agy][Major] AR-01:INV-19-forward-dep-violation (resolved 2026-05-28): applied DD-01 Recommended cure — _build_routing_node at convert_plan_dir.py:242-264 now resets legacy done-nodes (status:done + no source completion attestation) to status:in-progress instead of emitting fabricated migration-stamped attestation. Eliminates the INV-19 forward-dep tension: §02 no longer emits done-without-§06.4-verification; §02 stays independent of §03/§06 attestation policy. §02.2 success_criteria + body item updated to reflect the new canonical path; §02 success_criterion line 10 narrowed to “legacy done-nodes lacking a source attestation are RESET to status:in-progress (DD-01)”. 17 convert_plan_dir tests pass + 1 xfail (TPR-02-011 hash-collision separately tracked).

  • [TPR-02-004-audit][Medium] GROUNDEDNESS:stale-line-anchors (resolved 2026-05-28 — duplicate of line 94 entry; both rows mark the same finding [x] once cure landed): mechanical anchor refresh against HEAD per node_identity.py:45/51/90/30 + 010._run:129.

  • [TPR-02-005-audit][Medium] GROUNDEDNESS:unanchored-symbol-reference (resolved 2026-05-28 — duplicate of line 95 entry): clean_nodes reference re-anchored to convert_plan_dir.py:241 _build_routing_node; dangling /tmp/sfwa-poc/poc_convert.py reference removed.

  • [TPR-02-011-discovered-by-parity-test][Major] CONVERTER:duplicate-node-id-regression (resolved 2026-05-28): cure landed in scripts/plan_corpus/node_identity.py — (a) node_id signature extended to accept optional slug arg + hash width widened from 6 hex (24 bits) to 8 hex (32 bits) per _NODE_ID_HEX_WIDTH = 8; (b) migrate_plan restructured into 3 passes (slugify-first, ids-parallel-to-subs via slug-folded hash, build-nodes consuming parallel ids); (c) parallel id list replaces the prior id_map dict lookup at node-build time — id_map[sub_id] collapsed duplicate-sub_id rows to the LAST node’s id (the source 006 parser legitimately emits duplicate sub_id values like 02.A, 26.1, 27.1, 28.1 for distinct subs); (d) the id_map is retained for hint resolution (LAST-wins semantics on duplicate sub_ids unchanged). Test promotion: test_p_scripts_first_workflow_architecture_id_uniqueness (formerly _xfail) now positive-pins clean validation across all 184 nodes of scripts-first-workflow-architecture. New focused unit pin test_q_node_id_slug_disambiguates_collision covers (i) 2-arg back-compat determinism, (ii) slug-aware semantic disambiguation, (iii) idempotency, (iv) migrate_plan integration on duplicate-sub_id input. 19 convert_plan_dir tests pass.

  • [TPR-02-010-audit][Low] INTEGRITY:depends-on-vs-citation-drift (resolved 2026-05-28 — duplicate of line 96 entry): confirmed informational; §02 body mentions §[‘03’, ‘04’, ‘05’, ‘06’, ‘10’, ‘11’] as work-subjects (not predecessor claims). Citations correctly absent from depends_on:.

02.N Completion Checklist

  • 02.1-02.3 [x] and status: complete (frontmatter sections[] verified 2026-05-28).
  • All success criteria have [x] checkboxes — success_criteria assertions are met by §02.1-§02.3 deliverables.
  • pytest scripts/plan_corpus/tests/test_convert_plan_dir.py (converter suite) green — 19/19 pass.
  • python -m scripts.plan_corpus check plans/scripts-first-restructure/section-02-*.md exit 0.
  • /tpr-review passed (final, full-section) — 5-round invocation 2026-05-28; 13 actionable findings cured inline; third_party_review.status: cap_reached_clean.

References

  • Pass 1A inventory: 006_md_to_json.py:549 (build_plan_json_from_dir); node_identity.py:45 (node_id), :51 (slugify), :90 (migrate_plan), :30 (_STATUS_FWD) — extracted from 010_edges_to_route.py in §02.1; 010_edges_to_route.py:129 (_run CLI), :80 (migrate_plan_dir), :91 (reverse_to_plan); _sidecar_emit.py:50 (slugify def, 60-char cap), :25 (300-char items threshold), :475/:573/:587/:643/:666 (slugify call sites for layout decisions).
  • Validated POC retired: previous /tmp/sfwa-poc/poc_convert.py per-plan + flat + no-aliases proof landed as the production convert_plan_dir.py (see _build_routing_node at :241 for the routing-only field drop, formerly POC’s clean_nodes).
  • §01 invariants INV-3 (JSON routing-only), INV-4 (per-plan SSOT), INV-12 (content keyed by id) — IDs per §01.1 invariant→section table.

HISTORY

  • 2026-05-27 — Linear-execution rule #1/#4 auto-reversal: plan-cleanup detected out-of-order subsection completion (02.N marked complete while a predecessor was not). Reverted those subsections + completion checklist to not-started; flipped section reviewed: true → false. Re-run /review-plan to determine next steps.
  • 2026-05-28 — /review-plan autopilot run 3c56fac16e6740f6b1608e1c15e06688: Full Step 1–9 pipeline ran with /tpr-review × 5 rounds (codex+agy+claude-ds). Verdict: SIGNIFICANT REWORK APPLIED → reviewed:true flipped. 7 audit findings + 11 blind-spot findings + 2 architectural risks filed as §02.R checkboxes. Autopilot mechanical cures landed across commits 9e6cc931 (filings) → 59c0540b (line anchors + clean_nodes + INTEGRITY ack) → 3288c2c1 (overview/index resync) → a31577ac (EMPTY_BODY_PLACEHOLDER SSOT) → 96512f25 (—self-test subcommand) → 472beaf1 (_resolve_mission_ref invariant pin) → 64540a3e (conceptual_integrity_owner propagation through 006) → fbfed1ce (glob-collision fail-loud raise). 10/11 §02.R items resolved; remaining 3 architectural-decision items (TPR-02-006 INV-19 forward-dep DD-01 decision, TPR-02-007 (b) item-id pre-commit lint design, TPR-02-009 (c) workflow-architecture dry-run parity test) deferred to user touchpoint per §Mid-Execution Prompts exception-3 ban under autopilot.
  • 2026-05-28 — TPR-02-011 duplicate-node-id regression cured: /continue-roadmap --autopilot resume picked up the last open §02.R blocker. Root cause traced across three layers — (i) 24-bit truncated-sha1 namespace was too small (~1% birthday collision at 184 nodes), (ii) source 006 parser legitimately emits duplicate sub_id values (02.A, 26.1, 27.1, 28.1) for distinct subs, (iii) migrate_plan’s id_map[sub_id] dict lookup at node-build time silently collapsed duplicate-sub_id rows to the LAST node’s id (real root cause). Cure landed in node_identity.py: node_id extended to fold optional slug into hash input + hash widened to 8 hex (32 bits, 50000× larger namespace); migrate_plan restructured to slugify-first → ids-parallel-to-subs (NOT keyed on sub_id) → build-nodes consuming parallel ids. id_map retained for hint resolution only. Test pinning: test_p_scripts_first_workflow_architecture_id_uniqueness promoted from xfail to positive pin (validates 184-node clean conversion); new test_q_node_id_slug_disambiguates_collision unit pin covers back-compat + slug disambiguation + idempotency + migrate_plan duplicate-sub_id integration. 19 convert_plan_dir tests pass (17 prior + test_p promoted + test_q new = 19).
  • 2026-05-29 — §07A v7 route schema supersedes the v6 nodes[] model: the route schema v7 two-level plan.json (sections[] + flat work_items[], per §07A + decisions/04-sections-work-items-split.md) supersedes this section’s v6 flat-node nodes[] contract; §02’s v6 converter stays complete but the current emitter is scripts/plan_corpus/convert_plan_dir_v7.py.