97%

Section 07A — Route Schema v7 (Sections + Work_items Split)

Cites §01 north star (invariants 3, 10, 12, 15 — JSON-routing-only + single-update-point + content-keyed + flat-nodes): the v7 two-level schema realizes those invariants in the per-plan plan.json shape.

  • Supersedes the v6 flat-node nodes[] model for every per-plan plan.json.
  • Full design SSOT: decisions/04-sections-work-items-split.md (schema shape, resolved decisions, blocker-move illustration, invariant deltas, reshape footprint).
  • Inserted after §07, before §08 (linear chain §06 -> §07 -> §07A -> §08); the rest of the plan builds on the v7 shape.

Intelligence Reconnaissance

Run 2026-05-29:

  • scripts/intel-query.sh dag-ascii scripts-first-restructure — §07A linear-walk position §07 -> §07A -> §08; depends_on: ["07"] (sole direct predecessor per INV-19; §08 re-pointed to [“07A”]).
  • Remaining bullets are direct file citations (wrapper Python + JSON not reliably intel-indexed):
  • [ori:plans/scripts-first-restructure/decisions/04-sections-work-items-split.md] — the v7 schema SSOT this section delivers.
  • [ori:scripts/plan_corpus/migrations/006_md_to_json.py] — the §02 converter that must emit sections[] + work_items[] instead of nodes[].
  • [ori:scripts/plan_orchestrator/route_walk.py] — §06 engine serve_next to reshape (walk sections, return current work_item, own work_item status lifecycle).
  • [ori:plans/completed/scripts-first-workflow-architecture/plan.json] — the existing 184-node v6 plan.json to re-convert to v7 (engine-only per §08; user-typed unlock required to read).
  • [ori:plans/scripts-first-restructure/section-01-principles.md] — INV-10/12/15/19/20 definitions to update per decisions/04.

07A.1 — Schema v7 definition + validators

  • Confirm decisions/04-sections-work-items-split.md captures the final schema v7 (sections[] + work_items[]), the resolved decisions, the blocker-move illustration, and the invariant deltas (it is the SSOT for the rest of this section).
  • Define schema v7 in scripts/plan_corpus/ (the v6 schema validator + plan_json_write API): schema: scripts-first-restructure/plan-routing/v7; sections[] {id: s-, slug, key, status, hint_needs, body_ref}; work_items[] {id: w-, section_id, key, status, hint_needs}. No nodes[]. No body text in JSON (INV-3 preserved). Delivered: scripts/plan_corpus/schemas/v7/plan-routing.schema.json + scripts/plan_corpus/schema_loader.py:load_plan_routing_v7_schema.
  • Author the array==sorted(key) validator (scripts/plan_corpus/ v7 validator module, per decisions/04 resolved-decision 3 + 11): work_items[] physical order MUST equal sorted(work_items, key=lexorank), and sections[] likewise; AND lexorank keys MUST be strictly unique within each array (pairwise-distinct — duplicate keys rejected as a hard precondition, per decisions/04 resolved-decision 11). Reject (validation error) any out-of-order OR duplicate-key array. Pytest: out-of-order array rejected; in-order accepted; a between-key insert keeps it sorted; test_duplicate_lexorank_key_rejected (two work_items sharing a key -> rejected; two sections sharing a key -> rejected). Delivered: scripts/plan_corpus/route_v7.py:validate_array_sorted_and_unique + scripts/plan_corpus/tests/test_route_v7.py.
  • Author the JSON<->md item-coherence validator: every work_items[].id appears as an id-reference in its section_id’s body_ref md; every id-reference in a section md has a work_items[] entry. Reject orphans either side. Pytest: orphan-in-json rejected; orphan-in-md rejected; matched set accepted. Delivered: scripts/plan_corpus/route_v7.py:validate_md_item_coherence + scripts/plan_corpus/tests/test_route_v7_coherence.py.
  • Subsection close (07A.1) — all [x]; status: complete.

07A.2 — §02 converter reshape

  • Emit sections[] + work_items[] from the source markdown: one section per source markdown section (slug-first content/<slug>--s-<hex>.md), one work_item per checkbox item hoisted into work_items[] with a stable w-<hex> id + section_id pointer + lexorank key + status derived from the source checkbox state. Delivered ADDITIVE in scripts/plan_corpus/convert_plan_dir_v7.py (reuses the 006_md_to_json.build_plan_json_from_dir parser SSOT; the v6 nodes[] emitter convert_plan_dir.py stays intact for its live consumers route_check.py + plan_json_write.py — v7 additive until §07A.5 re-converts the subject, per decisions/04).
  • The converter strips checkboxes from the emitted content md (items live in work_items[]); the md retains the item DESCRIPTION lines keyed by the stable w-<hex> id (greppable, no - [ ]).
  • Acceptance (pytest): converting a fixture markdown plan yields a v7 plan.json whose array==sorted(key) + JSON<->md coherence validators (imported from §07A.1 route_v7.py) pass; item count in work_items[] equals source checkbox count; section count equals source section count. Delivered: scripts/plan_corpus/tests/test_convert_plan_dir_v7.py (16 tests).
  • Subsection close (07A.2) — all [x]; status: complete.

07A.3 — §03 substrate reshape

  • DEPRECATE the id-keyed checkbox scanner (the v6 INV-10 status-derivation path) — status is now work_items[].status, not a markdown checkbox scan. Deprecation marker added to scripts/plan_corpus/checkbox_scan.py citing §11B as the hard-deletion owner; deletion DEFERRED to §11B (the dedicated strip-parsers section) per decisions/04 additive-until-§07A.5 discipline (the still-v6 subject plan.json relies on the v6 status contract until §07A.5 re-converts). Zero live production consumers confirmed by caller-grep.
  • ADD the work_item status write API (atomic, fcntl.flock, mirrors the existing plan_json_write CAS discipline): flip a work_items[].status (not-started -> in-progress -> completed) + the move primitive (reassign section_id + re-key + re-splice the array to sorted position). The API is the SOLE status-write path; the LLM never edits plan.json (§08). Delivered: scripts/plan_corpus/work_item_status.py (flip_work_item_status forward-only + section-rollup recompute, move_work_item) reusing plan_json_write.mutate_routing_state CAS+flock (no new lock regime).
  • Wire both validators (07A.1) into plan_corpus check so v7 plans are validated at write-time + pre-commit. Delivered: route_check.check_route_dir dispatches by is_v7_plan_json → validate_v7_plan_json (array==sorted+unique + JSON<->md coherence), STRUCTURE:route-v7-schema-violation finding; v6 path unchanged (additive gating). Fixed an in-scope validation-bypass gap (is_route_migrated_dir check-branch widened to recognize v7).
  • Acceptance (pytest): status flip persists + array stays sorted; a move keeps hint_needs refs resolving + array sorted; coherence validator runs in plan_corpus check. Delivered: scripts/plan_corpus/tests/test_work_item_status.py (16 tests); CLI end-to-end verified (clean v7 exit 0; orphan-in-md exit 1 with v7 finding).
  • Subsection close (07A.3) — all [x]; status: complete.

07A.4 — §05 route model + §06 serve_next

  • Reshape the §05 route model to two levels: sections carry the linear chain (lexorank key), work_items are the flat movable units; hint_needs (both levels) is ADVISORY-ONLY per INV-6 + decisions/04 resolved-decision 8 — it informs authoring-time lexorank placement, never walk-time filtering.
  • Reshape scripts/plan_orchestrator/route_walk.py:serve_next: walk sections[] in array order (= key order) to the first non-completed section; within it walk work_items[] (by section_id) in array order to the first NON-COMPLETED work_item — NO hint_needs filtering at walk time. In-progress precedence (work_item-level INV-20, per decisions/04 resolved-decision 10): if that first non-completed work_item is in-progress, serve_next returns it AS-IS and NEVER walks past it to a later not-started work_item — at most one work_item per section is in-progress at a time (mirrors the INV-20 section-level “in-progress predecessor returned as-is, never skipped” clause at the work_item level). hint_needs is ADVISORY per INV-6 + decisions/04 resolved-decision 8: serve_next does NOT skip a work_item with unmet hint_needs; it serves the first non-completed work_item in array order and surfaces any unmet hints as advisory metadata, returning {section, body_ref, work_item_id, unmet_hints: ["w-<hex>"]}. Array order (lexorank key) is the sole walk-time ordering authority; the blocker-move pattern (re-key the blocker physically before the work it unblocks) keeps array order sufficient without a hard hint-gate (per decisions/04 resolved-decision 8).
  • serve_next is WRITE-ON-SERVE per decisions/04 resolved-decision 10: it routes (reads the next work_item) AND flips its status not-started -> in-progress as an atomic side effect, and flips in-progress -> completed only when INV-20 gates pass — it is the single status-transition entry point. Add a read-only peek_next variant that returns the identical routing payload ({section, body_ref, work_item_id, unmet_hints}) WITHOUT any status write, for dry-run / lookup callers (dag-ascii, resume-manifest builders) that must never mutate plan state. The query (read) and the transition (write) are separated by which function is called, never folded into one ambiguous call.
  • section.status is a DERIVED rollup of its work_items (never independently written): all completed -> completed; any in-progress/mixed -> in-progress; all not-started -> not-started; ZERO work_items -> completed (decisions/04 resolved-decision 9 — vacuous-truth so an empty section never deadlocks the walk). serve_next + the status API compute it.
  • Acceptance (pytest): serve_next returns the section + current work_item verbatim with unmet_hints advisory metadata (never skips on unmet hints); never runtime-sorts (trusts array order); engine flips work_item status (LLM never does); peek_next returns the same payload with zero status mutation; an in-progress work_item is returned verbatim, never skipped past to a later not-started one; test_serve_next_returns_whole_section_body + test_work_item_status_engine_only + test_serve_next_does_not_filter_on_hint_needs + test_peek_next_is_read_only + test_serve_next_returns_in_progress_work_item_as_is (work_item-level INV-20 precedence: a section with one in-progress + later not-started work_items serves the in-progress one, never the not-started one).
  • Acceptance (pytest) — section.status rollup matrix in scripts/plan_orchestrator/tests/ covering every rollup branch from decisions/04 resolved-decision 4+9: test_section_status_rollup_all_not_started (all work_items not-started -> section not-started); test_section_status_rollup_mixed_in_progress (any in-progress or mixed not-started/completed -> in-progress); test_section_status_rollup_all_completed (all work_items completed -> completed); test_section_status_rollup_empty_section_is_completed (zero work_items -> completed, walk advances past it); test_section_status_rejects_independent_write (a direct write to section.status bypassing the work_item rollup is rejected — status is derived, never independently written).
  • Acceptance (pytest) — preserve test_route_walk_exit_reason_parity (the v6 exit_reason contract in scripts/plan_orchestrator/tests/) under v7 by adding fixtures: a v7 plan whose serve_next yields a current work_item; a v7 plan with no not-started work_item left (no-ready / queue-drained exit_reason); a v7 plan whose served work_item lacks its deliverable (INV-17 quarantine exit_reason). Each fixture asserts the same exit_reason key the v6 walk produced for the equivalent node-level state, so the v7 sections/work_items walk is exit_reason-compatible with the v6 nodes walk.
  • Subsection close (07A.4) — all [x]; status: complete.

07A.5 — Re-convert subject plan.json + invariant-doc updates

  • Re-convert the existing scripts-first-workflow-architecture plan.json to v7 (sections[] + work_items[] + slug-first content/<slug>--<id>.md filenames); both validators green. Result on disk: 49 coarse sections[] + 184 work_items[]. Mechanism: the coarse shape was produced by the one-off scripts/plan_corpus/regroup_v7_coarse.py (v7->v7 coarsening; commit 3461f8c2), NOT by re-running the md->v7 converter — the subject’s v6 source markdown is archived under _archive/ and no longer available to feed the md path. SEPARATELY, the converter source was corrected so future md->v7 conversions emit coarse output: convert_plan_dir_v7.convert_plan_dir_v7 now groups subsections by parent section (commit 093c214e; verified — scripts-first-restructure md->v7 yields 15 coarse sections, not 93). The legacy v6-nodes path reshape_v6_to_v7 strips checkboxes but remains 1:1 (N nodes -> N sections); it is superseded by regroup_v7_coarse (v7->v7) and the fixed md path for coarse output. Engine-only access per §08 — the LLM never raw-edited plan.json. The INV-18 content-no-checkbox validator gate (route_v7.validate_content_no_checkbox, commit 6dff58e1) prevents any checkbox regression in the emitted content md.
  • Update section-01-principles.md + references/restructure-charter-seed.md §2 invariant text: INV-6 (hint_needs ADVISORY — never a serve_next gate, per decisions/04 resolved-decision 8), INV-10 (status = work_items[].status field, not md checkbox), INV-12 (id-SUFFIX glob content/*--<id>.md), INV-15 (two levels: sections + work_items), INV-19 (single-branch over sections; hint_needs advisory; empty-section rolls up completed), INV-20 (serve_next over sections + engine owns work_item lifecycle; write-on-serve with read-only peek_next) — each citing decisions/04.
  • Update 00-overview.md (umbrella) for the v7 contract: reword the Mission Success Criteria bullets + North-star INV-2/INV-3 summary + data-flow description that currently describe the v6 flat-node nodes[] model so they state the v7 two-level sections[] + work_items[] contract (sections = one md each / status DERIVED rollup; work_items = stable section-independent ids / engine-written status; slug-first content/<slug>--<id>.md), citing decisions/04-sections-work-items-split.md. The umbrella description must match the v7 shape the rest of the plan now builds on, not the superseded v6 flat-node description.
  • Confirm §08 depends_on: ["07A"] (linear chain §06 -> §07 -> §07A -> §08); the v6 design of §02/§03/§05/§06 stays complete (superseded by this section, not reopened) — note the supersession in each of their HISTORY blocks.
  • Acceptance: python -m scripts.plan_corpus check plans/completed/scripts-first-workflow-architecture/ exit 0 on the v7 plan.json; slug-first content filenames resolve via id-suffix glob.
  • Subsection close (07A.5) — all [x]; status: complete.

07A.R Third Party Review Findings

  • None.

07A.N Completion Checklist

  • 07A.1-07A.5 [x] and status: complete.
  • All success criteria have [x] checkboxes.
  • schema v7 validators (array==sorted(key) + JSON<->md coherence) green; converter + substrate + serve_next pytest green. TPR final passed: 3 rounds (2 with cures), 9/9 findings resolved+committed a8d24505+06e0ab3b pushed origin/main; route_v7+route_walk_v7 81-test suite PASS; plan_corpus check section-07A + subject scripts-first-workflow-architecture exit 0; reviewed:true verdict SIGNIFICANT REWORK APPLIED 2026-05-29
  • subject plan.json re-converted to v7; python -m scripts.plan_corpus check plans/completed/scripts-first-workflow-architecture/ exit 0. TPR final passed: 3 rounds (2 with cures), 9/9 findings resolved+committed a8d24505+06e0ab3b pushed origin/main; route_v7+route_walk_v7 81-test suite PASS; plan_corpus check section-07A + subject scripts-first-workflow-architecture exit 0; reviewed:true verdict SIGNIFICANT REWORK APPLIED 2026-05-29
  • invariant docs (section-01 + charter-seed §2) carry the v7 deltas (INV-6/10/12/15/19/20); 00-overview.md Mission Success Criteria + North-star + data-flow re-stated for the v7 sections/work_items contract; §08 depends_on re-pointed to 07A.
  • python -m scripts.plan_corpus check plans/scripts-first-restructure/section-07A-*.md exit 0. TPR final passed: 3 rounds (2 with cures), 9/9 findings resolved+committed a8d24505+06e0ab3b pushed origin/main; route_v7+route_walk_v7 81-test suite PASS; plan_corpus check section-07A + subject scripts-first-workflow-architecture exit 0; reviewed:true verdict SIGNIFICANT REWORK APPLIED 2026-05-29
  • /tpr-review passed (final, full-section). TPR final passed: 3 rounds (2 with cures), 9/9 findings resolved+committed a8d24505+06e0ab3b pushed origin/main; route_v7+route_walk_v7 81-test suite PASS; plan_corpus check section-07A + subject scripts-first-workflow-architecture exit 0; reviewed:true verdict SIGNIFICANT REWORK APPLIED 2026-05-29

References

  • decisions/04-sections-work-items-split.md — schema v7 SSOT (this section’s design record).
  • Pass: §02 converter scripts/plan_corpus/migrations/006_md_to_json.py; §03 substrate scripts/plan_corpus/ write API + validators; §06 engine scripts/plan_orchestrator/route_walk.py.
  • INV-6 (hint_needs advisory) + INV-19 (single-branch) reconciliation; INV-4 (cross-plan lexorank merge); INV-18 (whole-content-unit to LLM).