100%

Goal

R12 USER-CONFIRMED storage layout: ONE flat content/intel/ folder, sibling of the content/section-<sid>/ section dirs (this plan’s content/ holds content/section-s-<hex>/ dirs; content/overview/ is the write.py template-intended mission_ref home, not materialized in this plan on disk) — plan.intel.md (plan-level rollup) + <slug>--s-<hex>.intel.md per section, name-matched 1:1 to the section body. Retire the gitignored .intel/intel_package/ ephemeral cache AND the nested content/section-<sid>/intel/ per-section dirs.

Section ownership boundary (WBS — resolves the plan.intel.md rollup double-ownership)

storage-layout (s-983a8a1b) is the lexorank PREDECESSOR of schema-integration (s-e2117453); a successor cannot satisfy a predecessor’s hint_needs under INV-19. The plan-root rollup work is split so neither section double-migrates nor leaves a gap:

  • storage-layout (THIS section) owns the on-disk FILE layer: it owns the storage layout, so it owns (a) the write.py dest-template flattening, (b) the content/intel/plan.intel.md rollup-FILE write + its exact-filename INV-18 exemption registration, and (c) the on-disk FILE relocation (moving each content/section-<sid>/intel/package.md into the flat content/intel/ layout, plus the sibling .intel.json envelope).
  • schema-integration (successor s-e2117453) owns the plan.json SCHEMA layer ONLY: the structured Section.intel_package {path, generated_at_sha} shape, the new plan-root intel_package ref struct, the plan-routing.schema.json path pattern, the v2 PlanSectionSchema reconcile, and the forward-only plan.json ref backfill (w-d7213728). It builds ON this flat layout; it never writes or relocates a .intel.md/.intel.json FILE.
  • The boundary: storage-layout writes/moves the FILES (incl. content/intel/plan.intel.md itself); schema-integration backfills the plan.json REF struct {path, generated_at_sha} that POINTS at those files. The rollup FILE write is here; only the rollup REF struct is the successor’s.

Relocation-seam framing:

  • The seam is scripts/plan_corpus/write.py — the hardcoded engine-writer dest template _SIDECAR_PATH_TEMPLATE['intel_package'] (write.py:3593) currently maps to content/section-{sid}/intel/{name}.md.
  • The flat layout lands by changing THAT template, not by rewriting the runtime files that delegate to it.
  • This section folds in the MINIMAL schema-side changes its own writes need to land + validate (the write.py dest template + the EXACT content/intel/plan.intel.md plan-level exemption registration). It also owns the rollup FILE write (content/intel/plan.intel.md) since it owns the storage layout.
  • The broader v7 schema integration — the Section.intel_package {path, generated_at_sha} struct, the plan-root intel_package REF field, the plan-routing.schema.json path pattern, the v2 reconcile, and the plan.json REF backfill — is a forward-handoff to successor schema-integration--s-e2117453.md. Only the REF struct is the successor’s; the FILES are this section’s per the §Section ownership boundary.

Implementation Sketch

  • Change the engine-writer dest template scripts/plan_corpus/write.py:3593 _SIDECAR_PATH_TEMPLATE['intel_package'] from content/section-{sid}/intel/{name}.md to the flat content/intel/{name}.md layout. This template is the SINGLE relocation seam — update_sidecar (write.py:3601) formats it per write; every sidecar consumer routes through it.
  • Register the EXACT rollup path content/intel/plan.intel.md (NOT the bare directory prefix content/intel/) for the INV-18 checkbox-strip exemption in scripts/plan_corpus/route_v7.py:409. PLAN_LEVEL_SIDECAR_PREFIXES matches by startswith, so registering the content/intel/ prefix would ALSO exempt every per-section <slug>--<id>.intel.md — the over-exemption this section forbids. Register the exact rollup filename: if the registry only accepts directory prefixes, add an exact-match branch (or a dedicated PLAN_LEVEL_SIDECAR_FILES exact set) so only content/intel/plan.intel.md is plan-level and the per-section files stay per-section reference content.
  • Do NOT rewrite storage.py — store_v7_sidecar (storage.py:27) delegates to write.update_sidecar; it carries no destination knowledge and changes nothing once the write.py template is flat. Leave its delegation intact.
  • persist.py (98) — keep as-is for what it owns: persist_package validates the staged package.json (via consume.load_package), derives the sibling package.md, route-detects, and on the v7 path calls store_v7_sidecar; resolve_persistence_section_id validates the section. It already routes the package.md through the engine writer, so it inherits the flat dest automatically. No path-template logic lives here.
  • stage.py (91) — stage_for_plan_dir (stage.py:44) writes the ephemeral <plan_dir>/.intel/intel_package/ cache (package.json + package.md), keyed off target_for_plan_name(...). The DURABLE persistence path is what moves to the flat content/intel/ layout; the ephemeral staging SCRATCH dir is RETAINED for its run-time one-shot-staleness cache role (stage_for_target’s prior envelope cache key), NOT retired wholesale. Two distinct concerns:
    • Durable layout (this section’s deliverable): content/intel/<slug>--<id>.intel.{md,json} — the committed, name-matched sidecars.
    • Ephemeral run-time cache (retained, NOT durable): the .intel/intel_package/ scratch dir feeding prior-based staleness checks.
    • The ephemeral scratch PATH MUST disambiguate across target kinds — stage_for_plan_dir only resolves plan-section targets via target_for_plan_name, but stage_for_target serves all four kinds (plan-section / bug / diff / symbol-set per target_kinds.py). A bare <plan_dir>/.intel/intel_package/ path collides a diff-target or symbol-set-target cache against the plan-section cache. The retained scratch path keys on the target-kind + target-ref (e.g. .intel/intel_package/<kind>/<ref-stem>/) so distinct target kinds never share one cache slot. The durable content/intel/ flat layout is a SEPARATE namespace from this ephemeral cache and never collides with it.
  • Migrate the existing populated packages into content/intel/ (see Work Items for the count disambiguation).
  • The package md is REFERENCE CONTENT ONLY (no checkbox, no status) per plan-read-discipline.md §3.5.

JSON-envelope replacement before retiring .intel

  • Retiring the .intel/intel_package/ ephemeral cache removes the package.json envelope that consume.py:load_package (consume.py:20) parses for the /intel-package consume path.
  • The replacement is DURABLE but MUST be uniquely named per package — a flat content/intel/ directory holds one rollup + every section’s package, so a bare package.json per section would collide to a single file. Name each durable envelope to MIRROR its .intel.md sidecar: per-section <slug>--<id>.intel.json beside <slug>--<id>.intel.md, and the rollup plan.intel.json beside plan.intel.md. The .md sidecars are already uniquely named, so the .json envelopes adopt the same unique stem.
  • consume.py:load_package currently key-resolves by appending a bare package.json to a directory (the path.is_dir() → path / "package.json" branch at consume.py:30-31). In the shared flat content/intel/ dir that shortcut collides every package to one file — it MUST instead key on the unique <slug>--<id> STEM: given a content/intel/<slug>--<id>.intel.md ref (or its bare stem), resolve the sibling <slug>--<id>.intel.json; given the rollup content/intel/plan.intel.md ref, resolve content/intel/plan.intel.json. Stem-keyed resolution replaces directory-append for the flat layout.
  • The ref→JSON-envelope path mapping is centralized in ONE helper in the already-present scripts/intel_package_runtime/envelope.py (the module consume.py already imports validate_envelope from) as the SSOT across all 5 consumers (/continue-roadmap recon, /review-plan audit, /tpr-review grounding, /add-bug relatedness, /fix-bug RCA). Add an envelope.py function (e.g. envelope_path_for_ref(ref) -> Path) that maps a .intel.md ref / stem / dir to its sibling .intel.json; load_package and every other consumer call it rather than re-deriving the .md→.json mapping inline. Re-deriving the mapping anywhere else is LEAK:scattered-knowledge.
  • persist_package already derives the sibling package.md from the staged package.json (persist.py:89); persist BOTH the uniquely-named .intel.md (engine sidecar) and the uniquely-named .intel.json (envelope) into content/intel/ so every per-package envelope resolves with no collision. The s-15e599fa pins assert each flat package JSON resolves to its own envelope via the envelope.py mapping.

Spec References

  • scripts/plan_corpus/write.py:3593 (_SIDECAR_PATH_TEMPLATE['intel_package'] dest template — the relocation seam) + write.py:3601 (update_sidecar formats it)
  • scripts/plan_corpus/route_v7.py:409 (PLAN_LEVEL_SIDECAR_PREFIXES — register the EXACT filename content/intel/plan.intel.md via a new exact-match set/branch, NOT the bare content/intel/ prefix which would startswith-over-exempt per-section files)
  • scripts/intel_package_runtime/persist.py (persist_package derives package.md, calls store_v7_sidecar) + stage.py:44 (stage_for_plan_dir — repurpose .intel/intel_package/ away from durable output to a target-kind-disambiguated ephemeral cache) + consume.py:load_package (consume.py:30-31 directory-append → stem-keyed flat-ref resolution) + scripts/intel_package_runtime/envelope.py (centralized ref→.intel.json path-mapping SSOT for all 5 consumers; consume.py already imports validate_envelope from it)
  • scripts/intel_package_runtime/storage.py:27 (delegates to write.update_sidecar — NOT a rewrite target)
  • plans/intel-package-dossier-rebuild/content/schema-integration--s-e2117453.md (successor — owns the BROADER v7 schema integration as a forward-handoff; this section declares NO depends_on:)
  • intel-package-rebuild-notes.md R11, R12; the ASCII layout confirmed in session
  • .claude/rules/plan-read-discipline.md §3.5 (sidecars are reference content only)

Sequencing — forward-handoff, not backward dependency

  • This section declares no advisory dependency (hint_needs: [], the active v7 field — depends_on is retired per v7-plan-system.md §7) and is independently completable: the write.py dest-template change + the content/intel/ plan-level-prefix registration are the minimal schema-side changes its OWN writes need to land + validate.
  • It does NOT declare hint_needs: — an advisory edge to a lexorank SUCCESSOR is a backward/diamond edge that violates INV-19 (strict-linear-single-branch DAG).
  • The lexorank order storage-layout → schema-integration sequences the BROADER schema work AFTER this layout lands: the structured Section.intel_package {path, generated_at_sha} shape, the plan-root intel_package REF field (the {path, generated_at_sha} struct that POINTS at the content/intel/plan.intel.md FILE this section writes — the FILE is here, the REF is the successor’s), the path pattern in plan-routing.schema.json, the v2 PlanSectionSchema reconcile, the forward plan.json REF backfill.
  • schema-integration--s-e2117453.md builds ON this flat layout; this section hands it forward.

Work Items

  • Flat-layout writing content/intel/plan.intel.md (plan-level rollup) + content/intel/<slug>--<id>.intel.md (per-section), via the two seam changes (storage-layout owns the rollup FILE write per the §Section ownership boundary; the rollup REF struct is schema-integration’s):
    • Change scripts/plan_corpus/write.py:3593 _SIDECAR_PATH_TEMPLATE['intel_package'] to the flat content/intel/{name}.md dest.
    • Register the EXACT rollup filename content/intel/plan.intel.md for the INV-18 checkbox-strip exemption — NOT the bare directory prefix content/intel/. scripts/plan_corpus/route_v7.py:409 PLAN_LEVEL_SIDECAR_PREFIXES is a tuple matched by startswith; registering the content/intel/ prefix would startswith-match (and thus over-exempt) every per-section content/intel/<slug>--<id>.intel.md too. Add a dedicated exact-match set/branch (e.g. PLAN_LEVEL_SIDECAR_FILES = ("content/intel/plan.intel.md",) consulted by an == -equality check alongside the prefix startswith check) so ONLY content/intel/plan.intel.md is treated as plan-level; per-section .intel.md files stay per-section reference content.
    • Persist a durable .intel.json envelope beside each .intel.md in content/intel/ so consume.py:load_package resolves the flat ref via the centralized envelope.py stem→envelope mapping; leave storage.py delegation + persist.py derivation untouched (they inherit the flat dest).
  • Retire .intel/intel_package/ durable-write + nested content/section-<sid>/intel/ write paths:
    • Retire the stage.py:44 <plan_dir>/.intel/intel_package/ location AS A DURABLE-PERSISTENCE target; the ephemeral run-time staging cache is RETAINED with a target-kind-disambiguated path per the §stage.py clarification above (it is not deleted, only repurposed away from durable output).
    • The nested content/section-<sid>/intel/ path disappears as a consequence of the write.py:3593 template flattening — no separate code site retires it; verify the old prefix is absent after execution.
  • Relocate the existing on-disk packages into content/intel/ (storage-layout owns the FILE relocation; schema-integration’s w-d7213728 owns the disjoint plan.json REF backfill — no overlap):
    • Migration scope — explicit (resolves the understated “9 plans” figure): the migration target is the ref-populated set — the plans whose plan.json carries a populated intel_package ref (measured on disk: 15 plan.json files carry the ref; the prior “9” figure is stale and is superseded by the measured count). The on-disk file footprint of that set is content/section-<sid>/intel/package.md nested dirs (measured: 15 plans corpus-wide) plus .intel/intel_package/ ephemeral caches (measured: 10 plans). The 6 in-plan on-disk package.md files under THIS plan are a subset of that nested-dir footprint.
    • In-plan: relocate the 6 on-disk content/section-<sid>/intel/package.md files under this plan into the flat content/intel/ layout (each becomes content/intel/<slug>--<id>.intel.md + sibling .intel.json).
    • Cross-plan corpus: relocate every ref-populated plan’s nested content/section-<sid>/intel/package.md (the durable per-section packages) into that plan’s flat content/intel/ layout. Ephemeral .intel/intel_package/ caches are NOT relocated — they are run-time scratch (regenerated on next stage) and are out of the durable-relocation scope; the Verification “old paths absent” criterion below scopes “absent” precisely to the durable nested dirs of the ref-populated set, NOT to ephemeral caches that legitimately regenerate.

Verification

Pinned by the success criteria the engine evaluates (extending sc-0006 per learning-ledger.md-discipline criteria; the assertion pins live in plan.json and reference the s-15e599fa pin section). Execution is not complete until ALL hold:

  • content/intel/plan.intel.md rollup FILE is generated (the plan-level rollup exists at the flat location and the EXACT filename content/intel/plan.intel.md — not the bare content/intel/ prefix — is registered for the INV-18 exemption in route_v7.py:409). Per-section content/intel/<slug>--<id>.intel.md files are confirmed NOT plan-level-exempt (a per-section file with a checkbox still fails INV-18 validation).
  • New per-section path is populated: each section’s content/intel/<slug>--<id>.intel.md exists, name-matched 1:1 to the section body.
  • Old durable paths are ABSENT after execution, scoped to the ref-populated set: no content/section-<sid>/intel/ nested dir remains in any plan whose plan.json carries a populated intel_package ref (the migration target). The “absent” qualifier is the durable nested dirs of the ref-populated set; ephemeral .intel/intel_package/ run-time caches are out of scope (they regenerate on next stage and are not a durable old-path leak).
  • Each migrated package’s content matches its original: every relocated .intel.md is byte-identical (modulo the path move) to the source package.md it was migrated from — no content lost or altered in the relocation.
  • Persist + consume round-trip (not just file placement): for a generated plan-section package, the sibling .intel.json envelope persists beside the .intel.md AND consume.py:load_package, resolving via the centralized envelope.py stem→envelope mapping, returns a valid (non-degraded) envelope from the flat content/intel/ ref. The pytest pin in scripts/intel_package_runtime/tests/ exercises the full round-trip (persist .intel.md + .intel.json → resolve the flat ref → load_package returns the same non-degraded envelope), not merely that the .md file landed.

Reference the s-15e599fa pins, which assert the rollup-generation, old-paths-absent, migrated-content-match, and persist+consume-round-trip conditions.