Reversible cutover — new engine drives; ungate zeroed
Goal
- Select engine mode when a new RootClosure starts.
- Cut over invocation, state-authority validation, virtual batches, context control, client re-anchor projection, and preflight as one path.
- Never switch an active root by toggling the
WORKFLOW_ENGINE_LEGACYescape.
Self-hosting constraint
- Cutover is the ONLY moment the live drive path changes; it shipped as engine-default, with
WORKFLOW_ENGINE_LEGACY=1the single per-invocation escape restoring the legacy path verbatim. - Re-verify (NEVER re-fix) that per-item completion stays ungated across the cutover.
- The old
review_phase.pypath is ALREADY ungated today (no zeroed-gate to keep intact), so the job is confirming the engine matches that current behavior, not migrating away from a still-gated old path.
Cutover status — SHIPPED; remaining items are POST-cutover verification
The hard cutover has LANDED. Read the shipped contract, never the pre-cutover sketch:
- The engine serves + completes plan-targeted
roadmaporchestration andcomplete-work-item-v7BY DEFAULT (scripts/workflow_engine/drive_hook.py). WORKFLOW_ENGINE_LEGACY=1is the SINGLE documented per-invocation escape back to the legacy path; further reversibility is git-based (git revertthe cutover commit).- The rollback runbook is pinned by
scripts/workflow_engine/tests/test_cutover_reversibility.py. Discovery SSOT:.claude/rules/workflow-engine.mdDrive-route-default + Rollback-runbook rows. - BANNED: re-adding a
WORKFLOW_ENGINE_DRIVE-gated opt-IN route, or treating engine-drive as the non-default branch. The polarity is engine-default +WORKFLOW_ENGINE_LEGACYescape.
Implementation sketch
WORKFLOW_ENGINE_DRIVEgated the original opt-in cutover (w-ce6ed9ab, complete); the shipped default is engine-drive with theWORKFLOW_ENGINE_LEGACYescape above.- Re-verify the new engine’s per-item completion matches CURRENT
review_phase.py/route_walk.pybehavior (no zeroed-gate;ROUTE_REQUIREDretired) — mirroring the sec-9 required-verification pattern bug-flow-shared-engine’s w-954c5f62 runs against/fix-bug’s as-built phase mechanics. Assert noROUTE_REQUIREDon the new path for a non-zeroed plan. - Self-hosting proof: on the default drive route, drive THIS plan’s own post-cutover sections to completion through the engine (the north-star probe).
- Assert every post-cutover handoff carries this plan’s exact root/frame capability and rejects a command targeting another plan.
- Pin the selected engine bundle and ReentryPayload to the root at creation. Toggling
WORKFLOW_ENGINE_LEGACYaffects only future roots; an owned goal/loop for an active new-engine root continues to read that pinned bundle’s ContinueRuntimeAPI until it relays the matching TerminalReceipt, while foreign steering remains untouched. - Include ReentryPayload, ContinueRuntimeAPI/View, ClientTransportDirective, ReanchorGuard, and TerminalReceipt in the atomic cutover surface; reject a mixed path where goal/loop bypasses the continue runtime, the new engine runs under an old/client-authored prompt, or an old engine consumes a new transport receipt.
- Reversibility test:
WORKFLOW_ENGINE_LEGACY=1-> the legacy path’s emitted next_action + completion behavior is byte-identical to the verified shadow-parity baseline.
Gates: sc-0007 proves atomic new-root cutover; sc-116de6bf proves WORKFLOW_ENGINE_LEGACY=1 byte-identical reversibility.
Spec references
- plans/workflow-engine-rework/spec/workflow-state-machines.md sec 7 (historical zeroed-keying bug — verify only), sec 2 (validate-first per-item). Re-verify stable symbols
review_phase.py::ROUTE_REQUIRED,review_phase.py::complete_work_item_v7, and theroute_walk.pycompletion route;_section_is_verify_firstmust remain absent.
Work items
- LAST-MOMENT ABSORPTION FOLD (gates every cutover item in this section): the legacy workflow system keeps changing while this plan runs, so the real fold is deferred to here, the last responsible moment before the drive flag flips. (1) Disposition EVERY pending (disposition: null) row in content/bug-fold-manifest.json — each post-snapshot bug arrival becomes obe / absorb / release with per-row teeth per verify_bug_fold_manifest, driving pending to ZERO against the LIVE tracker at fold time; (2) legacy-delta reconciliation sweep: diff every superseded legacy path (scripts/plan_orchestrator/, scripts/fix_next_bug_runtime/, scripts/review_section_runtime/, scripts/review_plan_runtime/, scripts/commit_push/** and the rest of decisions/04-virtual-batch-supersession.md Superseded Paths) against each port’s baseline SHA, classify every behavior change landed since (ported / engine-already-covers / not-applicable with rationale), and port the uncovered deltas into the engine with pins; (3) re-run the full shadow-parity harness AFTER the fold so parity evidence reflects the folded state. Expect the pending set to have GROWN since any earlier snapshot — that is the designed shape, not drift.
- Commit-driven tooling-delta fold verification (user directive 2026-07-19): enumerate EVERY commit since plan start (db53b83fd, 2026-06-30) touching tooling paths (scripts/, .claude/rules/, .claude/skills/, .claude/hooks/, .claude/commands/**) — 273 commits confirmed in the db53b83fd..HEAD range — and disposition each against the new engine before cutover. Build content/tooling-delta-manifest.json (rows: {commit_sha, date, subject, tooling_paths, disposition: null|ported|engine-native|not-applicable|superseded-path|fold-required, evidence}) generated by a script (scripts/workflow_engine/tooling_delta_manifest.py) that git-log-scans the range and REFUSES unknown paths silently dropped. Build the verifier (verify_tooling_delta_manifest) by MIRRORING verify_bug_fold_manifest.py’s shape exactly — a fail-closed CLI probe with a main entry, canonical-API reads (git-log range scan; tracker reads reused via bug_tracker.api where applicable, never a hand-read of a raw JSON/git-log dump), exact-field rows, per-disposition + full-range-coverage teeth, exit-1-on-first-fail — reusing existing SSOT machinery (the same tracker-read API and the legacy deterministic helper for any byte-equal check) rather than authoring a second byte-equal or tracker-read machine. The verifier requires pending=0 and full commit-range coverage at fold time, wired beside verify_bug_fold_manifest into the w-c4a90002 last-moment fold gate; snapshot-coverage drift pins keep the manifest absorbing new tooling commits as parallel sessions land them (same append-only pattern as the bug-fold manifest).
- Add the
WORKFLOW_ENGINE_DRIVEfeature flag routing serve+completion through the new engine - Cut over RootClosure/InvocationFrame, closed state authority, virtual batch, ContextAssembler, RuleUnit/Reminder/ReanchorGuard, ReentryPayload/ContinueRuntimeView/ClientTransportDirective, TerminalReceipt, and preflight paths together; reject mixed-engine roots, goal/loop runtime bypass, prompts, and transport receipts
- Re-verify new-engine per-item completion matches CURRENT review_phase.py/route_walk.py behavior (no zeroed-gate; ROUTE_REQUIRED already retired) — confirm inheritance, not introduction; assert no ROUTE_REQUIRED for a non-zeroed plan
- Reversibility: the shipped escape
WORKFLOW_ENGINE_LEGACY=1restores the legacy path byte-identically against the verified shadow-parity baseline across bounded deterministic repetitions and every declared crash window; no elapsed-time waiting criterion - POST-CUTOVER PILOT (user directive 2026-07-19; rescoped — the cutover has shipped, so this item no longer gates
w-b6a03f29and carries no pre-cutover ordering claim): drive the real plans/bytecode-vm plan through the engine on its DEFAULT drive route, invoked from the CODEX client by the user (client-agnosticism proof: the engine’s ContinueRuntimeAPI + codex adapter serve the identical protocol; this session monitors read-only). Unit seam tests such astest_codex_default_serve_emits_native_goal_installprove the seam, NEVER this production execution. Deliverables: (a) a documented single-command codex invocation (entry command only — the drive route is the DEFAULT and the invocation carries NO engine env flag) requiring zero Claude-Code-specific surfaces; (b) a pilot evidence bundle — per-root correlated trajectory stream (trajectory.py), engine journals, transport/lifecycle records, and served-vs-completed unit log — collected under a runtime-owned pilot dir for post-run triage; (c) breakage protocol: any engine defect surfaced by the pilot is triaged from the trajectory/journal evidence, cured at source in this plan, and the pilot re-run; (d) acceptance: at least one full bytecode-vm serve->work->complete cycle lands end-to-end from codex with the evidence bundle checked in and zero unexplained divergence vs the legacy path’s byte-equal baseline where applicable. - Self-hosting proof: drive this plan’s own post-cutover sections to completion on the new engine
Test strategy / acceptance
Matrix: zeroed/non-zeroed x default-drive/WORKFLOW_ENGINE_LEGACY=1 x matching/wrong-plan target x adapter (Claude Code/Codex with owned/foreign native transport; OpenCode with explicit unavailable plus universal-floor-only recovery).
Pins:
- Per-item completion stays ungated.
- The default drive route uses one full script-prompted ContextAssembly/transport path.
- Active roots remain bundle-pinned across route changes.
- Wrong-plan or mixed-engine prompt/receipt rejects.
WORKFLOW_ENGINE_LEGACY=1invocations remain byte-identical to the legacy baseline.- spec-sec-11.4 legacy latency/resource comparators pass.
- Use fixed workloads, sample counts, and fault windows rather than elapsed-time waiting.
Intel dossier — pointer and tier disposition
Dossier: reversible-cutover--s-d343a923.intel.md (this section’s content/intel/ sidecar)
Read that dossier from line 1 through EOF before acting on this section. This block is a pointer and an audit record, never a substitute: not the BLUF, not a summary, not selected tiers.
| Dossier tier | Title | Lines | Disposition | Where / why |
|---|---|---|---|---|
| 0 | 0. Bottom line up front | 34 | pending | |
| 1 | 1. The target, verbatim | 30 | pending | |
| 2 | 2. Terrain — symbols and files | 37 | pending | |
| 3 | 3. Code-graph recon | 204 | pending | |
| 3D | 3D. Diagnostic / observability surface | 29 | pending | |
| 3H | 3H. Hygiene constraints the recommended direction inherits | 29 | pending | |
| 4 | 4. Cluster / family — the sibling sections and the fold gates | 42 | pending | |
| 5 | 5. Conformance audit — SUBSTRATE-DARK, with a source-confirmed substitute | 29 | pending | |
| 6 | 6. Plan ownership — the bridge does not reach this terrain | 29 | pending | |
| 7 | 7. Prior art | 23 | pending | |
| 8 | 8. Sentiment | 22 | pending | |
| 9 | 9. Declared coverage gaps — what is dark, and why | 31 | pending | |
| 10 | 10. Recon entry points — the ordered read path | 25 | pending |