Root closure durability + recovery
Goal
Harden the root closure and invocation tree before shadow/cutover. Make nested work crash-resumable, fenced, version-consistent, and safe under cancellation and concurrent discovery.
Implementation sketch
- Journal every child registration, mutation, acceptance, return, fold, freeze, finalization, commit, and report effect before execution.
- Use request IDs for root creation plus closure_id:invocation_id:node_id:action:attempt keys and monotonic fencing epochs.
- Reconcile root state against domain artifacts and repositories without replaying landed effects.
- Reject cycles, orphan returns, stale generations/attempts/engine epochs, duplicate non-identical returns, and cross-root capabilities.
- Pin immutable engine/workflow bundles at root creation as typed StoreReference values via the harness_dist release store; descendants and crash resume resolve the pinned reference (the engine never materializes bundle bytes).
- Fence frozen roots with repository/tracker/queue revisions under owning locks before the first finalizer.
- Persist Reminder lifecycle and closure obligations at invocation/subtree/root scopes.
- Journal task launch intent before provider dispatch; require idempotent launch/lookup/cancel request IDs, persist provider handles, and reattach through resume obligations.
- Persist runnable-turn leases and ShelvedReceipt origin/blocker/return state; fence and requeue/reattach when a client ends while runnable.
- Persist UTC ReminderLease due times/duration, boot identity, refresh generation, and acknowledgement; let engine-core ReminderScheduler derive monotonic deadlines, take over orphan leases, refresh on clock uncertainty, and fence stale delivery.
- Journal each script-generated ContinueRuntimeView/ClientTransportDirective plan/root/generation/state/prompt digest and each authorized setup/cleanup tool receipt. Keep provider goal/task handles and scratch sentinels as projection/idempotency evidence only; they never mirror plan status, current task, or upcoming task.
- On restart, reconstruct domain truth from the plan/tracker API and execution truth from the root journal, snapshot the actual client lifecycle, then repair or decline the projection. The next goal/loop trigger still performs only a fresh continue-runtime read; never recover a transition, current/next task, or completion verdict from goal/loop state, transcript prose, or scratch.
Test strategy / acceptance
Fault injection surrounds every durable boundary. Cover request/launch/result/client-yield, timer fire/inject/ack/crash/restart, transport directive/tool receipt, foreign/owned client lifecycle, stale TerminalReceipt, shelve/push/return/resume, cancellation, generations, revisions, bundles, publication, and finalizer replay.
Work Items
- Implement invocation-tree journals, reconcile-resume, and depth-independent traversal
- Adopt request/root/invocation/attempt idempotency keys, owner-bound capabilities, CAS generation/external-revision checks, and fencing epochs
- Journal and crash-harden the root-closure w-4bb72d45/w-3639089d child/subtree/root cancellation and compensation protocol (persist, never re-implement, its cancel/compensate/tombstone transitions): durable crash-survivable tombstones, orphan-return detection on restart (reject returns whose parent frame no longer exists), and safe durable release of accepted-pending-root-close children under concurrent discovery.
- Materialize and resolve immutable engine/workflow bundles per root; migrate or reject returns across self-upgrade epochs
- Pin and resolve immutable engine/workflow bundles per root as typed StoreReference values via the harness_dist release store (scripts/harness_dist/release_store.py owns publication/resolution/GC; the engine never materializes bundle bytes); migrate or reject returns across self-upgrade epochs
- Persist Reminder lifecycle, evidence receipts, closure obligations, and invalidation at invocation, subtree, and root scopes
- Implement durable task launch/lookup/cancel journaling, provider-handle reattachment, and duplicate-execution refusal across every dispatch crash boundary
- Persist runnable-turn and ReminderLease timers plus fix-interference shelve/blocker/return state and ClientTransportDirective/receipt digests; recover overdue refresh, client yield, disconnect, transport projection drift, and crash without user-authored continuation or reconstruction from client state
- Build fault-injection, concurrency, and arbitrary-depth property suites before enabling shadow or cutover
Intel dossier — pointer and tier disposition
Dossier: root-closure-durability-recovery--s-7aad8aa5.intel.md (this section’s content/intel/ sidecar)
Read that dossier from line 1 through EOF before acting on this section. This block is a pointer and an audit record, never a substitute: not the BLUF, not a summary, not selected tiers.
| Dossier tier | Title | Lines | Disposition | Where / why |
|---|---|---|---|---|
| 0 | 0. Bottom line up front | 133 | pending | |
| 1 | 1. The target, verbatim | 48 | pending | |
| 2 | 2. Terrain — symbols and files | 93 | pending | |
| 3 | 3. Code-graph recon | 152 | pending | |
| 3V | 3V. Vacuous-pass audit — the assigned defect-class sweep | 40 | pending | |
| 3D | 3D. Diagnostic / observability surface | 30 | pending | |
| 3H | 3H. Hygiene constraints the downstream work inherits | 35 | pending | |
| 4 | 4. Cluster / family — where this section sits | 34 | pending | |
| 5 | 5. Conformance audit — graph SUBSTRATE-DARK, source substitute supplied | 54 | pending | |
| 6 | 6. Plan ownership | 39 | pending | |
| 7 | 7. Prior art — interrogated, not emulated | 56 | pending | |
| 8 | 8. Sentiment | 19 | pending | |
| 9 | 9. Declared coverage gaps — what is dark, and why | 41 | pending | |
| 10 | 10. Recon entry points — the ordered read path | 39 | pending |