80%

Root closure durability + recovery

Goal

Harden the root closure and invocation tree before shadow/cutover. Make nested work crash-resumable, fenced, version-consistent, and safe under cancellation and concurrent discovery.

Implementation sketch

  • Journal every child registration, mutation, acceptance, return, fold, freeze, finalization, commit, and report effect before execution.
  • Use request IDs for root creation plus closure_id:invocation_id:node_id:action:attempt keys and monotonic fencing epochs.
  • Reconcile root state against domain artifacts and repositories without replaying landed effects.
  • Reject cycles, orphan returns, stale generations/attempts/engine epochs, duplicate non-identical returns, and cross-root capabilities.
  • Pin immutable engine/workflow bundles at root creation as typed StoreReference values via the harness_dist release store; descendants and crash resume resolve the pinned reference (the engine never materializes bundle bytes).
  • Fence frozen roots with repository/tracker/queue revisions under owning locks before the first finalizer.
  • Persist Reminder lifecycle and closure obligations at invocation/subtree/root scopes.
  • Journal task launch intent before provider dispatch; require idempotent launch/lookup/cancel request IDs, persist provider handles, and reattach through resume obligations.
  • Persist runnable-turn leases and ShelvedReceipt origin/blocker/return state; fence and requeue/reattach when a client ends while runnable.
  • Persist UTC ReminderLease due times/duration, boot identity, refresh generation, and acknowledgement; let engine-core ReminderScheduler derive monotonic deadlines, take over orphan leases, refresh on clock uncertainty, and fence stale delivery.
  • Journal each script-generated ContinueRuntimeView/ClientTransportDirective plan/root/generation/state/prompt digest and each authorized setup/cleanup tool receipt. Keep provider goal/task handles and scratch sentinels as projection/idempotency evidence only; they never mirror plan status, current task, or upcoming task.
  • On restart, reconstruct domain truth from the plan/tracker API and execution truth from the root journal, snapshot the actual client lifecycle, then repair or decline the projection. The next goal/loop trigger still performs only a fresh continue-runtime read; never recover a transition, current/next task, or completion verdict from goal/loop state, transcript prose, or scratch.

Test strategy / acceptance

Fault injection surrounds every durable boundary. Cover request/launch/result/client-yield, timer fire/inject/ack/crash/restart, transport directive/tool receipt, foreign/owned client lifecycle, stale TerminalReceipt, shelve/push/return/resume, cancellation, generations, revisions, bundles, publication, and finalizer replay.

Work Items

  • Implement invocation-tree journals, reconcile-resume, and depth-independent traversal
  • Adopt request/root/invocation/attempt idempotency keys, owner-bound capabilities, CAS generation/external-revision checks, and fencing epochs
  • Journal and crash-harden the root-closure w-4bb72d45/w-3639089d child/subtree/root cancellation and compensation protocol (persist, never re-implement, its cancel/compensate/tombstone transitions): durable crash-survivable tombstones, orphan-return detection on restart (reject returns whose parent frame no longer exists), and safe durable release of accepted-pending-root-close children under concurrent discovery.
  • Materialize and resolve immutable engine/workflow bundles per root; migrate or reject returns across self-upgrade epochs
  • Pin and resolve immutable engine/workflow bundles per root as typed StoreReference values via the harness_dist release store (scripts/harness_dist/release_store.py owns publication/resolution/GC; the engine never materializes bundle bytes); migrate or reject returns across self-upgrade epochs
  • Persist Reminder lifecycle, evidence receipts, closure obligations, and invalidation at invocation, subtree, and root scopes
  • Implement durable task launch/lookup/cancel journaling, provider-handle reattachment, and duplicate-execution refusal across every dispatch crash boundary
  • Persist runnable-turn and ReminderLease timers plus fix-interference shelve/blocker/return state and ClientTransportDirective/receipt digests; recover overdue refresh, client yield, disconnect, transport projection drift, and crash without user-authored continuation or reconstruction from client state
  • Build fault-injection, concurrency, and arbitrary-depth property suites before enabling shadow or cutover

Intel dossier — pointer and tier disposition

Dossier: root-closure-durability-recovery--s-7aad8aa5.intel.md (this section’s content/intel/ sidecar)

Read that dossier from line 1 through EOF before acting on this section. This block is a pointer and an audit record, never a substitute: not the BLUF, not a summary, not selected tiers.

Dossier tierTitleLinesDispositionWhere / why
00. Bottom line up front133pending
11. The target, verbatim48pending
22. Terrain — symbols and files93pending
33. Code-graph recon152pending
3V3V. Vacuous-pass audit — the assigned defect-class sweep40pending
3D3D. Diagnostic / observability surface30pending
3H3H. Hygiene constraints the downstream work inherits35pending
44. Cluster / family — where this section sits34pending
55. Conformance audit — graph SUBSTRATE-DARK, source substitute supplied54pending
66. Plan ownership39pending
77. Prior art — interrogated, not emulated56pending
88. Sentiment19pending
99. Declared coverage gaps — what is dark, and why41pending
1010. Recon entry points — the ordered read path39pending