89%

Virtual batch roots + queue execution

Goal

  • Support root targets without plan/bug artifacts.
  • Top-level batching creates a durable virtual RootClosure and closes once over member union.
  • Nested batching creates a descendant batch frame under the existing root.
  • Make this the sole state_authority=virtual_root workflow family; plan/bug roots can never select it as their state home.

Implementation sketch

  • Define RootTarget as domain artifact or VirtualBatchDescriptor with command, selector policy, membership mode, order, filters, mode, and provenance.
  • Declare state_authority=virtual_root only when RootTarget is a validated VirtualBatchDescriptor with no domain artifact. Reject plan/tracker mutation Actions in the batch workflow; member child workflows retain their own domain-artifact authority.
  • Persist admitted-child history, typed dispositions, queue revision receipts, workset/evidence union, and freeze generation.
  • Atomically select, claim, issue child capability, and capture queue revision under the queue writer lock; then suspend, fold return, and reconcile.
  • Treat fixed/OBE as accepted. In snapshot/live autopilot, dependency blockers become required descendants and retry the original member without changing snapshot membership. In one-item mode, persist blocked continuation and pause/return without auto-fixing. Route persistent ownership explicitly; pause local-gate and unresolved-required outcomes.
  • Preserve /fix-next-bug live-rescan semantics by default; compare-and-swap queue revision during frozen exhaustion checks.
  • Implement snapshot mode with an entry-frozen ordered member set and ignored later admissions. Autopilot exhausts the set; one-item deterministically selects one primary from it.
  • Derive completion from outer mode: autopilot requires snapshot exhaustion/live quiescence; top-level one-item closes after one accepted primary even when other members remain, while blocked pauses. Nested one-item returns accepted/blocked without closing the enclosing root.
  • Use request IDs for crash-idempotent virtual-root creation. Empty top-level roots emit one queue-empty terminal receipt; empty nested batches return typed empty without root effects.
  • Distinguish ephemeral virtual execution batches from persistent umbrella plans that own one shared cure.
  • Route every selector consumer through one canonical queue component; retire parallel fix-next-bug selectors and the historical cluster-promotion path.

Test strategy / acceptance

Exercise top-level/nested empty, one/thirty, snapshot/live membership, closed state-authority validation, plan-write rejection, concurrent claim, dynamic admission, filters/orders, every typed disposition, blocker-child/retry, one-item blocked pause/return, root-creation crashes, duplicate returns, queue-revision changes, local acceptance failure, foreign mutation, and one root closeout/report.

Work Items

  • Define VirtualBatchDescriptor and domain-artifact/virtual-batch RootTarget union; bind state_authority=virtual_root exclusively to a descriptor with no plan/bug artifact and reject plan/tracker mutation Actions
  • Implement generic reconcile/atomic-claim/invoke/fold batch loop with request-idempotent root entry, durable membership policy, typed dispositions, admitted-child history, queue revisions, and freeze generation
  • Migrate fix-next-bug’s REAL selection/orchestration logic — scripts/plan_orchestrator/bug_queue_scan.py (filters, section/severity ordering, lifecycle exclusions, live rescan) and scripts/plan_orchestrator/fix_next_bug.py’s independent-child dispatch loop, plus the smaller scripts/fix_next_bug_runtime/queue.py:select_next_bug stub — into ONE canonical queue/batch-member selector component owned by this section’s batch loop, per decisions/04-virtual-batch-supersession.md’s Superseded Paths table. This canonical queue selector is DISTINCT from workflow_engine.selectors.select (the ContextAssembly content-category selector for instructions/knowledge/memory/examples/tools/guardrails/reminders — a different responsibility); do not merge into that module or treat it as the migration target.
  • Fold accepted child deltas into the virtual root; run aggregate gates per frozen generation and global finalization/publication/status once
  • Generalize the batch node for future queue commands without adding workflow-specific orchestration code
  • Preserve outer batch mode across every select/consume/resume envelope so interactive one-item execution cannot silently become autopilot after the first child
  • Implement the full membership-mode x outer-mode x empty/blocked/multi-member matrix plus request replay, concurrent claim, queue-revision CAS, eligibility loss, and nested return paths
  • Distinguish virtual execution batches from explicit persistent umbrella ownership; keep graph relatedness advisory and supersede old cluster-promotion/parallel-selector paths

Fresh intel (regenerated)

This is NOT a greenfield build. The graph shows the virtual-root state machine already STANDS: state_authority=virtual_root is a live enum (contract_validation.py:77), the engine loader already rejects plan/tracker mutation actions in a virtual-root workflow (engine.py:284), create_root already serializes virtual-root admission per-request vs per-target (execution_store.py:185), and the canonical selector engine (selectors.select) is already wired into context_snapshot.build_context_snapshot. The section’s 8 work items are the LAST-MILE convergence: name VirtualBatchDescriptor/RootTarget as first-class types, generalize the batch loop, and — the load-bearing move — MIGRATE the legacy fix_next_bug_runtime.queue.select_next_bug into selectors.select and DELETE the parallel path.

The single most important [JOIN]: the legacy selector’s retirement blast radius is tiny. select_next_bug has ONE non-test production caller (queue_empty, same file) and 6 in-file tests — no cross-module consumer threads it (callers partial-but-consistent). Meanwhile selectors.select already has a production consumer (context_snapshot). So “route every selector consumer through one canonical queue component” is a small, contained migration, NOT a sprawling rewire — the risk is a LEAK:algorithmic-duplication left behind (two selectors coexisting), not a broken call graph.

Decisions this package forces before any code is written:

  1. The migration target is selectors.select + SelectorRule — do NOT author a third selector. The whole point of w-f125f992 + w-ff46589d is COLLAPSE, so the deliverable is select_next_bug DELETED, its 6 tests re-homed against selectors.select.
  2. Bind state_authority=virtual_root by EXTENDING the existing closed enum (STATE_AUTHORITIES) + the existing virtual_root rejection in engine.py/contract_validation.py — never a parallel authority path. The plan-write-rejection + shadow-state guards are ALREADY pinned by test_root_state_authority.py; extend those pins, do not re-implement the guard.
  3. Route virtual-root creation through the existing admit_and_publish_root admission SSOT (root_admission.py:66) and the existing per-request lock (execution_store.py:185) — the crash-idempotent request-ID path already exists; w-2da64786’s fold + w-010c902f’s concurrent-claim/queue-revision-CAS build ON it, they do not fork it.
  4. The queue-revision CAS surface is the existing RevisionSet typed fence (closeout_records.py:30 + closeout.py:_freeze_closeout) — the “compare-and-swap queue revision during frozen exhaustion” is a freeze against an identical-vs-different queue_revision. Bind to it.
  5. Generic-batch-node constraint (w-f2044f18) sits directly under script-first.md §11 (STRUCTURE:plan-specific-tooling-leak) — the batch node MUST NOT hard-code fix-next-bug/tooling-cli; it is a data-declared workflow.
["derive_capability_id", "admit_and_publish_root", "SHADOW_DOMAIN_STATE_KEYS"]

DIG DEEPER

sed -n '1,120p' plans/workflow-engine-rework/content/virtual-batch-roots-queue-execution--s-89b0c8d3.md
sed -n '70,100p' scripts/workflow_engine/contract_validation.py   # STATE_AUTHORITIES + SHADOW_DOMAIN_STATE_KEYS

(full dossier: virtual-batch-roots-queue-execution—s-89b0c8d3.intel.md)