engine-spec

Complete engine specification (user directive 2026-07-19)

Goal

• A full, self-contained, prescriptive specification of the workflow engine, in spec-clause format, that any consumer (especially /migrate-skill-to-engine and weaker models) can point at WITHOUT relying on this plan’s body, dossiers, or history for knowledge.
• Home: scripts/workflow_engine/spec/ (versioned beside the implementation); .claude/rules/workflow-engine.md gains the discovery pointer.

Implementation sketch

• Author numbered clause files under scripts/workflow_engine/spec/: (1) architecture + invariants (scripts drive / LLM codes, one SSOT per fact, closed vocabularies, typed refusals, read-only registries); (2) execution model (RootClosure, InvocationFrame, push/return/fold protocol, exactly-once receipts, and the closed domain_artifact | virtual_root | protocol_root authority set); (3) durability + recovery (journal-first writes, claim markers, epoch fences, reconcile_resume heal legs, crash-window catalog); (4) workflow definition format (workflows/*.json schema, states, comparators, gates, actions, banners, capability requests) incorporating spec/extension-contract.md by reference as the extension clause (never duplicated); (5) components + registration (LogicGate/Action/Banner folders, ComponentMetadata, vocabulary manifest); (6) context assembly + reminders + reference capabilities (selectors, all control catalogs, dynamic read-only lookup contracts, mapping rows, NOW-NEXT-BANNED, activation lifecycle, evidence requirements); (7) client protocol (ContinueRuntimeAPI read contract, adapters + capability floors, transports, lifecycle reconciliation, TerminalReceipt matching-only termination); (8) capability + authorization model (derive_capability_id, child capabilities, scopes, refusals); (9) batches + queues (virtual roots, membership modes, dispositions, selectors); (10) closure services + obligations (service contracts, finalizers, discharge, publication lifecycle); (11) provider dispatch + tasks (launch identity, journaling, transport digests, leases, timers); (12) observability plus bounded liveness/performance (shadow hook, trajectory stream, parity/eval surfaces, resource limits, comparator thresholds, fairness and recovery workloads from spec sec 11.4); (13) migration clause: EXACTLY how every public skill/command consumes one of the three target shapes (stateful workflow, one-shot service, context/reference), using the exhaustive spec/migration-targets.md inventory and requiring public-entrypoint cutover, scenario/performance proof, receipt, and old-owner retirement.
• Every clause prescriptive-only (skill-vocabulary.md); each clause cites the owning module/schema as its machine anchor; no history/rationale prose.
• A machine conformance check ties spec to code: every shipped workflow id, component kind, catalog, state authority, capability scope kind, and refusal type appears in the spec’s coverage index; drift refuses.

Test strategy / acceptance

• Conformance pytest: spec coverage index vs live registries (workflow ids, component registries, vocabulary manifest, every context catalog, adapters, state authorities including protocol_root, service/result contracts, resource bounds, public-surface inventory, refusal types) — missing/unknown entries fail; clause files parse (numbered, ASCII, prescriptive-lint clean).
• Deterministic performance/fault acceptance from spec sec 11.4 is a completion gate; every validated regression or required comparator miss is Critical.
• verify_extension untouched and green; .claude/rules/workflow-engine.md pointer updated.

Work Items

• Author scripts/workflow_engine/spec/ clause files 1-13, each prescriptive-only, ASCII, numbered, and citing its owning module/schema as machine anchor (no history/rationale prose): clauses 1-11 retain their existing owners while clause 2 includes the paired protocol_root reopen and clause 6 covers all context/reference catalogs and read-only lookup services; clause 12 owns observability plus spec sec 11.4’s explicit bounds, fairness, crash workloads, and performance comparator thresholds; clause 13 consumes the exhaustive migration-target inventory and prescribes the three target shapes, public-entrypoint cutover, scenario/performance evidence, receipt, and old-owner retirement rather than requiring a bespoke workflow JSON for every surface.
• Author the spec-to-code conformance check (scripts/workflow_engine/spec_conformance.py + pytest) as a pure read-only aggregator over the existing closed-registry accessor layer — catalog_binding.closed_registries()/ClosedRegistries, registry.discover_components(), extension_registry_checks.capability_scopes()/return_contract_ids()/_manifest_rows()/_shipped_workflow_ids()/check_vocabulary_coverage/check_new_kind_declarations/check_reminder_mappings, registry.VOCABULARY_KINDS/REGISTRABLE_KINDS/NON_REGISTRABLE_KINDS/COMPONENT_FOLDERS, contract_validation.STATE_AUTHORITIES — importing each closed accessor rather than re-enumerating it (never re-glob workflows/, never re-list a kind set, never re-hardcode a scope set — the exact LEAK:algorithmic-duplication this section’s own conformance mandate exists to prevent). Coverage index over: workflow ids (E1), component kinds (E2), component registries (E3), vocabulary manifest rows (E5), reminder catalogs (E6), engine-resolved reminder facts (E7), client adapters/capability floors (E8), state authorities (E9), capability scope kinds (E10), refusal/typed-error classes (E11), and service/result contracts (E12); missing/unknown entries in any axis fail typed. For the E11 refusal-type axis specifically — where gateway_rejections.GatewayRejection is the ONLY closed refusal enum while the remaining *Error/Rejection exception classes are scattered across scripts/workflow_engine/.py with no single enumeration — explicitly decide and implement ONE of: (a) add a closed refusal-type registry satisfying the engine’s own typed-refusal invariant (clause 1/8), or (b) scope refusal coverage to GatewayRejection plus a declared, pytest-pinned snapshot of the scattered exception classes; never an un-pinned grep. scripts/workflow_engine/verify_extension.py stays untouched and green — the new check is additive-only, sharing the same read-only registry accessors, never editing verify_extension or its extension_registry_checks composition. Ship scripts/workflow_engine/tests/test_spec_conformance.py exercising the missing/unknown-entry-fails matrix across all twelve axes.
• Wire discovery: .claude/rules/workflow-engine.md points at scripts/workflow_engine/spec/ as the engine-spec SSOT; migration-targets.md and extension-contract.md cross-reference the owning clauses; prose-lint clean over every authored clause

Intel dossier — pointer and tier disposition

Dossier: engine-spec--s-522dcca0.intel.md (this section’s content/intel/ sidecar)

Read that dossier from line 1 through EOF before acting on this section. This block is a pointer and an audit record, never a substitute: not the BLUF, not a summary, not selected tiers.