docs: add openspec proposals for memory v2 and orchestrator flow patterns
This commit is contained in:
70
openspec/changes/orchestrator-flow-advanced/design.md
Normal file
70
openspec/changes/orchestrator-flow-advanced/design.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Orchestrator Advanced Flows — Design
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌───────────── Step dispatch ─────────────────┐
|
||||
│ │
|
||||
│ Flow-runner resolves step: │
|
||||
│ 1. Check trigger_rule on deps │
|
||||
│ 2. Substitute $vars in prompt │
|
||||
│ 3. If approval gate → pause for user │
|
||||
│ 4. INSERT task row → dispatcher picks up │
|
||||
│ 5. On terminal: append to event log │
|
||||
│ 6. Advance next ready step │
|
||||
│ │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Type Changes
|
||||
|
||||
### `apps/coder/src/conductor/types.ts`
|
||||
|
||||
```typescript
|
||||
export type TriggerRule = 'all_success' | 'one_success' | 'all_done';
|
||||
|
||||
export interface Step {
|
||||
id: string;
|
||||
kind: StepKind | 'approval'; // + new kind
|
||||
deps?: string[];
|
||||
trigger_rule?: TriggerRule; // NEW: default 'all_success'
|
||||
agent?: string;
|
||||
run: (ctx: StepContext) => string | Promise<string>;
|
||||
when?: (ctx: StepContext) => boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### `apps/coder/src/services/flow-runner.ts`
|
||||
|
||||
| Change | Detail |
|
||||
|--------|--------|
|
||||
| Trigger evaluation | Before dispatching a step, check deps statuses against `trigger_rule`. Skip if conditions not met |
|
||||
| Variable substitution | Scan prompt for `$word.word` patterns, resolve from previous step outputs |
|
||||
| Approval gate | When `step.kind === 'approval'`, insert a `tasks` row with `state='blocked'` and publish a `permission_requested` WS frame. Wait for `permission_resolved` to unblock |
|
||||
| Event log | Append-only per-step events: `{ step_id, event: 'started'|'completed'|'failed'|'paused'|'resumed', at: timestamp }` in `flow_step_events` table |
|
||||
|
||||
## Schema
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS flow_step_events (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
run_id UUID NOT NULL REFERENCES flow_runs(id),
|
||||
step_id VARCHAR(64) NOT NULL,
|
||||
event VARCHAR(32) NOT NULL, -- started, completed, failed, paused, resumed, skipped
|
||||
payload JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
|
||||
);
|
||||
```
|
||||
|
||||
## Resolution Order
|
||||
|
||||
1. Collect all completed steps in the run
|
||||
2. For each unstarted step whose deps are met:
|
||||
- Evaluate `trigger_rule` against dep statuses
|
||||
- If met → advance the step (with variable substitution)
|
||||
- If not met → skip (for `one_success`, mark it complete when any dep succeeds)
|
||||
3. For approval gates: pause, publish frame, wait for user response
|
||||
|
||||
## Rollback
|
||||
|
||||
All changes are additive to the Step type. Existing flows without `trigger_rule` default to `all_success`, preserving current behavior. Approval gates are opt-in per step definition.
|
||||
42
openspec/changes/orchestrator-flow-advanced/proposal.md
Normal file
42
openspec/changes/orchestrator-flow-advanced/proposal.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# Orchestrator — Advanced Flow Patterns
|
||||
|
||||
**Status:** Proposed
|
||||
**Epic:** orchestrator-flow-advanced
|
||||
**Depends on:** v2.7.17-orchestrator (flow-runner already shipped)
|
||||
|
||||
## Why
|
||||
|
||||
The orchestrator (shipped v2.7.17) runs sequential research/analysis flows on local Qwen. Each step is a linear dependency chain: A → B → C. This works for analysis flows (code-review, investigate) but limits three high-value scenarios:
|
||||
|
||||
1. **Parallel research** — "Analyze this from 3 angles" currently requires 3 separate runs. A single flow with parallel branches would halve the wall-clock time.
|
||||
2. **Adaptive depth** — "Investigate bug, and if it's a security issue, escalate to security review" requires conditional branching. Currently all steps run unconditionally.
|
||||
3. **Human-in-the-loop** — "Review the diff and approve before applying" requires the orchestrator to pause and wait for user input before proceeding.
|
||||
|
||||
The patterns from the Ion hybrid workflow engine (trigger rules, event sourcing, approval gates, variable substitution) provide a proven vocabulary for these scenarios — we adapt the patterns without adopting the project.
|
||||
|
||||
## What Changes
|
||||
|
||||
### Trigger rules on step deps
|
||||
Add `trigger_rule?: 'all_success' | 'one_success' | 'all_done'` to the `Step` type. Default `all_success` preserves existing behavior.
|
||||
|
||||
- `all_success` — step runs when ALL dependencies complete successfully (current behavior)
|
||||
- `one_success` — step runs when ANY dependency completes (parallel research: whichever finishes first seeds the synthesis)
|
||||
- `all_done` — step runs when all deps finish regardless of status (cleanup/reporting steps)
|
||||
|
||||
### Variable substitution in step prompts
|
||||
Add `$stepId.output` and `$stepId.output.field` syntax in step prompts. The flow-runner resolves these before dispatching.
|
||||
|
||||
- `$research.output` — the full text output of step with id "research"
|
||||
- `$classify.output.severity` — the "severity" field from step output parsed as YAML/JSON frontmatter
|
||||
|
||||
### Human approval gate
|
||||
New `kind: 'approval'` step type that pauses the flow and publishes a permission frame to the user channel. Flow resumes when the user approves or rejects.
|
||||
|
||||
### Event-sourced step log
|
||||
Append-only event log for each step execution (start, complete, fail, skip, pause, resume). Enables deterministic resume after coder restart without polling.
|
||||
|
||||
## Non-Goals
|
||||
- No YAML DAG format (stay with TypeScript flow definitions)
|
||||
- No CLI tool (orchestrator stays in-app)
|
||||
- No replacement of the existing flow definitions — additive changes only
|
||||
- No VM sandbox or WASM
|
||||
40
openspec/changes/orchestrator-flow-advanced/tasks.md
Normal file
40
openspec/changes/orchestrator-flow-advanced/tasks.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Tasks — Orchestrator Advanced Flows
|
||||
|
||||
## Prerequisites
|
||||
- v2.7.17 on main (orchestrator + flow-runner shipped)
|
||||
- v2.8.0 on main (fork-lifts complete)
|
||||
|
||||
## Tasks
|
||||
|
||||
### 1. Trigger rules in Step type
|
||||
- [ ] 1.1 Add `TriggerRule` type to `conductor/types.ts`
|
||||
- [ ] 1.2 Add `trigger_rule?: TriggerRule` field to `Step` interface (defaults `all_success`)
|
||||
- [ ] 1.3 Write `evaluateTriggerRule(deps, rule): boolean` in `flow-runner-decisions.ts`
|
||||
- [ ] 1.4 Unit tests for each rule variant
|
||||
|
||||
### 2. Variable substitution
|
||||
- [ ] 2.1 Write `resolveVariables(prompt, completedSteps): string` in flow-runner
|
||||
- [ ] 2.2 Supports `$stepId.output` and `$stepId.output.field` (dot-path)
|
||||
- [ ] 2.3 Unit tests with multi-step outputs
|
||||
|
||||
### 3. Approval gate step kind
|
||||
- [ ] 3.1 Add `'approval'` to `StepKind` union
|
||||
- [ ] 3.2 Flow-runner: when step.kind === 'approval', pause and publish `permission_requested` frame
|
||||
- [ ] 3.3 Wire `permission_resolved` frame handler to unblock blocked step
|
||||
- [ ] 3.4 Test: approval gate pauses flow, approval resumes it
|
||||
|
||||
### 4. Event-sourced step log
|
||||
- [ ] 4.1 Create `flow_step_events` table in `apps/coder/src/schema.sql`
|
||||
- [ ] 4.2 Write `appendStepEvent(runId, stepId, event, payload?)` helper
|
||||
- [ ] 4.3 Wire events into flow-runner lifecycle hooks (start, complete, fail, skip, pause, resume)
|
||||
- [ ] 4.4 Unit test: events are recorded in order
|
||||
|
||||
### 5. Example flow with parallel branches
|
||||
- [ ] 5.1 Create `conductor/flows/parallel-research.ts` — splits into 3 parallel research steps, then joins with synthesis
|
||||
- [ ] 5.2 Uses `trigger_rule: 'one_success'` on the synthesis step
|
||||
- [ ] 5.3 Integration test: parallel flow completes correctly
|
||||
|
||||
### 6. Smoke
|
||||
- [ ] 6.1 Run parallel-research flow with 3 agents
|
||||
- [ ] 6.2 Verify synthesis step triggers on first completion
|
||||
- [ ] 6.3 Verify variable substitution in synthesis prompt
|
||||
Reference in New Issue
Block a user