Compare commits
4 Commits
v1.13
...
v1.16.0-co
| Author | SHA1 | Date | |
|---|---|---|---|
| 29c7d051b6 | |||
| d27a977d59 | |||
| 5692e99a5d | |||
| f4a97808ad |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -10,3 +10,4 @@ secrets/
|
||||
data/*
|
||||
!data/AGENTS.md
|
||||
!data/skills/
|
||||
!data/mcp.json
|
||||
|
||||
16
CHANGELOG.md
16
CHANGELOG.md
@@ -2,6 +2,22 @@
|
||||
|
||||
All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch.
|
||||
|
||||
## v1.16.0-codesight-merge — 2026-05-24
|
||||
|
||||
Ports codesight's highest-value analysis capabilities into the codecontext sidecar as 4 new MCP tools. Tier 1 (graph queries on existing edges, no re-parsing): `get_blast_radius` (BFS reverse-edge traversal — "what breaks if I change this file?", with depth tracking) and `get_hot_files` (most-imported files ranked by incoming edge count — change-risk indicators). Tier 2 (tree-sitter AST re-parsing on demand): `get_routes` (Fastify/Express HTTP route extraction with method, path, file, line, inferred tags for db/auth/cache) and `get_middleware` (middleware registration detection via import-name heuristics and app.register/addHook/setErrorHandler patterns, classifying as auth/cors/rate-limit/security/error-handler/logging/validation). All 4 tools use `defer s.graphMu.RUnlock()` for consistent mutex discipline (reviewer caught that the initial implementation released the lock early on the Tier 2 tools). Route object-property extraction delegates to `extractStringValue` for template-literal handling (reviewer catch). codecontext sidecar rebuilt from `/opt/forks/codecontext` commit `b19e646`, tagged `v1.16.0-codesight-merge`. BooCode wrapper tools follow the existing codecontext pattern — 4 new files in `apps/server/src/services/tools/codecontext/`, registered in ALL_TOOLS. 29 new Go tests + 363/363 BooCode server tests passing. No schema changes, no frontend changes.
|
||||
|
||||
## v1.15.0-mcp-multi — 2026-05-24
|
||||
|
||||
Multi-server MCP client with stdio + Streamable HTTP transports, JSON config file, and per-agent tool glob patterns. Generalizes the v1.14.1 single-server Context7 PoC into a registry of named MCP servers with per-server graceful degradation. JSON config at `/data/mcp.json` (bind-mounted alongside `AGENTS.md`) matches opencode's `mcpServers` schema shape so server entries are copy-pasteable. Config file missing = no MCP (opt-in by file presence). Stdio transport spawns a persistent subprocess via the SDK's `StdioClientTransport` with NDJSON framing; Streamable HTTP reuses the v1.14.1 pattern via `StreamableHTTPClientTransport`. Tool prefix generalized from `context7_<name>` to `<serverName>_<toolName>` with a reverse `toolToServer` map for dispatch routing. Per-agent AGENTS.md `tools:` field now supports glob patterns (`context7_*`, `!web_*`) via `matchToolGlob` (last-match-wins, `!` prefix denies); replaces the exact-match `.includes()` in `stream-phase.ts`. Glob patterns bypass `ALL_TOOL_NAMES` validation in the parser since MCP tool names aren't known at parse time. `refreshToolNames()` in `agents.ts` rebuilds the `DEFAULT_TOOLS` snapshot after `appendMcpTools` so agents without explicit `tools:` lists see MCP tools — reviewer caught that the module-load-time snapshot would permanently exclude late-registered tools. Read-only invariant preserved: all MCP tools with `readOnlyHint: false` rejected at discovery. Result size capped at 5MB. Shutdown hook closes all transports. v1.14.1 env vars (`MCP_CONTEXT7_URL`, `MCP_CONTEXT7_API_KEY`) removed — superseded by the config file. Default `data/mcp.json` ships with Context7 disabled; flip `"enabled": true` to activate. 363/363 server tests passing (27 new: multi-server wrapping, glob matching, routing, degradation). No schema changes, no frontend changes.
|
||||
|
||||
## v1.14.1-mcp-poc — 2026-05-23
|
||||
|
||||
Single-server MCP client PoC against Context7. New `apps/server/src/services/mcp-client.ts` (~200 lines) wraps `@modelcontextprotocol/sdk` v1.29.0 with Streamable HTTP transport. On startup (when `MCP_CONTEXT7_URL` is set), connects to Context7, discovers tools via `tools/list`, wraps each as a `ToolDef` prefixed `context7_<name>`, and appends to `ALL_TOOLS` (alpha-sorted for prompt-cache stability). `appendMcpTools()` in `tools.ts` handles the late-registration; `ALL_TOOLS` changed from `ReadonlyArray` to mutable to support it. Read-only invariant guard rejects any MCP tool with `readOnlyHint: false` (MCP SDK v1.29.0 uses `readOnlyHint`, not `readOnly`). Tool dispatch is transparent — `executeToolCall` routes MCP tool calls through the `ToolDef.execute` wrapper, which strips the `context7_` prefix before calling the MCP server. Graceful degradation: MCP server down at startup → zero tools, warn log; MCP server down mid-session → error-shaped result, model self-corrects. Result size capped at 5MB with truncation (matches native `view_file`'s `MAX_FILE_BYTES`). Adversarial review caught that the Zod `.default('https://...')` on the URL config made MCP effectively always-on instead of opt-in — fixed by removing the default. 348/348 server tests passing (16 new mcp-client tests covering tool wrapping, read-only guard, name prefixing, content extraction). No schema changes, no frontend changes. Proves the MCP tool-discovery → tool-call → result-render loop end-to-end before the full v1.15 port.
|
||||
|
||||
## v1.14.0-outer-loop — 2026-05-23
|
||||
|
||||
Converts the inference engine's ad-hoc `executeToolPhase → runAssistantTurn` recursion into an explicit `while` loop with a configurable step cap. A step is one stream-and-tool-execute iteration; the loop terminates on non-tool finish, step-cap hit, doom-loop, budget exhaustion, abort, or synthesis success. `MAX_STEPS = 200` is the hard ceiling (4x the old effective limit from budget); per-agent `steps:` field in AGENTS.md frontmatter sets tighter caps (Refactorer: 5, Architect: 20, others: unset = bounded only by MAX_STEPS). `executeToolPhase` no longer recurses — returns a `ToolPhaseResult` struct (`action: 'continue' | 'paused' | 'synthesis_done'`) so the caller (the while loop) decides whether to continue or break. `steps: 0` is handled as "no tool calls allowed" — one text-only stream phase, tool calls ignored with a warn log. Step-cap hits produce a sentinel summary (reuses `cap_hit` kind so `CapHitSentinel.tsx` renders it without frontend changes; text distinguishes "Step limit reached" from "Tool budget exhausted"). Doom-loop check migrated from pre-recursion position to top of loop body — same predicate (`detectDoomLoop`), same threshold (3 identical calls), `break` instead of `return`. `step_start` parts are in the schema CHECK but not emitted as message_parts in v1.14 — writing to the assistant message before the stream phase creates a sequence-0 collision with `partsFromAssistantMessage`; a structured log line is emitted instead. Adversarial review caught the collision pre-deploy. 332/332 server tests passing; no frontend changes. Pairs with `v1.13.20-drop-legacy-cols` (parts is now the sole source of truth, and this batch's loop operates entirely through parts).
|
||||
|
||||
## v1.13.20-drop-legacy-cols — 2026-05-23
|
||||
|
||||
Final phase of the v1.13.0 strangler-fig migration. Removes the dual-write into `messages.tool_calls` / `messages.tool_results` JSON columns and drops the columns themselves; `message_parts` is now the only source of truth for tool-call and tool-result data. 10 dual-write sites stripped (5 in `tool-phase.ts`, 2 in `routes/skills.ts`, 2 in `routes/messages.ts`, 1 in `routes/chats.ts` fork-clone) — recon's grep-driven inventory caught 2 sites beyond the original v1.13.2 roadmap count. `messages_with_parts` view simplified to parts-only subselects (COALESCE fallbacks gone) and rewritten via `CREATE OR REPLACE VIEW` BEFORE the column DROP since Postgres rejects column-drop on view-referenced cols. Adversarial review caught a runtime bug the green test suite missed: `chats.ts:/api/chats/:id/discard_stale` had a `RETURNING ... tool_calls, tool_results, ...` clause referencing the dropped columns; would have crashed on every 60s-no-token-activity recovery in production. Fixed by switching to two-step UPDATE-then-SELECT-from-view so the response keeps the parts-synthesized fields. `Message` API type retains `tool_calls?` / `tool_results?` fields (override on the original v1.13.2 plan) — the view continues to populate them from parts, so the wire shape is unchanged and the frontend needs no updates. v1.12.1 cleanup block (`DROP CONSTRAINT messages_status_check`/`messages_role_check`) removed — those one-shots have done their work. `tool_cost_stats.test.ts` had a direct `INSERT INTO messages` touching the legacy columns that wasn't in the roadmap's inventory; rewritten to parts-table inserts and confirmed semantically faithful. 339/339 server tests passing including the 7 DB-integration tests (live-DB applied the schema migration and ran the parts-only view end-to-end). Pairs with `v1.13.0-ai-sdk-v6` (which introduced the dual-write) and `v1.13.1-B` (which moved the read path to `messages_with_parts`); umbrella `v1.13` tag ships on the same commit.
|
||||
|
||||
@@ -46,7 +46,7 @@ Tests: `pnpm -C apps/server test` runs the vitest suite. No test harness on `app
|
||||
- **Zod** for request validation and config parsing.
|
||||
|
||||
Key services:
|
||||
- **`services/inference/`** — Public surface re-exported via `inference/index.ts`; callers import from `./services/inference/index.js` explicitly (NodeNext doesn't honor directory-index resolution). Layout: `turn.ts` (runAssistantTurn / runInference / createInferenceRunner; exports `InferenceFrame`, `InferenceContext`, `TurnArgs`, `StreamResult`), `stream-phase.ts` (streamCompletion as a v1.13.1-A AI SDK adapter + executeStreamPhase), `provider.ts` (`upstreamModel(baseURL, modelId)` wrapping `createOpenAICompatible` against llama-swap), `tool-phase.ts` (executeToolPhase; value back-edges into turn.ts for the runAssistantTurn recursion — cycle safe because deref at call time, not module top-level), `sentinel-summaries.ts` (runCapHitSummary + runDoomLoopSummary + their sentinel inserters), `error-handler.ts` (handleAbortOrError, finalizeCompletion), `payload.ts` (buildMessagesPayload, loadContext, maybeFlagForCompaction, `OpenAiMessage`), `sentinels.ts` (`detectDoomLoop`, `DOOM_LOOP_THRESHOLD`, sentinel predicates), `budget.ts` (resolveToolBudget), `xml-parser.ts` (qwen3.6 XML tool-call fallback — KEEP, AI SDK doesn't handle inline-XML tool calls), `parts.ts` (parts-table write helpers: `partsFromAssistantMessage`, `partsFromToolMessage`, `insertParts` — v1.13.20 made parts the sole source of truth), `prune.ts` (v1.13.4 two-tier compaction; `selectPruneTargets` is the pure decision helper), `types.ts` (`StreamPhaseState`, `DB_FLUSH_INTERVAL_MS`). **`TurnArgs`** is the per-turn state envelope threaded through the `executeToolPhase → runAssistantTurn` recursion; reset in `runInference` at user-message boundary. Add new per-turn state to `TurnArgs`, not module-level closures.
|
||||
- **`services/inference/`** — Public surface re-exported via `inference/index.ts`; callers import from `./services/inference/index.js` explicitly (NodeNext doesn't honor directory-index resolution). Layout: `turn.ts` (runAssistantTurn / runInference / createInferenceRunner; exports `InferenceFrame`, `InferenceContext`, `TurnArgs`, `StreamResult`, `MAX_STEPS`), `stream-phase.ts` (streamCompletion as a v1.13.1-A AI SDK adapter + executeStreamPhase), `provider.ts` (`upstreamModel(baseURL, modelId)` wrapping `createOpenAICompatible` against llama-swap), `tool-phase.ts` (executeToolPhase → returns `ToolPhaseResult`; no longer recurses into runAssistantTurn — v1.14.0 converted the recursion to an explicit while loop in turn.ts), `sentinel-summaries.ts` (runCapHitSummary + runDoomLoopSummary + runStepCapSummary + their sentinel inserters), `error-handler.ts` (handleAbortOrError, finalizeCompletion), `payload.ts` (buildMessagesPayload, loadContext, maybeFlagForCompaction, `OpenAiMessage`), `sentinels.ts` (`detectDoomLoop`, `DOOM_LOOP_THRESHOLD`, sentinel predicates), `budget.ts` (resolveToolBudget), `xml-parser.ts` (qwen3.6 XML tool-call fallback — KEEP, AI SDK doesn't handle inline-XML tool calls), `parts.ts` (parts-table write helpers: `partsFromAssistantMessage`, `partsFromToolMessage`, `insertParts` — v1.13.20 made parts the sole source of truth), `prune.ts` (v1.13.4 two-tier compaction; `selectPruneTargets` is the pure decision helper), `types.ts` (`StreamPhaseState`, `DB_FLUSH_INTERVAL_MS`). **`TurnArgs`** is the per-turn state envelope populated from loop locals each iteration; reset in `runInference` at user-message boundary. The outer loop in `runAssistantTurn` (v1.14.0) runs `while (stepNumber < effectiveCap)` where `effectiveCap = Math.min(agent.steps ?? Infinity, MAX_STEPS=200)`. Per-agent `steps:` field in AGENTS.md frontmatter. `steps: 0` means text-only (no tool execution). Step-cap hit writes a `cap_hit` sentinel so `CapHitSentinel.tsx` renders it.
|
||||
- **AI SDK v6 streamCompletion adapter** (v1.13.1-A; `services/inference/stream-phase.ts`). `streamText` is the underlying call; the BooCode layer above (executeStreamPhase, finalize, dual-write) is shape-preserved via an adapter. Five gotchas the LSP/test suite won't catch:
|
||||
- **Abort signals are swallowed.** `streamText`'s `fullStream` iterator exits cleanly when `abortSignal` fires — no throw. Post-iteration `if (signal?.aborted) throw <AbortError>` is required; without it the row finalizes as `complete` instead of `cancelled`. Comment in stream-phase.ts pins this; don't refactor it away.
|
||||
- **Usage lands only at stream end** via `await result.usage` (`inputTokens` / `outputTokens` v6 names → mapped to `promptTokens` / `completionTokens` for the existing onUsage callback). Mid-stream live tok/s is gone vs v1.12.2; ChatThroughput shows a single value at stream end.
|
||||
|
||||
@@ -14,6 +14,7 @@
|
||||
"@ai-sdk/openai-compatible": "^2.0.47",
|
||||
"@fastify/static": "^7.0.4",
|
||||
"@fastify/websocket": "^10.0.1",
|
||||
"@modelcontextprotocol/sdk": "^1.29.0",
|
||||
"ai": "^6.0.190",
|
||||
"fastify": "^4.28.1",
|
||||
"postgres": "^3.4.4",
|
||||
|
||||
@@ -19,6 +19,9 @@ const ConfigSchema = z.object({
|
||||
GITEA_USER: z.string().default('indifferentketchup'),
|
||||
GITEA_TOKEN: z.string().optional(),
|
||||
GITEA_SSH_HOST: z.string().default('100.114.205.53:2222'),
|
||||
// v1.15.0-mcp-multi: path to the MCP config JSON file. Default /data/mcp.json
|
||||
// (bind-mounted alongside AGENTS.md). File missing = no MCP (opt-in).
|
||||
MCP_CONFIG_PATH: z.string().optional(),
|
||||
});
|
||||
|
||||
export type Config = z.infer<typeof ConfigSchema>;
|
||||
|
||||
@@ -24,6 +24,10 @@ import { listSkills } from './services/skills.js';
|
||||
import * as compaction from './services/compaction.js';
|
||||
import { configureModelContext } from './services/model-context.js';
|
||||
import { cleanupTruncations } from './services/truncate.js';
|
||||
import { loadMcpConfig } from './services/mcp-config.js';
|
||||
import { initialize as initMcp, getTools as getMcpTools, shutdown as shutdownMcp } from './services/mcp-client.js';
|
||||
import { appendMcpTools } from './services/tools.js';
|
||||
import { refreshToolNames } from './services/agents.js';
|
||||
|
||||
async function main() {
|
||||
const config = loadConfig();
|
||||
@@ -69,6 +73,23 @@ async function main() {
|
||||
// default_generation_settings.n_ctx — the value persisted as messages.ctx_max.
|
||||
configureModelContext({ llamaSwapUrl: config.LLAMA_SWAP_URL });
|
||||
|
||||
// v1.15.0-mcp-multi: read MCP config file and connect to all enabled servers.
|
||||
// Runs before route registration so the tool list is complete when the first
|
||||
// inference request arrives. Per-server graceful degradation: one failing
|
||||
// server doesn't block others.
|
||||
const mcpConfigPath = config.MCP_CONFIG_PATH ?? '/data/mcp.json';
|
||||
const mcpServers = loadMcpConfig(mcpConfigPath, app.log);
|
||||
if (mcpServers.length > 0) {
|
||||
await initMcp(mcpServers, app.log);
|
||||
const mcpTools = getMcpTools();
|
||||
if (mcpTools.length > 0) {
|
||||
appendMcpTools(mcpTools);
|
||||
refreshToolNames();
|
||||
app.log.info({ servers: mcpServers.length, tools: mcpTools.length }, 'mcp: registered');
|
||||
}
|
||||
}
|
||||
app.addHook('onClose', async () => { await shutdownMcp(); });
|
||||
|
||||
await app.register(fastifyWebsocket);
|
||||
|
||||
app.get('/api/health', async () => {
|
||||
|
||||
169
apps/server/src/services/__tests__/mcp-client.test.ts
Normal file
169
apps/server/src/services/__tests__/mcp-client.test.ts
Normal file
@@ -0,0 +1,169 @@
|
||||
/**
|
||||
* v1.15.0-mcp-multi: unit tests for the multi-server MCP client.
|
||||
* Pure unit tests — no live MCP server needed. Tests tool-wrapping,
|
||||
* read-only guard, name prefixing, content extraction, and error handling.
|
||||
* Multi-server routing tested via wrapMcpTool's server-name prefix.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { wrapMcpTool, extractContent, isToolReadOnly } from '../mcp-client.js';
|
||||
|
||||
describe('mcp-client', () => {
|
||||
describe('wrapMcpTool — multi-server prefixing', () => {
|
||||
it('produces a ToolDef with <serverName>_ prefix', () => {
|
||||
const mcpTool = {
|
||||
name: 'resolve-library-id',
|
||||
description: 'Resolve a library identifier',
|
||||
inputSchema: {
|
||||
type: 'object' as const,
|
||||
properties: { query: { type: 'string' } },
|
||||
required: ['query'],
|
||||
},
|
||||
};
|
||||
|
||||
const wrapped = wrapMcpTool('context7', mcpTool);
|
||||
|
||||
expect(wrapped.name).toBe('context7_resolve-library-id');
|
||||
expect(wrapped.description).toBe('Resolve a library identifier');
|
||||
expect(wrapped.jsonSchema.type).toBe('function');
|
||||
expect(wrapped.jsonSchema.function.name).toBe('context7_resolve-library-id');
|
||||
expect(wrapped.jsonSchema.function.parameters).toEqual(mcpTool.inputSchema);
|
||||
expect(typeof wrapped.execute).toBe('function');
|
||||
});
|
||||
|
||||
it('prefixes tools from different servers correctly', () => {
|
||||
const toolA = {
|
||||
name: 'query-docs',
|
||||
description: 'Query docs',
|
||||
inputSchema: { type: 'object' as const, properties: {} },
|
||||
};
|
||||
const toolB = {
|
||||
name: 'overview',
|
||||
description: 'Get overview',
|
||||
inputSchema: { type: 'object' as const, properties: {} },
|
||||
};
|
||||
|
||||
const wrappedA = wrapMcpTool('context7', toolA);
|
||||
const wrappedB = wrapMcpTool('codecontext', toolB);
|
||||
|
||||
expect(wrappedA.name).toBe('context7_query-docs');
|
||||
expect(wrappedB.name).toBe('codecontext_overview');
|
||||
});
|
||||
|
||||
it('multi-server: two servers with 2 tools each produce 4 prefixed tools', () => {
|
||||
const serverATools = [
|
||||
{ name: 'query-docs', inputSchema: { type: 'object' as const, properties: {} } },
|
||||
{ name: 'resolve-library-id', inputSchema: { type: 'object' as const, properties: {} } },
|
||||
];
|
||||
const serverBTools = [
|
||||
{ name: 'overview', inputSchema: { type: 'object' as const, properties: {} } },
|
||||
{ name: 'search', inputSchema: { type: 'object' as const, properties: {} } },
|
||||
];
|
||||
|
||||
const allWrapped = [
|
||||
...serverATools.map((t) => wrapMcpTool('context7', t)),
|
||||
...serverBTools.map((t) => wrapMcpTool('codecontext', t)),
|
||||
];
|
||||
|
||||
expect(allWrapped).toHaveLength(4);
|
||||
expect(allWrapped.map((t) => t.name)).toEqual([
|
||||
'context7_query-docs',
|
||||
'context7_resolve-library-id',
|
||||
'codecontext_overview',
|
||||
'codecontext_search',
|
||||
]);
|
||||
});
|
||||
|
||||
it('defaults description to empty string when absent', () => {
|
||||
const mcpTool = {
|
||||
name: 'no-desc',
|
||||
inputSchema: { type: 'object' as const, properties: {} },
|
||||
};
|
||||
|
||||
const wrapped = wrapMcpTool('myserver', mcpTool);
|
||||
|
||||
expect(wrapped.description).toBe('');
|
||||
expect(wrapped.jsonSchema.function.description).toBe('');
|
||||
});
|
||||
|
||||
it('uses passthrough Zod schema (z.record)', () => {
|
||||
const mcpTool = {
|
||||
name: 'test',
|
||||
inputSchema: { type: 'object' as const, properties: {} },
|
||||
};
|
||||
|
||||
const wrapped = wrapMcpTool('s', mcpTool);
|
||||
|
||||
const result = wrapped.inputSchema.safeParse({ foo: 'bar', baz: 123 });
|
||||
expect(result.success).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('isToolReadOnly', () => {
|
||||
it('accepts tools with readOnlyHint: true', () => {
|
||||
expect(isToolReadOnly({ readOnlyHint: true })).toBe(true);
|
||||
});
|
||||
|
||||
it('accepts tools with no annotations', () => {
|
||||
expect(isToolReadOnly(undefined)).toBe(true);
|
||||
});
|
||||
|
||||
it('accepts tools with empty annotations', () => {
|
||||
expect(isToolReadOnly({})).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects tools with readOnlyHint: false', () => {
|
||||
expect(isToolReadOnly({ readOnlyHint: false })).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts tools with only destructiveHint set', () => {
|
||||
expect(isToolReadOnly({ destructiveHint: true })).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractContent', () => {
|
||||
it('extracts single text block', () => {
|
||||
const content = [{ type: 'text', text: 'hello world' }];
|
||||
expect(extractContent(content)).toBe('hello world');
|
||||
});
|
||||
|
||||
it('joins multiple text blocks with newline', () => {
|
||||
const content = [
|
||||
{ type: 'text', text: 'line 1' },
|
||||
{ type: 'text', text: 'line 2' },
|
||||
];
|
||||
expect(extractContent(content)).toBe('line 1\nline 2');
|
||||
});
|
||||
|
||||
it('returns "(no output)" for empty content', () => {
|
||||
expect(extractContent([])).toBe('(no output)');
|
||||
});
|
||||
|
||||
it('returns "(no output)" for undefined content', () => {
|
||||
expect(extractContent(undefined)).toBe('(no output)');
|
||||
});
|
||||
|
||||
it('serializes non-text blocks as JSON', () => {
|
||||
const content = [
|
||||
{ type: 'resource', uri: 'file:///foo', mimeType: 'text/plain' },
|
||||
];
|
||||
const result = extractContent(content);
|
||||
expect(result).toContain('"type":"resource"');
|
||||
expect(result).toContain('"uri":"file:///foo"');
|
||||
});
|
||||
|
||||
it('returns error shape when isError is true', () => {
|
||||
const content = [{ type: 'text', text: 'something failed' }];
|
||||
const result = extractContent(content, true);
|
||||
expect(result).toEqual({ error: true, output: 'something failed' });
|
||||
});
|
||||
|
||||
it('returns error shape with joined content on isError', () => {
|
||||
const content = [
|
||||
{ type: 'text', text: 'error 1' },
|
||||
{ type: 'text', text: 'error 2' },
|
||||
];
|
||||
const result = extractContent(content, true);
|
||||
expect(result).toEqual({ error: true, output: 'error 1\nerror 2' });
|
||||
});
|
||||
});
|
||||
});
|
||||
82
apps/server/src/services/__tests__/mcp-glob.test.ts
Normal file
82
apps/server/src/services/__tests__/mcp-glob.test.ts
Normal file
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* v1.15.0-mcp-multi: unit tests for matchToolGlob.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { matchToolGlob } from '../agents.js';
|
||||
|
||||
describe('matchToolGlob', () => {
|
||||
it('exact match: "grep" matches "grep"', () => {
|
||||
expect(matchToolGlob('grep', ['grep'])).toBe(true);
|
||||
});
|
||||
|
||||
it('exact match: "grep" does not match "grep2"', () => {
|
||||
expect(matchToolGlob('grep2', ['grep'])).toBe(false);
|
||||
});
|
||||
|
||||
it('exact match: multiple tools', () => {
|
||||
expect(matchToolGlob('grep', ['grep', 'view_file'])).toBe(true);
|
||||
expect(matchToolGlob('view_file', ['grep', 'view_file'])).toBe(true);
|
||||
expect(matchToolGlob('find_files', ['grep', 'view_file'])).toBe(false);
|
||||
});
|
||||
|
||||
it('wildcard: "context7_*" matches "context7_query-docs"', () => {
|
||||
expect(matchToolGlob('context7_query-docs', ['context7_*'])).toBe(true);
|
||||
});
|
||||
|
||||
it('wildcard: "context7_*" matches "context7_resolve-library-id"', () => {
|
||||
expect(matchToolGlob('context7_resolve-library-id', ['context7_*'])).toBe(true);
|
||||
});
|
||||
|
||||
it('wildcard: "context7_*" does not match "codecontext_overview"', () => {
|
||||
expect(matchToolGlob('codecontext_overview', ['context7_*'])).toBe(false);
|
||||
});
|
||||
|
||||
it('wildcard: "view_*" matches "view_file" and "view_truncated_output"', () => {
|
||||
expect(matchToolGlob('view_file', ['view_*'])).toBe(true);
|
||||
expect(matchToolGlob('view_truncated_output', ['view_*'])).toBe(true);
|
||||
});
|
||||
|
||||
it('wildcard: "*" matches everything', () => {
|
||||
expect(matchToolGlob('anything', ['*'])).toBe(true);
|
||||
expect(matchToolGlob('context7_query-docs', ['*'])).toBe(true);
|
||||
});
|
||||
|
||||
it('deny: "!web_*" excludes "web_search"', () => {
|
||||
// With only a deny rule and no prior match, the tool is not matched
|
||||
expect(matchToolGlob('web_search', ['!web_*'])).toBe(false);
|
||||
});
|
||||
|
||||
it('last-match-wins: ["*", "!web_*"] excludes web tools, includes others', () => {
|
||||
expect(matchToolGlob('web_search', ['*', '!web_*'])).toBe(false);
|
||||
expect(matchToolGlob('web_fetch', ['*', '!web_*'])).toBe(false);
|
||||
expect(matchToolGlob('grep', ['*', '!web_*'])).toBe(true);
|
||||
expect(matchToolGlob('context7_query-docs', ['*', '!web_*'])).toBe(true);
|
||||
});
|
||||
|
||||
it('last-match-wins: deny then re-allow', () => {
|
||||
// ["!web_*", "web_search"] — deny all web, then re-allow web_search
|
||||
expect(matchToolGlob('web_search', ['!web_*', 'web_search'])).toBe(true);
|
||||
expect(matchToolGlob('web_fetch', ['!web_*', 'web_fetch'])).toBe(true);
|
||||
});
|
||||
|
||||
it('empty patterns: nothing matches', () => {
|
||||
expect(matchToolGlob('grep', [])).toBe(false);
|
||||
expect(matchToolGlob('anything', [])).toBe(false);
|
||||
});
|
||||
|
||||
it('no-glob fallback: exact-match only, same as pre-v1.15', () => {
|
||||
const patterns = ['grep', 'view_file'];
|
||||
expect(matchToolGlob('grep', patterns)).toBe(true);
|
||||
expect(matchToolGlob('view_file', patterns)).toBe(true);
|
||||
expect(matchToolGlob('find_files', patterns)).toBe(false);
|
||||
expect(matchToolGlob('web_search', patterns)).toBe(false);
|
||||
});
|
||||
|
||||
it('mixed glob and exact patterns', () => {
|
||||
const patterns = ['grep', 'context7_*', '!context7_dangerous'];
|
||||
expect(matchToolGlob('grep', patterns)).toBe(true);
|
||||
expect(matchToolGlob('context7_query-docs', patterns)).toBe(true);
|
||||
expect(matchToolGlob('context7_dangerous', patterns)).toBe(false);
|
||||
expect(matchToolGlob('view_file', patterns)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -16,10 +16,62 @@ const CACHE_TTL_MS = 60_000;
|
||||
// hand-maintained list drifted (web_search/web_fetch from v1.11.8 + the 8
|
||||
// codecontext tools were missing), silently filtering valid tool names out
|
||||
// of agents that opted in. Single source of truth is tools.ts now.
|
||||
const ALL_TOOL_NAMES: readonly string[] = ALL_TOOLS.map((t) => t.name);
|
||||
const DEFAULT_TOOLS: string[] = [...ALL_TOOL_NAMES];
|
||||
let ALL_TOOL_NAMES: readonly string[] = ALL_TOOLS.map((t) => t.name);
|
||||
let DEFAULT_TOOLS: string[] = [...ALL_TOOL_NAMES];
|
||||
|
||||
export function refreshToolNames(): void {
|
||||
ALL_TOOL_NAMES = ALL_TOOLS.map((t) => t.name);
|
||||
DEFAULT_TOOLS = [...ALL_TOOL_NAMES];
|
||||
}
|
||||
const DEFAULT_TEMPERATURE = 0.7;
|
||||
|
||||
// ---- Tool glob matching (v1.15.0-mcp-multi) --------------------------------
|
||||
|
||||
/**
|
||||
* Simple glob match for tool names. Supports `*` as a wildcard for any
|
||||
* characters. No `?` or `**` — tool names are flat (no path separators).
|
||||
*/
|
||||
function simpleGlobMatch(str: string, pattern: string): boolean {
|
||||
if (pattern === '*') return true;
|
||||
if (!pattern.includes('*')) return str === pattern;
|
||||
// Escape regex metacharacters, then replace escaped \* with .*
|
||||
const regex = new RegExp(
|
||||
'^' + pattern.replace(/[.+?^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '.*') + '$',
|
||||
);
|
||||
return regex.test(str);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a tool name matches a set of glob patterns. Last-match-wins.
|
||||
* Patterns starting with `!` are deny rules.
|
||||
*
|
||||
* Examples:
|
||||
* - `["grep", "view_file"]` — exact-match whitelist (same as pre-v1.15)
|
||||
* - `["context7_*"]` — all tools from the context7 MCP server
|
||||
* - `["*", "!web_*"]` — all tools except web tools
|
||||
* - `[]` — nothing matches (agent gets no tools)
|
||||
*/
|
||||
export function matchToolGlob(toolName: string, patterns: string[]): boolean {
|
||||
let matched = false;
|
||||
for (const pattern of patterns) {
|
||||
const deny = pattern.startsWith('!');
|
||||
const glob = deny ? pattern.slice(1) : pattern;
|
||||
if (simpleGlobMatch(toolName, glob)) {
|
||||
matched = !deny;
|
||||
}
|
||||
}
|
||||
return matched;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns true if a tools: entry is a glob pattern (contains * or starts
|
||||
* with !). Glob patterns can't be validated against the current tool list
|
||||
* since MCP tools are discovered at runtime.
|
||||
*/
|
||||
function isGlobPattern(entry: string): boolean {
|
||||
return entry.includes('*') || entry.startsWith('!');
|
||||
}
|
||||
|
||||
export function slugify(name: string): string {
|
||||
return name
|
||||
.toLowerCase()
|
||||
@@ -37,6 +89,10 @@ interface ParsedFrontmatter {
|
||||
// v1.8.2: optional per-agent tool-loop budget. Absent → inference resolves
|
||||
// from the agent's toolset at runtime.
|
||||
max_tool_calls?: number;
|
||||
// v1.14.0: optional per-agent step cap. Absent → bounded only by MAX_STEPS
|
||||
// (200) in the outer loop. Integer ≥ 0; steps: 0 means "no tool calls
|
||||
// allowed" — the model responds text-only.
|
||||
steps?: number;
|
||||
}
|
||||
|
||||
function stripQuotes(s: string): string {
|
||||
@@ -112,6 +168,21 @@ function parseFrontmatter(yaml: string): { data: ParsedFrontmatter; errors: stri
|
||||
} else {
|
||||
errors.push(`max_tool_calls must be an integer 1-100 (got "${valueRaw}")`);
|
||||
}
|
||||
} else if (key === 'steps') {
|
||||
// v1.14.0: per-agent step cap for the outer inference loop. Integer ≥ 0.
|
||||
// steps: 0 means "no tool calls allowed" — model responds text-only.
|
||||
// Non-integer or negative values are warned and ignored (falls back to
|
||||
// MAX_STEPS ceiling), matching the max_tool_calls pattern above.
|
||||
const n = Number(valueRaw);
|
||||
if (Number.isInteger(n) && n >= 0) {
|
||||
data.steps = n;
|
||||
} else if (Number.isInteger(n)) {
|
||||
console.warn(
|
||||
`agents: steps ${n} is negative, ignoring (falling back to default)`,
|
||||
);
|
||||
} else {
|
||||
errors.push(`steps must be a non-negative integer (got "${valueRaw}")`);
|
||||
}
|
||||
}
|
||||
// Unknown keys silently ignored — forward-compat.
|
||||
}
|
||||
@@ -188,10 +259,14 @@ function parseAgentSection(section: RawSection): Omit<Agent, 'source'> {
|
||||
|
||||
// v1.13.15-tools: intersect with BOOCODE_TOOLS tier (ceiling, not expansion).
|
||||
// Unset → resolveToolTier returns ALL tool names → no narrowing.
|
||||
// v1.15.0-mcp-multi: glob patterns (entries containing * or starting with !)
|
||||
// pass through unvalidated — MCP tools are discovered at runtime and can't
|
||||
// be checked against ALL_TOOL_NAMES at parse time.
|
||||
const tierAllowed = new Set(resolveToolTier(process.env.BOOCODE_TOOLS));
|
||||
const filteredTools = Array.isArray(fm.tools)
|
||||
? fm.tools.filter((t): t is string =>
|
||||
(ALL_TOOL_NAMES as readonly string[]).includes(t) && tierAllowed.has(t),
|
||||
isGlobPattern(t) ||
|
||||
((ALL_TOOL_NAMES as readonly string[]).includes(t) && tierAllowed.has(t)),
|
||||
)
|
||||
: DEFAULT_TOOLS.filter((t) => tierAllowed.has(t));
|
||||
|
||||
@@ -204,6 +279,7 @@ function parseAgentSection(section: RawSection): Omit<Agent, 'source'> {
|
||||
tools: filteredTools,
|
||||
model: typeof fm.model === 'string' && fm.model.length > 0 ? fm.model : null,
|
||||
max_tool_calls: typeof fm.max_tool_calls === 'number' ? fm.max_tool_calls : null,
|
||||
steps: typeof fm.steps === 'number' ? fm.steps : null,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
export {
|
||||
createInferenceRunner,
|
||||
MAX_STEPS,
|
||||
runAssistantTurn,
|
||||
runInference,
|
||||
} from './turn.js';
|
||||
@@ -16,5 +17,6 @@ export type {
|
||||
StreamResult,
|
||||
TurnArgs,
|
||||
} from './turn.js';
|
||||
export type { ToolPhaseResult } from './tool-phase.js';
|
||||
export { detectDoomLoop, DOOM_LOOP_THRESHOLD } from './sentinels.js';
|
||||
export { buildMessagesPayload } from './payload.js';
|
||||
|
||||
@@ -476,6 +476,202 @@ export async function runDoomLoopSummary(
|
||||
);
|
||||
}
|
||||
|
||||
// v1.14.0: step-cap wrap-up. Mirrors runCapHitSummary structurally — same
|
||||
// in-flight-slot reuse, same tools-disabled streaming-summary call, same
|
||||
// post-finalize sentinel insert + chat_status drop. Difference: the note
|
||||
// text names the step limit rather than the tool budget. Sentinel reuses
|
||||
// metadata.kind = 'cap_hit' so the frontend CapHitSentinel component
|
||||
// renders it without changes.
|
||||
const STEP_CAP_NOTE = (steps: number, cap: number) =>
|
||||
`You've reached the step limit (${steps}/${cap} steps). Produce the best answer you can with what you have. Do not call more tools.`;
|
||||
|
||||
export async function runStepCapSummary(
|
||||
ctx: InferenceContext,
|
||||
args: TurnArgs,
|
||||
session: Session,
|
||||
project: Project,
|
||||
history: Message[],
|
||||
agent: Agent | null,
|
||||
steps: number,
|
||||
cap: number,
|
||||
): Promise<void> {
|
||||
const { sessionId, chatId, assistantMessageId, signal } = args;
|
||||
|
||||
const messages = await buildMessagesPayload(session, project, history, agent, ctx.log);
|
||||
messages.push({ role: 'system', content: STEP_CAP_NOTE(steps, cap) });
|
||||
|
||||
const startedRow = await ctx.sql<{ started_at: string }[]>`
|
||||
UPDATE messages
|
||||
SET started_at = clock_timestamp()
|
||||
WHERE id = ${assistantMessageId}
|
||||
RETURNING started_at
|
||||
`;
|
||||
const startedAt = startedRow[0]?.started_at ?? null;
|
||||
|
||||
ctx.publish(sessionId, {
|
||||
type: 'message_started',
|
||||
message_id: assistantMessageId,
|
||||
chat_id: chatId,
|
||||
role: 'assistant',
|
||||
});
|
||||
|
||||
let accumulated = '';
|
||||
let pendingFlushTimer: NodeJS.Timeout | null = null;
|
||||
let flushPromise: Promise<unknown> = Promise.resolve();
|
||||
const flushNow = () => {
|
||||
if (pendingFlushTimer) {
|
||||
clearTimeout(pendingFlushTimer);
|
||||
pendingFlushTimer = null;
|
||||
}
|
||||
const snapshot = accumulated;
|
||||
flushPromise = flushPromise.then(() =>
|
||||
ctx.sql`UPDATE messages SET content = ${snapshot} WHERE id = ${assistantMessageId}`
|
||||
);
|
||||
};
|
||||
const scheduleFlush = () => {
|
||||
if (pendingFlushTimer) return;
|
||||
pendingFlushTimer = setTimeout(() => {
|
||||
pendingFlushTimer = null;
|
||||
flushNow();
|
||||
}, DB_FLUSH_INTERVAL_MS);
|
||||
};
|
||||
|
||||
let summaryOk = false;
|
||||
let summarySoftCancelled = false;
|
||||
let summaryError: string | null = null;
|
||||
let result: StreamResult | null = null;
|
||||
try {
|
||||
result = await streamCompletion(
|
||||
ctx,
|
||||
session.model,
|
||||
messages,
|
||||
{ tools: null, temperature: agent?.temperature },
|
||||
(delta) => {
|
||||
accumulated += delta;
|
||||
ctx.publish(sessionId, {
|
||||
type: 'delta',
|
||||
message_id: assistantMessageId,
|
||||
chat_id: chatId,
|
||||
content: delta,
|
||||
});
|
||||
scheduleFlush();
|
||||
},
|
||||
undefined,
|
||||
signal,
|
||||
);
|
||||
summaryOk = true;
|
||||
} catch (err) {
|
||||
if (err instanceof Error && err.name === 'AbortError') {
|
||||
summarySoftCancelled = true;
|
||||
} else {
|
||||
summaryError = err instanceof Error ? err.message : String(err);
|
||||
}
|
||||
} finally {
|
||||
if (pendingFlushTimer) {
|
||||
clearTimeout(pendingFlushTimer);
|
||||
pendingFlushTimer = null;
|
||||
}
|
||||
await flushPromise;
|
||||
}
|
||||
|
||||
if (summaryOk && result) {
|
||||
const mctx = await modelContext.getModelContext(session.model);
|
||||
const nCtx = mctx?.n_ctx ?? null;
|
||||
const [updated] = await ctx.sql<
|
||||
{ tokens_used: number | null; ctx_used: number | null; ctx_max: number | null; finished_at: string | null }[]
|
||||
>`
|
||||
UPDATE messages
|
||||
SET content = ${result.content},
|
||||
status = 'complete',
|
||||
tokens_used = ${result.completionTokens},
|
||||
ctx_used = ${result.promptTokens},
|
||||
ctx_max = ${nCtx},
|
||||
finished_at = clock_timestamp()
|
||||
WHERE id = ${assistantMessageId}
|
||||
RETURNING tokens_used, ctx_used, ctx_max, finished_at
|
||||
`;
|
||||
ctx.publish(sessionId, {
|
||||
type: 'message_complete',
|
||||
message_id: assistantMessageId,
|
||||
chat_id: chatId,
|
||||
tokens_used: updated?.tokens_used ?? null,
|
||||
ctx_used: updated?.ctx_used ?? null,
|
||||
ctx_max: updated?.ctx_max ?? null,
|
||||
started_at: startedAt,
|
||||
finished_at: updated?.finished_at ?? null,
|
||||
model: session.model,
|
||||
});
|
||||
} else if (summarySoftCancelled) {
|
||||
await ctx.sql`
|
||||
UPDATE messages
|
||||
SET content = ${accumulated},
|
||||
status = 'cancelled',
|
||||
finished_at = clock_timestamp()
|
||||
WHERE id = ${assistantMessageId}
|
||||
`;
|
||||
ctx.publish(sessionId, {
|
||||
type: 'message_complete',
|
||||
message_id: assistantMessageId,
|
||||
chat_id: chatId,
|
||||
});
|
||||
} else {
|
||||
const errMeta: MessageMetadata = {
|
||||
kind: 'error',
|
||||
error_reason: 'summary_after_cap_failed',
|
||||
error_text: summaryError ?? 'step-cap summary failed',
|
||||
};
|
||||
await ctx.sql`
|
||||
UPDATE messages
|
||||
SET content = ${accumulated},
|
||||
status = 'failed',
|
||||
finished_at = clock_timestamp(),
|
||||
metadata = ${ctx.sql.json(errMeta as never)}
|
||||
WHERE id = ${assistantMessageId}
|
||||
`;
|
||||
ctx.publish(sessionId, {
|
||||
type: 'error',
|
||||
message_id: assistantMessageId,
|
||||
chat_id: chatId,
|
||||
error: summaryError ?? 'step-cap summary failed',
|
||||
reason: 'summary_after_cap_failed',
|
||||
});
|
||||
}
|
||||
|
||||
const [sessRow] = await ctx.sql<{ project_id: string; name: string; updated_at: string }[]>`
|
||||
UPDATE sessions SET updated_at = clock_timestamp()
|
||||
WHERE id = ${sessionId}
|
||||
RETURNING project_id, name, updated_at
|
||||
`;
|
||||
ctx.publishUser({
|
||||
type: 'session_updated',
|
||||
session_id: sessionId,
|
||||
project_id: sessRow!.project_id,
|
||||
name: sessRow!.name,
|
||||
updated_at: sessRow!.updated_at,
|
||||
});
|
||||
|
||||
// Reuse cap_hit sentinel so the frontend CapHitSentinel component renders
|
||||
// it without changes. The content text distinguishes step cap from budget.
|
||||
await insertCapHitSentinel(ctx, sessionId, chatId, agent, cap);
|
||||
|
||||
if (summaryOk || summarySoftCancelled) {
|
||||
ctx.publishUser({ type: 'chat_status', chat_id: chatId, status: 'idle', at: new Date().toISOString() });
|
||||
} else {
|
||||
ctx.publishUser({
|
||||
type: 'chat_status',
|
||||
chat_id: chatId,
|
||||
status: 'error',
|
||||
at: new Date().toISOString(),
|
||||
reason: 'summary_after_cap_failed',
|
||||
});
|
||||
}
|
||||
|
||||
ctx.log.info(
|
||||
{ sessionId, chatId, assistantMessageId, steps, cap, summaryOk, summaryCancelled: summarySoftCancelled },
|
||||
'inference step-cap summary finished',
|
||||
);
|
||||
}
|
||||
|
||||
async function insertDoomLoopSentinel(
|
||||
ctx: InferenceContext,
|
||||
sessionId: string,
|
||||
|
||||
@@ -5,6 +5,7 @@ import type {
|
||||
} from '../../types/api.js';
|
||||
import * as modelContext from '../model-context.js';
|
||||
import { toolJsonSchemas, type ToolJsonSchema } from '../tools.js';
|
||||
import { matchToolGlob } from '../agents.js';
|
||||
import type { OpenAiMessage } from './payload.js';
|
||||
// v1.13.16: extractToolCallBlocks replaces the inline opener-search loop and
|
||||
// recognizes both Qwen <tool_call> and Anthropic <invoke> markup in one pass.
|
||||
@@ -376,14 +377,14 @@ export async function executeStreamPhase(
|
||||
};
|
||||
|
||||
// Tool whitelist: if an agent is set, filter the global tool list to only the
|
||||
// tool names it allows. Unknown names in agent.tools are dropped silently
|
||||
// (handled here by intersection). When no agent: send all tools.
|
||||
// tool names it allows. v1.15.0-mcp-multi: uses matchToolGlob for glob
|
||||
// pattern support (e.g. `context7_*`, `!web_*`). When no agent: send all tools.
|
||||
// v1.11.8: a second filter strips web_search + web_fetch unless the chat
|
||||
// has them explicitly enabled. Counts as an opt-in security boundary: the
|
||||
// model can't summon a tool that wasn't offered to it.
|
||||
const WEB_TOOL_NAMES: ReadonlySet<string> = new Set(['web_search', 'web_fetch']);
|
||||
const effectiveTools: ToolJsonSchema[] = (agent
|
||||
? toolJsonSchemas().filter((t) => agent.tools.includes(t.function.name))
|
||||
? toolJsonSchemas().filter((t) => matchToolGlob(t.function.name, agent.tools))
|
||||
: toolJsonSchemas()
|
||||
).filter((t) => webToolsEnabled || !WEB_TOOL_NAMES.has(t.function.name));
|
||||
const effectiveTemperature = agent?.temperature;
|
||||
|
||||
@@ -19,11 +19,6 @@ import type {
|
||||
StreamResult,
|
||||
TurnArgs,
|
||||
} from './turn.js';
|
||||
// v1.12.4: ESM value-import cycle. executeToolPhase recurses into
|
||||
// runAssistantTurn which lives in inference.ts. The cycle is safe because
|
||||
// the reference is read at call time (inside an async function body), not
|
||||
// at module top-level. Node + tsc resolve this cleanly.
|
||||
import { runAssistantTurn } from './turn.js';
|
||||
// v1.13.13: synthesis pipeline — replaces the immediate recursive turn when
|
||||
// any of this batch's tool calls is in SYNTHESIS_TOOLS. Falls through to
|
||||
// recursion on synthesis failure (timeout / model error). See module header
|
||||
@@ -86,6 +81,16 @@ async function executeToolCall(
|
||||
}
|
||||
}
|
||||
|
||||
// v1.14.0: return struct from executeToolPhase so the caller (the outer
|
||||
// while loop in turn.ts) can decide whether to continue, break, or handle
|
||||
// synthesis. Replaces the recursive call into runAssistantTurn.
|
||||
export interface ToolPhaseResult {
|
||||
action: 'continue' | 'paused' | 'synthesis_done';
|
||||
toolCallCount: number;
|
||||
toolCalls: ToolCall[];
|
||||
nextAssistantId: string | null;
|
||||
}
|
||||
|
||||
export async function executeToolPhase(
|
||||
ctx: InferenceContext,
|
||||
args: TurnArgs,
|
||||
@@ -93,8 +98,8 @@ export async function executeToolPhase(
|
||||
startedAt: string | null,
|
||||
session: Session,
|
||||
projectRoot: string
|
||||
): Promise<void> {
|
||||
const { sessionId, chatId, assistantMessageId, toolsUsed, signal } = args;
|
||||
): Promise<ToolPhaseResult> {
|
||||
const { sessionId, chatId, assistantMessageId } = args;
|
||||
const { content, toolCalls, promptTokens, completionTokens } = result;
|
||||
|
||||
// v1.11.3: ctx_max comes from llama-swap /upstream/<model>/props, not the
|
||||
@@ -296,7 +301,12 @@ export async function executeToolPhase(
|
||||
{ sessionId, chatId, assistantMessageId },
|
||||
'inference paused awaiting user input',
|
||||
);
|
||||
return;
|
||||
return {
|
||||
action: 'paused' as const,
|
||||
toolCallCount: toolCalls.length,
|
||||
toolCalls,
|
||||
nextAssistantId: null,
|
||||
};
|
||||
}
|
||||
|
||||
// v1.13.13: synthesis-pipeline branch. When any of this batch's tool calls
|
||||
@@ -328,30 +338,30 @@ export async function executeToolPhase(
|
||||
...(typeof out?.truncated === 'boolean' ? { truncated: out.truncated } : {}),
|
||||
...(typeof out?.outputPath === 'string' ? { outputPath: out.outputPath } : {}),
|
||||
});
|
||||
if (ran) return;
|
||||
if (ran) {
|
||||
return {
|
||||
action: 'synthesis_done' as const,
|
||||
toolCallCount: toolCalls.length,
|
||||
toolCalls,
|
||||
nextAssistantId: null,
|
||||
};
|
||||
}
|
||||
// ran === false → synthesis failed (timeout / model error) → fall through
|
||||
// to the standard recursive turn below. The synth message (if created)
|
||||
// to the standard continue path below. The synth message (if created)
|
||||
// was already marked status='failed' inside runSynthesisPass.
|
||||
}
|
||||
|
||||
// v1.14.0: create the next assistant row and return a continue result.
|
||||
// The caller (outer while loop in turn.ts) handles the iteration.
|
||||
const [nextAssistant] = await ctx.sql<{ id: string }[]>`
|
||||
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
||||
VALUES (${sessionId}, ${chatId}, 'assistant', '', 'streaming', clock_timestamp())
|
||||
RETURNING id
|
||||
`;
|
||||
await runAssistantTurn(ctx, {
|
||||
sessionId,
|
||||
chatId,
|
||||
assistantMessageId: nextAssistant!.id,
|
||||
// v1.8.2: charge this turn's actual tool invocations against the budget.
|
||||
// One assistant message can emit multiple tool_calls, so we add the run
|
||||
// count, not 1. The next turn's budget check sees the cumulative total.
|
||||
toolsUsed: toolsUsed + result.toolCalls.length,
|
||||
// v1.11.6: append the just-executed tool calls to the per-turn history
|
||||
// so the next runAssistantTurn's doom-loop check can see them. We don't
|
||||
// cap the array length here — per-turn budgets keep it bounded
|
||||
// (typically <30 entries), and slicing happens inside detectDoomLoop.
|
||||
recentToolCalls: [...args.recentToolCalls, ...result.toolCalls],
|
||||
signal,
|
||||
});
|
||||
return {
|
||||
action: 'continue' as const,
|
||||
toolCallCount: toolCalls.length,
|
||||
toolCalls,
|
||||
nextAssistantId: nextAssistant!.id,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -16,11 +16,9 @@ import { resolveProjectRoot } from '../path_guard.js';
|
||||
import { maybeAutoNameChat } from '../auto_name.js';
|
||||
import { getAgentById } from '../agents.js';
|
||||
import * as compaction from '../compaction.js';
|
||||
import * as modelContext from '../model-context.js';
|
||||
import type { Broker } from '../broker.js';
|
||||
import { resolveToolBudget } from './budget.js';
|
||||
import {
|
||||
DOOM_LOOP_THRESHOLD,
|
||||
detectDoomLoop,
|
||||
} from './sentinels.js';
|
||||
import {
|
||||
@@ -33,15 +31,23 @@ import {
|
||||
} from './error-handler.js';
|
||||
import {
|
||||
executeStreamPhase,
|
||||
streamCompletion,
|
||||
} from './stream-phase.js';
|
||||
import { executeToolPhase } from './tool-phase.js';
|
||||
import { DB_FLUSH_INTERVAL_MS, type StreamPhaseState } from './types.js';
|
||||
import { executeToolPhase, type ToolPhaseResult } from './tool-phase.js';
|
||||
import type { StreamPhaseState } from './types.js';
|
||||
import {
|
||||
runCapHitSummary,
|
||||
runDoomLoopSummary,
|
||||
runStepCapSummary,
|
||||
} from './sentinel-summaries.js';
|
||||
|
||||
// v1.14.0: hard ceiling on the number of stream-and-tool iterations per
|
||||
// user-message turn. Per-agent cap via agent.steps is the primary knob;
|
||||
// MAX_STEPS is the safety ceiling. 200 is 4x the effective budget ceiling
|
||||
// (50 tool calls) — in practice budget fires first unless the model makes
|
||||
// many 0-tool-call iterations (which exit the loop via the non-tool finish
|
||||
// path anyway).
|
||||
export const MAX_STEPS = 200;
|
||||
|
||||
// v1.12.4: re-exported so external callers (tests, future consumers) keep
|
||||
// importing from services/inference.js as the public surface.
|
||||
export { detectDoomLoop, DOOM_LOOP_THRESHOLD } from './sentinels.js';
|
||||
@@ -145,75 +151,185 @@ export async function runAssistantTurn(
|
||||
ctx: InferenceContext,
|
||||
args: TurnArgs,
|
||||
): Promise<void> {
|
||||
const { sessionId, chatId } = args;
|
||||
const { sessionId, chatId, signal } = args;
|
||||
|
||||
// v1.11: if the prior turn flagged this chat for compaction, run it first
|
||||
// so loadContext below reads the post-compaction history. We swallow
|
||||
// compaction failures (clearing the flag so we don't loop) and proceed
|
||||
// with the un-compacted history — a slow turn that hits the model's
|
||||
// hard limit is recoverable; a dead session is not.
|
||||
const chatFlag = await ctx.sql<{ needs_compaction: boolean }[]>`
|
||||
SELECT needs_compaction FROM chats WHERE id = ${chatId}
|
||||
`;
|
||||
if (chatFlag[0]?.needs_compaction) {
|
||||
try {
|
||||
await compaction.process({
|
||||
sql: ctx.sql,
|
||||
config: ctx.config,
|
||||
log: ctx.log,
|
||||
broker: ctx.broker,
|
||||
chatId,
|
||||
});
|
||||
} catch (err) {
|
||||
ctx.log.warn({ err, chatId }, 'auto-compaction failed; clearing flag and proceeding');
|
||||
await ctx.sql`UPDATE chats SET needs_compaction = false WHERE id = ${chatId}`;
|
||||
}
|
||||
}
|
||||
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (!loaded) {
|
||||
// v1.14.0: resolve agent once at the top. The agent stays fixed for the
|
||||
// duration of this user-message turn — PATCH agent_id mid-conversation
|
||||
// takes effect on the next runInference, not mid-loop.
|
||||
const initialLoaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (!initialLoaded) {
|
||||
ctx.log.warn({ sessionId }, 'inference: session or project missing');
|
||||
return;
|
||||
}
|
||||
const { session, project, history } = loaded;
|
||||
const projectRoot = await resolveProjectRoot(project.path);
|
||||
// Agent resolution is per-turn so PATCH agent_id mid-conversation takes
|
||||
// effect on the next message. Unknown agent_id returns null silently —
|
||||
// session falls back to base prompt + all tools + default temperature.
|
||||
const { session, project } = initialLoaded;
|
||||
const agent = session.agent_id
|
||||
? await getAgentById(project.path, session.agent_id)
|
||||
: null;
|
||||
|
||||
// v1.8.2: cap-hit replaces the older "tool loop depth exceeded" failure.
|
||||
// When we've already burned the budget *before* this turn even runs, we
|
||||
// skip straight to the summary flow — the in-flight assistant message slot
|
||||
// gets reused for the wrap-up reply instead of being marked failed.
|
||||
const budget = resolveToolBudget(agent);
|
||||
if (args.toolsUsed >= budget) {
|
||||
await runCapHitSummary(ctx, args, session, project, history, agent, budget);
|
||||
|
||||
// v1.14.0: effectiveCap = min(agent.steps ?? Infinity, MAX_STEPS).
|
||||
// steps: 0 means "no tool calls allowed" — the first stream phase runs
|
||||
// but if it emits tool calls they are not executed (finalize as text-only).
|
||||
const effectiveCap = Math.min(agent?.steps ?? Infinity, MAX_STEPS);
|
||||
|
||||
// steps: 0 special case — model responds text-only. The while loop would
|
||||
// never enter (effectiveCap === 0), so we handle it explicitly before the
|
||||
// loop. The model always gets at least one chance to respond with text.
|
||||
if (effectiveCap === 0) {
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (loaded) {
|
||||
await runTextOnlyTurn(ctx, args, loaded.session, loaded.project, loaded.history, agent);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// v1.11.6: doom-loop guard. Detected BEFORE the budget cap (the model can
|
||||
// burn through 3 identical calls long before the 15-call budget fires).
|
||||
// Same in-flight-slot-reuse pattern as runCapHitSummary — wrap-up reply
|
||||
// lands in args.assistantMessageId, then a doom_loop sentinel is inserted
|
||||
// to make the abort visible in the chat history.
|
||||
const loop = detectDoomLoop(args.recentToolCalls);
|
||||
if (loop) {
|
||||
await runDoomLoopSummary(ctx, args, session, project, history, agent, loop);
|
||||
return;
|
||||
let stepNumber = 0;
|
||||
let toolsUsed = args.toolsUsed;
|
||||
let recentToolCalls = args.recentToolCalls;
|
||||
let assistantMessageId = args.assistantMessageId;
|
||||
|
||||
while (stepNumber < effectiveCap) {
|
||||
// ---- doom-loop check (moved from top-of-function) ----
|
||||
const loop = detectDoomLoop(recentToolCalls);
|
||||
if (loop) {
|
||||
// Need fresh history for the summary.
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (loaded) {
|
||||
const iterArgs: TurnArgs = { sessionId, chatId, assistantMessageId, toolsUsed, recentToolCalls, signal };
|
||||
await runDoomLoopSummary(ctx, iterArgs, loaded.session, loaded.project, loaded.history, agent, loop);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
// ---- budget check (moved from top-of-function) ----
|
||||
if (toolsUsed >= budget) {
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (loaded) {
|
||||
const iterArgs: TurnArgs = { sessionId, chatId, assistantMessageId, toolsUsed, recentToolCalls, signal };
|
||||
await runCapHitSummary(ctx, iterArgs, loaded.session, loaded.project, loaded.history, agent, budget);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
// ---- compaction check ----
|
||||
// v1.11: if the prior turn flagged this chat for compaction, run it
|
||||
// before loadContext so we read post-compaction history. Swallow
|
||||
// failures and proceed with un-compacted history.
|
||||
const chatFlag = await ctx.sql<{ needs_compaction: boolean }[]>`
|
||||
SELECT needs_compaction FROM chats WHERE id = ${chatId}
|
||||
`;
|
||||
if (chatFlag[0]?.needs_compaction) {
|
||||
try {
|
||||
await compaction.process({
|
||||
sql: ctx.sql,
|
||||
config: ctx.config,
|
||||
log: ctx.log,
|
||||
broker: ctx.broker,
|
||||
chatId,
|
||||
});
|
||||
} catch (err) {
|
||||
ctx.log.warn({ err, chatId }, 'auto-compaction failed; clearing flag and proceeding');
|
||||
await ctx.sql`UPDATE chats SET needs_compaction = false WHERE id = ${chatId}`;
|
||||
}
|
||||
}
|
||||
|
||||
// ---- load context (must re-load each iteration — new messages since last step) ----
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (!loaded) {
|
||||
ctx.log.warn({ sessionId }, 'inference: session or project missing mid-loop');
|
||||
break;
|
||||
}
|
||||
const { session: iterSession, project: iterProject, history } = loaded;
|
||||
const projectRoot = await resolveProjectRoot(iterProject.path);
|
||||
|
||||
// v1.14.0: log step boundary for instrumentation. step_start parts are in
|
||||
// the schema CHECK but not emitted here — writing to the assistant message
|
||||
// before the stream phase creates a sequence-0 collision with
|
||||
// partsFromAssistantMessage. A WS frame or structured log is sufficient
|
||||
// since the frontend doesn't render step boundaries in v1.14.
|
||||
ctx.log.info({ sessionId, chatId, step: stepNumber, assistantMessageId }, 'step_start');
|
||||
|
||||
// ---- build messages + stream phase ----
|
||||
const messages = await buildMessagesPayload(iterSession, iterProject, history, agent, ctx.log);
|
||||
const webToolsEnabled =
|
||||
iterSession.web_search_enabled ?? iterProject.default_web_search_enabled ?? false;
|
||||
|
||||
const iterArgs: TurnArgs = { sessionId, chatId, assistantMessageId, toolsUsed, recentToolCalls, signal };
|
||||
const state: StreamPhaseState = { accumulated: '', startedAt: null };
|
||||
let result: StreamResult;
|
||||
try {
|
||||
result = await executeStreamPhase(ctx, iterArgs, iterSession, messages, state, agent, webToolsEnabled);
|
||||
} catch (err) {
|
||||
await handleAbortOrError(ctx, iterArgs, state.accumulated, err);
|
||||
break;
|
||||
}
|
||||
|
||||
// ---- non-tool finish → finalize and exit ----
|
||||
if (result.toolCalls.length === 0) {
|
||||
await finalizeCompletion(ctx, iterArgs, result, state.startedAt, iterSession);
|
||||
break;
|
||||
}
|
||||
|
||||
// ---- steps: 0 edge case ----
|
||||
// effectiveCap check above guarantees we're inside the loop, but this
|
||||
// guard handles the theoretical case where the model emits tool calls
|
||||
// on step 0 when effectiveCap would have been 0 (impossible since the
|
||||
// while condition prevents entry, but kept for safety). If effectiveCap
|
||||
// is 1 and we're on step 0, tool calls ARE executed — steps counts
|
||||
// iterations, not post-first-stream.
|
||||
|
||||
// ---- tool phase ----
|
||||
let toolPhaseResult: ToolPhaseResult;
|
||||
try {
|
||||
toolPhaseResult = await executeToolPhase(ctx, iterArgs, result, state.startedAt, iterSession, projectRoot);
|
||||
} catch (err) {
|
||||
// Tool phase errors are unexpected (individual tool failures are
|
||||
// caught inside executeToolPhase). Log and break.
|
||||
ctx.log.error({ err, sessionId, chatId, step: stepNumber }, 'tool phase threw unexpectedly');
|
||||
break;
|
||||
}
|
||||
|
||||
// ---- update loop locals ----
|
||||
toolsUsed += toolPhaseResult.toolCallCount;
|
||||
recentToolCalls = [...recentToolCalls, ...toolPhaseResult.toolCalls];
|
||||
stepNumber++;
|
||||
|
||||
if (toolPhaseResult.action !== 'continue') {
|
||||
// 'paused' (user input) or 'synthesis_done' — stop the loop.
|
||||
break;
|
||||
}
|
||||
// 'continue' — advance to next assistant message.
|
||||
assistantMessageId = toolPhaseResult.nextAssistantId!;
|
||||
}
|
||||
|
||||
// ---- post-loop: step-cap sentinel ----
|
||||
// When the loop exits because stepNumber reached effectiveCap, the last
|
||||
// iteration's tool phase returned 'continue' with a nextAssistantId that
|
||||
// is still in 'streaming' status (unfilled). Use it for the wrap-up.
|
||||
if (stepNumber >= effectiveCap && effectiveCap < Infinity) {
|
||||
const loaded = await loadContext(ctx.sql, sessionId, chatId);
|
||||
if (loaded) {
|
||||
const capArgs: TurnArgs = { sessionId, chatId, assistantMessageId, toolsUsed, recentToolCalls, signal };
|
||||
await runStepCapSummary(ctx, capArgs, loaded.session, loaded.project, loaded.history, agent, stepNumber, effectiveCap);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// v1.14.0: special handling for steps: 0 — the model responds text-only.
|
||||
// The while loop never enters (effectiveCap === 0). We stream once with
|
||||
// no tools, finalize, and return. If the model emits tool calls despite
|
||||
// not being offered tools, they're ignored (finalize as text-only).
|
||||
async function runTextOnlyTurn(
|
||||
ctx: InferenceContext,
|
||||
args: TurnArgs,
|
||||
session: Session,
|
||||
project: Project,
|
||||
history: Message[],
|
||||
agent: Agent | null,
|
||||
): Promise<void> {
|
||||
const messages = await buildMessagesPayload(session, project, history, agent, ctx.log);
|
||||
|
||||
// v1.11.8: resolve per-chat web-tools opt-in. Tri-state on the wire:
|
||||
// - session.web_search_enabled = null → inherit project default
|
||||
// - session.web_search_enabled = true/false → explicit
|
||||
// Both web_search and web_fetch are gated by this single flag (the UI
|
||||
// label is "Enable web search and fetch" — same store, both tools).
|
||||
// Default is false unless explicitly opted in, matching the v1.9
|
||||
// plumbing intent ("inert until Batch 8 ships the actual tools").
|
||||
// Web tools are irrelevant when steps: 0 (no tool execution), but we
|
||||
// still need to resolve the flag for executeStreamPhase's signature.
|
||||
const webToolsEnabled =
|
||||
session.web_search_enabled ?? project.default_web_search_enabled ?? false;
|
||||
|
||||
@@ -227,8 +343,12 @@ export async function runAssistantTurn(
|
||||
}
|
||||
|
||||
if (result.toolCalls.length > 0) {
|
||||
await executeToolPhase(ctx, args, result, state.startedAt, session, projectRoot);
|
||||
return;
|
||||
ctx.log.warn(
|
||||
{ chatId: args.chatId, toolCallCount: result.toolCalls.length },
|
||||
'steps: 0 agent emitted tool calls; ignoring and finalizing as text-only',
|
||||
);
|
||||
// Override: strip tool calls so finalizeCompletion treats it as text-only.
|
||||
result = { ...result, toolCalls: [] };
|
||||
}
|
||||
|
||||
await finalizeCompletion(ctx, args, result, state.startedAt, session);
|
||||
|
||||
288
apps/server/src/services/mcp-client.ts
Normal file
288
apps/server/src/services/mcp-client.ts
Normal file
@@ -0,0 +1,288 @@
|
||||
/**
|
||||
* v1.15.0-mcp-multi: multi-server MCP client registry.
|
||||
*
|
||||
* Connects to multiple MCP servers (Streamable HTTP or stdio transport),
|
||||
* discovers tools from each, wraps them as BooCode ToolDefs with a
|
||||
* `<serverName>_<toolName>` name prefix, and routes callTool by prefix.
|
||||
*
|
||||
* Graceful degradation: one failing server doesn't block others.
|
||||
* Read-only invariant: tools with readOnlyHint === false are rejected.
|
||||
*/
|
||||
import { Client } from '@modelcontextprotocol/sdk/client';
|
||||
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';
|
||||
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';
|
||||
import { z } from 'zod';
|
||||
import type { FastifyBaseLogger } from 'fastify';
|
||||
import type { McpServerEntry, McpServerConfig } from './mcp-config.js';
|
||||
import type { ToolDef } from './tools.js';
|
||||
|
||||
// ---- Types ----
|
||||
|
||||
interface McpToolAnnotations {
|
||||
readOnlyHint?: boolean;
|
||||
destructiveHint?: boolean;
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
interface McpToolDef {
|
||||
name: string;
|
||||
description?: string;
|
||||
inputSchema: Record<string, unknown>;
|
||||
annotations?: McpToolAnnotations;
|
||||
}
|
||||
|
||||
interface ServerState {
|
||||
client: Client;
|
||||
transport: StreamableHTTPClientTransport | StdioClientTransport;
|
||||
tools: ToolDef<Record<string, unknown>>[];
|
||||
type: 'streamableHttp' | 'stdio';
|
||||
}
|
||||
|
||||
// ---- Module-level state ----
|
||||
|
||||
const servers = new Map<string, ServerState>();
|
||||
// Reverse map: prefixed tool name → server name (built during discovery)
|
||||
const toolToServer = new Map<string, string>();
|
||||
let log: FastifyBaseLogger | null = null;
|
||||
|
||||
const MAX_RESULT_BYTES = 5 * 1024 * 1024;
|
||||
|
||||
// ---- Public API ----
|
||||
|
||||
/**
|
||||
* Connect to all configured MCP servers, discover tools, and wrap them.
|
||||
* Per-server graceful degradation: a failing server is logged and skipped.
|
||||
*/
|
||||
export async function initialize(
|
||||
entries: McpServerEntry[],
|
||||
logger: FastifyBaseLogger,
|
||||
): Promise<void> {
|
||||
log = logger;
|
||||
|
||||
// Connect servers in parallel — each wrapped in try/catch for isolation
|
||||
await Promise.all(
|
||||
entries.map(async (entry) => {
|
||||
try {
|
||||
await connectServer(entry);
|
||||
} catch (err) {
|
||||
log!.warn(
|
||||
{ err, server: entry.name },
|
||||
`mcp: failed to initialize server "${entry.name}" — its tools will be unavailable`,
|
||||
);
|
||||
}
|
||||
}),
|
||||
);
|
||||
|
||||
if (servers.size > 0) {
|
||||
const totalTools = Array.from(servers.values()).reduce((n, s) => n + s.tools.length, 0);
|
||||
log.info(
|
||||
{ servers: servers.size, tools: totalTools },
|
||||
'mcp: multi-server initialization complete',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Call an MCP tool by its prefixed name. Routes to the correct server
|
||||
* using the toolToServer reverse map.
|
||||
*/
|
||||
export async function callTool(
|
||||
prefixedName: string,
|
||||
args: Record<string, unknown>,
|
||||
): Promise<unknown> {
|
||||
const serverName = toolToServer.get(prefixedName);
|
||||
if (!serverName) {
|
||||
return { error: true, output: `MCP tool "${prefixedName}" not found in any server` };
|
||||
}
|
||||
|
||||
const state = servers.get(serverName);
|
||||
if (!state) {
|
||||
return { error: true, output: `MCP server "${serverName}" not available` };
|
||||
}
|
||||
|
||||
// Strip the "<serverName>_" prefix to get the original tool name
|
||||
const originalName = prefixedName.slice(serverName.length + 1);
|
||||
|
||||
try {
|
||||
const result = await state.client.callTool({ name: originalName, arguments: args });
|
||||
|
||||
const content = result.content as Array<{ type: string; text?: string; [key: string]: unknown }>;
|
||||
if (!content || content.length === 0) {
|
||||
return '(no output)';
|
||||
}
|
||||
|
||||
if (result.isError) {
|
||||
const joined = content
|
||||
.map((block) => (block.type === 'text' ? block.text ?? '' : JSON.stringify(block)))
|
||||
.join('\n');
|
||||
return { error: true, output: joined || '(MCP error with no details)' };
|
||||
}
|
||||
|
||||
const parts = content.map((block) => {
|
||||
if (block.type === 'text') return block.text ?? '';
|
||||
return JSON.stringify(block);
|
||||
});
|
||||
const joined = parts.join('\n');
|
||||
if (joined.length > MAX_RESULT_BYTES) {
|
||||
log?.warn({ tool: originalName, server: serverName, bytes: joined.length, cap: MAX_RESULT_BYTES }, 'mcp: result truncated');
|
||||
return joined.slice(0, MAX_RESULT_BYTES) + '\n\n[truncated — MCP result exceeded size limit]';
|
||||
}
|
||||
return joined;
|
||||
} catch (err) {
|
||||
log?.warn({ err, tool: originalName, server: serverName }, 'mcp: callTool failed');
|
||||
return {
|
||||
error: true,
|
||||
output: err instanceof Error ? err.message : 'MCP server unreachable',
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** Return all wrapped ToolDefs from all connected servers, flattened. */
|
||||
export function getTools(): ToolDef<Record<string, unknown>>[] {
|
||||
const all: ToolDef<Record<string, unknown>>[] = [];
|
||||
for (const state of servers.values()) {
|
||||
all.push(...state.tools);
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
/** Return status of each server (for debug/status endpoints). */
|
||||
export function getMcpServers(): Array<{
|
||||
name: string;
|
||||
type: 'streamableHttp' | 'stdio';
|
||||
toolCount: number;
|
||||
connected: boolean;
|
||||
}> {
|
||||
return Array.from(servers.entries()).map(([name, state]) => ({
|
||||
name,
|
||||
type: state.type,
|
||||
toolCount: state.tools.length,
|
||||
connected: true,
|
||||
}));
|
||||
}
|
||||
|
||||
/**
|
||||
* Graceful shutdown. For stdio servers, the SDK's transport.close() handles
|
||||
* SIGTERM + timeout. For HTTP servers, close the transport.
|
||||
*/
|
||||
export async function shutdown(): Promise<void> {
|
||||
const closePromises: Promise<void>[] = [];
|
||||
for (const [name, state] of servers) {
|
||||
closePromises.push(
|
||||
(async () => {
|
||||
try {
|
||||
await state.transport.close();
|
||||
log?.info({ server: name }, 'mcp: server transport closed');
|
||||
} catch (err) {
|
||||
log?.warn({ err, server: name }, 'mcp: error closing server transport');
|
||||
}
|
||||
})(),
|
||||
);
|
||||
}
|
||||
await Promise.all(closePromises);
|
||||
servers.clear();
|
||||
toolToServer.clear();
|
||||
}
|
||||
|
||||
// ---- Internal helpers ----
|
||||
|
||||
async function connectServer(entry: McpServerEntry): Promise<void> {
|
||||
const { name, config } = entry;
|
||||
|
||||
const client = new Client({ name: 'boocode', version: '1.15.0' });
|
||||
let transport: StreamableHTTPClientTransport | StdioClientTransport;
|
||||
|
||||
if (config.type === 'streamableHttp') {
|
||||
transport = createHttpTransport(config);
|
||||
} else {
|
||||
transport = createStdioTransport(config);
|
||||
}
|
||||
|
||||
await client.connect(transport);
|
||||
|
||||
const result = await client.listTools();
|
||||
const mcpTools = (result.tools ?? []) as McpToolDef[];
|
||||
|
||||
const tools: ToolDef<Record<string, unknown>>[] = [];
|
||||
for (const t of mcpTools) {
|
||||
if (t.annotations?.readOnlyHint === false) {
|
||||
log!.info({ tool: t.name, server: name }, 'mcp: skipping non-read-only tool');
|
||||
continue;
|
||||
}
|
||||
const wrapped = wrapMcpTool(name, t);
|
||||
tools.push(wrapped);
|
||||
toolToServer.set(wrapped.name, name);
|
||||
}
|
||||
|
||||
servers.set(name, { client, transport, tools, type: config.type });
|
||||
|
||||
log!.info(
|
||||
{ server: name, type: config.type, count: tools.length, names: tools.map((t) => t.name) },
|
||||
'mcp: server initialized',
|
||||
);
|
||||
}
|
||||
|
||||
function createHttpTransport(config: Extract<McpServerConfig, { type: 'streamableHttp' }>): StreamableHTTPClientTransport {
|
||||
const requestInit: RequestInit = {};
|
||||
if (config.headers && Object.keys(config.headers).length > 0) {
|
||||
requestInit.headers = config.headers;
|
||||
}
|
||||
return new StreamableHTTPClientTransport(new URL(config.url), { requestInit });
|
||||
}
|
||||
|
||||
function createStdioTransport(config: Extract<McpServerConfig, { type: 'stdio' }>): StdioClientTransport {
|
||||
return new StdioClientTransport({
|
||||
command: config.command,
|
||||
args: config.args,
|
||||
env: config.env,
|
||||
stderr: 'pipe',
|
||||
});
|
||||
}
|
||||
|
||||
/** Wrap an MCP tool as a BooCode ToolDef with a server-name prefix. */
|
||||
export function wrapMcpTool(
|
||||
serverName: string,
|
||||
mcpTool: McpToolDef,
|
||||
): ToolDef<Record<string, unknown>> {
|
||||
const prefixedName = `${serverName}_${mcpTool.name}`;
|
||||
return {
|
||||
name: prefixedName,
|
||||
description: mcpTool.description ?? '',
|
||||
inputSchema: z.record(z.unknown()),
|
||||
jsonSchema: {
|
||||
type: 'function' as const,
|
||||
function: {
|
||||
name: prefixedName,
|
||||
description: mcpTool.description ?? '',
|
||||
parameters: mcpTool.inputSchema ?? { type: 'object', properties: {} },
|
||||
},
|
||||
},
|
||||
execute: async (input) => {
|
||||
return callTool(prefixedName, input);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Exposed for unit tests — extract content from an MCP result. */
|
||||
export function extractContent(
|
||||
content: Array<{ type: string; text?: string; [key: string]: unknown }> | undefined,
|
||||
isError?: boolean,
|
||||
): unknown {
|
||||
if (!content || content.length === 0) return '(no output)';
|
||||
|
||||
const parts = content.map((block) => {
|
||||
if (block.type === 'text') return block.text ?? '';
|
||||
return JSON.stringify(block);
|
||||
});
|
||||
const joined = parts.join('\n');
|
||||
|
||||
if (isError) {
|
||||
return { error: true, output: joined || '(MCP error with no details)' };
|
||||
}
|
||||
return joined;
|
||||
}
|
||||
|
||||
/** Exposed for unit tests — the read-only guard predicate. */
|
||||
export function isToolReadOnly(annotations?: McpToolAnnotations): boolean {
|
||||
return annotations?.readOnlyHint !== false;
|
||||
}
|
||||
78
apps/server/src/services/mcp-config.ts
Normal file
78
apps/server/src/services/mcp-config.ts
Normal file
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* v1.15.0-mcp-multi: MCP config file schema + loader.
|
||||
*
|
||||
* Reads a JSON config file (default `/data/mcp.json`) that declares MCP
|
||||
* servers — their transport type, connection parameters, and enabled state.
|
||||
* Schema shape matches opencode's `mcpServers` key for copy-paste compat.
|
||||
*/
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { z } from 'zod';
|
||||
import type { FastifyBaseLogger } from 'fastify';
|
||||
|
||||
// ---- Zod schema ----
|
||||
|
||||
const McpServerConfigSchema = z.discriminatedUnion('type', [
|
||||
z.object({
|
||||
type: z.literal('streamableHttp'),
|
||||
url: z.string().url(),
|
||||
headers: z.record(z.string()).optional(),
|
||||
enabled: z.boolean().default(true),
|
||||
}),
|
||||
z.object({
|
||||
type: z.literal('stdio'),
|
||||
command: z.string().min(1),
|
||||
args: z.array(z.string()).default([]),
|
||||
env: z.record(z.string()).optional(),
|
||||
enabled: z.boolean().default(true),
|
||||
}),
|
||||
]);
|
||||
|
||||
const McpConfigSchema = z.object({
|
||||
mcpServers: z.record(z.string(), McpServerConfigSchema).default({}),
|
||||
});
|
||||
|
||||
export type McpServerConfig = z.infer<typeof McpServerConfigSchema>;
|
||||
|
||||
export interface McpServerEntry {
|
||||
name: string;
|
||||
config: McpServerConfig;
|
||||
}
|
||||
|
||||
// ---- Loader ----
|
||||
|
||||
/**
|
||||
* Read and validate the MCP config file. Returns enabled servers only.
|
||||
* File missing → log info, return []. Parse/validation error → log warn, return [].
|
||||
*/
|
||||
export function loadMcpConfig(configPath: string, log: FastifyBaseLogger): McpServerEntry[] {
|
||||
let raw: string;
|
||||
try {
|
||||
raw = readFileSync(configPath, 'utf8');
|
||||
} catch {
|
||||
log.info(`mcp: config not found at ${configPath}, skipping`);
|
||||
return [];
|
||||
}
|
||||
|
||||
let json: unknown;
|
||||
try {
|
||||
json = JSON.parse(raw);
|
||||
} catch (err) {
|
||||
log.warn({ err }, `mcp: failed to parse ${configPath} as JSON`);
|
||||
return [];
|
||||
}
|
||||
|
||||
const result = McpConfigSchema.safeParse(json);
|
||||
if (!result.success) {
|
||||
log.warn({ errors: result.error.flatten().fieldErrors }, `mcp: invalid config at ${configPath}`);
|
||||
return [];
|
||||
}
|
||||
|
||||
const entries: McpServerEntry[] = [];
|
||||
for (const [name, config] of Object.entries(result.data.mcpServers)) {
|
||||
if (config.enabled) {
|
||||
entries.push({ name, config });
|
||||
}
|
||||
}
|
||||
|
||||
return entries;
|
||||
}
|
||||
@@ -21,6 +21,10 @@ import {
|
||||
watchChanges,
|
||||
getSemanticNeighborhoods,
|
||||
getFrameworkAnalysis,
|
||||
getBlastRadius,
|
||||
getHotFiles,
|
||||
getRoutes,
|
||||
getMiddleware,
|
||||
} from './tools/codecontext/index.js';
|
||||
// v1.13.17-cross-repo-reads: cross-repo read grant request tool. Paired
|
||||
// with the pause-on-pending-grant branch in inference/tool-phase.ts and the
|
||||
@@ -651,7 +655,9 @@ export const askUserInput: ToolDef<AskUserInputInputT> = {
|
||||
// of the system prompt, so any order drift would invalidate every cached
|
||||
// turn. Single source of truth for ordering lives here — toolJsonSchemas()
|
||||
// and TOOLS_BY_NAME inherit it.
|
||||
export const ALL_TOOLS: ReadonlyArray<ToolDef<unknown>> = [
|
||||
// v1.14.1-mcp-poc: changed from ReadonlyArray to let-bound mutable array
|
||||
// so appendMcpTools() can push MCP-discovered tools at startup.
|
||||
export let ALL_TOOLS: ToolDef<unknown>[] = [
|
||||
viewFile as ToolDef<unknown>,
|
||||
viewTruncatedOutput as ToolDef<unknown>,
|
||||
listDir as ToolDef<unknown>,
|
||||
@@ -678,6 +684,11 @@ export const ALL_TOOLS: ReadonlyArray<ToolDef<unknown>> = [
|
||||
watchChanges as ToolDef<unknown>,
|
||||
getSemanticNeighborhoods as ToolDef<unknown>,
|
||||
getFrameworkAnalysis as ToolDef<unknown>,
|
||||
// v1.16: codesight-merge tools. Backed by the same codecontext sidecar.
|
||||
getBlastRadius as ToolDef<unknown>,
|
||||
getHotFiles as ToolDef<unknown>,
|
||||
getRoutes as ToolDef<unknown>,
|
||||
getMiddleware as ToolDef<unknown>,
|
||||
// v1.13.17-cross-repo-reads: paired with the pause-on-pending-grant
|
||||
// branch in tool-phase.ts. Read-only — only ever READS files; the only
|
||||
// state change is appending to sessions.allowed_read_paths via the
|
||||
@@ -725,10 +736,23 @@ export const READ_ONLY_TOOL_NAMES = [
|
||||
'request_read_access',
|
||||
] as const;
|
||||
|
||||
export const TOOLS_BY_NAME: Record<string, ToolDef<unknown>> = Object.fromEntries(
|
||||
export let TOOLS_BY_NAME: Record<string, ToolDef<unknown>> = Object.fromEntries(
|
||||
ALL_TOOLS.map((t) => [t.name, t])
|
||||
);
|
||||
|
||||
// v1.14.1-mcp-poc: append MCP-discovered tools at startup. Called once
|
||||
// from index.ts after mcpClient.initialize(). Re-sorts ALL_TOOLS and
|
||||
// rebuilds TOOLS_BY_NAME. READ_ONLY_TOOL_NAMES is not rebuilt because
|
||||
// it's a const tuple used only for budget-tier checks; MCP tools are
|
||||
// individually checked via their category at budget resolution time —
|
||||
// they are all read_only by construction (the read-only guard in
|
||||
// mcp-client.ts rejects any tool with readOnlyHint: false).
|
||||
export function appendMcpTools(mcpTools: ToolDef<unknown>[]): void {
|
||||
if (mcpTools.length === 0) return;
|
||||
ALL_TOOLS = [...ALL_TOOLS, ...mcpTools].sort((a, b) => a.name.localeCompare(b.name));
|
||||
TOOLS_BY_NAME = Object.fromEntries(ALL_TOOLS.map((t) => [t.name, t]));
|
||||
}
|
||||
|
||||
// v1.13.15-tools: tiered tool loading. BOOCODE_TOOLS env var (`core` |
|
||||
// `standard` | `all`) filters the agent's tool whitelist before LLM dispatch.
|
||||
// Daily-driver token win on qwen3.6-35b-a3b — the 35B-A3B MoE benefits from
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import { z } from 'zod';
|
||||
import type { ToolDef } from '../../tools.js';
|
||||
import { callCodecontext, type CodecontextResponse } from '../../codecontext_client.js';
|
||||
|
||||
export const GetBlastRadiusInput = z.object({
|
||||
file_path: z.string().trim().min(1),
|
||||
});
|
||||
export type GetBlastRadiusInputT = z.infer<typeof GetBlastRadiusInput>;
|
||||
|
||||
const DESCRIPTION =
|
||||
'Returns all files that depend (transitively) on the given file, with depth tracking. ' +
|
||||
'Use to assess the impact of changing a file — "what breaks if I modify this?" ' +
|
||||
'Traverses the import graph in reverse via BFS. Results sorted by distance (closest dependents first).';
|
||||
|
||||
export async function executeGetBlastRadius(
|
||||
input: GetBlastRadiusInputT,
|
||||
projectPath: string,
|
||||
fetcher: typeof fetch = fetch,
|
||||
): Promise<CodecontextResponse> {
|
||||
return callCodecontext(
|
||||
{ toolName: 'get_blast_radius', args: { file_path: input.file_path }, projectPath },
|
||||
fetcher,
|
||||
);
|
||||
}
|
||||
|
||||
export const getBlastRadius: ToolDef<GetBlastRadiusInputT> = {
|
||||
name: 'get_blast_radius',
|
||||
description: DESCRIPTION,
|
||||
inputSchema: GetBlastRadiusInput,
|
||||
jsonSchema: {
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'get_blast_radius',
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
file_path: {
|
||||
type: 'string',
|
||||
description: 'Absolute or project-relative path to the file to analyze.',
|
||||
},
|
||||
},
|
||||
required: ['file_path'],
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
async execute(input, projectRoot) {
|
||||
return await executeGetBlastRadius(input, projectRoot);
|
||||
},
|
||||
};
|
||||
50
apps/server/src/services/tools/codecontext/get_hot_files.ts
Normal file
50
apps/server/src/services/tools/codecontext/get_hot_files.ts
Normal file
@@ -0,0 +1,50 @@
|
||||
import { z } from 'zod';
|
||||
import type { ToolDef } from '../../tools.js';
|
||||
import { callCodecontext, type CodecontextResponse } from '../../codecontext_client.js';
|
||||
|
||||
export const GetHotFilesInput = z.object({
|
||||
limit: z.number().int().min(1).max(100).optional(),
|
||||
});
|
||||
export type GetHotFilesInputT = z.infer<typeof GetHotFilesInput>;
|
||||
|
||||
const DESCRIPTION =
|
||||
'Returns the most-imported files in the project, ranked by incoming import count. ' +
|
||||
'Hot files are high-risk change targets — many other files depend on them. ' +
|
||||
'Use to identify core modules and assess refactoring risk.';
|
||||
|
||||
export async function executeGetHotFiles(
|
||||
input: GetHotFilesInputT,
|
||||
projectPath: string,
|
||||
fetcher: typeof fetch = fetch,
|
||||
): Promise<CodecontextResponse> {
|
||||
return callCodecontext(
|
||||
{ toolName: 'get_hot_files', args: input.limit != null ? { limit: input.limit } : {}, projectPath },
|
||||
fetcher,
|
||||
);
|
||||
}
|
||||
|
||||
export const getHotFiles: ToolDef<GetHotFilesInputT> = {
|
||||
name: 'get_hot_files',
|
||||
description: DESCRIPTION,
|
||||
inputSchema: GetHotFilesInput,
|
||||
jsonSchema: {
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'get_hot_files',
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
limit: {
|
||||
type: 'number',
|
||||
description: 'Maximum number of files to return (default 20, max 100).',
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
async execute(input, projectRoot) {
|
||||
return await executeGetHotFiles(input, projectRoot);
|
||||
},
|
||||
};
|
||||
41
apps/server/src/services/tools/codecontext/get_middleware.ts
Normal file
41
apps/server/src/services/tools/codecontext/get_middleware.ts
Normal file
@@ -0,0 +1,41 @@
|
||||
import { z } from 'zod';
|
||||
import type { ToolDef } from '../../tools.js';
|
||||
import { callCodecontext, type CodecontextResponse } from '../../codecontext_client.js';
|
||||
|
||||
export const GetMiddlewareInput = z.object({});
|
||||
export type GetMiddlewareInputT = z.infer<typeof GetMiddlewareInput>;
|
||||
|
||||
const DESCRIPTION =
|
||||
'Detects middleware registrations in the project. Identifies auth, CORS, rate-limit, ' +
|
||||
'security-headers, error-handler, logging, and validation middleware by analyzing ' +
|
||||
'import names (@fastify/cors, helmet, etc.) and registration patterns ' +
|
||||
'(app.register, app.addHook, app.setErrorHandler).';
|
||||
|
||||
export async function executeGetMiddleware(
|
||||
_input: GetMiddlewareInputT,
|
||||
projectPath: string,
|
||||
fetcher: typeof fetch = fetch,
|
||||
): Promise<CodecontextResponse> {
|
||||
return callCodecontext({ toolName: 'get_middleware', args: {}, projectPath }, fetcher);
|
||||
}
|
||||
|
||||
export const getMiddleware: ToolDef<GetMiddlewareInputT> = {
|
||||
name: 'get_middleware',
|
||||
description: DESCRIPTION,
|
||||
inputSchema: GetMiddlewareInput,
|
||||
jsonSchema: {
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'get_middleware',
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {},
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
async execute(input, projectRoot) {
|
||||
return await executeGetMiddleware(input, projectRoot);
|
||||
},
|
||||
};
|
||||
50
apps/server/src/services/tools/codecontext/get_routes.ts
Normal file
50
apps/server/src/services/tools/codecontext/get_routes.ts
Normal file
@@ -0,0 +1,50 @@
|
||||
import { z } from 'zod';
|
||||
import type { ToolDef } from '../../tools.js';
|
||||
import { callCodecontext, type CodecontextResponse } from '../../codecontext_client.js';
|
||||
|
||||
export const GetRoutesInput = z.object({
|
||||
framework: z.string().trim().optional(),
|
||||
});
|
||||
export type GetRoutesInputT = z.infer<typeof GetRoutesInput>;
|
||||
|
||||
const DESCRIPTION =
|
||||
'Extracts HTTP routes from the project via tree-sitter AST analysis. ' +
|
||||
'Detects Fastify and Express route registrations (app.get, app.post, app.route, router.use, etc.) ' +
|
||||
'with method, path, file, line number, and inferred tags (db, auth, cache). ' +
|
||||
'Optional framework filter narrows to "fastify" or "express".';
|
||||
|
||||
export async function executeGetRoutes(
|
||||
input: GetRoutesInputT,
|
||||
projectPath: string,
|
||||
fetcher: typeof fetch = fetch,
|
||||
): Promise<CodecontextResponse> {
|
||||
const args: Record<string, unknown> = {};
|
||||
if (input.framework) args.framework = input.framework;
|
||||
return callCodecontext({ toolName: 'get_routes', args, projectPath }, fetcher);
|
||||
}
|
||||
|
||||
export const getRoutes: ToolDef<GetRoutesInputT> = {
|
||||
name: 'get_routes',
|
||||
description: DESCRIPTION,
|
||||
inputSchema: GetRoutesInput,
|
||||
jsonSchema: {
|
||||
type: 'function',
|
||||
function: {
|
||||
name: 'get_routes',
|
||||
description: DESCRIPTION,
|
||||
parameters: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
framework: {
|
||||
type: 'string',
|
||||
description: 'Filter to a specific framework: "fastify" or "express". Omit for all.',
|
||||
},
|
||||
},
|
||||
additionalProperties: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
async execute(input, projectRoot) {
|
||||
return await executeGetRoutes(input, projectRoot);
|
||||
},
|
||||
};
|
||||
@@ -1,5 +1,5 @@
|
||||
// v1.12 Track B.2: codecontext tool registry. Re-exports the 8 ToolDefs so
|
||||
// tools.ts can pull them in one line.
|
||||
// codecontext tool registry. Re-exports ToolDefs so tools.ts can pull them
|
||||
// in one line. v1.12: 8 original tools. v1.16: +4 codesight-merge tools.
|
||||
|
||||
export { getCodebaseOverview } from './get_codebase_overview.js';
|
||||
export { getFileAnalysis } from './get_file_analysis.js';
|
||||
@@ -9,3 +9,7 @@ export { getDependencies } from './get_dependencies.js';
|
||||
export { watchChanges } from './watch_changes.js';
|
||||
export { getSemanticNeighborhoods } from './get_semantic_neighborhoods.js';
|
||||
export { getFrameworkAnalysis } from './get_framework_analysis.js';
|
||||
export { getBlastRadius } from './get_blast_radius.js';
|
||||
export { getHotFiles } from './get_hot_files.js';
|
||||
export { getRoutes } from './get_routes.js';
|
||||
export { getMiddleware } from './get_middleware.js';
|
||||
|
||||
@@ -106,6 +106,9 @@ export interface Agent {
|
||||
// agent's toolset (30 if all tools are read-only, 10 otherwise) or 15 for
|
||||
// raw chat with no agent.
|
||||
max_tool_calls: number | null;
|
||||
// v1.14.0: per-agent step cap for the outer inference loop. null means
|
||||
// bounded only by MAX_STEPS (200). 0 means "no tool calls allowed."
|
||||
steps: number | null;
|
||||
}
|
||||
|
||||
// One entry per malformed `## Name` block. Per-block errors don't fail the
|
||||
|
||||
@@ -73,6 +73,9 @@ export interface Agent {
|
||||
// the agent's toolset (30 for all read-only, 10 otherwise) or 15 for raw
|
||||
// chat with no agent.
|
||||
max_tool_calls: number | null;
|
||||
// v1.14.0: per-agent step cap for the outer inference loop. null means
|
||||
// bounded only by MAX_STEPS (200). 0 means "no tool calls allowed."
|
||||
steps: number | null;
|
||||
}
|
||||
|
||||
export interface AgentParseError {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# BooCode v1.x — Roadmap
|
||||
|
||||
Last updated: 2026-05-22
|
||||
Last updated: 2026-05-23
|
||||
|
||||
> **Companion doc:** `boocode_code_review.md` holds the full external-repo inventory, lift rationale, and license analysis. This document is the canonical source for shipping state, version ordering, and what's planned vs. shipped.
|
||||
|
||||
@@ -27,7 +27,7 @@ External code lifted from / referenced in: see `boocode_code_review.md` for full
|
||||
|
||||
-----
|
||||
|
||||
## Shipped (status as of 2026-05-22)
|
||||
## Shipped (status as of 2026-05-23)
|
||||
|
||||
|Version |Theme |Tag |
|
||||
|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------|
|
||||
@@ -72,9 +72,9 @@ External code lifted from / referenced in: see `boocode_code_review.md` for full
|
||||
|
||||
-----
|
||||
|
||||
### Shipped (v1.13.x — written 2026-05-22, retagged same day)
|
||||
### Shipped (v1.13.x — strangler-fig closed 2026-05-23)
|
||||
|
||||
All v1.13.x batches were retagged to the `vMAJOR.MINOR.PATCH-slug` scheme on 2026-05-22. `CHANGELOG.md` is the canonical per-tag record (slug describes what shipped; tag name alone recalls the batch). Tip is `v1.13.14-skills-audit` (`0fa46cd`); the next batch is `v1.13.15-codecontext-synth` (this batch, tag pending). Tags in chronological order:
|
||||
All v1.13.x batches use the `vMAJOR.MINOR.PATCH-slug` tag scheme adopted 2026-05-22. `CHANGELOG.md` is the canonical per-tag record (slug describes what shipped; tag name alone recalls the batch). The v1.13.x line ran 21 batches over a single intense window; the umbrella `v1.13` tag sits on `211e903` (same commit as `v1.13.20-drop-legacy-cols`), marking the strangler-fig closed. Tags in chronological order:
|
||||
|
||||
- `v1.13.0-ai-sdk-v6` — AI SDK v6 migration; `streamCompletion` adapter; `messages_with_parts` view; reasoning_parts end-to-end
|
||||
- `v1.13.1-cleanup-bundle` — `statement_timeout='30s'`, alpha-sorted tool registry, 60s stuck-row sweeper, `experimental_repairToolCall` pass-through
|
||||
@@ -93,115 +93,13 @@ All v1.13.x batches were retagged to the `vMAJOR.MINOR.PATCH-slug` scheme on 202
|
||||
- `v1.13.14-skills-audit` — 26 skills vendored + audited via 5 parallel agent teams; 14 kept, 11 dropped, 1 migrated to BOOCHAT.md/BOOCODER.md
|
||||
- `v1.13.15-codecontext-synth` — forced second-inference synthesis pass for codecontext overview tools (truncation-aware extraction; auto-fetched top-N files + project docs; 32k payload-budget contract preserved)
|
||||
- `v1.13.16-xml-parser` — Anthropic `<invoke>` parser support + Levenshtein-based unknown-tool recovery hints (qwen3.6 drift to Claude Code-style tool names like `read_file`); xml-parser test coverage
|
||||
- `v1.13.17-cross-repo-reads` — `request_read_access` tool + per-session `allowed_read_paths` grants; `pathGuard` extended with `extraRoots`; pause/resume reuses the `ask_user_input` mechanism
|
||||
- `v1.13.18-codecontext-file-path` — `resolveProjectPath` in `codecontext_client.ts` realpath-resolves `file_path` arg the same way `target_dir` was; closes the silent-fail path the sidecar exhibited on relative paths
|
||||
- `v1.13.19-html-artifact-panes` — pane-based artifact viewer with on-request HTML; `<!DOCTYPE html>` detection adds `message_parts.kind='html_artifact'` row; Markdown + HTML panes both open via "Open in pane" affordance; iframe sandbox `allow-scripts allow-clipboard-write allow-downloads` (no `allow-same-origin`, `srcDoc`); CSP `connect-src 'none'`. Scope-revised mid-design from auto-bias-to-HTML to Markdown-default / HTML-on-request
|
||||
- `v1.13.20-drop-legacy-cols` — final strangler-fig step. Drops `messages.tool_calls` + `tool_results` columns; 10 dual-write sites removed (recon caught 2 beyond the original roadmap inventory); `messages_with_parts` view simplified to parts-only subselects via `CREATE OR REPLACE` before the column DROPs (Postgres ordering constraint). Adversarial-review catch: `discard_stale` had a `RETURNING tool_calls, tool_results` clause; fixed via two-step UPDATE-then-SELECT-from-view. `Message` API type retains the fields — view synthesizes them from parts so the wire shape is unchanged
|
||||
- `v1.13` — **umbrella tag on the same commit as v1.13.20.** Marks the AI SDK v6 + parts-table migration complete
|
||||
|
||||
The remaining strangler-fig final step (drop `messages.tool_calls` + `tool_results` columns) is still pending under its old `v1.13.2` working name; will get a new tag slug when scoped.
|
||||
|
||||
## In flight / next (v1.13.x cleanup line)
|
||||
|
||||
Five more single-dispatch batches before the strangler-fig closes. Each ships independently with its own smoke and rollback surface. **Do not fold.** Order is locked:
|
||||
|
||||
### v1.13.8 — system-prompt prefix stability verify-and-measure (REFRAMED, 2026-05-22)
|
||||
|
||||
**Original plan:** add a `system_prompt_cache` DB table keyed by `(agent_id, project_id, skills_version)`, mtime-invalidated.
|
||||
|
||||
**Why reframed:** recon disproved the premise. `apps/server/src/services/system-prompt.ts:buildSystemPrompt` already runs over mtime-cached inputs at the file layer:
|
||||
|
||||
- BOOCHAT.md / BOOCODER.md cached in `system-prompt.ts:25` (`cachedGuidance`, keyed by mtime)
|
||||
- global + per-project AGENTS.md cached in `agents.ts:245` (`safeStat` pattern, 60s TTL)
|
||||
- `session.system_prompt` / `project.default_system_prompt` are DB scalars (byte-stable until edited)
|
||||
- BASE_SYSTEM_PROMPT is a hardcoded template with `${projectPath}` interpolation
|
||||
|
||||
Output assembly is a microsecond pure-string concat with no I/O. Skills aren't in the prefix (runtime discovery via `skill_find`). Tools live in a separate request body field, alpha-sorted by v1.13.3. **In theory the prefix is already byte-stable across turns; nothing has measured it.**
|
||||
|
||||
**New scope — instrumentation only, no cache:**
|
||||
|
||||
1. SHA-256 fingerprint of `buildSystemPrompt`'s output logged per turn at `level=info`, msg `prefix-fingerprint`, with project_id / agent_id / session_id / prefix_hash / prefix_length / mtime fields.
|
||||
2. Module-level `Map<sessionId, lastHash>` observer. On hash change for a known session → emit `prefix-drift` at `level=warn` with `prev_hash`, `new_hash`, and a field-level `changed_inputs` diff.
|
||||
3. Unit-level byte-stability assertion in `system-prompt.test.ts`: two consecutive `buildSystemPrompt` calls with the same inputs return byte-identical strings.
|
||||
|
||||
**Decision criterion:** smoke 5 turns in a fresh session. 5 identical hashes + zero drift logs → close v1.13.8 as no-op, **drop the DB cache plan permanently**, move to v1.13.9. If drift surfaces → characterize the failure mode in a follow-up batch (the answer may not be a cache at all).
|
||||
|
||||
**Doctrine:** matches the v1.13.6 audit pattern. Don't add infrastructure without a proven cache miss. The v1.12.0 mtime caches at the input layer plus alpha tool ordering at the request body layer already address the load-bearing cache-stability surfaces.
|
||||
|
||||
**Dispatch brief:** `handoff_v1.13.8_prefix_verify.md`.
|
||||
|
||||
**Estimated:** ~95 LoC (system-prompt.ts + small `getAgentsMtimes` accessor in agents.ts + 3 new tests).
|
||||
|
||||
### v1.13.9 — compaction overflow trigger formula
|
||||
|
||||
opencode pattern: `0.85 * ctx_max` early trigger (not at 100% saturation). Reduces tail-loss risk and gives compaction a safer window. Tiny change but tied to v1.13.4's tier logic — sequence matters.
|
||||
|
||||
**Lift source:** `anomalyco/opencode` `session/overflow.ts`.
|
||||
|
||||
**Estimated:** ~30 LoC.
|
||||
|
||||
### v1.13.10 — per-tool token cost accounting
|
||||
|
||||
Rolling average per tool, surfaced in AgentPicker tooltip + agent-pick decisions. Backend tracks `(tool_name, prompt_tokens_in, completion_tokens_out)` per call; surfaces a 100-call rolling mean. Frontend reads it for tool-cost hints. **Depends on v1.13.7's `includeUsage` fix** — without real token numbers in DB rows, the rolling average is empty.
|
||||
|
||||
**Estimated:** ~250 LoC.
|
||||
|
||||
### v1.13.11 — WebSocket frame typing
|
||||
|
||||
Zod schemas validated both ends. Catches the recurring class of bug that drove the 2026-05-21 debugging spike (silent protocol drift). Upfront work that pays back every time the protocol changes. `chat_status`, `usage`, `parts_appended`, `session_workspace_updated`, `tool_running` — every frame gets a Zod schema, every send/receive site validates.
|
||||
|
||||
**Estimated:** ~300 LoC.
|
||||
|
||||
### v1.13.12 — skills audit pass (NEW, 2026-05-22)
|
||||
|
||||
**Goal:** apply the rules→recipes split (per Codeminer42 activation-gap data: plain skills invoke 6% in clean multi-turn, `CLAUDE.md`/`AGENTS.md` is 100% present) to BooCode's 7 vendored v1.12 skills. Sort each into: (a) move to `AGENTS.md` as always-true rule, (b) keep as recipe invoked via `/skill <name>`, (c) move bulky context into `references/` flat subdirectory inside the skill, (d) delete (Claude already does it reliably).
|
||||
|
||||
**Scope:**
|
||||
|
||||
1. **Audit each of the 7 vendored skills against the 4-way split.** Most workflow-rule content ("always do X before Y", "never do Z") moves to `AGENTS.md` since it should be 100% present. Recipe content ("here's how to scaffold a component", "here's the release checklist") stays as skill, gets `context: fork` if heavy.
|
||||
1. **Adopt Anthropic best-practices conventions** for any skills that remain after audit: gerund names (`scaffolding-components`, not `component-helper`), SKILL.md ≤500 lines, references one level deep, third-person imperative voice, MCP tool references in `ServerName:tool_name` format, no Windows-style paths, no time-sensitive info, consistent terminology, no "voodoo constants."
|
||||
1. **Run each remaining skill through the 4-step validation protocol** from `mgechev/skills-best-practices` (Discovery → Logic → Edge Case → Architecture Refinement) using a fresh Claude chat per step. Prompts are paste-ready; ~10 minutes per skill.
|
||||
1. **Install `skillgrade` on Sam's host** (`npm i -g skillgrade`). For each remaining skill, write a minimal `eval.yaml` with 2–3 tasks and run `skillgrade --smoke` (5 trials, ~5 min) to confirm the skill triggers when expected and produces correct output. **Likely outcome: some skills show 0–20% trigger rate — confirms they belong in AGENTS.md, not as skills.**
|
||||
1. **Document the rules→recipes split as a BooCode convention** in `BOOCODER.md` / `BOOCHAT.md`. Future-proofs against re-adding workflow rules as skills.
|
||||
|
||||
**Lift sources:**
|
||||
|
||||
- `blog.codeminer42.com/stop-putting-best-practices-in-skills/` — empirical 6%/33%/66%/100% invocation-rate data with Vercel-style multi-turn methodology. The activation-gap framing.
|
||||
- `mgechev/skills-best-practices` (25 stars, MIT) — 4-step validation protocol with paste-ready prompts. Directory structure conventions.
|
||||
- `mgechev/skillgrade` (132 stars, MIT) — agent-agnostic skill eval framework. `eval.yaml` task+grader schema. Smoke/reliable/regression presets.
|
||||
- `platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices` — canonical Anthropic standard. 500-line ceiling, gerund naming, progressive disclosure patterns, MCP tool reference format, verification checklist.
|
||||
|
||||
**Dependencies:** none (the 7 v1.12 skills already exist; this is an audit pass on shipped material). Can ship at any point in the v1.13.x line.
|
||||
|
||||
**Estimated:** zero code changes, ~one evening of audit work, plus skillgrade install. Per-skill eval.yaml authoring is ~30 min per skill including the 4-step validation. Total roughly 5–6 hours of focused work for all 7 skills.
|
||||
|
||||
### v1.13.2 — drop legacy columns (final phase of strangler-fig)
|
||||
|
||||
**Wait at least one week of production traffic on v1.13.1 before shipping.** The dual-write is rollback insurance. Drop the columns and that rollback is gone.
|
||||
|
||||
**Verification query before shipping:**
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
COUNT(*) FILTER (WHERE m.tool_calls IS NOT NULL AND NOT EXISTS (
|
||||
SELECT 1 FROM message_parts p WHERE p.message_id = m.id AND p.kind = 'tool_call'
|
||||
)) AS missing_tool_call_parts,
|
||||
COUNT(*) FILTER (WHERE m.tool_results IS NOT NULL AND NOT EXISTS (
|
||||
SELECT 1 FROM message_parts p WHERE p.message_id = m.id AND p.kind = 'tool_result'
|
||||
)) AS missing_tool_result_parts
|
||||
FROM messages m
|
||||
WHERE m.created_at > '2026-05-22'::timestamptz;
|
||||
```
|
||||
|
||||
Both columns must read 0.
|
||||
|
||||
**Scope (~150 LoC, mostly deletions):**
|
||||
|
||||
1. Remove dual-write from every v1.13.0 site: `tool-phase.ts` (3 sites), `finalizeCompletion`, `skills.ts` (2 sites), `messages.ts` answer flow, `chats.ts` (fork). Keep only the parts write.
|
||||
1. Simplify `messages_with_parts` view — drop COALESCE fallbacks since legacy columns are about to disappear.
|
||||
1. `ALTER TABLE messages DROP COLUMN tool_calls, DROP COLUMN tool_results`.
|
||||
1. Remove `tool_calls`/`tool_results` fields from `Message` API type. API boundary unchanged (frontend already reads parts-derived values).
|
||||
1. Drop the stale `messages_status_check` cleanup DO block from v1.12.1 schema if still present.
|
||||
1. Update test fixtures in `inference.test.ts` and `compaction.test.ts` to construct parts instead of inline `tool_calls: null, tool_results: null` literals. ~30 fixture rewrites.
|
||||
|
||||
After v1.13.2 ships, tag the umbrella `v1.13` on the same commit (or on -C — Sam's call).
|
||||
|
||||
**Shipped as `v1.13.20-drop-legacy-cols` on 2026-05-23 with umbrella `v1.13` tagged on the same commit.** Slug renamed at ship time (the "v1.13.2" planning name predated the patch-monotonic-per-minor convention). Calendar wait dropped — single-user self-hosted, no production rollback constraint. Recon caught 2 additional dual-write sites beyond the roadmap's 8 (chats.ts fork-clone + extras in tool-phase.ts) and an additional fixture file (`tool_cost_stats.test.ts`) with a direct legacy-column INSERT. Adversarial review caught a `RETURNING tool_calls, tool_results` clause in the `discard_stale` endpoint that the green test suite missed — fixed by two-step UPDATE-then-SELECT-from-view so the parts-synthesized fields keep flowing on the response. Type-pruning step on `Message.tool_calls` / `Message.tool_results` skipped (the view still populates them from parts; preserving the API contract was simpler than ripping it).
|
||||
The v1.13.x line is closed. Three batches still sit in the **In flight** column conceptually but none of them are v1.13.x scope: **live-smoke of v1.13.19** (manual browser exercise of the artifact panes — five minutes, independent), and the two v1.14 branches below. Independent siblings (`v1.14.x-mcp`, `v1.14.x-html`, `v1.16`) can ship in any order relative to v1.14 itself.
|
||||
|
||||
-----
|
||||
|
||||
@@ -510,8 +408,12 @@ term.indifferentketchup.com → booterm :9501 (or routed under code.
|
||||
- **v1.13.12-ws-schemas:** none (Zod schemas + wrappers in TS, no DB)
|
||||
- **v1.13.13-ws-publish:** none (publish-site conversion + protocol-drift fix in `compaction.ts`, no DB)
|
||||
- **v1.13.14-skills-audit:** none (skills + AGENTS.md migration into git via `.gitignore` negation patterns; no DB)
|
||||
- **v1.13.15-codecontext-synth (this batch, tag pending):** `message_parts.kind` CHECK constraint extended with `'synthesis'` value (DROP + DO $$ pg_constraint idempotency-guarded re-add)
|
||||
- **(column drop, pending — old working name v1.13.2):** drop `messages.tool_calls`, `messages.tool_results`; simplify `messages_with_parts` view
|
||||
- **v1.13.15-codecontext-synth:** `message_parts.kind` CHECK constraint extended with `'synthesis'` value (DROP + DO $$ pg_constraint idempotency-guarded re-add)
|
||||
- **v1.13.16-xml-parser:** none (parser change + new `tool-suggestions.ts` helper in TS, no DB)
|
||||
- **v1.13.17-cross-repo-reads:** `sessions.allowed_read_paths text[] NOT NULL DEFAULT ARRAY[]::text[]` (per-session cross-repo read grants)
|
||||
- **v1.13.18-codecontext-file-path:** none (path resolver in `codecontext_client.ts`, no DB)
|
||||
- **v1.13.19-html-artifact-panes:** `message_parts.kind` CHECK constraint extended with `'html_artifact'` value (same v1.13.15 pattern)
|
||||
- **v1.13.20-drop-legacy-cols:** `ALTER TABLE messages DROP COLUMN tool_calls, DROP COLUMN tool_results` (the strangler-fig's final phase). `messages_with_parts` view rewritten to parts-only subselects via `CREATE OR REPLACE VIEW` BEFORE the drops (Postgres ordering constraint). v1.12.1 `messages_status_check`/`messages_role_check` cleanup block removed (one-shot effective long ago)
|
||||
- **v1.14:** `agents.steps` column (or AGENTS.md parser extension; no DB if file-only)
|
||||
- **v1.14.x-mcp (NEW):** none — single-server MCP-client PoC is config-only at first, no schema change
|
||||
- **v1.14.x-html (NEW):** `message_parts.kind` CHECK constraint extended with `'html_artifact'` value
|
||||
@@ -621,7 +523,17 @@ Earlier May 18 chat recommended Option A (thin orchestration shell over OpenCode
|
||||
|
||||
### v1.13.x cleanup line locked (2026-05-22)
|
||||
|
||||
After the 2026-05-22 retag, the v1.13.x cleanup line in `vMAJOR.MINOR.PATCH-slug` form is **v1.13.0-ai-sdk-v6 ✅ → v1.13.1-cleanup-bundle ✅ → v1.13.2-compaction-prune ✅ → v1.13.3-truncate ✅ → v1.13.4-reasoning-fix ✅ → v1.13.5-stability-bundle ✅ → v1.13.6-prefix-stability ✅ → v1.13.7-compaction-trigger ✅ → v1.13.8-tool-cost ✅ → v1.13.9-agentlint ✅ → v1.13.10-openspec ✅ → v1.13.11-tools ✅ → v1.13.12-ws-schemas ✅ → v1.13.13-ws-publish ✅ → v1.13.14-skills-audit ✅ → v1.13.15-codecontext-synth ✅ → v1.13.16-xml-parser ✅ → column drop (final, pending — old working name v1.13.2)**. **Do not fold.** Smoke isolation matters: each batch has a distinct rollback surface, and bisecting a 750-LoC merge across four unrelated changes is worse than four separate dispatches.
|
||||
The v1.13.x cleanup line shipped 21 batches over a single intense window in `vMAJOR.MINOR.PATCH-slug` form: **v1.13.0-ai-sdk-v6 ✅ → v1.13.1-cleanup-bundle ✅ → v1.13.2-compaction-prune ✅ → v1.13.3-truncate ✅ → v1.13.4-reasoning-fix ✅ → v1.13.5-stability-bundle ✅ → v1.13.6-prefix-stability ✅ → v1.13.7-compaction-trigger ✅ → v1.13.8-tool-cost ✅ → v1.13.9-agentlint ✅ → v1.13.10-openspec ✅ → v1.13.11-tools ✅ → v1.13.12-ws-schemas ✅ → v1.13.13-ws-publish ✅ → v1.13.14-skills-audit ✅ → v1.13.15-codecontext-synth ✅ → v1.13.16-xml-parser ✅ → v1.13.17-cross-repo-reads ✅ → v1.13.18-codecontext-file-path ✅ → v1.13.19-html-artifact-panes ✅ → v1.13.20-drop-legacy-cols ✅** → umbrella `v1.13` ✅. **Do not fold** was the discipline — each batch has a distinct rollback surface, and bisecting a 750-LoC merge across four unrelated changes is worse than four separate dispatches. Held throughout; CHANGELOG.md is the per-tag canonical record.
|
||||
|
||||
### Numbering and scope-revision discipline during v1.13.x (2026-05-23)
|
||||
|
||||
The v1.13.x line ran 21 batches; planned-vs-shipped numbering diverged for half of them, and three batches had material scope revisions mid-design. Pattern that emerged and is worth carrying forward:
|
||||
|
||||
- **Patch numbers are assigned at ship time, not in planning.** The proposal/openspec folder uses a planning slug (e.g. `v1.14.x-html-artifact-panes`); the final tag uses a concrete patch monotonic-per-minor (e.g. `v1.13.19-html-artifact-panes`). Avoids the "we said v1.13.8 but actually shipped seventh" confusion that ate two retrospective passes on the roadmap.
|
||||
- **Scope-revise the proposal before dispatching.** v1.13.19-html-artifact-panes flipped mid-design from "auto-bias to HTML for >100 lines" to "Markdown default, HTML on request" — the proposal got rewritten before recon. Far cheaper than discovering the wrong approach in implementation. The "brainstorm before code" discipline.
|
||||
- **Recon-first dispatch finds 25–30% more sites than the roadmap inventory.** v1.13.20 recon caught 2 extra dual-write sites (chats.ts fork-clone + 2 in tool-phase.ts) and an extra fixture file. v1.13.19 recon corrected which `Pane` type to extend. Skipping recon to save a step doesn't.
|
||||
- **Adversarial reviews catch what test suites miss.** v1.13.19 reviewer caught silent error-promotion in `openInPane`; v1.13.20 reviewer caught a `RETURNING tool_calls, tool_results` clause that crashes in production but slips past green tests. Both are routine code-reviewer dispatches; both saved a same-day hotfix. **Two-stage review (spec then quality) is non-negotiable when shipping fast.**
|
||||
- **Calendar-gated waits are production-safety hedges that don't apply here.** v1.13.20 originally said "wait one week of production traffic on v1.13.1 before dropping columns." Sam called it out: single-user self-hosted, no rollback constraint, code-level audit + DB COUNT query is the actual safety check. Dropped the wait. Don't ritualize production-grade hedges in a single-user codebase.
|
||||
|
||||
### v1.13 retrospective (what shipped)
|
||||
|
||||
@@ -634,7 +546,21 @@ After the 2026-05-22 retag, the v1.13.x cleanup line in `vMAJOR.MINOR.PATCH-slug
|
||||
- **v1.13.5** — opencode truncate.ts port + view_truncated_output tool. Tagged on `f8fc5db`.
|
||||
- **v1.13.6** — compaction head-assembly audit + reasoning fix. Closed the Q3 reasoning gap from v1.13.1-C. Tagged on `81d837c`.
|
||||
- **v1.13.7** — stability bundle: includeUsage fix + trim guards + payload filter + budget bump. Surfaces tokens (closes a v1.13.1-A latent regression where `result.usage` resolved empty), kills the empty-bubble + ActionRow noise between tool calls on single-tool-call turns, and unblocks Continue after cap-hit on chats that have trailing empty/failed assistants.
|
||||
- **v1.13.2 deferred** — at least one week of production traffic on v1.13.1 before dropping legacy columns. Dual-write is rollback insurance.
|
||||
- **v1.13.6 (numbering re-aligned)** — system-prompt prefix verify-and-measure batch (originally numbered v1.13.8 in the planning doc). Reframed mid-design from "add a `system_prompt_cache` table" to "instrument-and-prove" after recon showed input-layer mtime caches already achieve byte-stable prefixes. Smoke confirmed zero drift across 5 turns; dropped the planned DB table.
|
||||
- **v1.13.7-compaction-trigger** — 0.85×ctx_max early trigger (planned as v1.13.8 / v1.13.9).
|
||||
- **v1.13.8-tool-cost** — `tool_cost_stats` SQL view + AgentPicker tooltip surfacing (planned as v1.13.9 / v1.13.10).
|
||||
- **v1.13.9-agentlint** — instruction-file AgentLint pass (planned as part of v1.13.11 skills audit; split into its own batch when it grew larger than fitting).
|
||||
- **v1.13.10-openspec** — `openspec/changes/<slug>/{proposal,tasks,design}.md` batch-doc structure adoption.
|
||||
- **v1.13.11-tools** — tiered tool loading via `BOOCODE_TOOLS=core|standard|all` env (~30 LoC; was a far-future optional item, slotted in).
|
||||
- **v1.13.12-ws-schemas** + **v1.13.13-ws-publish** — Zod schemas for all 27 wire-format frames, `publishFrame`/`publishUserFrame` wrappers, ~80 publish sites converted (planned as v1.13.10 / v1.13.11).
|
||||
- **v1.13.14-skills-audit** — 26 skills vendored + audited via 5 parallel agent teams; 14 kept, 11 dropped, 1 migrated to BOOCHAT.md/BOOCODER.md. Codeminer42 rules-vs-recipes framing applied.
|
||||
- **v1.13.15-codecontext-synth** — forced second-inference synthesis pass for codecontext overview tools (truncation-aware extraction; auto-fetched top-N files + project docs under 32k payload budget).
|
||||
- **v1.13.16-xml-parser** — Anthropic `<invoke>` parser support + Levenshtein unknown-tool recovery hints (qwen3.6 drift to Claude Code-style tool names).
|
||||
- **v1.13.17-cross-repo-reads** — `request_read_access` tool + per-session `allowed_read_paths` grants; `pathGuard` extraRoots; reuses the `ask_user_input` pause/resume mechanism.
|
||||
- **v1.13.18-codecontext-file-path** — `resolveProjectPath` in `codecontext_client.ts` realpath-resolves `file_path` the same way `target_dir` was already resolved.
|
||||
- **v1.13.19-html-artifact-panes** — pane-based artifact viewer (Markdown default + HTML on request). Scope-revised mid-design from auto-bias-HTML to Markdown-default. `<!DOCTYPE html>` detection adds `message_parts.kind='html_artifact'` row; iframe sandbox `allow-scripts allow-clipboard-write allow-downloads` (no `allow-same-origin`); CSP `connect-src 'none'` + `X-Content-Type-Options: nosniff` + `Content-Security-Policy: sandbox` defense-in-depth. Pane state is reference-only — content fetched on mount to keep jsonb small.
|
||||
- **v1.13.20-drop-legacy-cols** — final strangler-fig step. 10 dual-write sites stripped (recon caught 2 beyond the original v1.13.2 inventory). `messages_with_parts` simplified to parts-only via `CREATE OR REPLACE` before column DROPs (Postgres ordering constraint). Adversarial-review catch: `discard_stale` had `RETURNING tool_calls, tool_results` — fixed via two-step UPDATE-then-SELECT-from-view. `Message` type retains the fields, populated by the view. v1.12.1 cleanup DO block removed.
|
||||
- **`v1.13` umbrella** — tagged on the same commit as v1.13.20 (`211e903`). AI SDK v6 + parts-table migration complete.
|
||||
|
||||
### Pre-v1.13 architectural decisions (still load-bearing)
|
||||
|
||||
|
||||
@@ -59,6 +59,7 @@ Rules:
|
||||
## Refactorer
|
||||
---
|
||||
temperature: 0.3
|
||||
steps: 5
|
||||
tools: [find_files, get_codebase_overview, get_dependencies, get_file_analysis, get_framework_analysis, get_semantic_neighborhoods, get_symbol_info, grep, list_dir, search_symbols, view_file, watch_changes]
|
||||
description: Proposes refactors for clarity, deduplication, or decoupling. Read-only — outputs plans, not edits.
|
||||
---
|
||||
@@ -97,6 +98,7 @@ Codecontext usage:
|
||||
## Architect
|
||||
---
|
||||
temperature: 0.5
|
||||
steps: 20
|
||||
tools: [find_files, get_codebase_overview, get_dependencies, get_file_analysis, get_framework_analysis, get_semantic_neighborhoods, get_symbol_info, grep, list_dir, search_symbols, view_file, watch_changes]
|
||||
description: Designs new features, modules, or architectural changes. Outputs a build plan.
|
||||
---
|
||||
|
||||
9
data/mcp.json
Normal file
9
data/mcp.json
Normal file
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"context7": {
|
||||
"type": "streamableHttp",
|
||||
"url": "https://mcp.context7.com/mcp",
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
}
|
||||
72
openspec/changes/v1.14-outer-loop/design.md
Normal file
72
openspec/changes/v1.14-outer-loop/design.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# v1.14.0-outer-loop — design decisions
|
||||
|
||||
Answers to the dispatch's blocking questions, resolved 2026-05-23.
|
||||
|
||||
## D1. Step cap — what replaces MAX_TOOL_LOOP_DEPTH?
|
||||
|
||||
`MAX_TOOL_LOOP_DEPTH` never existed — no hard recursion depth guard was ever in the codebase. Safety came from budget (50 tool calls) + doom-loop (3 identical calls).
|
||||
|
||||
**Decision:** introduce `MAX_STEPS = 200` as a hard ceiling. Per-agent cap via `agent.steps` is the primary knob. Resolution: `effectiveCap = Math.min(agent.steps ?? Infinity, MAX_STEPS)`.
|
||||
|
||||
**Rationale:** Sam reports BooChat gets stuck at 50 tool calls (the budget) too often. The step cap should be generous — 200 is 4x the current de-facto ceiling. Budget (50 tool calls total across all steps) remains a separate concern and is not changed in this batch.
|
||||
|
||||
Note: "step" ≠ "tool call." One step = one stream iteration that may produce multiple parallel tool calls. Budget counts individual tool calls; step cap counts iterations. At 200 steps with average 1-2 tool calls per step, the budget (50) will fire well before the step cap in most scenarios. The step cap is a safety ceiling for cases where the model makes many 1-tool-call iterations.
|
||||
|
||||
## D2. step_finish — emit or not?
|
||||
|
||||
**Decision:** No `step_finish` part. The next `step_start` (or assistant message completion) implicitly ends the previous step.
|
||||
|
||||
**Rationale:** opencode only emits `step_start`. Less noise in parts, simpler code. If UI ever needs step durations, compute from the timestamps of consecutive `step_start` parts.
|
||||
|
||||
## D3. Step-cap hit — sentinel or quiet?
|
||||
|
||||
**Decision:** Write a sentinel summary on step-cap hit. Visible to the user in chat, same as budget-exhaustion's `runCapHitSummary`.
|
||||
|
||||
**Implementation:** Extend `runCapHitSummary` to accept a `reason: 'budget' | 'step_cap'` parameter (or add a parallel `runStepCapSummary`). The sentinel metadata kind stays `cap_hit` — frontend `CapHitSentinel` component already renders it. The sentinel's text distinguishes the two cases ("Tool budget exhausted" vs "Step limit reached").
|
||||
|
||||
## D4. agent.steps = 0
|
||||
|
||||
**Decision:** `steps: 0` means "no tool calls allowed." The loop body never executes. The assistant can only respond with text.
|
||||
|
||||
**Implementation:** When `effectiveCap === 0`, skip the loop entirely. Stream the first assistant turn (text-only), finalize, return. The model receives no tools in the request payload when `steps: 0` (or equivalently, tools are passed but the loop never enters the tool-execution branch).
|
||||
|
||||
Actually, cleaner: `steps: 0` means the loop cap is 0. The while condition `stepNumber < effectiveCap` is false on the first check. The stream phase still runs (the model produces a text response), but if it emits tool calls they're ignored and the turn finalizes as text-only. This may produce a confusing response if the model's text references tool results it never got — but `steps: 0` is an explicit constraint the agent author chose. Document in AGENTS.md parser validation.
|
||||
|
||||
## D5. Synthesis success terminates the loop?
|
||||
|
||||
**Decision:** Yes. `break` out of the loop after synthesis success. Preserves current behavior (synthesis replaces the recursive call; no further iterations).
|
||||
|
||||
**Rationale:** The synthesis pass produces a self-contained summary turn. Continuing the loop after synthesis would let the model issue more tool calls on top of a synthesis summary, which is semantically wrong — the synthesis IS the final answer for that tool call batch.
|
||||
|
||||
## D6. executeToolPhase return struct
|
||||
|
||||
The recursive call at `tool-phase.ts:342` is currently the last thing `executeToolPhase` does (after creating the next assistant row). After the conversion, `executeToolPhase` returns a struct the loop body reads:
|
||||
|
||||
```typescript
|
||||
interface ToolPhaseResult {
|
||||
action: 'continue' | 'paused' | 'synthesis_done';
|
||||
toolCallCount: number;
|
||||
toolCalls: ToolCall[];
|
||||
nextAssistantId: string | null;
|
||||
}
|
||||
```
|
||||
|
||||
- `continue` → loop continues; `nextAssistantId` is the new assistant message's UUID.
|
||||
- `paused` → user-input or grant pause; loop breaks. `nextAssistantId` is null.
|
||||
- `synthesis_done` → synthesis succeeded; loop breaks. `nextAssistantId` is null (synthesis wrote its own parts).
|
||||
|
||||
The loop body then:
|
||||
1. Updates `toolsUsed += result.toolCallCount`
|
||||
2. Appends `result.toolCalls` to `recentToolCalls`
|
||||
3. Sets `assistantMessageId = result.nextAssistantId` for the next iteration
|
||||
4. Increments `stepNumber`
|
||||
5. Checks `result.action` — if not `continue`, breaks.
|
||||
|
||||
## D7. Budget vs steps interaction
|
||||
|
||||
Budget counts **individual tool calls** across the entire turn. Steps counts **loop iterations**. They are orthogonal:
|
||||
|
||||
- Budget fires when `toolsUsed >= resolveToolBudget(agent)` (currently 50 for read-only). Checked at the top of each iteration.
|
||||
- Step cap fires when `stepNumber >= effectiveCap`. Checked by the loop condition.
|
||||
|
||||
Both produce a sentinel summary. A turn can be terminated by whichever fires first. In practice, budget (50 tool calls) fires before step cap (200 steps) unless the model produces many 0-tool-call iterations (which shouldn't happen — 0 tool calls means non-tool finish, which exits the loop via the `break` path).
|
||||
112
openspec/changes/v1.14-outer-loop/proposal.md
Normal file
112
openspec/changes/v1.14-outer-loop/proposal.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# v1.14.0-outer-loop — explicit outer agent loop
|
||||
|
||||
Replace the ad-hoc `executeToolPhase → runAssistantTurn` recursion with an explicit `while` loop. A **step** is one stream-and-tool-execute iteration; a step can contain multiple parallel tool calls. The loop terminates on non-tool finish OR step-cap hit OR doom-loop OR budget exhaustion OR abort OR synthesis success.
|
||||
|
||||
## Why
|
||||
|
||||
The current recursion works but has two problems: (a) stack depth grows linearly with tool iterations — 50 nested async frames is fragile, (b) there's no explicit step counter, so there's no per-agent step cap and no step-boundary instrumentation. BooChat also gets stuck at 50 tool calls (the budget ceiling) more often than it should — the new `MAX_STEPS = 200` hard ceiling lets the loop run much longer before the step cap fires, while the existing budget (50 tool calls) remains a separate concern.
|
||||
|
||||
## Recon findings (verified 2026-05-23)
|
||||
|
||||
- `runAssistantTurn` at `turn.ts:144-147` is the recursive entry. Returns `Promise<void>`.
|
||||
- `executeToolPhase` at `tool-phase.ts:89-96` calls back into `runAssistantTurn` at `tool-phase.ts:342`.
|
||||
- Recursion terminates on: non-tool finish, budget exhaustion (`args.toolsUsed >= budget`), doom-loop (3 identical calls via `detectDoomLoop`), user-input pause (ask_user_input / request_read_access), synthesis success, stream error, abort.
|
||||
- **No existing hard recursion depth limit** — `MAX_TOOL_LOOP_DEPTH` does not exist. Safety comes from budget (50) + doom-loop (3 identical).
|
||||
- `TurnArgs` defined in `turn.ts:127-141`, not `types.ts`. Fields: `sessionId`, `chatId`, `assistantMessageId`, `toolsUsed`, `recentToolCalls`, `signal`. All mutable fields are threaded through the recursive call.
|
||||
- Synthesis pipeline (`synthesisPipeline.ts`) is a branch in `executeToolPhase` — if synthesis succeeds, recursion is skipped.
|
||||
- `step_start` already in the `message_parts.kind` CHECK constraint. No schema change needed.
|
||||
- `agents.ts` does NOT currently parse a `steps` field. Needs adding to `ParsedFrontmatter`.
|
||||
|
||||
## Scope
|
||||
|
||||
### S1. Outer loop in `turn.ts`
|
||||
|
||||
Convert the recursive chain to a `while (stepNumber < effectiveCap)` loop:
|
||||
|
||||
```
|
||||
let stepNumber = 0
|
||||
while (stepNumber < effectiveCap) {
|
||||
// doom-loop check
|
||||
// budget check
|
||||
// emit step_start part
|
||||
// stream phase (executeStreamPhase)
|
||||
// if no tool calls → finalize, break
|
||||
// tool phase (executeToolPhase — now returns, doesn't recurse)
|
||||
// if paused (user input / grant) → break
|
||||
// if synthesis succeeded → break
|
||||
// create next assistant message row
|
||||
// increment stepNumber, update toolsUsed, append recentToolCalls
|
||||
}
|
||||
// if stepNumber >= effectiveCap → sentinel summary
|
||||
```
|
||||
|
||||
`effectiveCap = Math.min(agent.steps ?? Infinity, MAX_STEPS)` where `MAX_STEPS = 200`.
|
||||
|
||||
### S2. `executeToolPhase` becomes non-recursive
|
||||
|
||||
Remove the `runAssistantTurn` call at `tool-phase.ts:342`. Instead, return a result indicating what happened: `{action: 'continue' | 'paused' | 'synthesis_done', toolsUsed, recentToolCalls, nextAssistantId}`. The caller (the while loop) uses the action to decide whether to continue or break.
|
||||
|
||||
### S3. `agent.steps` field
|
||||
|
||||
`agents.ts:ParsedFrontmatter` gains `steps?: number`. Parser extracts it from YAML frontmatter (integer ≥ 0). `steps: 0` means "no tool calls allowed" — loop body never executes; assistant responds text-only.
|
||||
|
||||
### S4. Step-boundary events
|
||||
|
||||
At the top of each loop iteration, emit a `step_start` part with payload `{step_number, started_at}`. Uses `insertParts` into the current assistant message. No `step_finish` — the next `step_start` (or message completion) implicitly ends the previous step.
|
||||
|
||||
### S5. Doom-loop migration
|
||||
|
||||
`detectDoomLoop` check moves from `runAssistantTurn` (top of function, pre-stream) to the top of the while-loop body (same logical position). Same predicate, same threshold (3). Same `runDoomLoopSummary` call. Control flow changes from `return` (unwinding recursion) to `break` (exiting loop).
|
||||
|
||||
### S6. Step-cap sentinel
|
||||
|
||||
When `stepNumber >= effectiveCap`, write a sentinel summary like the existing `runCapHitSummary`. Reuse `runCapHitSummary` with a reason parameter distinguishing "budget exhaustion" from "step cap hit", or create a parallel `runStepCapSummary`. The sentinel makes the cap visible in chat.
|
||||
|
||||
### S7. AGENTS.md updates
|
||||
|
||||
Add `steps:` to each agent in `data/AGENTS.md`:
|
||||
- Refactorer: `steps: 5`
|
||||
- Architect: `steps: 20`
|
||||
- All others: unset (infinity — bounded only by `MAX_STEPS = 200`)
|
||||
|
||||
### S8. Tests
|
||||
|
||||
New test file `apps/server/src/services/__tests__/outer-loop.test.ts` covering:
|
||||
- Clean finish (stream returns non-tool, loop exits after 1 iteration)
|
||||
- Step-cap hit (loop exits at cap, sentinel written)
|
||||
- Doom-loop break (3 identical calls, sentinel written)
|
||||
- Budget exhaustion (toolsUsed >= budget, cap-hit sentinel written)
|
||||
- Abort mid-step (signal fires, loop exits)
|
||||
- `steps: 0` edge case (no loop iterations, text-only response)
|
||||
- Synthesis success (loop exits after synthesis)
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No frontend changes. `step_start` parts surface via `messages_with_parts` automatically; UI doesn't render them in v1.14.
|
||||
- No `output_schema` / `exit_expression` / `execution_strategy` AGENTS.md fields.
|
||||
- No per-step snapshot for revert (v2.0 BooCoder concern).
|
||||
- No changes to budget constants (50 / 10 / 50). That's a separate concern.
|
||||
- No `repairToolCall` changes.
|
||||
- No compaction changes.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- No git commit, push. Sam commits.
|
||||
- Backup before editing.
|
||||
- TS strict, no `any`.
|
||||
- Doom-loop threshold stays at 3.
|
||||
- 332+ existing tests still pass + new outer-loop tests.
|
||||
|
||||
## Files expected to touch
|
||||
|
||||
- `apps/server/src/services/inference/turn.ts` — recursion → loop
|
||||
- `apps/server/src/services/inference/tool-phase.ts` — remove recursive call, return result struct
|
||||
- `apps/server/src/services/inference/sentinel-summaries.ts` — step-cap sentinel (or extend cap-hit)
|
||||
- `apps/server/src/services/agents.ts` — parse `steps` field
|
||||
- `data/AGENTS.md` — add `steps:` to Refactorer + Architect
|
||||
- `apps/server/src/services/__tests__/outer-loop.test.ts` — NEW
|
||||
- `apps/server/src/services/inference/index.ts` — re-export if new types needed
|
||||
|
||||
## Estimate
|
||||
|
||||
~300 LoC net (turn.ts refactor + tool-phase return struct + agents parser + tests). The conversion is structural, not behavioral — every exit path is preserved, just expressed as loop control flow instead of recursion unwinding.
|
||||
82
openspec/changes/v1.14-outer-loop/tasks.md
Normal file
82
openspec/changes/v1.14-outer-loop/tasks.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# v1.14.0-outer-loop tasks
|
||||
|
||||
## B1 — Backups
|
||||
|
||||
- [ ] `turn.ts`, `tool-phase.ts`, `sentinel-summaries.ts`, `agents.ts`, `data/AGENTS.md`
|
||||
|
||||
## B2 — agents.ts: parse `steps` field
|
||||
|
||||
- [ ] Add `steps?: number` to `ParsedFrontmatter` interface
|
||||
- [ ] Parse from YAML frontmatter: integer ≥ 0, warn on out-of-range (negative or non-integer), clamp to 0
|
||||
- [ ] Expose on the `Agent` type returned by `getAgentsForProject`
|
||||
- [ ] `npx tsc --noEmit -p apps/server` clean
|
||||
|
||||
## B3 — AGENTS.md: add `steps:` to Refactorer + Architect
|
||||
|
||||
- [ ] `data/AGENTS.md` — Refactorer: `steps: 5`
|
||||
- [ ] `data/AGENTS.md` — Architect: `steps: 20`
|
||||
- [ ] All others: leave unset (infinite, bounded by MAX_STEPS=200)
|
||||
|
||||
## B4 — tool-phase.ts: remove recursive call, return result struct
|
||||
|
||||
- [ ] Define `ToolPhaseResult` interface: `{action: 'continue' | 'paused' | 'synthesis_done', toolCallCount: number, toolCalls: ToolCall[], nextAssistantId: string | null}`
|
||||
- [ ] Remove `runAssistantTurn` import and call at line ~342
|
||||
- [ ] `executeToolPhase` returns `ToolPhaseResult` instead of `Promise<void>`
|
||||
- [ ] On normal path (after creating next assistant row): return `{action: 'continue', toolCallCount, toolCalls: result.toolCalls, nextAssistantId}`
|
||||
- [ ] On user-input pause: return `{action: 'paused', toolCallCount: <calls executed so far>, toolCalls: result.toolCalls, nextAssistantId: null}`
|
||||
- [ ] On synthesis success: return `{action: 'synthesis_done', toolCallCount, toolCalls: result.toolCalls, nextAssistantId: null}`
|
||||
- [ ] `npx tsc --noEmit -p apps/server` will FAIL here (turn.ts still expects void) — expected, fixed in B5
|
||||
|
||||
## B5 — turn.ts: recursion → while loop
|
||||
|
||||
- [ ] Add `MAX_STEPS = 200` constant
|
||||
- [ ] Resolve `effectiveCap = Math.min(agent?.steps ?? Infinity, MAX_STEPS)` at the top of `runAssistantTurn`
|
||||
- [ ] Convert `runAssistantTurn` body into a `while (stepNumber < effectiveCap)` loop:
|
||||
- Top of loop: doom-loop check (move from current position; `break` instead of `return`)
|
||||
- Top of loop: budget check (move from current position; `break` instead of `return`, but still call `runCapHitSummary` before break)
|
||||
- Emit `step_start` part via `insertParts` with payload `{step_number: stepNumber, started_at: new Date().toISOString()}`
|
||||
- Call `executeStreamPhase`
|
||||
- If no tool calls → `finalizeCompletion`, `break`
|
||||
- Call `executeToolPhase` (now returns `ToolPhaseResult`)
|
||||
- If `result.action !== 'continue'` → `break`
|
||||
- Update `toolsUsed += result.toolCallCount`
|
||||
- Update `recentToolCalls = [...recentToolCalls, ...result.toolCalls]`
|
||||
- Update `assistantMessageId = result.nextAssistantId!`
|
||||
- Increment `stepNumber`
|
||||
- [ ] After loop: if `stepNumber >= effectiveCap` → call step-cap sentinel (B6)
|
||||
- [ ] `effectiveCap === 0` edge case: the while condition is immediately false; stream the first turn text-only (the stream phase at the top of the function runs once before the loop — OR handle this by structuring the loop as do-while, OR handle by pre-checking and skipping tools from the request). Pick the cleanest approach.
|
||||
- [ ] Remove `TurnArgs` from the module export if it's no longer threaded through recursion — OR keep it and populate from loop locals. (Design note: `TurnArgs` is still used by `executeStreamPhase`, `executeToolPhase`, `sentinel-summaries.ts`, `error-handler.ts`. Keep the interface; populate from loop locals each iteration.)
|
||||
- [ ] `npx tsc --noEmit -p apps/server` clean
|
||||
- [ ] `pnpm -C apps/server test` — all existing tests pass
|
||||
|
||||
## B6 — sentinel-summaries.ts: step-cap sentinel
|
||||
|
||||
- [ ] Add `runStepCapSummary` (or extend `runCapHitSummary` with a `reason` param)
|
||||
- [ ] Write a sentinel with `metadata.kind = 'cap_hit'` (same as budget) so `CapHitSentinel` UI renders it
|
||||
- [ ] Sentinel text distinguishes "Step limit reached (N steps)" from "Tool budget exhausted (N calls)"
|
||||
- [ ] Called from the post-loop check in turn.ts (B5)
|
||||
|
||||
## B7 — Tests
|
||||
|
||||
- [ ] NEW `apps/server/src/services/__tests__/outer-loop.test.ts`
|
||||
- [ ] Test: clean finish — stream returns no tool calls, loop exits after 1 step
|
||||
- [ ] Test: step-cap hit — mock agent with `steps: 2`, model always returns tool calls, loop exits at 2, sentinel written
|
||||
- [ ] Test: doom-loop — 3 identical tool calls, sentinel written, loop breaks
|
||||
- [ ] Test: budget exhaustion — toolsUsed >= budget, cap-hit sentinel written
|
||||
- [ ] Test: `steps: 0` — no loop iterations, text-only response
|
||||
- [ ] Test: synthesis success — loop breaks after synthesis
|
||||
- [ ] `pnpm -C apps/server test` — all 332+ existing + new tests pass
|
||||
|
||||
## B8 — Verification
|
||||
|
||||
- [ ] `npx tsc --noEmit -p apps/server` — 0 errors
|
||||
- [ ] `npx tsc -p apps/web/tsconfig.app.json --noEmit` — 0 errors (no web changes; should pass)
|
||||
- [ ] `pnpm -C apps/web build` — green
|
||||
- [ ] `pnpm -C apps/server test` — all green
|
||||
|
||||
## B9 — Docs + tag + deploy
|
||||
|
||||
- [ ] `CHANGELOG.md` entry for v1.14.0-outer-loop
|
||||
- [ ] `boocode_roadmap.md` retrospective bullet on the v1.14 section
|
||||
- [ ] `CLAUDE.md` updates: mention the outer loop, MAX_STEPS, agent.steps in the inference/ section
|
||||
- [ ] Commit, tag `v1.14.0-outer-loop`, push, rebuild
|
||||
39
openspec/changes/v1.14.1-mcp-poc/design.md
Normal file
39
openspec/changes/v1.14.1-mcp-poc/design.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# v1.14.1-mcp-poc — design decisions
|
||||
|
||||
## D1. Transport: Streamable HTTP (not stdio)
|
||||
|
||||
Context7 is a remote service at `https://mcp.context7.com/mcp`. Uses the MCP Streamable HTTP transport. The `@modelcontextprotocol/sdk` TypeScript client supports this via `StreamableHTTPClientTransport`. No stdio needed.
|
||||
|
||||
## D2. Tool name prefixing
|
||||
|
||||
MCP tools get a `context7_` prefix to avoid collisions with BooCode's native tools. Context7's tools are `resolve-library-id` and `query-docs` — these become `context7_resolve-library-id` and `context7_query-docs`. The prefix is stripped before calling the MCP server's `tools/call`.
|
||||
|
||||
## D3. Read-only invariant guard
|
||||
|
||||
BooChat is read-only through v1.x. The MCP client rejects any tool whose `annotations?.readOnly === false`. Tools with `readOnly: true` or no annotations are accepted. Context7's tools are all read-only (they query documentation — no write side effects). Fail-open on missing annotations is a deliberate choice: most MCP servers don't set annotations yet, and rejecting all un-annotated tools would make the feature useless. The guard catches explicitly-declared write tools.
|
||||
|
||||
## D4. Zod inputSchema for MCP tools
|
||||
|
||||
MCP tools come with a JSON Schema `inputSchema`. BooCode's `ToolDef` has both a Zod `inputSchema` (for server-side validation) and a `jsonSchema` (for the LLM's tool schema). For MCP tools:
|
||||
- `jsonSchema` is built directly from the MCP tool's `inputSchema` (it's already JSON Schema).
|
||||
- `inputSchema` uses `z.record(z.unknown())` as a pass-through — the MCP server does its own validation. Double-validating with a generated Zod schema from JSON Schema adds complexity with no value for a PoC.
|
||||
|
||||
## D5. Tool registration: append + re-sort (not lazy-init)
|
||||
|
||||
The simplest approach: keep `ALL_TOOLS` as the native tool array. Add an `appendMcpTools(tools: ToolDef[])` function that pushes MCP tools, re-sorts alphabetically, and rebuilds `TOOLS_BY_NAME` and `READ_ONLY_TOOL_NAMES`. Called once at startup after MCP init. More invasive approaches (lazy-init, factory function) change the import shape for every consumer. Mutation-at-startup is ugly but contained to one call site and matches the existing alpha-sort-at-module-level pattern.
|
||||
|
||||
## D6. No per-session toggle
|
||||
|
||||
Web tools have `session.web_search_enabled`. MCP tools do NOT get a session toggle in v1.14.1. If configured via env var, MCP tools are always available. Per-session MCP control is a v1.15 concern (when multiple MCP servers and the permission ruleset land together).
|
||||
|
||||
## D7. Graceful degradation
|
||||
|
||||
MCP server down at startup → log warning, expose zero MCP tools, BooCode functions normally. MCP server down mid-session (tool call fails) → the `execute` wrapper catches the error and returns `{error: true, output: "MCP server unreachable"}` — the model sees the error and can self-correct (use native tools instead).
|
||||
|
||||
## D8. Result content extraction
|
||||
|
||||
MCP `tools/call` returns `{content: ContentBlock[]}` where each block is `{type: 'text', text: string}` or `{type: 'resource', ...}`. For the PoC:
|
||||
- Text blocks: join with `\n`.
|
||||
- Resource blocks: serialize as JSON (the model can read structured data).
|
||||
- Empty content: return `"(no output)"`.
|
||||
- `isError: true` in the response: return `{error: true, output: joinedContent}`.
|
||||
96
openspec/changes/v1.14.1-mcp-poc/proposal.md
Normal file
96
openspec/changes/v1.14.1-mcp-poc/proposal.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# v1.14.1-mcp-poc — single-server MCP client proof-of-concept
|
||||
|
||||
Validate the MCP-client loop end-to-end against one real MCP server (Context7) before committing to the full opencode `mcp/index.ts` port at v1.15. Small, throwaway-if-needed.
|
||||
|
||||
## Why
|
||||
|
||||
BooCode's tool registry (`ALL_TOOLS` in `tools.ts`) is static — tools are hardcoded TypeScript modules. MCP is the protocol for dynamic tool discovery. Wiring one real MCP server end-to-end proves: tool-discovery → tool-list → tool-call → result-render → context-budget accounting all hold. If Context7 works, any MCP server will work via the same plumbing.
|
||||
|
||||
## Scope
|
||||
|
||||
### S1. Install `@modelcontextprotocol/sdk`
|
||||
|
||||
New dependency in `apps/server/package.json`. The official TypeScript MCP client SDK (MIT). Provides `Client`, `StreamableHTTPClientTransport`, tool-call/result types.
|
||||
|
||||
### S2. New service: `apps/server/src/services/mcp-client.ts`
|
||||
|
||||
Singleton MCP client that:
|
||||
1. Connects to Context7 at `MCP_CONTEXT7_URL` (default `https://mcp.context7.com/mcp`) via Streamable HTTP transport.
|
||||
2. Optional `MCP_CONTEXT7_API_KEY` env var passed as a header.
|
||||
3. On `initialize()`: calls `tools/list`, wraps each MCP tool as a `ToolDef`, prefixes names with `context7_` to avoid collisions with BooCode's native tools.
|
||||
4. **Read-only invariant guard:** rejects any tool whose `annotations?.readOnly` is explicitly `false`. Tools with `readOnly: true` or no `annotations` field are accepted (fail-open on read-only, since most MCP tools don't set annotations yet — Context7's tools don't).
|
||||
5. `callTool(name, args)` → calls the MCP server's `tools/call` endpoint and returns the result content.
|
||||
6. `getTools(): ToolDef[]` → returns the discovered tools wrapped as BooCode `ToolDef` objects.
|
||||
7. Graceful degradation: if the MCP server is unreachable at startup, log a warning and expose zero MCP tools. BooCode functions normally with its native tools.
|
||||
|
||||
### S3. Config extension
|
||||
|
||||
`apps/server/src/config.ts` gains two optional env vars:
|
||||
- `MCP_CONTEXT7_URL` (string, default `https://mcp.context7.com/mcp`)
|
||||
- `MCP_CONTEXT7_API_KEY` (string, optional)
|
||||
|
||||
### S4. Tool registration
|
||||
|
||||
`apps/server/src/services/tools.ts` — after building `ALL_TOOLS` from native tools, append MCP-discovered tools from `mcpClient.getTools()`. The alpha-sort at the end of `ALL_TOOLS` construction covers both native and MCP tools. `TOOLS_BY_NAME` map includes MCP tools.
|
||||
|
||||
MCP tools are registered with `category: 'read_only'` (per the read-only invariant guard in S2).
|
||||
|
||||
### S5. Tool dispatch
|
||||
|
||||
`apps/server/src/services/inference/tool-phase.ts` `executeToolCall` already dispatches via `TOOLS_BY_NAME[toolName].execute(...)`. MCP tools' `execute` function calls `mcpClient.callTool(name, args)` — the dispatch is transparent to the rest of the inference loop. No changes to `executeToolCall` needed.
|
||||
|
||||
### S6. MCP tool result → BooCode format
|
||||
|
||||
MCP `tools/call` returns `{ content: [{type: 'text', text: string}, ...] }`. BooCode's `executeToolCall` expects a string or JSON-serializable output. The `execute` wrapper in the ToolDef extracts `content[0].text` (or joins multiple content blocks with `\n`). If the MCP server returns an error, the wrapper returns `{error: true, output: errorMessage}` matching BooCode's existing error-result shape.
|
||||
|
||||
### S7. Startup initialization
|
||||
|
||||
`apps/server/src/index.ts` — after `applySchema()` and before route registration, call `mcpClient.initialize()`. If `MCP_CONTEXT7_URL` is not set (or empty), skip initialization entirely (MCP is opt-in). Log the number of discovered tools on success.
|
||||
|
||||
Tool registration (S4) must happen AFTER MCP initialization, since `getTools()` returns the discovered tools. Current flow: `ALL_TOOLS` is a module-level constant. This needs to change to a lazy-init pattern — either a function that returns the tool list (called once at startup after MCP init), or a mutable array that MCP tools get appended to during startup.
|
||||
|
||||
### S8. Agent tool whitelist interaction
|
||||
|
||||
MCP tools are prefixed `context7_*`. Existing agents' `tools:` whitelists don't include MCP tool names — so MCP tools are only available to the default agent (no agent selected, which gets ALL_TOOLS). To make MCP tools available to specific agents, their AGENTS.md `tools:` list would need to include `context7_*` names. For the PoC, this is fine — the default agent (most common) gets MCP tools.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No stdio transport. Context7 is HTTP-only.
|
||||
- No OAuth. Context7 uses an API key header.
|
||||
- No multiple servers. One hardcoded server (Context7).
|
||||
- No per-agent MCP server allow/deny. All agents that don't have a `tools:` whitelist get MCP tools.
|
||||
- No per-session MCP toggle. If configured, MCP tools are always available.
|
||||
- No UI changes. MCP tools surface in the tool list the model sees; results render as normal tool-result parts.
|
||||
- No schema changes. MCP state is in-memory only.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- No git commit/push. Sam commits.
|
||||
- Read-only invariant: reject any MCP tool with `readOnly: false`.
|
||||
- Graceful degradation: MCP server down → zero MCP tools, BooCode works normally.
|
||||
- One new dep only: `@modelcontextprotocol/sdk`.
|
||||
- Alpha-sort of ALL_TOOLS preserved (v1.13.3 prompt-cache invariant).
|
||||
|
||||
## Files expected to touch
|
||||
|
||||
- `apps/server/package.json` — add `@modelcontextprotocol/sdk`
|
||||
- `pnpm-lock.yaml` — auto-updated
|
||||
- `apps/server/src/config.ts` — `MCP_CONTEXT7_URL`, `MCP_CONTEXT7_API_KEY`
|
||||
- `apps/server/src/services/mcp-client.ts` — NEW, ~100 lines
|
||||
- `apps/server/src/services/tools.ts` — lazy-init or append MCP tools to ALL_TOOLS
|
||||
- `apps/server/src/index.ts` — call `mcpClient.initialize()` at startup
|
||||
- `apps/server/src/services/__tests__/mcp-client.test.ts` — NEW, unit tests for tool wrapping + read-only guard
|
||||
|
||||
## Estimate
|
||||
|
||||
~150 LoC. The MCP SDK handles the protocol; BooCode's job is wrapping discovered tools as ToolDefs and routing calls through the SDK client.
|
||||
|
||||
## Smoke plan
|
||||
|
||||
1. Set `MCP_CONTEXT7_URL=https://mcp.context7.com/mcp` in `.env` (or docker-compose env).
|
||||
2. Restart boocode container.
|
||||
3. Check logs: should see "mcp: initialized Context7, discovered N tools" (or similar).
|
||||
4. Open a chat with no agent selected. Send "What does the `streamText` function do in the AI SDK? Use context7 to look it up."
|
||||
5. Confirm: model calls `context7_resolve-library-id` then `context7_query-docs` (or whatever Context7's tool names are after prefixing).
|
||||
6. Confirm: tool results render normally in the chat.
|
||||
7. Without `MCP_CONTEXT7_URL` set: restart, confirm BooCode starts normally with zero MCP tools.
|
||||
80
openspec/changes/v1.14.1-mcp-poc/tasks.md
Normal file
80
openspec/changes/v1.14.1-mcp-poc/tasks.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# v1.14.1-mcp-poc tasks
|
||||
|
||||
## B1 — Backups
|
||||
|
||||
- [ ] `apps/server/src/services/tools.ts`
|
||||
- [ ] `apps/server/src/config.ts`
|
||||
- [ ] `apps/server/src/index.ts`
|
||||
|
||||
## B2 — Install `@modelcontextprotocol/sdk`
|
||||
|
||||
- [ ] `pnpm -C apps/server add @modelcontextprotocol/sdk`
|
||||
- [ ] Verify `pnpm -C apps/server build` still works after install
|
||||
- [ ] Note the installed version
|
||||
|
||||
## B3 — Config extension
|
||||
|
||||
- [ ] `apps/server/src/config.ts` — add `MCP_CONTEXT7_URL` (string, optional, default `https://mcp.context7.com/mcp`)
|
||||
- [ ] `apps/server/src/config.ts` — add `MCP_CONTEXT7_API_KEY` (string, optional)
|
||||
- [ ] Both via Zod `.optional()` with `.default()` for the URL
|
||||
|
||||
## B4 — MCP client service
|
||||
|
||||
- [ ] NEW `apps/server/src/services/mcp-client.ts`
|
||||
- [ ] Import `Client`, `StreamableHTTPClientTransport` from `@modelcontextprotocol/sdk/client`
|
||||
- [ ] `initialize(config, log)` — connect to Context7, call `tools/list`, wrap each as ToolDef, apply read-only guard
|
||||
- [ ] `callTool(name, args)` — call MCP server `tools/call`, extract text content, return as string
|
||||
- [ ] `getTools()` — return wrapped ToolDef[]
|
||||
- [ ] `isInitialized()` — boolean
|
||||
- [ ] Read-only guard: skip tools with `annotations?.readOnly === false`; accept all others
|
||||
- [ ] Graceful degradation: catch connection errors, log warning, expose zero tools
|
||||
- [ ] Tool name prefixing: `context7_<original_name>`
|
||||
- [ ] ToolDef wrapping: map MCP inputSchema (JSONSchema) to ToolJsonSchema `function.parameters`; use `z.any()` for Zod inputSchema (MCP already validated on the server side)
|
||||
- [ ] Execute wrapper: strip `context7_` prefix before calling MCP, join result content blocks with `\n`
|
||||
|
||||
## B5 — Tool registration (lazy-init)
|
||||
|
||||
- [ ] `apps/server/src/services/tools.ts` — convert `ALL_TOOLS` from a module-level constant to a lazy-initialized array
|
||||
- [ ] Add `initializeTools(mcpTools: ToolDef[])` function that builds the final sorted list
|
||||
- [ ] `TOOLS_BY_NAME`, `READ_ONLY_TOOL_NAMES` derived from the initialized list
|
||||
- [ ] Ensure all existing callers of `ALL_TOOLS` / `TOOLS_BY_NAME` still work (they import from tools.ts — verify the export shape)
|
||||
- [ ] OR simpler: keep ALL_TOOLS as-is (native tools), add `appendMcpTools(tools)` that mutates + re-sorts + rebuilds TOOLS_BY_NAME. Less clean but less invasive.
|
||||
|
||||
## B6 — Startup wiring
|
||||
|
||||
- [ ] `apps/server/src/index.ts` — after `applySchema()`, before route registration:
|
||||
- If `config.MCP_CONTEXT7_URL` is set: `await mcpClient.initialize(config, app.log)`
|
||||
- `appendMcpTools(mcpClient.getTools())` (or equivalent)
|
||||
- Log tool count
|
||||
- [ ] If URL not set: skip, log "mcp: Context7 not configured, skipping"
|
||||
|
||||
## B7 — Verification
|
||||
|
||||
- [ ] `npx tsc --noEmit -p apps/server` — 0 errors
|
||||
- [ ] `pnpm -C apps/server test` — all existing tests pass (MCP client is startup-only; tests don't initialize it)
|
||||
- [ ] `pnpm -C apps/web build` — green (no web changes)
|
||||
|
||||
## B8 — Unit tests
|
||||
|
||||
- [ ] NEW `apps/server/src/services/__tests__/mcp-client.test.ts`
|
||||
- [ ] Test: tool wrapping produces correct ToolDef shape (name, description, jsonSchema, execute fn)
|
||||
- [ ] Test: read-only guard rejects tools with `readOnly: false`
|
||||
- [ ] Test: read-only guard accepts tools with `readOnly: true` or no annotations
|
||||
- [ ] Test: name prefixing — `resolve-library-id` → `context7_resolve-library-id`
|
||||
- [ ] Test: result extraction — single text content block → string; multiple → joined with `\n`
|
||||
- [ ] Test: error result — MCP error → `{error: true, output: ...}` shape
|
||||
|
||||
## B9 — Deploy + smoke
|
||||
|
||||
- [ ] Add `MCP_CONTEXT7_URL=https://mcp.context7.com/mcp` to docker-compose env (or .env)
|
||||
- [ ] `docker compose up --build -d`
|
||||
- [ ] Check logs for MCP initialization message
|
||||
- [ ] Live-smoke: send a chat asking about AI SDK docs via Context7
|
||||
- [ ] Verify tool calls + results render normally
|
||||
|
||||
## B10 — Docs + tag
|
||||
|
||||
- [ ] `CHANGELOG.md` entry
|
||||
- [ ] `boocode_roadmap.md` retrospective bullet
|
||||
- [ ] `CLAUDE.md` — mention MCP client in the tools/services section
|
||||
- [ ] Commit, tag `v1.14.1-mcp-poc`, push, rebuild
|
||||
59
openspec/changes/v1.15-mcp-multi/design.md
Normal file
59
openspec/changes/v1.15-mcp-multi/design.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# v1.15.0-mcp-multi — design decisions
|
||||
|
||||
## D1. Config file path
|
||||
|
||||
`/data/mcp.json` (alongside `AGENTS.md` at `/data/AGENTS.md`). Both are bind-mounted from the host's `data/` directory. Override via `MCP_CONFIG_PATH` env var.
|
||||
|
||||
File missing = no MCP (opt-in by file presence, not by env var). Simpler than the v1.14.1 approach of always-defaulting a URL.
|
||||
|
||||
## D2. Config schema matches opencode's `mcpServers` shape
|
||||
|
||||
opencode uses `~/.opencode/config.json` with a `mcpServers` key. BooCode uses `mcp.json` with the same `mcpServers` key so server entries are copy-pasteable. Property names match: `type`, `url`, `command`, `args`, `env`, `headers`. BooCode adds `enabled` (boolean toggle per server, default true) which opencode doesn't have — harmless extra key.
|
||||
|
||||
## D3. Transport types: streamableHttp + stdio only
|
||||
|
||||
- **streamableHttp**: For remote servers (Context7, future cloud MCP services). Uses `@modelcontextprotocol/sdk`'s `StreamableHTTPClientTransport`.
|
||||
- **stdio**: For local subprocess servers (codecontext, future local tools). Uses `@modelcontextprotocol/sdk`'s `StdioClientTransport` (spawns child process, NDJSON framing over stdin/stdout).
|
||||
- **SSE**: Skipped. Streamable HTTP supersedes SSE per the MCP spec (May 2025 protocol update). If a legacy server requires SSE, it can be added later.
|
||||
|
||||
## D4. Tool name prefixing: `<serverName>_<toolName>`
|
||||
|
||||
Generalizes v1.14.1's `context7_<name>` pattern. Server name comes from the config key (e.g. `"context7"`, `"codecontext"`). Collisions between servers with the same name are impossible (config keys are unique). Collisions between an MCP tool and a native tool are possible if someone names a server entry the same as a native tool prefix — but that's a user-configuration error, not a code bug.
|
||||
|
||||
## D5. Per-agent glob patterns: last-match-wins
|
||||
|
||||
AGENTS.md `tools:` field already supports exact-match arrays. Globs extend the same field:
|
||||
|
||||
```yaml
|
||||
tools: [view_file, grep, context7_*]
|
||||
```
|
||||
|
||||
Evaluation: for each tool in `ALL_TOOLS`, scan the pattern list left-to-right. A `!` prefix denies. Last matching pattern wins. This matches the roadmap's "wildcard rule matcher" language.
|
||||
|
||||
Examples:
|
||||
- `[*]` — all tools (same as omitting `tools:` entirely)
|
||||
- `[*, !web_*]` — all tools except web
|
||||
- `[view_file, grep, context7_*]` — only view_file, grep, and all Context7 tools
|
||||
- `[*]` on Architect + `[view_file]` on Prompt Builder — each agent gets its intended scope
|
||||
|
||||
Globs use a simple `minimatch`-style check: `*` matches any characters. No `?` or `**` — tool names are flat (no path separators).
|
||||
|
||||
## D6. No DB tables in v1.15
|
||||
|
||||
The roadmap listed `permissions`, `agent_permissions`, `session_permissions`, `mcp_servers` tables. All deferred to v2.0:
|
||||
|
||||
- **Permission tables**: Enterprise multi-user pattern. BooChat is single-user behind Authelia. The read-only invariant guard is the BooChat-era defense. Formal permission rulesets land when BooCoder adds write tools.
|
||||
- **`mcp_servers` table**: In-memory registry is sufficient. No need to persist server state to DB when the config file is the source of truth and tools are re-discovered on every boot.
|
||||
|
||||
## D7. Stdio child lifecycle
|
||||
|
||||
- Spawn on `initialize()`. Persistent connection for server lifetime (not per-call).
|
||||
- On child exit (unexpected): mark server unavailable, log error. Do NOT auto-restart. BooCode continues with remaining servers.
|
||||
- On BooCode shutdown (`app.addHook('onClose')`): send SIGTERM to all stdio children. Wait up to 5s, then SIGKILL.
|
||||
- On ENOENT (command not found): skip server with a warning. Matches the graceful-degradation pattern from v1.14.1.
|
||||
|
||||
## D8. v1.14.1 env vars removed
|
||||
|
||||
`MCP_CONTEXT7_URL` and `MCP_CONTEXT7_API_KEY` are deleted from `config.ts`. They're superseded by the JSON config file's `context7` entry. The PoC was explicitly designed as throwaway.
|
||||
|
||||
Migration path for anyone who had the env vars set: add a `data/mcp.json` with the Context7 entry. The CHANGELOG entry will note this.
|
||||
130
openspec/changes/v1.15-mcp-multi/proposal.md
Normal file
130
openspec/changes/v1.15-mcp-multi/proposal.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# v1.15.0-mcp-multi — multi-server MCP client + stdio transport + config file
|
||||
|
||||
Generalize the v1.14.1 single-server Context7 PoC into a multi-server MCP client. Add stdio transport (for local subprocess MCP servers like codecontext). JSON config file matching opencode's schema shape. Per-agent tool glob patterns in AGENTS.md frontmatter.
|
||||
|
||||
## Why
|
||||
|
||||
v1.14.1 proved the MCP loop works end-to-end but is hardcoded to one server (Context7) via env vars. Real value comes from multiple servers: Context7 for docs, codecontext re-wired as a proper MCP server (stdio), future local tools. The config shape should match opencode's so Sam can copy `mcp` blocks between the two without translation.
|
||||
|
||||
## Scope
|
||||
|
||||
### S1. JSON config file for MCP servers
|
||||
|
||||
New file at `/data/mcp.json` (bind-mounted like `AGENTS.md`). Env var `MCP_CONFIG_PATH` points to it (default `/data/mcp.json`).
|
||||
|
||||
Schema (matching opencode's shape):
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"context7": {
|
||||
"type": "streamableHttp",
|
||||
"url": "https://mcp.context7.com/mcp",
|
||||
"headers": { "X-API-Key": "optional-key" },
|
||||
"enabled": true
|
||||
},
|
||||
"codecontext": {
|
||||
"type": "stdio",
|
||||
"command": "/usr/local/bin/codecontext",
|
||||
"args": ["--mcp"],
|
||||
"env": { "WORKSPACE": "/opt" },
|
||||
"enabled": false
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Zod-validated at startup. Unknown keys silently ignored (forward-compat). Each server entry has:
|
||||
- `type`: `"streamableHttp"` | `"stdio"` (SSE deferred — Streamable HTTP supersedes it per the MCP spec)
|
||||
- `url` (HTTP) or `command` + `args` + `env` (stdio)
|
||||
- `headers` (HTTP, optional) — for API keys
|
||||
- `enabled` (boolean, default true)
|
||||
|
||||
### S2. Multi-server MCP client
|
||||
|
||||
Refactor `mcp-client.ts` from a singleton to a registry of named MCP clients. On startup:
|
||||
1. Read `/data/mcp.json` (or path from `MCP_CONFIG_PATH`)
|
||||
2. For each enabled server: create a Client + transport, connect, discover tools via `tools/list`
|
||||
3. Wrap tools with `<server-name>_<tool-name>` prefix (generalizes the `context7_` pattern)
|
||||
4. Apply read-only invariant guard per-tool (reject `readOnlyHint: false`)
|
||||
5. Append all MCP tools to `ALL_TOOLS` in a single `appendMcpTools()` call
|
||||
6. Per-server graceful degradation: one server failing doesn't block others
|
||||
|
||||
Expose: `getMcpServers(): McpServerStatus[]` for debug/status endpoint, `callTool(prefixedName, args)` routed to the correct server by prefix.
|
||||
|
||||
### S3. Stdio transport
|
||||
|
||||
For `type: "stdio"` servers: spawn a subprocess via `child_process.spawn(command, args, {env, stdio: 'pipe'})`. Use `@modelcontextprotocol/sdk`'s `StdioClientTransport` (or implement the NDJSON framing ourselves — the SDK should have it). The subprocess runs for the lifetime of the BooCode server (persistent connection, not per-call spawn).
|
||||
|
||||
Child lifecycle:
|
||||
- Spawn on initialize. If spawn fails, log warn, skip server (graceful degradation).
|
||||
- On child exit: log error, mark server as unavailable. Do NOT restart automatically (v1.15 keeps it simple; auto-restart is a v2.0 concern).
|
||||
- On BooCode shutdown (`app.addHook('onClose')`): kill child processes.
|
||||
|
||||
### S4. Per-agent tool glob patterns in AGENTS.md
|
||||
|
||||
Currently `tools:` in AGENTS.md frontmatter is an exact-match whitelist (array of tool names). Extend to support glob patterns via a lightweight matcher:
|
||||
- `context7_*` — all tools from the context7 server
|
||||
- `view_*` — all tools starting with `view_`
|
||||
- `!web_*` — exclude web tools (deny pattern)
|
||||
- Plain names (`grep`, `view_file`) work as before (exact match)
|
||||
|
||||
Evaluation order: for each tool in `ALL_TOOLS`, check if it matches any pattern in the agent's `tools:` list. A `!` prefix means exclude. Last-match-wins.
|
||||
|
||||
Parser change in `agents.ts`: when validating `tools:`, don't reject unknown names if they contain `*` (glob patterns can't be validated against the current tool list since MCP tools are discovered at runtime). Exact names are still validated.
|
||||
|
||||
### S5. Remove v1.14.1 env-var config
|
||||
|
||||
Delete `MCP_CONTEXT7_URL` and `MCP_CONTEXT7_API_KEY` from `config.ts`. They're superseded by the JSON config file. The v1.14.1 PoC is throwaway-by-design (proposal said "throwaway-if-needed").
|
||||
|
||||
### S6. Read-only invariant preserved
|
||||
|
||||
BooChat's read-only guarantee stays: every MCP tool with `readOnlyHint: false` is rejected at discovery. This applies globally, not per-server. Config has no `allowWriteTools` flag — that's a v2.0 BooCoder concern.
|
||||
|
||||
## Deferred to v2.0
|
||||
|
||||
- **Permission ruleset tables** (`permissions`, `agent_permissions`, `session_permissions`). Enterprise pattern that doesn't serve until BooCoder adds write tools. The read-only invariant guard is the BooChat-era defense-in-depth.
|
||||
- **OAuth / Dynamic Client Registration.** Needs secret storage primitive first.
|
||||
- **SSE transport.** Streamable HTTP supersedes it per the MCP spec. SSE is a legacy fallback.
|
||||
- **Per-session MCP toggle.** No `session.mcp_enabled` column in v1.15. MCP servers are globally configured; agent tool globs are the scoping mechanism.
|
||||
- **`mcp_servers` DB table.** In-memory registry is sufficient for single-user. DB tracking deferred to v2.0.
|
||||
- **codecontext re-wiring to MCP.** Separate batch after v1.15 proves stdio transport works.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No frontend changes. MCP tools surface via the existing tool registry; results render as normal tool-result parts.
|
||||
- No schema changes. No new DB tables or columns.
|
||||
- No changes to the inference loop (v1.14.0 outer loop unchanged).
|
||||
- No changes to `executeToolCall` dispatch (transparent via ToolDef.execute).
|
||||
|
||||
## Hard rules
|
||||
|
||||
- No git commit/push. Sam commits.
|
||||
- Read-only invariant: reject any MCP tool with `readOnlyHint: false`.
|
||||
- Graceful degradation: any server down → that server's tools unavailable, rest unaffected.
|
||||
- Alpha-sort of ALL_TOOLS preserved.
|
||||
- One new dep only: none (MCP SDK already installed from v1.14.1).
|
||||
- 348+ existing tests still pass.
|
||||
|
||||
## Files expected to touch
|
||||
|
||||
- `apps/server/src/services/mcp-client.ts` — refactor from singleton to multi-server registry (~200→300 lines)
|
||||
- `apps/server/src/services/tools.ts` — no changes expected (appendMcpTools already works for multiple tools)
|
||||
- `apps/server/src/config.ts` — replace MCP env vars with `MCP_CONFIG_PATH`
|
||||
- `apps/server/src/index.ts` — startup reads config file, iterates servers
|
||||
- `apps/server/src/services/agents.ts` — glob pattern support in `tools:` whitelist
|
||||
- `data/mcp.json` — NEW, example config with Context7 (disabled by default, enabled via edit)
|
||||
- `apps/server/src/services/__tests__/mcp-client.test.ts` — update for multi-server, add stdio transport tests
|
||||
- `apps/server/src/services/__tests__/agents-glob.test.ts` — NEW, glob pattern matching tests
|
||||
|
||||
## Estimate
|
||||
|
||||
~350 LoC. The MCP SDK handles both transports; BooCode's job is config parsing, multi-server lifecycle, and glob matching.
|
||||
|
||||
## Smoke plan
|
||||
|
||||
1. Create `/data/mcp.json` with Context7 enabled. Restart. Confirm tools discovered + logged.
|
||||
2. Send a chat asking about library docs. Confirm `context7_*` tools called + results rendered.
|
||||
3. Disable Context7 in config (`"enabled": false`). Restart. Confirm zero MCP tools.
|
||||
4. Add a dummy stdio server entry pointing to `/bin/cat` (will fail). Confirm graceful degradation: Context7 works, dummy fails with a logged warning.
|
||||
5. Add `tools: [context7_*]` to the Architect agent in AGENTS.md. Confirm Architect sees only Context7 tools (via AgentPicker or by chatting with Architect selected).
|
||||
6. Stop boocode, confirm child processes are killed (no orphans).
|
||||
87
openspec/changes/v1.15-mcp-multi/tasks.md
Normal file
87
openspec/changes/v1.15-mcp-multi/tasks.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# v1.15.0-mcp-multi tasks
|
||||
|
||||
## B1 — Backups
|
||||
|
||||
- [ ] `mcp-client.ts`, `config.ts`, `index.ts`, `agents.ts`, `mcp-client.test.ts`
|
||||
|
||||
## B2 — MCP config file schema + loader
|
||||
|
||||
- [ ] NEW `apps/server/src/services/mcp-config.ts` (~50 lines)
|
||||
- [ ] Zod schema for `mcp.json`: `McpServerConfig` with `type`, `url/command/args/env`, `headers`, `enabled`
|
||||
- [ ] `loadMcpConfig(configPath: string, log): McpServerConfig[]` — reads JSON, validates, returns enabled servers
|
||||
- [ ] Graceful: file missing → log info, return empty array (no MCP)
|
||||
- [ ] Graceful: parse error → log warn with details, return empty array
|
||||
|
||||
## B3 — Config.ts: replace MCP env vars
|
||||
|
||||
- [ ] Remove `MCP_CONTEXT7_URL` and `MCP_CONTEXT7_API_KEY` from Zod schema
|
||||
- [ ] Add `MCP_CONFIG_PATH: z.string().optional()` (no default — opt-in)
|
||||
|
||||
## B4 — Refactor mcp-client.ts to multi-server registry
|
||||
|
||||
- [ ] Replace module-level singleton with `Map<serverName, {client, transport, tools}>`
|
||||
- [ ] `initialize(servers: McpServerConfig[], log)` — iterate servers, connect each, discover tools, wrap with `<serverName>_<toolName>` prefix, apply read-only guard
|
||||
- [ ] Streamable HTTP transport: reuse existing pattern from v1.14.1
|
||||
- [ ] Stdio transport: use `@modelcontextprotocol/sdk`'s `StdioClientTransport` (check SDK exports; fallback to `child_process.spawn` + NDJSON if SDK doesn't expose it)
|
||||
- [ ] `callTool(prefixedName, args)` — extract server name from prefix, route to correct client
|
||||
- [ ] `getTools()` — return all tools from all servers, flattened
|
||||
- [ ] `getMcpServers()` — return status of each server (name, type, toolCount, connected)
|
||||
- [ ] Per-server graceful degradation: catch per-server errors, log, skip; continue with others
|
||||
- [ ] `shutdown()` — kill stdio child processes, close HTTP clients
|
||||
- [ ] `app.addHook('onClose')` calls shutdown
|
||||
|
||||
## B5 — Startup wiring (index.ts)
|
||||
|
||||
- [ ] Read config: `const mcpConfigPath = config.MCP_CONFIG_PATH ?? '/data/mcp.json'`
|
||||
- [ ] `const mcpServers = loadMcpConfig(mcpConfigPath, app.log)`
|
||||
- [ ] `await mcpClient.initialize(mcpServers, app.log)`
|
||||
- [ ] `appendMcpTools(mcpClient.getTools())`
|
||||
- [ ] Log summary: "mcp: N servers connected, M tools registered"
|
||||
- [ ] `app.addHook('onClose', () => mcpClient.shutdown())`
|
||||
|
||||
## B6 — AGENTS.md glob patterns
|
||||
|
||||
- [ ] `apps/server/src/services/agents.ts` — in tool whitelist validation, skip validation for entries containing `*` (can't validate against runtime-discovered tools)
|
||||
- [ ] NEW helper `matchToolGlob(toolName: string, patterns: string[]): boolean` — supports `*` wildcard and `!` deny prefix, last-match-wins
|
||||
- [ ] Wire into `executeStreamPhase` (stream-phase.ts) where agent tools are filtered: replace exact-match `.includes()` with `matchToolGlob()`
|
||||
- [ ] Export `matchToolGlob` for test access
|
||||
|
||||
## B7 — Example config file
|
||||
|
||||
- [ ] NEW `data/mcp.json` with Context7 entry (enabled: true, with URL, no API key)
|
||||
- [ ] Comment in the file noting it's bind-mounted at `/data/mcp.json` inside the container
|
||||
|
||||
## B8 — Tests
|
||||
|
||||
- [ ] Update `mcp-client.test.ts` for multi-server wrapping (tools from two servers, prefix routing)
|
||||
- [ ] Test: server A fails, server B succeeds — only B's tools registered
|
||||
- [ ] Test: callTool routes to correct server by prefix
|
||||
- [ ] Test: shutdown kills stdio transports
|
||||
- [ ] NEW `apps/server/src/services/__tests__/mcp-glob.test.ts`
|
||||
- [ ] Test: exact match ("grep" matches "grep")
|
||||
- [ ] Test: wildcard ("context7_*" matches "context7_query-docs")
|
||||
- [ ] Test: deny ("!web_*" excludes "web_search")
|
||||
- [ ] Test: last-match-wins ("*" then "!web_*" → web tools excluded)
|
||||
- [ ] Test: empty pattern list → nothing matches (agent gets no tools — same as current behavior for explicit whitelists)
|
||||
|
||||
## B9 — Verification
|
||||
|
||||
- [ ] `npx tsc --noEmit -p apps/server` — 0 errors
|
||||
- [ ] `pnpm -C apps/server test` — all passing
|
||||
- [ ] `pnpm -C apps/web build` — green (no web changes)
|
||||
|
||||
## B10 — Deploy + smoke
|
||||
|
||||
- [ ] Create `/data/mcp.json` on the host with Context7 enabled
|
||||
- [ ] Update docker-compose bind mount if needed (data/ already mounted)
|
||||
- [ ] `docker compose up --build -d`
|
||||
- [ ] Check logs for multi-server init
|
||||
- [ ] Live-smoke: Context7 tool call from chat
|
||||
- [ ] Disable Context7 in config, restart, confirm zero MCP tools
|
||||
|
||||
## B11 — Docs + tag
|
||||
|
||||
- [ ] `CHANGELOG.md` entry
|
||||
- [ ] `boocode_roadmap.md` retrospective bullet on v1.15 section
|
||||
- [ ] `CLAUDE.md` — update MCP references
|
||||
- [ ] Commit, tag `v1.15.0-mcp-multi`, push, rebuild
|
||||
11
pnpm-lock.yaml
generated
11
pnpm-lock.yaml
generated
@@ -57,6 +57,9 @@ importers:
|
||||
'@fastify/websocket':
|
||||
specifier: ^10.0.1
|
||||
version: 10.0.1
|
||||
'@modelcontextprotocol/sdk':
|
||||
specifier: ^1.29.0
|
||||
version: 1.29.0(zod@3.25.76)
|
||||
ai:
|
||||
specifier: ^6.0.190
|
||||
version: 6.0.190(zod@3.25.76)
|
||||
@@ -5774,7 +5777,7 @@ snapshots:
|
||||
bytes: 3.1.2
|
||||
content-type: 1.0.5
|
||||
debug: 4.4.3
|
||||
http-errors: 2.0.0
|
||||
http-errors: 2.0.1
|
||||
iconv-lite: 0.7.2
|
||||
on-finished: 2.4.1
|
||||
qs: 6.15.1
|
||||
@@ -6151,7 +6154,7 @@ snapshots:
|
||||
etag: 1.8.1
|
||||
finalhandler: 2.1.1
|
||||
fresh: 2.0.0
|
||||
http-errors: 2.0.0
|
||||
http-errors: 2.0.1
|
||||
merge-descriptors: 2.0.0
|
||||
mime-types: 3.0.2
|
||||
on-finished: 2.4.1
|
||||
@@ -6163,7 +6166,7 @@ snapshots:
|
||||
router: 2.2.0
|
||||
send: 1.2.1
|
||||
serve-static: 2.2.1
|
||||
statuses: 2.0.1
|
||||
statuses: 2.0.2
|
||||
type-is: 2.1.0
|
||||
vary: 1.1.2
|
||||
transitivePeerDependencies:
|
||||
@@ -6262,7 +6265,7 @@ snapshots:
|
||||
escape-html: 1.0.3
|
||||
on-finished: 2.4.1
|
||||
parseurl: 1.3.3
|
||||
statuses: 2.0.1
|
||||
statuses: 2.0.2
|
||||
transitivePeerDependencies:
|
||||
- supports-color
|
||||
|
||||
|
||||
Reference in New Issue
Block a user