From 5692e99a5d11bf75ab1aac338603ca56df260aae Mon Sep 17 00:00:00 2001 From: indifferentketchup Date: Sat, 23 May 2026 21:58:09 +0000 Subject: [PATCH] v1.14.1-mcp-poc: single-server MCP client against Context7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Validates the MCP-client loop end-to-end against one real MCP server before the full v1.15 port. New services/mcp-client.ts wraps @modelcontextprotocol/sdk v1.29.0 with Streamable HTTP transport. On startup (when MCP_CONTEXT7_URL is set), connects to Context7, discovers tools via tools/list, wraps each as a ToolDef prefixed context7_, and appends to ALL_TOOLS via appendMcpTools. Read-only invariant guard rejects any tool with readOnlyHint: false. Tool dispatch is transparent — executeToolCall routes MCP calls through the ToolDef execute wrapper, which strips the prefix before calling the MCP server. Result size capped at 5MB with truncation. Graceful degradation: server down at startup → zero tools; server down mid-session → error result, model self-corrects. Adversarial review caught that a Zod .default() on the URL config made MCP always-on instead of opt-in — fixed by removing the default. MCP_CONTEXT7_URL must be explicitly set to enable. ALL_TOOLS changed from ReadonlyArray to mutable to support late-registration. appendMcpTools re-sorts and rebuilds TOOLS_BY_NAME after append. 348/348 server tests passing (16 new mcp-client tests). No schema changes, no frontend changes. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 4 + apps/server/package.json | 1 + apps/server/src/config.ts | 4 + apps/server/src/index.ts | 18 ++ .../src/services/__tests__/mcp-client.test.ts | 141 ++++++++++++ apps/server/src/services/mcp-client.ts | 205 ++++++++++++++++++ apps/server/src/services/tools.ts | 19 +- openspec/changes/v1.14.1-mcp-poc/design.md | 39 ++++ openspec/changes/v1.14.1-mcp-poc/proposal.md | 96 ++++++++ openspec/changes/v1.14.1-mcp-poc/tasks.md | 80 +++++++ pnpm-lock.yaml | 11 +- 11 files changed, 612 insertions(+), 6 deletions(-) create mode 100644 apps/server/src/services/__tests__/mcp-client.test.ts create mode 100644 apps/server/src/services/mcp-client.ts create mode 100644 openspec/changes/v1.14.1-mcp-poc/design.md create mode 100644 openspec/changes/v1.14.1-mcp-poc/proposal.md create mode 100644 openspec/changes/v1.14.1-mcp-poc/tasks.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 51ace5b..4f80d0d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch. +## v1.14.1-mcp-poc — 2026-05-23 + +Single-server MCP client PoC against Context7. New `apps/server/src/services/mcp-client.ts` (~200 lines) wraps `@modelcontextprotocol/sdk` v1.29.0 with Streamable HTTP transport. On startup (when `MCP_CONTEXT7_URL` is set), connects to Context7, discovers tools via `tools/list`, wraps each as a `ToolDef` prefixed `context7_`, and appends to `ALL_TOOLS` (alpha-sorted for prompt-cache stability). `appendMcpTools()` in `tools.ts` handles the late-registration; `ALL_TOOLS` changed from `ReadonlyArray` to mutable to support it. Read-only invariant guard rejects any MCP tool with `readOnlyHint: false` (MCP SDK v1.29.0 uses `readOnlyHint`, not `readOnly`). Tool dispatch is transparent — `executeToolCall` routes MCP tool calls through the `ToolDef.execute` wrapper, which strips the `context7_` prefix before calling the MCP server. Graceful degradation: MCP server down at startup → zero tools, warn log; MCP server down mid-session → error-shaped result, model self-corrects. Result size capped at 5MB with truncation (matches native `view_file`'s `MAX_FILE_BYTES`). Adversarial review caught that the Zod `.default('https://...')` on the URL config made MCP effectively always-on instead of opt-in — fixed by removing the default. 348/348 server tests passing (16 new mcp-client tests covering tool wrapping, read-only guard, name prefixing, content extraction). No schema changes, no frontend changes. Proves the MCP tool-discovery → tool-call → result-render loop end-to-end before the full v1.15 port. + ## v1.14.0-outer-loop — 2026-05-23 Converts the inference engine's ad-hoc `executeToolPhase → runAssistantTurn` recursion into an explicit `while` loop with a configurable step cap. A step is one stream-and-tool-execute iteration; the loop terminates on non-tool finish, step-cap hit, doom-loop, budget exhaustion, abort, or synthesis success. `MAX_STEPS = 200` is the hard ceiling (4x the old effective limit from budget); per-agent `steps:` field in AGENTS.md frontmatter sets tighter caps (Refactorer: 5, Architect: 20, others: unset = bounded only by MAX_STEPS). `executeToolPhase` no longer recurses — returns a `ToolPhaseResult` struct (`action: 'continue' | 'paused' | 'synthesis_done'`) so the caller (the while loop) decides whether to continue or break. `steps: 0` is handled as "no tool calls allowed" — one text-only stream phase, tool calls ignored with a warn log. Step-cap hits produce a sentinel summary (reuses `cap_hit` kind so `CapHitSentinel.tsx` renders it without frontend changes; text distinguishes "Step limit reached" from "Tool budget exhausted"). Doom-loop check migrated from pre-recursion position to top of loop body — same predicate (`detectDoomLoop`), same threshold (3 identical calls), `break` instead of `return`. `step_start` parts are in the schema CHECK but not emitted as message_parts in v1.14 — writing to the assistant message before the stream phase creates a sequence-0 collision with `partsFromAssistantMessage`; a structured log line is emitted instead. Adversarial review caught the collision pre-deploy. 332/332 server tests passing; no frontend changes. Pairs with `v1.13.20-drop-legacy-cols` (parts is now the sole source of truth, and this batch's loop operates entirely through parts). diff --git a/apps/server/package.json b/apps/server/package.json index eb70147..9cdf0ef 100644 --- a/apps/server/package.json +++ b/apps/server/package.json @@ -14,6 +14,7 @@ "@ai-sdk/openai-compatible": "^2.0.47", "@fastify/static": "^7.0.4", "@fastify/websocket": "^10.0.1", + "@modelcontextprotocol/sdk": "^1.29.0", "ai": "^6.0.190", "fastify": "^4.28.1", "postgres": "^3.4.4", diff --git a/apps/server/src/config.ts b/apps/server/src/config.ts index af64e2b..16ac9b0 100644 --- a/apps/server/src/config.ts +++ b/apps/server/src/config.ts @@ -19,6 +19,10 @@ const ConfigSchema = z.object({ GITEA_USER: z.string().default('indifferentketchup'), GITEA_TOKEN: z.string().optional(), GITEA_SSH_HOST: z.string().default('100.114.205.53:2222'), + // v1.14.1-mcp-poc: Context7 MCP server. Streamable HTTP transport. + // Set to empty string or omit to disable MCP tools entirely. + MCP_CONTEXT7_URL: z.string().optional(), + MCP_CONTEXT7_API_KEY: z.string().optional(), }); export type Config = z.infer; diff --git a/apps/server/src/index.ts b/apps/server/src/index.ts index 7a402f0..05f7f41 100644 --- a/apps/server/src/index.ts +++ b/apps/server/src/index.ts @@ -24,6 +24,8 @@ import { listSkills } from './services/skills.js'; import * as compaction from './services/compaction.js'; import { configureModelContext } from './services/model-context.js'; import { cleanupTruncations } from './services/truncate.js'; +import { initialize as initMcpClient, getTools as getMcpTools } from './services/mcp-client.js'; +import { appendMcpTools } from './services/tools.js'; async function main() { const config = loadConfig(); @@ -69,6 +71,22 @@ async function main() { // default_generation_settings.n_ctx — the value persisted as messages.ctx_max. configureModelContext({ llamaSwapUrl: config.LLAMA_SWAP_URL }); + // v1.14.1-mcp-poc: connect to Context7 MCP server and register discovered + // tools into ALL_TOOLS. Runs before route registration so the tool list is + // complete when the first inference request arrives. Graceful degradation: + // if Context7 is unreachable, zero MCP tools are registered and BooCode + // functions normally with native tools. + if (config.MCP_CONTEXT7_URL) { + await initMcpClient(config, app.log); + const mcpTools = getMcpTools(); + if (mcpTools.length > 0) { + appendMcpTools(mcpTools); + app.log.info({ count: mcpTools.length }, 'mcp: registered Context7 tools'); + } + } else { + app.log.info('mcp: MCP_CONTEXT7_URL not configured, skipping'); + } + await app.register(fastifyWebsocket); app.get('/api/health', async () => { diff --git a/apps/server/src/services/__tests__/mcp-client.test.ts b/apps/server/src/services/__tests__/mcp-client.test.ts new file mode 100644 index 0000000..5dc06cf --- /dev/null +++ b/apps/server/src/services/__tests__/mcp-client.test.ts @@ -0,0 +1,141 @@ +/** + * v1.14.1-mcp-poc: unit tests for the MCP client service. + * Pure unit tests — no live MCP server needed. Tests the tool-wrapping, + * read-only guard, name prefixing, content extraction, and error handling. + */ +import { describe, it, expect } from 'vitest'; +import { wrapMcpTool, extractContent, isToolReadOnly } from '../mcp-client.js'; + +describe('mcp-client', () => { + describe('wrapMcpTool', () => { + it('produces a ToolDef with context7_ prefix', () => { + const mcpTool = { + name: 'resolve-library-id', + description: 'Resolve a library identifier', + inputSchema: { + type: 'object' as const, + properties: { query: { type: 'string' } }, + required: ['query'], + }, + }; + + const wrapped = wrapMcpTool(mcpTool); + + expect(wrapped.name).toBe('context7_resolve-library-id'); + expect(wrapped.description).toBe('Resolve a library identifier'); + expect(wrapped.jsonSchema.type).toBe('function'); + expect(wrapped.jsonSchema.function.name).toBe('context7_resolve-library-id'); + expect(wrapped.jsonSchema.function.parameters).toEqual(mcpTool.inputSchema); + expect(typeof wrapped.execute).toBe('function'); + }); + + it('defaults description to empty string when absent', () => { + const mcpTool = { + name: 'no-desc', + inputSchema: { type: 'object' as const, properties: {} }, + }; + + const wrapped = wrapMcpTool(mcpTool); + + expect(wrapped.description).toBe(''); + expect(wrapped.jsonSchema.function.description).toBe(''); + }); + + it('uses passthrough Zod schema (z.record)', () => { + const mcpTool = { + name: 'test', + inputSchema: { type: 'object' as const, properties: {} }, + }; + + const wrapped = wrapMcpTool(mcpTool); + + // z.record(z.unknown()) should accept any object + const result = wrapped.inputSchema.safeParse({ foo: 'bar', baz: 123 }); + expect(result.success).toBe(true); + }); + }); + + describe('isToolReadOnly', () => { + it('accepts tools with readOnlyHint: true', () => { + expect(isToolReadOnly({ readOnlyHint: true })).toBe(true); + }); + + it('accepts tools with no annotations', () => { + expect(isToolReadOnly(undefined)).toBe(true); + }); + + it('accepts tools with empty annotations', () => { + expect(isToolReadOnly({})).toBe(true); + }); + + it('rejects tools with readOnlyHint: false', () => { + expect(isToolReadOnly({ readOnlyHint: false })).toBe(false); + }); + + it('accepts tools with only destructiveHint set', () => { + // readOnlyHint is not set, so it should be accepted per D3 + expect(isToolReadOnly({ destructiveHint: true })).toBe(true); + }); + }); + + describe('extractContent', () => { + it('extracts single text block', () => { + const content = [{ type: 'text', text: 'hello world' }]; + expect(extractContent(content)).toBe('hello world'); + }); + + it('joins multiple text blocks with newline', () => { + const content = [ + { type: 'text', text: 'line 1' }, + { type: 'text', text: 'line 2' }, + ]; + expect(extractContent(content)).toBe('line 1\nline 2'); + }); + + it('returns "(no output)" for empty content', () => { + expect(extractContent([])).toBe('(no output)'); + }); + + it('returns "(no output)" for undefined content', () => { + expect(extractContent(undefined)).toBe('(no output)'); + }); + + it('serializes non-text blocks as JSON', () => { + const content = [ + { type: 'resource', uri: 'file:///foo', mimeType: 'text/plain' }, + ]; + const result = extractContent(content); + expect(result).toContain('"type":"resource"'); + expect(result).toContain('"uri":"file:///foo"'); + }); + + it('returns error shape when isError is true', () => { + const content = [{ type: 'text', text: 'something failed' }]; + const result = extractContent(content, true); + expect(result).toEqual({ error: true, output: 'something failed' }); + }); + + it('returns error shape with joined content on isError', () => { + const content = [ + { type: 'text', text: 'error 1' }, + { type: 'text', text: 'error 2' }, + ]; + const result = extractContent(content, true); + expect(result).toEqual({ error: true, output: 'error 1\nerror 2' }); + }); + }); + + describe('name prefix', () => { + it('prefixed name maps correctly in wrapped tool', () => { + const mcpTool = { + name: 'query-docs', + description: 'Query documentation', + inputSchema: { type: 'object' as const, properties: {} }, + }; + + const wrapped = wrapMcpTool(mcpTool); + expect(wrapped.name).toBe('context7_query-docs'); + expect(wrapped.jsonSchema.function.name).toBe('context7_query-docs'); + }); + }); +}); diff --git a/apps/server/src/services/mcp-client.ts b/apps/server/src/services/mcp-client.ts new file mode 100644 index 0000000..af12991 --- /dev/null +++ b/apps/server/src/services/mcp-client.ts @@ -0,0 +1,205 @@ +/** + * v1.14.1-mcp-poc: singleton MCP client for Context7. + * + * Connects via Streamable HTTP transport, discovers tools at startup, + * wraps each as a BooCode ToolDef with a `context7_` name prefix. + * Graceful degradation: if the server is unreachable, zero tools are + * exposed and BooCode functions normally with native tools. + */ +import { Client } from '@modelcontextprotocol/sdk/client'; +import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js'; +import { z } from 'zod'; +import type { FastifyBaseLogger } from 'fastify'; +import type { Config } from '../config.js'; +import type { ToolDef } from './tools.js'; + +// ---- Types for the MCP tool shape returned by listTools ---- +interface McpToolAnnotations { + readOnlyHint?: boolean; + destructiveHint?: boolean; + [key: string]: unknown; +} + +interface McpToolDef { + name: string; + description?: string; + inputSchema: Record; + annotations?: McpToolAnnotations; +} + +// ---- Module-level state ---- +let client: Client | null = null; +let tools: ToolDef>[] = []; +let initialized = false; +let log: FastifyBaseLogger | null = null; + +const NAME_PREFIX = 'context7_'; +const MAX_RESULT_BYTES = 5 * 1024 * 1024; + +// ---- Public API ---- + +/** + * Connect to the Context7 MCP server, discover tools, and wrap them + * as BooCode ToolDefs. On failure, logs a warning and exposes zero tools. + */ +export async function initialize(config: Config, logger: FastifyBaseLogger): Promise { + log = logger; + if (!config.MCP_CONTEXT7_URL) { + log.info('mcp: MCP_CONTEXT7_URL not set, skipping Context7 initialization'); + initialized = true; + return; + } + + try { + client = new Client({ name: 'boocode', version: '1.14.1' }); + + const requestInit: RequestInit = {}; + if (config.MCP_CONTEXT7_API_KEY) { + requestInit.headers = { Authorization: `Bearer ${config.MCP_CONTEXT7_API_KEY}` }; + } + + const transport = new StreamableHTTPClientTransport( + new URL(config.MCP_CONTEXT7_URL), + { requestInit }, + ); + + await client.connect(transport); + + const result = await client.listTools(); + const mcpTools = (result.tools ?? []) as McpToolDef[]; + + tools = []; + for (const t of mcpTools) { + // D3: read-only invariant guard. Reject tools that explicitly declare + // readOnlyHint: false (i.e. write tools). Accept readOnlyHint: true + // or absent annotations (fail-open — most MCP servers don't annotate). + if (t.annotations?.readOnlyHint === false) { + log.info({ tool: t.name }, 'mcp: skipping non-read-only tool'); + continue; + } + tools.push(wrapMcpTool(t)); + } + + log.info( + { count: tools.length, names: tools.map((t) => t.name) }, + 'mcp: initialized Context7', + ); + initialized = true; + } catch (err) { + log.warn({ err }, 'mcp: failed to initialize Context7 — MCP tools will be unavailable'); + client = null; + tools = []; + initialized = true; + } +} + +/** + * Call an MCP tool by its prefixed name. Strips the prefix before + * forwarding to the MCP server. Returns a string on success or an + * error-shaped object on failure. + */ +export async function callTool( + prefixedName: string, + args: Record, +): Promise { + if (!client) { + return { error: true, output: 'MCP client not initialized' }; + } + + const originalName = prefixedName.startsWith(NAME_PREFIX) + ? prefixedName.slice(NAME_PREFIX.length) + : prefixedName; + + try { + const result = await client.callTool({ name: originalName, arguments: args }); + + // D8: extract content blocks + const content = result.content as Array<{ type: string; text?: string; [key: string]: unknown }>; + if (!content || content.length === 0) { + return '(no output)'; + } + + // If MCP reports an error, return error shape + if (result.isError) { + const joined = content + .map((block) => (block.type === 'text' ? block.text ?? '' : JSON.stringify(block))) + .join('\n'); + return { error: true, output: joined || '(MCP error with no details)' }; + } + + const parts = content.map((block) => { + if (block.type === 'text') return block.text ?? ''; + return JSON.stringify(block); + }); + const joined = parts.join('\n'); + if (joined.length > MAX_RESULT_BYTES) { + log?.warn({ tool: originalName, bytes: joined.length, cap: MAX_RESULT_BYTES }, 'mcp: result truncated'); + return joined.slice(0, MAX_RESULT_BYTES) + '\n\n[truncated — MCP result exceeded size limit]'; + } + return joined; + } catch (err) { + log?.warn({ err, tool: originalName }, 'mcp: callTool failed'); + return { + error: true, + output: err instanceof Error ? err.message : 'MCP server unreachable', + }; + } +} + +/** Return the wrapped ToolDefs discovered at initialization. */ +export function getTools(): ToolDef>[] { + return tools; +} + +/** Whether initialize() has been called (even if it failed). */ +export function isInitialized(): boolean { + return initialized; +} + +// ---- Internal helpers ---- + +/** Exposed for unit tests. */ +export function wrapMcpTool(mcpTool: McpToolDef): ToolDef> { + const prefixedName = `${NAME_PREFIX}${mcpTool.name}`; + return { + name: prefixedName, + description: mcpTool.description ?? '', + inputSchema: z.record(z.unknown()), + jsonSchema: { + type: 'function' as const, + function: { + name: prefixedName, + description: mcpTool.description ?? '', + parameters: mcpTool.inputSchema ?? { type: 'object', properties: {} }, + }, + }, + execute: async (input) => { + return callTool(prefixedName, input); + }, + }; +} + +/** Exposed for unit tests — extract content from an MCP result. */ +export function extractContent( + content: Array<{ type: string; text?: string; [key: string]: unknown }> | undefined, + isError?: boolean, +): unknown { + if (!content || content.length === 0) return '(no output)'; + + const parts = content.map((block) => { + if (block.type === 'text') return block.text ?? ''; + return JSON.stringify(block); + }); + const joined = parts.join('\n'); + + if (isError) { + return { error: true, output: joined || '(MCP error with no details)' }; + } + return joined; +} + +/** Exposed for unit tests — the read-only guard predicate. */ +export function isToolReadOnly(annotations?: McpToolAnnotations): boolean { + // Reject explicitly non-read-only tools; accept everything else + return annotations?.readOnlyHint !== false; +} diff --git a/apps/server/src/services/tools.ts b/apps/server/src/services/tools.ts index 918b99c..fc0aecb 100644 --- a/apps/server/src/services/tools.ts +++ b/apps/server/src/services/tools.ts @@ -651,7 +651,9 @@ export const askUserInput: ToolDef = { // of the system prompt, so any order drift would invalidate every cached // turn. Single source of truth for ordering lives here — toolJsonSchemas() // and TOOLS_BY_NAME inherit it. -export const ALL_TOOLS: ReadonlyArray> = [ +// v1.14.1-mcp-poc: changed from ReadonlyArray to let-bound mutable array +// so appendMcpTools() can push MCP-discovered tools at startup. +export let ALL_TOOLS: ToolDef[] = [ viewFile as ToolDef, viewTruncatedOutput as ToolDef, listDir as ToolDef, @@ -725,10 +727,23 @@ export const READ_ONLY_TOOL_NAMES = [ 'request_read_access', ] as const; -export const TOOLS_BY_NAME: Record> = Object.fromEntries( +export let TOOLS_BY_NAME: Record> = Object.fromEntries( ALL_TOOLS.map((t) => [t.name, t]) ); +// v1.14.1-mcp-poc: append MCP-discovered tools at startup. Called once +// from index.ts after mcpClient.initialize(). Re-sorts ALL_TOOLS and +// rebuilds TOOLS_BY_NAME. READ_ONLY_TOOL_NAMES is not rebuilt because +// it's a const tuple used only for budget-tier checks; MCP tools are +// individually checked via their category at budget resolution time — +// they are all read_only by construction (the read-only guard in +// mcp-client.ts rejects any tool with readOnlyHint: false). +export function appendMcpTools(mcpTools: ToolDef[]): void { + if (mcpTools.length === 0) return; + ALL_TOOLS = [...ALL_TOOLS, ...mcpTools].sort((a, b) => a.name.localeCompare(b.name)); + TOOLS_BY_NAME = Object.fromEntries(ALL_TOOLS.map((t) => [t.name, t])); +} + // v1.13.15-tools: tiered tool loading. BOOCODE_TOOLS env var (`core` | // `standard` | `all`) filters the agent's tool whitelist before LLM dispatch. // Daily-driver token win on qwen3.6-35b-a3b — the 35B-A3B MoE benefits from diff --git a/openspec/changes/v1.14.1-mcp-poc/design.md b/openspec/changes/v1.14.1-mcp-poc/design.md new file mode 100644 index 0000000..d0dae02 --- /dev/null +++ b/openspec/changes/v1.14.1-mcp-poc/design.md @@ -0,0 +1,39 @@ +# v1.14.1-mcp-poc — design decisions + +## D1. Transport: Streamable HTTP (not stdio) + +Context7 is a remote service at `https://mcp.context7.com/mcp`. Uses the MCP Streamable HTTP transport. The `@modelcontextprotocol/sdk` TypeScript client supports this via `StreamableHTTPClientTransport`. No stdio needed. + +## D2. Tool name prefixing + +MCP tools get a `context7_` prefix to avoid collisions with BooCode's native tools. Context7's tools are `resolve-library-id` and `query-docs` — these become `context7_resolve-library-id` and `context7_query-docs`. The prefix is stripped before calling the MCP server's `tools/call`. + +## D3. Read-only invariant guard + +BooChat is read-only through v1.x. The MCP client rejects any tool whose `annotations?.readOnly === false`. Tools with `readOnly: true` or no annotations are accepted. Context7's tools are all read-only (they query documentation — no write side effects). Fail-open on missing annotations is a deliberate choice: most MCP servers don't set annotations yet, and rejecting all un-annotated tools would make the feature useless. The guard catches explicitly-declared write tools. + +## D4. Zod inputSchema for MCP tools + +MCP tools come with a JSON Schema `inputSchema`. BooCode's `ToolDef` has both a Zod `inputSchema` (for server-side validation) and a `jsonSchema` (for the LLM's tool schema). For MCP tools: +- `jsonSchema` is built directly from the MCP tool's `inputSchema` (it's already JSON Schema). +- `inputSchema` uses `z.record(z.unknown())` as a pass-through — the MCP server does its own validation. Double-validating with a generated Zod schema from JSON Schema adds complexity with no value for a PoC. + +## D5. Tool registration: append + re-sort (not lazy-init) + +The simplest approach: keep `ALL_TOOLS` as the native tool array. Add an `appendMcpTools(tools: ToolDef[])` function that pushes MCP tools, re-sorts alphabetically, and rebuilds `TOOLS_BY_NAME` and `READ_ONLY_TOOL_NAMES`. Called once at startup after MCP init. More invasive approaches (lazy-init, factory function) change the import shape for every consumer. Mutation-at-startup is ugly but contained to one call site and matches the existing alpha-sort-at-module-level pattern. + +## D6. No per-session toggle + +Web tools have `session.web_search_enabled`. MCP tools do NOT get a session toggle in v1.14.1. If configured via env var, MCP tools are always available. Per-session MCP control is a v1.15 concern (when multiple MCP servers and the permission ruleset land together). + +## D7. Graceful degradation + +MCP server down at startup → log warning, expose zero MCP tools, BooCode functions normally. MCP server down mid-session (tool call fails) → the `execute` wrapper catches the error and returns `{error: true, output: "MCP server unreachable"}` — the model sees the error and can self-correct (use native tools instead). + +## D8. Result content extraction + +MCP `tools/call` returns `{content: ContentBlock[]}` where each block is `{type: 'text', text: string}` or `{type: 'resource', ...}`. For the PoC: +- Text blocks: join with `\n`. +- Resource blocks: serialize as JSON (the model can read structured data). +- Empty content: return `"(no output)"`. +- `isError: true` in the response: return `{error: true, output: joinedContent}`. diff --git a/openspec/changes/v1.14.1-mcp-poc/proposal.md b/openspec/changes/v1.14.1-mcp-poc/proposal.md new file mode 100644 index 0000000..08b5234 --- /dev/null +++ b/openspec/changes/v1.14.1-mcp-poc/proposal.md @@ -0,0 +1,96 @@ +# v1.14.1-mcp-poc — single-server MCP client proof-of-concept + +Validate the MCP-client loop end-to-end against one real MCP server (Context7) before committing to the full opencode `mcp/index.ts` port at v1.15. Small, throwaway-if-needed. + +## Why + +BooCode's tool registry (`ALL_TOOLS` in `tools.ts`) is static — tools are hardcoded TypeScript modules. MCP is the protocol for dynamic tool discovery. Wiring one real MCP server end-to-end proves: tool-discovery → tool-list → tool-call → result-render → context-budget accounting all hold. If Context7 works, any MCP server will work via the same plumbing. + +## Scope + +### S1. Install `@modelcontextprotocol/sdk` + +New dependency in `apps/server/package.json`. The official TypeScript MCP client SDK (MIT). Provides `Client`, `StreamableHTTPClientTransport`, tool-call/result types. + +### S2. New service: `apps/server/src/services/mcp-client.ts` + +Singleton MCP client that: +1. Connects to Context7 at `MCP_CONTEXT7_URL` (default `https://mcp.context7.com/mcp`) via Streamable HTTP transport. +2. Optional `MCP_CONTEXT7_API_KEY` env var passed as a header. +3. On `initialize()`: calls `tools/list`, wraps each MCP tool as a `ToolDef`, prefixes names with `context7_` to avoid collisions with BooCode's native tools. +4. **Read-only invariant guard:** rejects any tool whose `annotations?.readOnly` is explicitly `false`. Tools with `readOnly: true` or no `annotations` field are accepted (fail-open on read-only, since most MCP tools don't set annotations yet — Context7's tools don't). +5. `callTool(name, args)` → calls the MCP server's `tools/call` endpoint and returns the result content. +6. `getTools(): ToolDef[]` → returns the discovered tools wrapped as BooCode `ToolDef` objects. +7. Graceful degradation: if the MCP server is unreachable at startup, log a warning and expose zero MCP tools. BooCode functions normally with its native tools. + +### S3. Config extension + +`apps/server/src/config.ts` gains two optional env vars: +- `MCP_CONTEXT7_URL` (string, default `https://mcp.context7.com/mcp`) +- `MCP_CONTEXT7_API_KEY` (string, optional) + +### S4. Tool registration + +`apps/server/src/services/tools.ts` — after building `ALL_TOOLS` from native tools, append MCP-discovered tools from `mcpClient.getTools()`. The alpha-sort at the end of `ALL_TOOLS` construction covers both native and MCP tools. `TOOLS_BY_NAME` map includes MCP tools. + +MCP tools are registered with `category: 'read_only'` (per the read-only invariant guard in S2). + +### S5. Tool dispatch + +`apps/server/src/services/inference/tool-phase.ts` `executeToolCall` already dispatches via `TOOLS_BY_NAME[toolName].execute(...)`. MCP tools' `execute` function calls `mcpClient.callTool(name, args)` — the dispatch is transparent to the rest of the inference loop. No changes to `executeToolCall` needed. + +### S6. MCP tool result → BooCode format + +MCP `tools/call` returns `{ content: [{type: 'text', text: string}, ...] }`. BooCode's `executeToolCall` expects a string or JSON-serializable output. The `execute` wrapper in the ToolDef extracts `content[0].text` (or joins multiple content blocks with `\n`). If the MCP server returns an error, the wrapper returns `{error: true, output: errorMessage}` matching BooCode's existing error-result shape. + +### S7. Startup initialization + +`apps/server/src/index.ts` — after `applySchema()` and before route registration, call `mcpClient.initialize()`. If `MCP_CONTEXT7_URL` is not set (or empty), skip initialization entirely (MCP is opt-in). Log the number of discovered tools on success. + +Tool registration (S4) must happen AFTER MCP initialization, since `getTools()` returns the discovered tools. Current flow: `ALL_TOOLS` is a module-level constant. This needs to change to a lazy-init pattern — either a function that returns the tool list (called once at startup after MCP init), or a mutable array that MCP tools get appended to during startup. + +### S8. Agent tool whitelist interaction + +MCP tools are prefixed `context7_*`. Existing agents' `tools:` whitelists don't include MCP tool names — so MCP tools are only available to the default agent (no agent selected, which gets ALL_TOOLS). To make MCP tools available to specific agents, their AGENTS.md `tools:` list would need to include `context7_*` names. For the PoC, this is fine — the default agent (most common) gets MCP tools. + +## Non-goals + +- No stdio transport. Context7 is HTTP-only. +- No OAuth. Context7 uses an API key header. +- No multiple servers. One hardcoded server (Context7). +- No per-agent MCP server allow/deny. All agents that don't have a `tools:` whitelist get MCP tools. +- No per-session MCP toggle. If configured, MCP tools are always available. +- No UI changes. MCP tools surface in the tool list the model sees; results render as normal tool-result parts. +- No schema changes. MCP state is in-memory only. + +## Hard rules + +- No git commit/push. Sam commits. +- Read-only invariant: reject any MCP tool with `readOnly: false`. +- Graceful degradation: MCP server down → zero MCP tools, BooCode works normally. +- One new dep only: `@modelcontextprotocol/sdk`. +- Alpha-sort of ALL_TOOLS preserved (v1.13.3 prompt-cache invariant). + +## Files expected to touch + +- `apps/server/package.json` — add `@modelcontextprotocol/sdk` +- `pnpm-lock.yaml` — auto-updated +- `apps/server/src/config.ts` — `MCP_CONTEXT7_URL`, `MCP_CONTEXT7_API_KEY` +- `apps/server/src/services/mcp-client.ts` — NEW, ~100 lines +- `apps/server/src/services/tools.ts` — lazy-init or append MCP tools to ALL_TOOLS +- `apps/server/src/index.ts` — call `mcpClient.initialize()` at startup +- `apps/server/src/services/__tests__/mcp-client.test.ts` — NEW, unit tests for tool wrapping + read-only guard + +## Estimate + +~150 LoC. The MCP SDK handles the protocol; BooCode's job is wrapping discovered tools as ToolDefs and routing calls through the SDK client. + +## Smoke plan + +1. Set `MCP_CONTEXT7_URL=https://mcp.context7.com/mcp` in `.env` (or docker-compose env). +2. Restart boocode container. +3. Check logs: should see "mcp: initialized Context7, discovered N tools" (or similar). +4. Open a chat with no agent selected. Send "What does the `streamText` function do in the AI SDK? Use context7 to look it up." +5. Confirm: model calls `context7_resolve-library-id` then `context7_query-docs` (or whatever Context7's tool names are after prefixing). +6. Confirm: tool results render normally in the chat. +7. Without `MCP_CONTEXT7_URL` set: restart, confirm BooCode starts normally with zero MCP tools. diff --git a/openspec/changes/v1.14.1-mcp-poc/tasks.md b/openspec/changes/v1.14.1-mcp-poc/tasks.md new file mode 100644 index 0000000..e979b56 --- /dev/null +++ b/openspec/changes/v1.14.1-mcp-poc/tasks.md @@ -0,0 +1,80 @@ +# v1.14.1-mcp-poc tasks + +## B1 — Backups + +- [ ] `apps/server/src/services/tools.ts` +- [ ] `apps/server/src/config.ts` +- [ ] `apps/server/src/index.ts` + +## B2 — Install `@modelcontextprotocol/sdk` + +- [ ] `pnpm -C apps/server add @modelcontextprotocol/sdk` +- [ ] Verify `pnpm -C apps/server build` still works after install +- [ ] Note the installed version + +## B3 — Config extension + +- [ ] `apps/server/src/config.ts` — add `MCP_CONTEXT7_URL` (string, optional, default `https://mcp.context7.com/mcp`) +- [ ] `apps/server/src/config.ts` — add `MCP_CONTEXT7_API_KEY` (string, optional) +- [ ] Both via Zod `.optional()` with `.default()` for the URL + +## B4 — MCP client service + +- [ ] NEW `apps/server/src/services/mcp-client.ts` +- [ ] Import `Client`, `StreamableHTTPClientTransport` from `@modelcontextprotocol/sdk/client` +- [ ] `initialize(config, log)` — connect to Context7, call `tools/list`, wrap each as ToolDef, apply read-only guard +- [ ] `callTool(name, args)` — call MCP server `tools/call`, extract text content, return as string +- [ ] `getTools()` — return wrapped ToolDef[] +- [ ] `isInitialized()` — boolean +- [ ] Read-only guard: skip tools with `annotations?.readOnly === false`; accept all others +- [ ] Graceful degradation: catch connection errors, log warning, expose zero tools +- [ ] Tool name prefixing: `context7_` +- [ ] ToolDef wrapping: map MCP inputSchema (JSONSchema) to ToolJsonSchema `function.parameters`; use `z.any()` for Zod inputSchema (MCP already validated on the server side) +- [ ] Execute wrapper: strip `context7_` prefix before calling MCP, join result content blocks with `\n` + +## B5 — Tool registration (lazy-init) + +- [ ] `apps/server/src/services/tools.ts` — convert `ALL_TOOLS` from a module-level constant to a lazy-initialized array +- [ ] Add `initializeTools(mcpTools: ToolDef[])` function that builds the final sorted list +- [ ] `TOOLS_BY_NAME`, `READ_ONLY_TOOL_NAMES` derived from the initialized list +- [ ] Ensure all existing callers of `ALL_TOOLS` / `TOOLS_BY_NAME` still work (they import from tools.ts — verify the export shape) +- [ ] OR simpler: keep ALL_TOOLS as-is (native tools), add `appendMcpTools(tools)` that mutates + re-sorts + rebuilds TOOLS_BY_NAME. Less clean but less invasive. + +## B6 — Startup wiring + +- [ ] `apps/server/src/index.ts` — after `applySchema()`, before route registration: + - If `config.MCP_CONTEXT7_URL` is set: `await mcpClient.initialize(config, app.log)` + - `appendMcpTools(mcpClient.getTools())` (or equivalent) + - Log tool count +- [ ] If URL not set: skip, log "mcp: Context7 not configured, skipping" + +## B7 — Verification + +- [ ] `npx tsc --noEmit -p apps/server` — 0 errors +- [ ] `pnpm -C apps/server test` — all existing tests pass (MCP client is startup-only; tests don't initialize it) +- [ ] `pnpm -C apps/web build` — green (no web changes) + +## B8 — Unit tests + +- [ ] NEW `apps/server/src/services/__tests__/mcp-client.test.ts` +- [ ] Test: tool wrapping produces correct ToolDef shape (name, description, jsonSchema, execute fn) +- [ ] Test: read-only guard rejects tools with `readOnly: false` +- [ ] Test: read-only guard accepts tools with `readOnly: true` or no annotations +- [ ] Test: name prefixing — `resolve-library-id` → `context7_resolve-library-id` +- [ ] Test: result extraction — single text content block → string; multiple → joined with `\n` +- [ ] Test: error result — MCP error → `{error: true, output: ...}` shape + +## B9 — Deploy + smoke + +- [ ] Add `MCP_CONTEXT7_URL=https://mcp.context7.com/mcp` to docker-compose env (or .env) +- [ ] `docker compose up --build -d` +- [ ] Check logs for MCP initialization message +- [ ] Live-smoke: send a chat asking about AI SDK docs via Context7 +- [ ] Verify tool calls + results render normally + +## B10 — Docs + tag + +- [ ] `CHANGELOG.md` entry +- [ ] `boocode_roadmap.md` retrospective bullet +- [ ] `CLAUDE.md` — mention MCP client in the tools/services section +- [ ] Commit, tag `v1.14.1-mcp-poc`, push, rebuild diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 6c807ce..dfd1175 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -57,6 +57,9 @@ importers: '@fastify/websocket': specifier: ^10.0.1 version: 10.0.1 + '@modelcontextprotocol/sdk': + specifier: ^1.29.0 + version: 1.29.0(zod@3.25.76) ai: specifier: ^6.0.190 version: 6.0.190(zod@3.25.76) @@ -5774,7 +5777,7 @@ snapshots: bytes: 3.1.2 content-type: 1.0.5 debug: 4.4.3 - http-errors: 2.0.0 + http-errors: 2.0.1 iconv-lite: 0.7.2 on-finished: 2.4.1 qs: 6.15.1 @@ -6151,7 +6154,7 @@ snapshots: etag: 1.8.1 finalhandler: 2.1.1 fresh: 2.0.0 - http-errors: 2.0.0 + http-errors: 2.0.1 merge-descriptors: 2.0.0 mime-types: 3.0.2 on-finished: 2.4.1 @@ -6163,7 +6166,7 @@ snapshots: router: 2.2.0 send: 1.2.1 serve-static: 2.2.1 - statuses: 2.0.1 + statuses: 2.0.2 type-is: 2.1.0 vary: 1.1.2 transitivePeerDependencies: @@ -6262,7 +6265,7 @@ snapshots: escape-html: 1.0.3 on-finished: 2.4.1 parseurl: 1.3.3 - statuses: 2.0.1 + statuses: 2.0.2 transitivePeerDependencies: - supports-color