Compare commits
6 Commits
v1.12.4-in
...
v1.13.1-cl
| Author | SHA1 | Date | |
|---|---|---|---|
| a08d809b73 | |||
| ac1a71f583 | |||
| 13c3aa5b4e | |||
| c2c4f78a26 | |||
| 1cb6eee24c | |||
| ca64bf9f0a |
10
CLAUDE.md
10
CLAUDE.md
@@ -46,7 +46,9 @@ Tests: `pnpm -C apps/server test` runs the vitest suite. No test harness on `app
|
||||
- **Zod** for request validation and config parsing.
|
||||
|
||||
Key services:
|
||||
- **`services/inference.ts`** — Streams LLM responses, executes tool loops (max depth 15, see `MAX_TOOL_LOOP_DEPTH`), flushes to DB every 500ms. Publishes `InferenceFrame` events through the broker. **`TurnArgs`** is the per-turn state envelope threaded through the `executeToolPhase → runAssistantTurn` recursion (`toolsUsed`, `recentToolCalls`, `assistantMessageId`, `signal`); reset to defaults in `runInference` at the user-message boundary. Cap-hit (`toolsUsed >= budget`) and doom-loop (`detectDoomLoop(recentToolCalls)`) checks both read from this envelope. Add new per-turn state here, not in module-level closures.
|
||||
- **`services/inference/`** (v1.12.4 split — was a single `inference.ts` file). Public surface re-exported via `inference/index.ts`; callers import from `./services/inference/index.js`. Layout: `turn.ts` (runAssistantTurn / runInference / createInferenceRunner orchestration, plus `InferenceFrame`, `InferenceContext`, `TurnArgs`, `StreamResult` exported), `stream-phase.ts` (streamCompletion + executeStreamPhase + SSE parsing), `tool-phase.ts` (executeToolPhase; back-edges into turn.ts for the runAssistantTurn recursion — cycle is safe because dereferenced at call time, not module top-level), `sentinel-summaries.ts` (runCapHitSummary + runDoomLoopSummary + their sentinel inserters; two near-clones kept side-by-side until a third sentinel justifies factoring out runWrapUpSummary), `error-handler.ts` (handleAbortOrError, finalizeCompletion), `payload.ts` (buildMessagesPayload, loadContext, maybeFlagForCompaction, `OpenAiMessage`), `sentinels.ts` (`detectDoomLoop`, `DOOM_LOOP_THRESHOLD`, sentinel predicates), `budget.ts` (resolveToolBudget), `xml-parser.ts` (Qwen-coder XML tool-call fallback), `types.ts` (`StreamPhaseState`, `DB_FLUSH_INTERVAL_MS` shared between stream-phase and sentinel-summaries). **`TurnArgs`** is the per-turn state envelope threaded through the `executeToolPhase → runAssistantTurn` recursion (`toolsUsed`, `recentToolCalls`, `assistantMessageId`, `signal`); reset to defaults in `runInference` at the user-message boundary. Cap-hit (`toolsUsed >= budget`) and doom-loop (`detectDoomLoop(recentToolCalls)`) checks both read from this envelope. Add new per-turn state to `TurnArgs` in `turn.ts`, not module-level closures.
|
||||
- **`chat_status` frame shape** (published via `broker.publishUser`) — `status: 'streaming' | 'tool_running' | 'waiting_for_input' | 'idle' | 'error'` (widened from `working|idle|error` in v1.12.1). Frontend `useChatStatus` derives `idle_warm` (<30s since idle) vs `idle_cold`. `ChatThroughput` renders inline beside `StatusDot` only when streaming or tool_running, fed by 500ms-throttled `'usage'` WS frames (`completion_tokens` + `ctx_used` + `ctx_max`). The `POST /api/chats/:id/discard_stale` endpoint exists to mark a stuck-streaming row as `failed` when the frontend's 60s no-token-activity timer (`ChatPane` content-length watcher) gives up.
|
||||
- **Boot-time stale-streaming sweep** in `apps/server/src/index.ts` after `applySchema()`: any `messages.status='streaming'` older than 5 minutes flips to `'failed'`. Logs only on non-zero count. Recovers from container restart while inference was mid-stream (v1.12.1).
|
||||
- **`services/broker.ts`** — In-memory pub/sub with two channel types: per-session (message streaming) and per-user (sidebar updates). No persistence; clients reconnect on restart.
|
||||
- **`services/tools.ts`** — Tool registry (`ALL_TOOLS`, `READ_ONLY_TOOL_NAMES`, `TOOLS_BY_NAME`). Filesystem tools (view_file/list_dir/grep/find_files) go through three guard layers: `path_guard.ts` (workspace scope), `secret_guard.ts` (filename deny list), `url_guard.ts` (SSRF/private-IP block for web_fetch). v1.11.8+ web tools (`web_search`, `web_fetch`) are opt-in per chat via `session.web_search_enabled` (resolved with `project.default_web_search_enabled` fallback) and filtered out of the LLM's tool schema when false.
|
||||
- **`services/compaction.ts`** + **`services/model-context.ts`** — v1.11.0 anchored rolling summary (single `summary=true` assistant row per chat, supersedes itself on each compaction). Triggered when `chats.needs_compaction` is set after an inference turn exceeds `usable(ctx_max) = ctx_max - 20k`. **`ctx_max` comes from `model-context.getModelContext()` which fetches `${LLAMA_SWAP_URL}/upstream/<model>/props`** — NOT from `parsed.timings.n_ctx` (the stream completion's `timings` doesn't carry n_ctx; that read was dead code until v1.11.3 ripped it out).
|
||||
@@ -87,15 +89,14 @@ Font / CSS pipeline (apps/web):
|
||||
|
||||
### Multi-pane workspace
|
||||
|
||||
Sessions hold 1–5 panes (chat / empty / placeholder terminal+agent). Workspace pane state is **client-side only** (localStorage key `boocode.workspace.panes.<sessionId>`); the legacy `session_panes` table and its REST endpoints are deprecated — no `/api/panes/*` routes exist. Each chat lives in at most one pane; tab strip is per-pane and tracks `chatIds[]` + `activeChatIdx`. Sessions 1:N chats; chats own messages. Tab reorder via native HTML5 drag events.
|
||||
Sessions hold 1–5 panes (chat / empty / placeholder terminal+agent). v1.12.1 moved pane state from per-device localStorage to `sessions.workspace_panes jsonb` for cross-device sync. `PATCH /api/sessions/:id/workspace` persists; `session_workspace_updated` user-channel frame broadcasts to every device watching the session. `useWorkspacePanes` debounces saves 300ms and dedups echoes by JSON string. Legacy localStorage key `boocode.workspace.panes.<sessionId>` is read once on first hydrate (one-time seed-and-delete migration when server is empty but localStorage has data); no longer written. The deprecated `session_panes` table was dropped. `validatePanes(validChatIds)` prunes panes referencing chat IDs that no longer exist (called by `useSessionChats` after the chat list fetch lands). Each chat lives in at most one pane; tab strip is per-pane and tracks `chatIds[]` + `activeChatIdx`. Tab reorder via native HTML5 drag events.
|
||||
|
||||
## Database
|
||||
|
||||
PostgreSQL 16. Tables: `projects`, `sessions`, `chats`, `messages`, `settings`, `session_panes` (deprecated). Schema applied idempotently on startup via `applySchema()`. Use `clock_timestamp()` (not `NOW()`) inside transactions. CHECK constraints in place: `projects_status_chk` ('open'|'archived'), `sessions_status_chk` (same), `chats_status_chk` (same), `messages_role_chk`, `messages_status_chk` — keep in sync with the `*_STATUSES` const arrays in `apps/server/src/types/api.ts`.
|
||||
PostgreSQL 16. Tables: `projects`, `sessions`, `chats`, `messages`, `settings`. (`session_panes` was dropped in v1.12.1; workspace pane state lives in `sessions.workspace_panes jsonb`.) Schema applied idempotently on startup via `applySchema()`. Use `clock_timestamp()` (not `NOW()`) inside transactions. CHECK constraints in place: `projects_status_chk` ('open'|'archived'), `sessions_status_chk` (same), `chats_status_chk` (same), `messages_role_chk`, `messages_status_chk` — keep in sync with the `*_STATUSES` const arrays in `apps/server/src/types/api.ts`. The older anonymous `messages_status_check` (without 'cancelled') and `messages_role_check` (without 'system') were dropped in v1.12.1; only the `_chk` variants remain.
|
||||
|
||||
Schema CHECK migration order when renaming allowed values: (1) `ALTER TABLE ... DROP CONSTRAINT IF EXISTS <system_name>` (inline `CREATE TABLE` checks get `<table>_<column>_check`), (2) `UPDATE` rows to new values, (3) wrap new constraint ADD in `DO $$ ... pg_constraint` guard — that block is the only way to get `ADD CONSTRAINT IF NOT EXISTS`.
|
||||
|
||||
Position-shift pattern for panes (legacy `session_panes` table): negate-and-restore to avoid UNIQUE(session_id, position) collisions during reorder/insert/delete. Sentinel value -100 for the moving pane.
|
||||
|
||||
## Environment
|
||||
|
||||
@@ -125,6 +126,7 @@ Required: `DATABASE_URL`, `LLAMA_SWAP_URL`. Optional: `PORT` (3000), `HOST` (0.0
|
||||
- TypeScript strict mode. Both apps share `tsconfig.base.json`.
|
||||
- Server uses NodeNext module resolution (`.js` extensions in imports).
|
||||
- Discriminated unions for type narrowing: `Pane` (by `kind`), `SessionEvent` (by `type`), `InferenceFrame` (by `type`).
|
||||
- **Adding a new WS frame type** requires updating BOTH the server's `InferenceFrame` (loose `type:` union + optional fields in `services/inference/turn.ts`) AND the web `WsFrame` (strict discriminated union in `apps/web/src/api/types.ts`). Server publish is permissive; the frontend type is the wire-format gate. The `'usage'` frame added in v1.12.2 needed both sides; missing the web side silently drops the frame at JSON-parse.
|
||||
- shadcn primitives live in `components/ui/`. Don't modify them unless adding a new primitive.
|
||||
- `inferLanguage()` from `lib/attachments.ts` is the canonical file-extension-to-language map. `CodeBlock.tsx` keeps its own `LANG_MAP` because it also resolves markdown fence names.
|
||||
- Two UI event buses: `hooks/sessionEvents.ts` for DB-state events (chat_created, session_updated); `lib/events.ts` for ephemeral UI (`sendToTerminal`, `terminalsRegistry`). Don't merge — different subscriber lifecycles.
|
||||
|
||||
@@ -11,8 +11,10 @@
|
||||
"test": "vitest run"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai-compatible": "^2.0.47",
|
||||
"@fastify/static": "^7.0.4",
|
||||
"@fastify/websocket": "^10.0.1",
|
||||
"ai": "^6.0.190",
|
||||
"fastify": "^4.28.1",
|
||||
"postgres": "^3.4.4",
|
||||
"ws": "^8.18.0",
|
||||
|
||||
@@ -201,6 +201,46 @@ async function main() {
|
||||
app.log.info(`serving static frontend from ${webDist}`);
|
||||
}
|
||||
|
||||
// v1.13.3: periodic in-process sweeper for streaming rows orphaned by a
|
||||
// mid-session crash. The boot sweep (above) only fires once at startup;
|
||||
// this loop catches the in-flight case. 60s cadence + 5-min threshold
|
||||
// matches the boot sweep so behavior is consistent. Publishes
|
||||
// chat_status='idle' on the user channel so the UI dot drops without a
|
||||
// refresh — same pattern as handleAbortOrError.
|
||||
const SWEEP_INTERVAL_MS = 60_000;
|
||||
const sweepStaleStreaming = async (): Promise<void> => {
|
||||
try {
|
||||
const rows = await sql<{ id: string; chat_id: string }[]>`
|
||||
UPDATE messages
|
||||
SET status = 'failed', finished_at = clock_timestamp()
|
||||
WHERE status = 'streaming'
|
||||
AND created_at < NOW() - INTERVAL '5 minutes'
|
||||
RETURNING id, chat_id
|
||||
`;
|
||||
if (rows.length === 0) return;
|
||||
app.log.warn(
|
||||
{ swept: rows.length, ids: rows.map((r) => r.id) },
|
||||
'swept stale streaming rows',
|
||||
);
|
||||
const seenChats = new Set<string>();
|
||||
const now = new Date().toISOString();
|
||||
for (const row of rows) {
|
||||
if (seenChats.has(row.chat_id)) continue;
|
||||
seenChats.add(row.chat_id);
|
||||
broker.publishUser('default', {
|
||||
type: 'chat_status',
|
||||
chat_id: row.chat_id,
|
||||
status: 'idle',
|
||||
at: now,
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
app.log.error({ err }, 'stuck-row sweeper failed');
|
||||
}
|
||||
};
|
||||
const sweepTimer = setInterval(() => { void sweepStaleStreaming(); }, SWEEP_INTERVAL_MS);
|
||||
app.addHook('onClose', async () => { clearInterval(sweepTimer); });
|
||||
|
||||
const shutdown = async (signal: string) => {
|
||||
app.log.info(`received ${signal}, shutting down`);
|
||||
try {
|
||||
|
||||
@@ -313,6 +313,28 @@ export function registerChatRoutes(
|
||||
AND created_at <= ${target.created_at}::timestamptz
|
||||
AND status = 'complete'
|
||||
`;
|
||||
// v1.13.0: clone message_parts for the forked messages. Source and
|
||||
// destination preserve ordering (the INSERT above orders by created_at,
|
||||
// id) so a ROW_NUMBER pairing maps source.id → dest.id deterministically.
|
||||
await tx`
|
||||
WITH src AS (
|
||||
SELECT id, ROW_NUMBER() OVER (ORDER BY created_at ASC, id ASC) AS rn
|
||||
FROM messages
|
||||
WHERE chat_id = ${source.id}
|
||||
AND created_at <= ${target.created_at}::timestamptz
|
||||
AND status = 'complete'
|
||||
),
|
||||
dst AS (
|
||||
SELECT id, ROW_NUMBER() OVER (ORDER BY created_at ASC, id ASC) AS rn
|
||||
FROM messages
|
||||
WHERE chat_id = ${chat!.id}
|
||||
)
|
||||
INSERT INTO message_parts (message_id, sequence, kind, payload)
|
||||
SELECT dst.id, p.sequence, p.kind, p.payload
|
||||
FROM message_parts p
|
||||
JOIN src ON p.message_id = src.id
|
||||
JOIN dst ON dst.rn = src.rn
|
||||
`;
|
||||
return chat!;
|
||||
});
|
||||
|
||||
@@ -401,11 +423,12 @@ export function registerChatRoutes(
|
||||
reply.code(404);
|
||||
return { error: 'chat not found' };
|
||||
}
|
||||
// v1.13.1-B: reads tool_calls/tool_results via the parts-merged view.
|
||||
const rows = await sql<Message[]>`
|
||||
SELECT id, session_id, chat_id, role, content, kind, tool_calls, tool_results, status, last_seq,
|
||||
tokens_used, ctx_used, ctx_max, started_at, finished_at, created_at, metadata,
|
||||
summary, tail_start_id, compacted_at
|
||||
FROM messages
|
||||
FROM messages_with_parts
|
||||
WHERE chat_id = ${req.params.id}
|
||||
ORDER BY created_at ASC, id ASC
|
||||
`;
|
||||
|
||||
@@ -91,11 +91,12 @@ export function registerMessageRoutes(
|
||||
// SummaryCard) and shows compacted_at-stamped rows inline for context.
|
||||
// Internal inference assembly filters compacted_at IS NULL separately —
|
||||
// see services/inference.ts loadContext + services/compaction.ts.
|
||||
// v1.13.1-B: reads tool_calls/tool_results via the parts-merged view.
|
||||
const rows = await sql<Message[]>`
|
||||
SELECT id, session_id, chat_id, role, content, kind, tool_calls, tool_results, status, last_seq,
|
||||
tokens_used, ctx_used, ctx_max, started_at, finished_at, created_at, metadata,
|
||||
summary, tail_start_id, compacted_at
|
||||
FROM messages
|
||||
FROM messages_with_parts
|
||||
WHERE session_id = ${req.params.id}
|
||||
ORDER BY created_at ASC, id ASC
|
||||
`;
|
||||
@@ -469,30 +470,36 @@ export function registerMessageRoutes(
|
||||
const chat = chatRows[0]!;
|
||||
const sessionId = chat.session_id;
|
||||
|
||||
// Find the assistant message that emitted this tool_call. Scoped by
|
||||
// chat_id + role to avoid cross-chat lookups; ordered by created_at DESC
|
||||
// because the most recent issuance wins when an LLM reuses call IDs
|
||||
// across turns (the older, already-answered one is a different row with
|
||||
// populated tool_results downstream).
|
||||
const callerRows = await sql<{ id: string; tool_calls: ToolCall[] | null }[]>`
|
||||
SELECT id, tool_calls FROM messages
|
||||
WHERE chat_id = ${chat.id}
|
||||
AND role = 'assistant'
|
||||
AND tool_calls IS NOT NULL
|
||||
ORDER BY created_at DESC
|
||||
// v1.13.1-C: find the assistant's tool_call by indexing message_parts
|
||||
// directly on payload->>'id'. Scoped by chat_id + role via the JOIN.
|
||||
// Pre-v1.13.0 history has no parts rows — those tool_calls become
|
||||
// unreachable here (404). Acceptable per the dispatch decision: any
|
||||
// pending elicitation from before v1.13.0 is long timed out by now;
|
||||
// promote to a hotfix with a JSON-column fallback if it ever surfaces.
|
||||
const callerRows = await sql<{
|
||||
message_id: string;
|
||||
payload: { id: string; name: string; args: Record<string, unknown> };
|
||||
}[]>`
|
||||
SELECT p.message_id, p.payload
|
||||
FROM message_parts p
|
||||
JOIN messages m ON m.id = p.message_id
|
||||
WHERE m.chat_id = ${chat.id}
|
||||
AND m.role = 'assistant'
|
||||
AND p.kind = 'tool_call'
|
||||
AND p.payload->>'id' = ${tool_call_id}
|
||||
ORDER BY m.created_at DESC
|
||||
LIMIT 1
|
||||
`;
|
||||
let foundCall: ToolCall | null = null;
|
||||
for (const row of callerRows) {
|
||||
const match = row.tool_calls?.find((tc) => tc.id === tool_call_id);
|
||||
if (match) {
|
||||
foundCall = match;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!foundCall) {
|
||||
const callerRow = callerRows[0];
|
||||
if (!callerRow) {
|
||||
reply.code(404);
|
||||
return { error: 'unknown_tool_call_id' };
|
||||
}
|
||||
const foundCall: ToolCall = {
|
||||
id: callerRow.payload.id,
|
||||
name: callerRow.payload.name,
|
||||
args: callerRow.payload.args,
|
||||
};
|
||||
if (foundCall.name !== 'ask_user_input') {
|
||||
reply.code(400);
|
||||
return { error: 'tool_call_not_ask_user_input' };
|
||||
@@ -539,18 +546,21 @@ export function registerMessageRoutes(
|
||||
}
|
||||
}
|
||||
|
||||
// Find the pending tool row. ORDER BY created_at DESC + LIMIT 1 picks
|
||||
// the most recent row with this tool_call_id; the already-answered
|
||||
// check below guards against UPDATE-ing a stale answer.
|
||||
// v1.13.1-C: find the pending tool row via message_parts on
|
||||
// payload->>'tool_call_id'. Same fallback caveat as the caller lookup
|
||||
// above — pre-v1.13.0 rows are unreachable here.
|
||||
const toolRows = await sql<{
|
||||
id: string;
|
||||
tool_results: { tool_call_id: string; output: unknown } | null;
|
||||
message_id: string;
|
||||
payload: { tool_call_id: string; output: unknown };
|
||||
}[]>`
|
||||
SELECT id, tool_results FROM messages
|
||||
WHERE chat_id = ${chat.id}
|
||||
AND role = 'tool'
|
||||
AND tool_results->>'tool_call_id' = ${tool_call_id}
|
||||
ORDER BY created_at DESC
|
||||
SELECT p.message_id, p.payload
|
||||
FROM message_parts p
|
||||
JOIN messages m ON m.id = p.message_id
|
||||
WHERE m.chat_id = ${chat.id}
|
||||
AND m.role = 'tool'
|
||||
AND p.kind = 'tool_result'
|
||||
AND p.payload->>'tool_call_id' = ${tool_call_id}
|
||||
ORDER BY m.created_at DESC
|
||||
LIMIT 1
|
||||
`;
|
||||
const toolRow = toolRows[0];
|
||||
@@ -558,7 +568,7 @@ export function registerMessageRoutes(
|
||||
reply.code(404);
|
||||
return { error: 'unknown_tool_call_id', detail: 'tool message not found' };
|
||||
}
|
||||
if (toolRow.tool_results && toolRow.tool_results.output !== null) {
|
||||
if (toolRow.payload && toolRow.payload.output !== null) {
|
||||
reply.code(409);
|
||||
return { error: 'tool_call_already_answered' };
|
||||
}
|
||||
@@ -570,11 +580,21 @@ export function registerMessageRoutes(
|
||||
truncated: false,
|
||||
};
|
||||
|
||||
const toolMessageId = toolRow.message_id;
|
||||
const result = await sql.begin(async (tx) => {
|
||||
await tx`
|
||||
UPDATE messages
|
||||
SET tool_results = ${tx.json(newToolResults as never)}
|
||||
WHERE id = ${toolRow.id}
|
||||
WHERE id = ${toolMessageId}
|
||||
`;
|
||||
// v1.13.0: replace the pending tool_result part inserted at message
|
||||
// creation (tool-phase.ts) with the answered one. Delete-then-insert
|
||||
// is simpler than UPDATE because parts are append-style elsewhere;
|
||||
// the UNIQUE (message_id, sequence) constraint blocks plain insert.
|
||||
await tx`DELETE FROM message_parts WHERE message_id = ${toolMessageId} AND kind = 'tool_result'`;
|
||||
await tx`
|
||||
INSERT INTO message_parts (message_id, sequence, kind, payload)
|
||||
VALUES (${toolMessageId}, 0, 'tool_result', ${tx.json(newToolResults as never)})
|
||||
`;
|
||||
const [assistantMsg] = await tx<{ id: string }[]>`
|
||||
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
||||
@@ -584,7 +604,7 @@ export function registerMessageRoutes(
|
||||
await tx`UPDATE sessions SET updated_at = clock_timestamp() WHERE id = ${sessionId}`;
|
||||
await tx`UPDATE chats SET updated_at = clock_timestamp() WHERE id = ${chat.id}`;
|
||||
return {
|
||||
tool_message_id: toolRow.id,
|
||||
tool_message_id: toolMessageId,
|
||||
assistant_message_id: assistantMsg!.id,
|
||||
};
|
||||
});
|
||||
|
||||
@@ -90,11 +90,26 @@ export function registerSkillsRoutes(
|
||||
VALUES (${sessionId}, ${chat.id}, 'assistant', '', ${sql.json(toolCalls as never)}, 'complete', clock_timestamp())
|
||||
RETURNING id
|
||||
`;
|
||||
// v1.13.0: dual-write the synthetic assistant message's tool_call.
|
||||
// Single skill_use tool_call, no text content, so one part at seq 0.
|
||||
await tx`
|
||||
INSERT INTO message_parts (message_id, sequence, kind, payload)
|
||||
VALUES (${synthAssistant!.id}, 0, 'tool_call', ${tx.json({
|
||||
id: toolCallId,
|
||||
name: 'skill_use',
|
||||
args: { name: skill_name },
|
||||
} as never)})
|
||||
`;
|
||||
const [toolMsg] = await tx<{ id: string }[]>`
|
||||
INSERT INTO messages (session_id, chat_id, role, content, tool_results, status, created_at)
|
||||
VALUES (${sessionId}, ${chat.id}, 'tool', '', ${sql.json(toolResults as never)}, 'complete', clock_timestamp())
|
||||
RETURNING id
|
||||
`;
|
||||
// v1.13.0: dual-write the synthetic tool result (the skill body).
|
||||
await tx`
|
||||
INSERT INTO message_parts (message_id, sequence, kind, payload)
|
||||
VALUES (${toolMsg!.id}, 0, 'tool_result', ${tx.json(toolResults as never)})
|
||||
`;
|
||||
const [userMsg] = await tx<{ id: string }[]>`
|
||||
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
||||
VALUES (${sessionId}, ${chat.id}, 'user', ${userText}, 'complete', clock_timestamp())
|
||||
|
||||
@@ -23,11 +23,12 @@ export function registerWebSocket(
|
||||
|
||||
// v1.11: snapshot includes compaction fields so MessageBubble can
|
||||
// render the SummaryCard for summary=true rows on first connect.
|
||||
// v1.13.1-B: reads tool_calls/tool_results via the parts-merged view.
|
||||
const messages = await sql<Message[]>`
|
||||
SELECT id, session_id, chat_id, role, content, kind, tool_calls, tool_results, status, last_seq,
|
||||
tokens_used, ctx_used, ctx_max, started_at, finished_at, created_at, metadata,
|
||||
summary, tail_start_id, compacted_at
|
||||
FROM messages
|
||||
FROM messages_with_parts
|
||||
WHERE session_id = ${sessionId}
|
||||
ORDER BY created_at ASC, id ASC
|
||||
`;
|
||||
|
||||
@@ -1,3 +1,10 @@
|
||||
-- v1.13.3: statement_timeout is set at database level via:
|
||||
-- ALTER DATABASE boocode SET statement_timeout = '30s';
|
||||
-- ALTER DATABASE can't run inside a DO block, so this is an operational
|
||||
-- step rather than schema. Re-apply after a volume reset (the setting
|
||||
-- lives in pg_db which survives `docker compose up --build` but NOT a
|
||||
-- `docker volume rm boocode_pgdata`).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS projects (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
name TEXT NOT NULL,
|
||||
@@ -32,6 +39,59 @@ CREATE TABLE IF NOT EXISTS messages (
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_session ON messages(session_id, created_at);
|
||||
|
||||
-- v1.13.0: granular message parts table for AI SDK migration. Old
|
||||
-- messages.content / tool_calls / tool_results columns stay authoritative
|
||||
-- for reads in v1.13.0; this table is dual-written so the swap can happen
|
||||
-- in a later dispatch without a backfill window. ON DELETE CASCADE means
|
||||
-- removing a message removes its parts in one go.
|
||||
CREATE TABLE IF NOT EXISTS message_parts (
|
||||
id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
message_id uuid NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
|
||||
sequence int NOT NULL,
|
||||
kind text NOT NULL,
|
||||
payload jsonb NOT NULL,
|
||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||
CONSTRAINT message_parts_kind_chk CHECK (kind IN ('text', 'tool_call', 'tool_result', 'reasoning', 'step_start')),
|
||||
CONSTRAINT message_parts_seq_uniq UNIQUE (message_id, sequence)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS message_parts_msg_seq_idx ON message_parts (message_id, sequence);
|
||||
|
||||
-- v1.13.1-B: read-path view. Read sites SELECT FROM messages_with_parts
|
||||
-- instead of messages so tool_calls / tool_results / reasoning_parts come
|
||||
-- from the granular message_parts table. The COALESCE means pre-v1.13.0
|
||||
-- history (no parts rows) still resolves via the legacy JSON columns; the
|
||||
-- dual-write from v1.13.0 keeps both in sync for all rows written since.
|
||||
-- Writes continue to target `messages` directly — the view is read-only.
|
||||
-- Shapes match the in-memory ToolCall / ToolResult types: tool_calls is a
|
||||
-- jsonb array of {id, name, args}, tool_results is a single jsonb object
|
||||
-- {tool_call_id, output, truncated, error?}. reasoning_parts is new — only
|
||||
-- consumed by the inference history fetch (payload.ts) so v1.13.1-C can
|
||||
-- wire reasoning into the model payload. Not surfaced in external APIs yet.
|
||||
CREATE OR REPLACE VIEW messages_with_parts AS
|
||||
SELECT
|
||||
m.id, m.session_id, m.chat_id, m.role, m.content, m.kind, m.status,
|
||||
m.last_seq, m.tokens_used, m.ctx_used, m.ctx_max,
|
||||
m.started_at, m.finished_at, m.created_at, m.metadata,
|
||||
m.summary, m.tail_start_id, m.compacted_at,
|
||||
COALESCE(
|
||||
(SELECT jsonb_agg(p.payload ORDER BY p.sequence)
|
||||
FROM message_parts p
|
||||
WHERE p.message_id = m.id AND p.kind = 'tool_call'),
|
||||
m.tool_calls
|
||||
) AS tool_calls,
|
||||
COALESCE(
|
||||
(SELECT p.payload
|
||||
FROM message_parts p
|
||||
WHERE p.message_id = m.id AND p.kind = 'tool_result'
|
||||
ORDER BY p.sequence
|
||||
LIMIT 1),
|
||||
m.tool_results
|
||||
) AS tool_results,
|
||||
(SELECT jsonb_agg(p.payload ORDER BY p.sequence)
|
||||
FROM message_parts p
|
||||
WHERE p.message_id = m.id AND p.kind = 'reasoning') AS reasoning_parts
|
||||
FROM messages m;
|
||||
|
||||
ALTER TABLE messages ADD COLUMN IF NOT EXISTS tokens_used INTEGER;
|
||||
ALTER TABLE messages ADD COLUMN IF NOT EXISTS ctx_used INTEGER;
|
||||
ALTER TABLE messages ADD COLUMN IF NOT EXISTS ctx_max INTEGER;
|
||||
|
||||
121
apps/server/src/services/__tests__/parts.test.ts
Normal file
121
apps/server/src/services/__tests__/parts.test.ts
Normal file
@@ -0,0 +1,121 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { partsFromAssistantMessage, partsFromToolMessage } from '../inference/parts.js';
|
||||
import type { ToolCall, ToolResult } from '../../types/api.js';
|
||||
|
||||
describe('partsFromAssistantMessage', () => {
|
||||
it('emits one text part for content-only assistant', () => {
|
||||
const parts = partsFromAssistantMessage({ content: 'hello world', tool_calls: null });
|
||||
expect(parts).toHaveLength(1);
|
||||
expect(parts[0]).toEqual({
|
||||
sequence: 0,
|
||||
kind: 'text',
|
||||
payload: { text: 'hello world' },
|
||||
});
|
||||
});
|
||||
|
||||
it('emits one tool_call part for empty-content + single tool_call', () => {
|
||||
const tc: ToolCall = { id: 'call_1', name: 'view_file', args: { path: 'src/a.ts' } };
|
||||
const parts = partsFromAssistantMessage({ content: '', tool_calls: [tc] });
|
||||
expect(parts).toHaveLength(1);
|
||||
expect(parts[0]).toEqual({
|
||||
sequence: 0,
|
||||
kind: 'tool_call',
|
||||
payload: { id: 'call_1', name: 'view_file', args: { path: 'src/a.ts' } },
|
||||
});
|
||||
});
|
||||
|
||||
it('emits text then tool_call parts in order when both present', () => {
|
||||
const tc: ToolCall = { id: 'call_2', name: 'grep', args: { pattern: 'foo' } };
|
||||
const parts = partsFromAssistantMessage({ content: 'let me search', tool_calls: [tc] });
|
||||
expect(parts.map((p) => [p.sequence, p.kind])).toEqual([
|
||||
[0, 'text'],
|
||||
[1, 'tool_call'],
|
||||
]);
|
||||
});
|
||||
|
||||
it('preserves tool_call order with multiple calls', () => {
|
||||
const calls: ToolCall[] = [
|
||||
{ id: 'a', name: 'list_dir', args: { path: '.' } },
|
||||
{ id: 'b', name: 'view_file', args: { path: 'x.ts' } },
|
||||
{ id: 'c', name: 'grep', args: { pattern: 'y' } },
|
||||
];
|
||||
const parts = partsFromAssistantMessage({ content: '', tool_calls: calls });
|
||||
expect(parts).toHaveLength(3);
|
||||
expect(parts.map((p) => p.payload)).toEqual([
|
||||
{ id: 'a', name: 'list_dir', args: { path: '.' } },
|
||||
{ id: 'b', name: 'view_file', args: { path: 'x.ts' } },
|
||||
{ id: 'c', name: 'grep', args: { pattern: 'y' } },
|
||||
]);
|
||||
expect(parts.map((p) => p.sequence)).toEqual([0, 1, 2]);
|
||||
});
|
||||
|
||||
it('returns empty array for empty content + null tool_calls', () => {
|
||||
expect(partsFromAssistantMessage({ content: '', tool_calls: null })).toEqual([]);
|
||||
});
|
||||
|
||||
it('v1.13.1-C: reasoning lands at sequence 0 before text + tool_calls', () => {
|
||||
const tc: ToolCall = { id: 'call_r', name: 'view_file', args: { path: 'x.ts' } };
|
||||
const parts = partsFromAssistantMessage({
|
||||
content: 'inspecting now',
|
||||
tool_calls: [tc],
|
||||
reasoning: 'user asked about x.ts; I should view it',
|
||||
});
|
||||
expect(parts.map((p) => [p.sequence, p.kind])).toEqual([
|
||||
[0, 'reasoning'],
|
||||
[1, 'text'],
|
||||
[2, 'tool_call'],
|
||||
]);
|
||||
expect(parts[0]!.payload).toEqual({
|
||||
text: 'user asked about x.ts; I should view it',
|
||||
});
|
||||
});
|
||||
|
||||
it('v1.13.1-C: reasoning + empty content + tool_calls preserves seq 0 reasoning', () => {
|
||||
const tc: ToolCall = { id: 'call_r2', name: 'grep', args: { pattern: 'foo' } };
|
||||
const parts = partsFromAssistantMessage({
|
||||
content: '',
|
||||
tool_calls: [tc],
|
||||
reasoning: 'jumping straight to grep',
|
||||
});
|
||||
expect(parts.map((p) => [p.sequence, p.kind])).toEqual([
|
||||
[0, 'reasoning'],
|
||||
[1, 'tool_call'],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('partsFromToolMessage', () => {
|
||||
it('emits a single tool_result part at sequence 0', () => {
|
||||
const tr: ToolResult = {
|
||||
tool_call_id: 'call_1',
|
||||
output: { contents: 'console.log(1)' },
|
||||
truncated: false,
|
||||
};
|
||||
const parts = partsFromToolMessage({ tool_results: tr });
|
||||
expect(parts).toHaveLength(1);
|
||||
expect(parts[0]).toEqual({
|
||||
sequence: 0,
|
||||
kind: 'tool_result',
|
||||
payload: {
|
||||
tool_call_id: 'call_1',
|
||||
output: { contents: 'console.log(1)' },
|
||||
truncated: false,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('includes error in payload when present', () => {
|
||||
const tr: ToolResult = {
|
||||
tool_call_id: 'call_2',
|
||||
output: null,
|
||||
truncated: false,
|
||||
error: 'permission denied',
|
||||
};
|
||||
const parts = partsFromToolMessage({ tool_results: tr });
|
||||
expect(parts[0]!.payload).toMatchObject({ error: 'permission denied' });
|
||||
});
|
||||
|
||||
it('returns empty array when tool_results is null', () => {
|
||||
expect(partsFromToolMessage({ tool_results: null })).toEqual([]);
|
||||
});
|
||||
});
|
||||
14
apps/server/src/services/__tests__/tools.test.ts
Normal file
14
apps/server/src/services/__tests__/tools.test.ts
Normal file
@@ -0,0 +1,14 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { ALL_TOOLS } from '../tools.js';
|
||||
|
||||
describe('ALL_TOOLS registry', () => {
|
||||
// v1.13.3: tools must be alpha-sorted at module load. llama.cpp's prompt
|
||||
// cache hits on byte-identical prefixes; the tool list lives near the
|
||||
// top of the system prompt, so any order drift invalidates every cached
|
||||
// turn. The registry sort is the single source of truth; downstream
|
||||
// helpers (toolJsonSchemas, TOOLS_BY_NAME, buildAiTools) inherit it.
|
||||
it('exports tools in alphabetical order by name', () => {
|
||||
const names = ALL_TOOLS.map((t) => t.name);
|
||||
expect(names).toEqual([...names].sort((a, b) => a.localeCompare(b)));
|
||||
});
|
||||
});
|
||||
@@ -342,9 +342,11 @@ export async function process(input: ProcessInput): Promise<void> {
|
||||
// 2. All currently-active messages in this chat (compacted_at IS NULL).
|
||||
// ORDER BY (created_at, id) matches loadContext in inference.ts so the
|
||||
// turns() boundary logic sees the same sequence the LLM will.
|
||||
// v1.13.1-B: reads tool_calls/tool_results via the parts-merged view so
|
||||
// the compaction payload matches what the LLM saw on the original turn.
|
||||
const messages = await sql<CompactionMessage[]>`
|
||||
SELECT id, role, content, kind, summary, status, tool_calls, tool_results, metadata, created_at
|
||||
FROM messages
|
||||
FROM messages_with_parts
|
||||
WHERE chat_id = ${chatId} AND compacted_at IS NULL
|
||||
ORDER BY created_at ASC, id ASC
|
||||
`;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { MessageMetadata, Session } from '../../types/api.js';
|
||||
import * as modelContext from '../model-context.js';
|
||||
import { maybeFlagForCompaction } from './payload.js';
|
||||
import { insertParts, partsFromAssistantMessage } from './parts.js';
|
||||
import type { InferenceContext, StreamResult, TurnArgs } from './turn.js';
|
||||
|
||||
export async function handleAbortOrError(
|
||||
@@ -112,6 +113,24 @@ export async function finalizeCompletion(
|
||||
WHERE id = ${assistantMessageId}
|
||||
RETURNING tokens_used, ctx_used, ctx_max, finished_at
|
||||
`;
|
||||
// v1.13.0: dual-write the text part. finalizeCompletion is the terminal
|
||||
// path for text-only assistant turns (no tool calls); tool_calls are null
|
||||
// here by construction (the tool-bearing path goes through executeToolPhase).
|
||||
// v1.13.1-C: include result.reasoning so reasoning-channel models capture
|
||||
// a kind='reasoning' part alongside the text.
|
||||
// TODO(v1.13.1): wrap the UPDATE above and this insertParts in a single
|
||||
// sql.begin before flipping read authority to message_parts.
|
||||
await insertParts(
|
||||
ctx.sql,
|
||||
partsFromAssistantMessage({
|
||||
content,
|
||||
tool_calls: null,
|
||||
reasoning: result.reasoning,
|
||||
}).map((p) => ({
|
||||
...p,
|
||||
message_id: assistantMessageId,
|
||||
})),
|
||||
);
|
||||
// v1.11: flag for compaction on the terminal turn too. Catches the common
|
||||
// case of a turn that hit the limit without invoking tools.
|
||||
await maybeFlagForCompaction(ctx, chatId, updated);
|
||||
|
||||
95
apps/server/src/services/inference/parts.ts
Normal file
95
apps/server/src/services/inference/parts.ts
Normal file
@@ -0,0 +1,95 @@
|
||||
import type { Sql } from '../../db.js';
|
||||
import type { ToolCall, ToolResult } from '../../types/api.js';
|
||||
|
||||
// v1.13.0: dual-write helper. Every site that writes the legacy
|
||||
// messages.tool_calls / messages.tool_results JSON columns calls into here
|
||||
// to mirror the same data into message_parts rows. Reads still go to the
|
||||
// JSON columns; the swap to parts-as-source-of-truth happens in a later
|
||||
// v1.13 dispatch alongside the AI SDK streamText migration.
|
||||
|
||||
export type PartKind = 'text' | 'tool_call' | 'tool_result' | 'reasoning' | 'step_start';
|
||||
|
||||
export interface PartInsert {
|
||||
message_id: string;
|
||||
sequence: number;
|
||||
kind: PartKind;
|
||||
payload: unknown;
|
||||
}
|
||||
|
||||
export async function insertParts(sql: Sql, parts: PartInsert[]): Promise<void> {
|
||||
if (parts.length === 0) return;
|
||||
// postgres-js fans out an array of objects to a multi-row INSERT. Each
|
||||
// payload field needs sql.json() so jsonb storage receives a JSON value
|
||||
// rather than a quoted string.
|
||||
await sql`
|
||||
INSERT INTO message_parts ${sql(
|
||||
parts.map((p) => ({
|
||||
message_id: p.message_id,
|
||||
sequence: p.sequence,
|
||||
kind: p.kind,
|
||||
payload: sql.json(p.payload as never),
|
||||
})),
|
||||
'message_id',
|
||||
'sequence',
|
||||
'kind',
|
||||
'payload',
|
||||
)}
|
||||
`;
|
||||
}
|
||||
|
||||
// Derive parts from the canonical messages row for an assistant message.
|
||||
// reasoning (when non-empty) becomes a 'reasoning' part at sequence 0 —
|
||||
// it precedes user-visible content logically. content (when non-empty)
|
||||
// becomes a 'text' part next; each tool_call becomes a 'tool_call' part
|
||||
// with payload { id, name, args } where args is the parsed object (we
|
||||
// use the in-memory ToolCall shape, not the OpenAI stringified one).
|
||||
export function partsFromAssistantMessage(args: {
|
||||
content: string;
|
||||
tool_calls: ToolCall[] | null;
|
||||
// v1.13.1-C: optional reasoning text streamed alongside the answer.
|
||||
// Most rows have none — only models with separate reasoning channels
|
||||
// (qwen3.6 etc.) populate this.
|
||||
reasoning?: string;
|
||||
}): Omit<PartInsert, 'message_id'>[] {
|
||||
const out: Omit<PartInsert, 'message_id'>[] = [];
|
||||
let seq = 0;
|
||||
if (args.reasoning && args.reasoning.length > 0) {
|
||||
out.push({ sequence: seq, kind: 'reasoning', payload: { text: args.reasoning } });
|
||||
seq += 1;
|
||||
}
|
||||
if (args.content && args.content.length > 0) {
|
||||
out.push({ sequence: seq, kind: 'text', payload: { text: args.content } });
|
||||
seq += 1;
|
||||
}
|
||||
for (const tc of args.tool_calls ?? []) {
|
||||
out.push({
|
||||
sequence: seq,
|
||||
kind: 'tool_call',
|
||||
payload: { id: tc.id, name: tc.name, args: tc.args },
|
||||
});
|
||||
seq += 1;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Derive a single tool_result part from a tool message's tool_results JSON.
|
||||
// The payload includes the same shape that buildMessagesPayload reads from
|
||||
// later: tool_call_id, output, optional error/truncated metadata.
|
||||
export function partsFromToolMessage(args: {
|
||||
tool_results: ToolResult | null;
|
||||
}): Omit<PartInsert, 'message_id'>[] {
|
||||
if (!args.tool_results) return [];
|
||||
const tr = args.tool_results;
|
||||
return [
|
||||
{
|
||||
sequence: 0,
|
||||
kind: 'tool_result',
|
||||
payload: {
|
||||
tool_call_id: tr.tool_call_id,
|
||||
output: tr.output,
|
||||
truncated: tr.truncated,
|
||||
...(tr.error ? { error: tr.error } : {}),
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
@@ -19,6 +19,12 @@ export interface OpenAiMessage {
|
||||
function: { name: string; arguments: string };
|
||||
}>;
|
||||
tool_call_id?: string;
|
||||
// v1.13.1-C: reasoning text from a prior assistant turn, sourced from
|
||||
// message_parts kind='reasoning' rows joined in via reasoning_parts on
|
||||
// the messages_with_parts view. stream-phase.ts/toModelMessages threads
|
||||
// this into the AI SDK ReasoningPart when forwarding to the model so
|
||||
// reasoning models can resume mid-thought across tool-call boundaries.
|
||||
reasoning?: string;
|
||||
}
|
||||
|
||||
// v1.12: buildSystemPrompt lives in services/system-prompt.ts. It awaits the
|
||||
@@ -83,6 +89,12 @@ export async function buildMessagesPayload(
|
||||
function: { name: tc.name, arguments: JSON.stringify(tc.args) },
|
||||
}));
|
||||
}
|
||||
// v1.13.1-C: collapse reasoning_parts into a single string. The view
|
||||
// returns them ordered by sequence; multiple reasoning parts on one
|
||||
// message are rare but concat preserves ordering. Skip when absent.
|
||||
if (m.reasoning_parts && m.reasoning_parts.length > 0) {
|
||||
msg.reasoning = m.reasoning_parts.map((p) => p.text ?? '').join('');
|
||||
}
|
||||
out.push(msg);
|
||||
continue;
|
||||
}
|
||||
@@ -116,10 +128,14 @@ export async function loadContext(
|
||||
// /api/sessions/:id/messages endpoint still returns everything (so the UI
|
||||
// can show history with the summary card inline); only LLM payloads skip
|
||||
// compacted rows. compacted_at IS NULL keeps the active summary + tail.
|
||||
// v1.13.1-B: reads tool_calls/tool_results via the parts-merged view.
|
||||
// v1.13.1-C: also pull reasoning_parts so assistant messages from
|
||||
// reasoning models can be replayed with their reasoning context preserved.
|
||||
const history = await sql<Message[]>`
|
||||
SELECT id, session_id, chat_id, role, content, kind, tool_calls, tool_results, status, last_seq,
|
||||
tokens_used, ctx_used, ctx_max, started_at, finished_at, created_at, metadata
|
||||
FROM messages
|
||||
tokens_used, ctx_used, ctx_max, started_at, finished_at, created_at, metadata,
|
||||
reasoning_parts
|
||||
FROM messages_with_parts
|
||||
WHERE chat_id = ${chatId} AND compacted_at IS NULL
|
||||
ORDER BY created_at ASC, id ASC
|
||||
`;
|
||||
|
||||
26
apps/server/src/services/inference/provider.ts
Normal file
26
apps/server/src/services/inference/provider.ts
Normal file
@@ -0,0 +1,26 @@
|
||||
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
|
||||
import type { LanguageModel } from 'ai';
|
||||
|
||||
// v1.13.1-A: AI SDK provider against llama-swap. baseURL is threaded from
|
||||
// config.LLAMA_SWAP_URL at call time (not module-load) so tests can stub the
|
||||
// upstream without touching env vars. No apiKey — llama-swap is unauth in our
|
||||
// Tailscale topology and exposing it over the public internet is gated by
|
||||
// Authelia at the Caddy layer, not by API keys.
|
||||
|
||||
const cache = new Map<string, ReturnType<typeof createOpenAICompatible>>();
|
||||
|
||||
function getProvider(baseURL: string): ReturnType<typeof createOpenAICompatible> {
|
||||
let provider = cache.get(baseURL);
|
||||
if (!provider) {
|
||||
provider = createOpenAICompatible({
|
||||
name: 'llama-swap',
|
||||
baseURL: baseURL.endsWith('/v1') ? baseURL : `${baseURL}/v1`,
|
||||
});
|
||||
cache.set(baseURL, provider);
|
||||
}
|
||||
return provider;
|
||||
}
|
||||
|
||||
export function upstreamModel(baseURL: string, modelId: string): LanguageModel {
|
||||
return getProvider(baseURL).chatModel(modelId);
|
||||
}
|
||||
@@ -18,29 +18,15 @@ import type {
|
||||
StreamResult,
|
||||
TurnArgs,
|
||||
} from './turn.js';
|
||||
|
||||
interface ChatCompletionDelta {
|
||||
role?: string;
|
||||
content?: string | null;
|
||||
tool_calls?: Array<{
|
||||
index: number;
|
||||
id?: string;
|
||||
type?: 'function';
|
||||
function?: { name?: string; arguments?: string };
|
||||
}>;
|
||||
}
|
||||
|
||||
interface ChatCompletionChunk {
|
||||
choices?: Array<{
|
||||
delta: ChatCompletionDelta;
|
||||
finish_reason: string | null;
|
||||
}>;
|
||||
usage?: {
|
||||
prompt_tokens?: number;
|
||||
completion_tokens?: number;
|
||||
total_tokens?: number;
|
||||
};
|
||||
}
|
||||
import { upstreamModel } from './provider.js';
|
||||
import {
|
||||
jsonSchema,
|
||||
streamText,
|
||||
tool,
|
||||
type JSONValue,
|
||||
type ModelMessage,
|
||||
type ToolCallRepairFunction,
|
||||
} from 'ai';
|
||||
|
||||
interface StreamOptions {
|
||||
// null = omit tools entirely (compact phase); [] = caller stripped all tools
|
||||
@@ -49,44 +35,113 @@ interface StreamOptions {
|
||||
temperature?: number;
|
||||
}
|
||||
|
||||
async function* sseLines(stream: ReadableStream<Uint8Array>): AsyncGenerator<string> {
|
||||
const reader = stream.getReader();
|
||||
const decoder = new TextDecoder('utf-8');
|
||||
let buffer = '';
|
||||
try {
|
||||
while (true) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done) break;
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
let idx;
|
||||
while ((idx = buffer.indexOf('\n')) >= 0) {
|
||||
const line = buffer.slice(0, idx).replace(/\r$/, '');
|
||||
buffer = buffer.slice(idx + 1);
|
||||
if (line.length === 0) continue;
|
||||
yield line;
|
||||
// v1.13.1-A: convert BooCode's OpenAI-shaped history into AI SDK
|
||||
// ModelMessage[]. Tool result messages need a `toolName` field that the
|
||||
// OpenAI shape doesn't carry; we look it up by scanning earlier assistant
|
||||
// `tool_calls` entries for a matching id.
|
||||
function toModelMessages(messages: OpenAiMessage[]): ModelMessage[] {
|
||||
const toolNameById = new Map<string, string>();
|
||||
for (const m of messages) {
|
||||
if (m.role === 'assistant' && m.tool_calls) {
|
||||
for (const tc of m.tool_calls) {
|
||||
toolNameById.set(tc.id, tc.function.name);
|
||||
}
|
||||
}
|
||||
if (buffer.length > 0) yield buffer;
|
||||
} finally {
|
||||
reader.releaseLock();
|
||||
}
|
||||
const out: ModelMessage[] = [];
|
||||
for (const m of messages) {
|
||||
if (m.role === 'system' || m.role === 'user') {
|
||||
out.push({ role: m.role, content: m.content ?? '' });
|
||||
continue;
|
||||
}
|
||||
if (m.role === 'assistant') {
|
||||
const hasTools = m.tool_calls && m.tool_calls.length > 0;
|
||||
const hasReasoning = typeof m.reasoning === 'string' && m.reasoning.length > 0;
|
||||
if (!hasTools && !hasReasoning) {
|
||||
// Bare text assistant (string content). null content + no tool_calls
|
||||
// is degenerate but harmless to forward.
|
||||
out.push({ role: 'assistant', content: m.content ?? '' });
|
||||
continue;
|
||||
}
|
||||
// v1.13.1-C: AI SDK ReasoningPart precedes text + tool-calls in the
|
||||
// assistant content array. Reasoning models (qwen3.6) consume their
|
||||
// prior reasoning context to resume mid-thought across tool boundaries.
|
||||
const parts: Array<
|
||||
| { type: 'reasoning'; text: string }
|
||||
| { type: 'text'; text: string }
|
||||
| { type: 'tool-call'; toolCallId: string; toolName: string; input: unknown }
|
||||
> = [];
|
||||
if (hasReasoning) {
|
||||
parts.push({ type: 'reasoning', text: m.reasoning! });
|
||||
}
|
||||
if (m.content && m.content.length > 0) {
|
||||
parts.push({ type: 'text', text: m.content });
|
||||
}
|
||||
for (const tc of m.tool_calls ?? []) {
|
||||
let input: unknown = {};
|
||||
try {
|
||||
input = tc.function.arguments.length > 0 ? JSON.parse(tc.function.arguments) : {};
|
||||
} catch {
|
||||
// Malformed args from a prior turn: pass through as a raw blob so
|
||||
// the model sees the same shape it emitted. Wraps the string under
|
||||
// _raw to match the buildMessagesPayload upstream convention.
|
||||
input = { _raw: tc.function.arguments };
|
||||
}
|
||||
parts.push({ type: 'tool-call', toolCallId: tc.id, toolName: tc.function.name, input });
|
||||
}
|
||||
out.push({ role: 'assistant', content: parts });
|
||||
continue;
|
||||
}
|
||||
if (m.role === 'tool') {
|
||||
const toolCallId = m.tool_call_id ?? '';
|
||||
const toolName = toolNameById.get(toolCallId) ?? 'unknown';
|
||||
const raw = m.content ?? '';
|
||||
let output: { type: 'text'; value: string } | { type: 'json'; value: JSONValue };
|
||||
try {
|
||||
// JSON.parse returns `any`; cast to JSONValue since the upstream
|
||||
// tool_results column is already JSON-serializable by construction.
|
||||
output = { type: 'json', value: JSON.parse(raw) as JSONValue };
|
||||
} catch {
|
||||
output = { type: 'text', value: raw };
|
||||
}
|
||||
out.push({
|
||||
role: 'tool',
|
||||
content: [{ type: 'tool-result', toolCallId, toolName, output }],
|
||||
});
|
||||
continue;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Build the AI SDK tools record from BooCode's JSON-schema tool definitions.
|
||||
// No `execute` field: BooCode runs tools itself in tool-phase.ts; streamText
|
||||
// surfaces the tool-call parts via fullStream and we capture them for the
|
||||
// outer loop to dispatch.
|
||||
function buildAiTools(schemas: ToolJsonSchema[]): Record<string, ReturnType<typeof tool>> {
|
||||
const out: Record<string, ReturnType<typeof tool>> = {};
|
||||
for (const s of schemas) {
|
||||
out[s.function.name] = tool({
|
||||
description: s.function.description,
|
||||
inputSchema: jsonSchema(s.function.parameters),
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// v1.10.5 Qwen-coder XML fallback. Some local models (notably qwen3-coder via
|
||||
// llama-swap) emit tool calls as inline XML inside delta.content rather than
|
||||
// the structured delta.tool_calls field. The XML shape is:
|
||||
// the structured tool_calls field. We extract them out of the streamed text
|
||||
// before flushing it to the client, mirroring the pre-AI-SDK behavior.
|
||||
//
|
||||
// XML shape:
|
||||
// <tool_call>
|
||||
// <function=NAME>
|
||||
// <parameter=KEY>
|
||||
// VALUE
|
||||
// </parameter>
|
||||
// ...more parameters...
|
||||
// <parameter=KEY>VALUE</parameter>
|
||||
// ...
|
||||
// </function>
|
||||
// </tool_call>
|
||||
// Multiple <tool_call> blocks may appear back-to-back; they never nest.
|
||||
// streamCompletion buffers delta.content, extracts complete blocks, parses
|
||||
// them via parseXmlToolCall, and pushes synthetic entries into the existing
|
||||
// toolCallsBuffer alongside any native JSON-format tool calls.
|
||||
export async function streamCompletion(
|
||||
ctx: InferenceContext,
|
||||
model: string,
|
||||
@@ -96,152 +151,196 @@ export async function streamCompletion(
|
||||
onUsage: ((prompt: number | null, completion: number | null) => void) | undefined,
|
||||
signal?: AbortSignal
|
||||
): Promise<StreamResult> {
|
||||
const body: Record<string, unknown> = {
|
||||
model,
|
||||
messages,
|
||||
stream: true,
|
||||
stream_options: { include_usage: true },
|
||||
};
|
||||
if (opts.tools && opts.tools.length > 0) {
|
||||
body['tools'] = opts.tools;
|
||||
body['tool_choice'] = 'auto';
|
||||
}
|
||||
if (typeof opts.temperature === 'number') {
|
||||
body['temperature'] = opts.temperature;
|
||||
}
|
||||
const aiMessages = toModelMessages(messages);
|
||||
const hasTools = opts.tools !== null && opts.tools.length > 0;
|
||||
const aiTools = hasTools ? buildAiTools(opts.tools!) : undefined;
|
||||
|
||||
const res = await fetch(`${ctx.config.LLAMA_SWAP_URL}/v1/chat/completions`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(body),
|
||||
signal,
|
||||
const startedAt = Date.now();
|
||||
// v1.13.1-C: accumulate reasoning text across reasoning-delta parts.
|
||||
// qwen3.6 emits these on a separate channel from text content; we capture
|
||||
// them per stream so finalizeCompletion can dual-write a 'reasoning' part.
|
||||
// Replaces the v1.13.1-A counter-only diagnostic.
|
||||
let reasoningAccumulated = '';
|
||||
|
||||
// v1.13.3: experimental_repairToolCall keeps the stream alive when the
|
||||
// model emits a malformed tool call (bad JSON args, unknown name, etc.).
|
||||
// Without a repair function streamText throws and the WHOLE stream dies;
|
||||
// with one, the SDK invokes us and we route the bad call through normally.
|
||||
// Strategy: pass through unmodified. executeToolPhase's existing error
|
||||
// path (unknown tool name → "unknown tool: X" result; zod-reject → tool
|
||||
// 'X' rejected — fieldname: required) already gives the model a clean
|
||||
// recovery surface on the next turn. Logging gives us visibility into
|
||||
// how often qwen3.6 actually emits broken calls.
|
||||
const repairToolCall: ToolCallRepairFunction<NonNullable<typeof aiTools>> = async ({
|
||||
toolCall,
|
||||
error,
|
||||
}) => {
|
||||
ctx.log.warn(
|
||||
{
|
||||
toolCallId: toolCall.toolCallId,
|
||||
toolName: toolCall.toolName,
|
||||
error: error.message,
|
||||
},
|
||||
'malformed tool call surfaced via repairToolCall',
|
||||
);
|
||||
return toolCall;
|
||||
};
|
||||
|
||||
const result = streamText({
|
||||
model: upstreamModel(ctx.config.LLAMA_SWAP_URL, model),
|
||||
messages: aiMessages,
|
||||
...(aiTools
|
||||
? { tools: aiTools, toolChoice: 'auto' as const, experimental_repairToolCall: repairToolCall }
|
||||
: {}),
|
||||
...(typeof opts.temperature === 'number' ? { temperature: opts.temperature } : {}),
|
||||
abortSignal: signal,
|
||||
});
|
||||
if (!res.ok || !res.body) {
|
||||
const text = await res.text().catch(() => '');
|
||||
throw new Error(`llama-swap returned ${res.status}: ${text.slice(0, 200)}`);
|
||||
}
|
||||
|
||||
let content = '';
|
||||
// v1.10.5: holds delta.content bytes that may contain a partial XML tool
|
||||
// call. Anything not part of a (possibly forming) <tool_call>…</tool_call>
|
||||
// pair is flushed to content + onDelta as soon as we know it's safe.
|
||||
let pendingBuffer = '';
|
||||
let finishReason: string | null = null;
|
||||
let promptTokens: number | null = null;
|
||||
let completionTokens: number | null = null;
|
||||
const toolCallsBuffer = new Map<number, { id: string; name: string; argsText: string }>();
|
||||
// v1.13.1-A: AI SDK emits one `tool-call` part per fully-aggregated call,
|
||||
// so we no longer need the OpenAI-index reassembly map the manual SSE
|
||||
// parser used. XML tool calls extracted from text content go into the
|
||||
// same flat list and keep the v1.10.5 synthetic id convention.
|
||||
const toolCalls: ToolCall[] = [];
|
||||
|
||||
for await (const line of sseLines(res.body)) {
|
||||
if (!line.startsWith('data:')) continue;
|
||||
const payload = line.slice(5).trim();
|
||||
if (payload === '[DONE]') break;
|
||||
let parsed: ChatCompletionChunk;
|
||||
try {
|
||||
parsed = JSON.parse(payload);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (parsed.usage) {
|
||||
if (typeof parsed.usage.prompt_tokens === 'number') {
|
||||
promptTokens = parsed.usage.prompt_tokens;
|
||||
}
|
||||
if (typeof parsed.usage.completion_tokens === 'number') {
|
||||
completionTokens = parsed.usage.completion_tokens;
|
||||
}
|
||||
onUsage?.(promptTokens, completionTokens);
|
||||
}
|
||||
// v1.11.3: removed dead `parsed.timings.n_ctx` read. llama-server's
|
||||
// streaming completion does NOT emit n_ctx in timings (verified
|
||||
// empirically); the authoritative source is llama-swap's
|
||||
// /upstream/<model>/props endpoint, fetched per-turn via
|
||||
// model-context.getModelContext() at the finalization sites below.
|
||||
|
||||
const choice = parsed.choices?.[0];
|
||||
if (!choice) continue;
|
||||
const delta = choice.delta ?? {};
|
||||
if (typeof delta.content === 'string' && delta.content.length > 0) {
|
||||
// v1.10.5 XML fallback. Append, then extract any complete tool_call
|
||||
// blocks before deciding what's safe to flush as visible content.
|
||||
pendingBuffer += delta.content;
|
||||
while (true) {
|
||||
const startIdx = pendingBuffer.indexOf(XML_TOOL_OPEN);
|
||||
if (startIdx === -1) break;
|
||||
const closeIdx = pendingBuffer.indexOf(XML_TOOL_CLOSE, startIdx);
|
||||
if (closeIdx === -1) break;
|
||||
const blockEnd = closeIdx + XML_TOOL_CLOSE.length;
|
||||
const block = pendingBuffer.slice(startIdx, blockEnd);
|
||||
// Any text before the opener is plain content — flush it now.
|
||||
if (startIdx > 0) {
|
||||
const before = pendingBuffer.slice(0, startIdx);
|
||||
content += before;
|
||||
onDelta(before);
|
||||
for await (const part of result.fullStream) {
|
||||
switch (part.type) {
|
||||
case 'text-delta': {
|
||||
pendingBuffer += part.text;
|
||||
// Extract any complete <tool_call>...</tool_call> blocks before
|
||||
// flushing visible text.
|
||||
while (true) {
|
||||
const startIdx = pendingBuffer.indexOf(XML_TOOL_OPEN);
|
||||
if (startIdx === -1) break;
|
||||
const closeIdx = pendingBuffer.indexOf(XML_TOOL_CLOSE, startIdx);
|
||||
if (closeIdx === -1) break;
|
||||
const blockEnd = closeIdx + XML_TOOL_CLOSE.length;
|
||||
const block = pendingBuffer.slice(startIdx, blockEnd);
|
||||
if (startIdx > 0) {
|
||||
const before = pendingBuffer.slice(0, startIdx);
|
||||
content += before;
|
||||
onDelta(before);
|
||||
}
|
||||
const parsedCall = parseXmlToolCall(block);
|
||||
if (parsedCall) {
|
||||
const synthIdx = toolCalls.length;
|
||||
toolCalls.push({
|
||||
id: `xml_call_${synthIdx}`,
|
||||
name: parsedCall.name,
|
||||
args: parsedCall.args,
|
||||
});
|
||||
}
|
||||
// Parse failures still drop the block — leaking <tool_call> XML to
|
||||
// the chat would look worse than silently swallowing the bad block.
|
||||
pendingBuffer = pendingBuffer.slice(blockEnd);
|
||||
}
|
||||
const parsedCall = parseXmlToolCall(block);
|
||||
if (parsedCall) {
|
||||
const synthIdx = toolCallsBuffer.size;
|
||||
toolCallsBuffer.set(synthIdx, {
|
||||
id: `xml_call_${synthIdx}`,
|
||||
name: parsedCall.name,
|
||||
argsText: JSON.stringify(parsedCall.args),
|
||||
});
|
||||
// Hold back any (partial or full) unclosed opener; flush the rest.
|
||||
const partialIdx = partialXmlOpenerStart(pendingBuffer);
|
||||
if (partialIdx >= 0) {
|
||||
if (partialIdx > 0) {
|
||||
const flush = pendingBuffer.slice(0, partialIdx);
|
||||
content += flush;
|
||||
onDelta(flush);
|
||||
}
|
||||
pendingBuffer = pendingBuffer.slice(partialIdx);
|
||||
} else if (pendingBuffer.length > 0) {
|
||||
content += pendingBuffer;
|
||||
onDelta(pendingBuffer);
|
||||
pendingBuffer = '';
|
||||
}
|
||||
// If parsing failed we still drop the block — emitting unparseable
|
||||
// XML to the chat would look worse than silently swallowing it.
|
||||
pendingBuffer = pendingBuffer.slice(blockEnd);
|
||||
break;
|
||||
}
|
||||
// After all complete blocks are out, hold back any (partial or full)
|
||||
// unclosed opener; flush the rest.
|
||||
const partialIdx = partialXmlOpenerStart(pendingBuffer);
|
||||
if (partialIdx >= 0) {
|
||||
if (partialIdx > 0) {
|
||||
const flush = pendingBuffer.slice(0, partialIdx);
|
||||
content += flush;
|
||||
onDelta(flush);
|
||||
case 'tool-call': {
|
||||
// AI SDK has already parsed the input into an object. Match the
|
||||
// ToolCall shape BooCode passes around in toolCallsBuffer downstream.
|
||||
toolCalls.push({
|
||||
id: part.toolCallId,
|
||||
name: part.toolName,
|
||||
args: (part.input ?? {}) as Record<string, unknown>,
|
||||
});
|
||||
break;
|
||||
}
|
||||
case 'reasoning-delta': {
|
||||
// v1.13.1-C: accumulate; finalizeCompletion / executeToolPhase
|
||||
// dual-write the resulting text as a kind='reasoning' part.
|
||||
if (typeof part.text === 'string') {
|
||||
reasoningAccumulated += part.text;
|
||||
}
|
||||
pendingBuffer = pendingBuffer.slice(partialIdx);
|
||||
} else if (pendingBuffer.length > 0) {
|
||||
content += pendingBuffer;
|
||||
onDelta(pendingBuffer);
|
||||
pendingBuffer = '';
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (Array.isArray(delta.tool_calls)) {
|
||||
for (const tc of delta.tool_calls) {
|
||||
const idx = tc.index;
|
||||
const existing = toolCallsBuffer.get(idx) ?? { id: '', name: '', argsText: '' };
|
||||
if (tc.id) existing.id = tc.id;
|
||||
if (tc.function?.name) existing.name = tc.function.name;
|
||||
if (typeof tc.function?.arguments === 'string') existing.argsText += tc.function.arguments;
|
||||
toolCallsBuffer.set(idx, existing);
|
||||
case 'finish': {
|
||||
if (typeof part.finishReason === 'string') {
|
||||
finishReason = part.finishReason;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'error': {
|
||||
const err = part.error;
|
||||
throw err instanceof Error ? err : new Error(String(err));
|
||||
}
|
||||
// Intentional no-op: start, start-step, text-start, text-end,
|
||||
// reasoning-start, reasoning-end, source, file, tool-input-start,
|
||||
// tool-input-delta, tool-input-end, tool-result, tool-error,
|
||||
// finish-step, raw. We only care about the aggregated tool-call and
|
||||
// text-delta paths above; the rest are AI SDK lifecycle/streaming
|
||||
// breadcrumbs that don't change BooCode's persistence or WS contract.
|
||||
default:
|
||||
break;
|
||||
}
|
||||
if (choice.finish_reason) finishReason = choice.finish_reason;
|
||||
}
|
||||
|
||||
// v1.10.5: if the stream ended mid-XML (e.g. model truncated, no closer
|
||||
// ever arrived), flush whatever was buffered as plain content so it isn't
|
||||
// silently dropped. Better to show a stray `<tool_call>` than vanish text.
|
||||
// v1.13.1-A: drain any buffered partial XML opener as plain text. The
|
||||
// pre-AI-SDK path did this on stream end too — better to leak `<tool_c`
|
||||
// than vanish the text.
|
||||
if (pendingBuffer.length > 0) {
|
||||
content += pendingBuffer;
|
||||
onDelta(pendingBuffer);
|
||||
pendingBuffer = '';
|
||||
}
|
||||
|
||||
const toolCalls: ToolCall[] = [];
|
||||
for (const [, t] of [...toolCallsBuffer.entries()].sort(([a], [b]) => a - b)) {
|
||||
let args: Record<string, unknown> = {};
|
||||
if (t.argsText.length > 0) {
|
||||
try {
|
||||
args = JSON.parse(t.argsText);
|
||||
} catch {
|
||||
args = { _raw: t.argsText };
|
||||
}
|
||||
}
|
||||
toolCalls.push({ id: t.id || `call_${toolCalls.length}`, name: t.name, args });
|
||||
// AI SDK v6 fullStream returns normally on abort; check signal explicitly.
|
||||
// Without this throw the row would land as status='complete' with partial
|
||||
// content instead of going through handleAbortOrError → status='cancelled'.
|
||||
// Smoke D caught this in v1.13.1-A — don't refactor it away.
|
||||
if (signal?.aborted) {
|
||||
const abortErr = new Error('aborted');
|
||||
abortErr.name = 'AbortError';
|
||||
throw abortErr;
|
||||
}
|
||||
|
||||
return { finishReason, content, toolCalls, promptTokens, completionTokens };
|
||||
// Usage lands as a promise on the result; awaiting after fullStream is
|
||||
// drained is safe. AI SDK v6 names: `inputTokens` / `outputTokens`.
|
||||
let promptTokens: number | null = null;
|
||||
let completionTokens: number | null = null;
|
||||
try {
|
||||
const usage = await result.usage;
|
||||
if (typeof usage.inputTokens === 'number') promptTokens = usage.inputTokens;
|
||||
if (typeof usage.outputTokens === 'number') completionTokens = usage.outputTokens;
|
||||
} catch {
|
||||
// Some providers omit usage on partial streams; leave both null.
|
||||
}
|
||||
|
||||
if (onUsage && (promptTokens !== null || completionTokens !== null)) {
|
||||
onUsage(promptTokens, completionTokens);
|
||||
}
|
||||
|
||||
if (reasoningAccumulated.length > 0) {
|
||||
ctx.log.debug(
|
||||
{ reasoningChars: reasoningAccumulated.length, model, elapsed_ms: Date.now() - startedAt },
|
||||
'streamCompletion: captured reasoning',
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
finishReason,
|
||||
content,
|
||||
toolCalls,
|
||||
promptTokens,
|
||||
completionTokens,
|
||||
reasoning: reasoningAccumulated,
|
||||
};
|
||||
}
|
||||
|
||||
export async function executeStreamPhase(
|
||||
@@ -313,9 +412,12 @@ export async function executeStreamPhase(
|
||||
const mctxForStream = await modelContext.getModelContext(session.model);
|
||||
const nCtxForStream = mctxForStream?.n_ctx ?? null;
|
||||
|
||||
// v1.12.2: throttle live usage publishes to ~500ms. The model can land
|
||||
// dozens of usage frames per second; without a throttle the WS turns into
|
||||
// a firehose for a few KB savings on each render.
|
||||
// v1.12.2 → v1.13.1-A: live usage publishes were throttled to ~500ms when
|
||||
// the manual SSE parser saw `parsed.usage` per chunk. AI SDK v6 surfaces
|
||||
// usage only at stream end (result.usage promise), so the throttle is
|
||||
// effectively a single trailing publish. ChatThroughput will tick once at
|
||||
// stream completion rather than mid-stream — known regression vs v1.12.2,
|
||||
// recovered if a future dispatch interpolates from delta cadence.
|
||||
const USAGE_THROTTLE_MS = 500;
|
||||
let lastUsageAt = 0;
|
||||
let pendingUsage: { p: number | null; c: number | null } | null = null;
|
||||
|
||||
@@ -3,6 +3,7 @@ import * as modelContext from '../model-context.js';
|
||||
import { PathScopeError } from '../path_guard.js';
|
||||
import { TOOLS_BY_NAME } from '../tools.js';
|
||||
import { maybeFlagForCompaction } from './payload.js';
|
||||
import { insertParts, partsFromAssistantMessage, partsFromToolMessage } from './parts.js';
|
||||
import type {
|
||||
InferenceContext,
|
||||
StreamResult,
|
||||
@@ -97,6 +98,26 @@ export async function executeToolPhase(
|
||||
WHERE id = ${assistantMessageId}
|
||||
RETURNING tokens_used, ctx_used, ctx_max, finished_at
|
||||
`;
|
||||
// v1.13.0: dual-write to message_parts. v1.13.1-B made parts authoritative
|
||||
// for reads via the messages_with_parts view; the JSON column write above
|
||||
// remains for v1.13.1 fallback compatibility (dropped in v1.13.2).
|
||||
// v1.13.1-C: include result.reasoning so models with separate reasoning
|
||||
// channels (qwen3.6) get a kind='reasoning' part at sequence 0.
|
||||
// TODO(v1.13.1): wrap the UPDATE above and this insertParts in a single
|
||||
// sql.begin before flipping read authority to message_parts. Without the
|
||||
// transaction, a crash between the two leaves an orphan message that
|
||||
// becomes invisible in the parts-authoritative read path.
|
||||
await insertParts(
|
||||
ctx.sql,
|
||||
partsFromAssistantMessage({
|
||||
content,
|
||||
tool_calls: toolCalls,
|
||||
reasoning: result.reasoning,
|
||||
}).map((p) => ({
|
||||
...p,
|
||||
message_id: assistantMessageId,
|
||||
})),
|
||||
);
|
||||
// v1.11: flag for compaction if this turn pushed us over the usable budget.
|
||||
// We never compact mid-loop (the recursive runAssistantTurn keeps tools
|
||||
// flowing); the flag fires on the NEXT turn's pre-fetch hook above.
|
||||
@@ -150,6 +171,18 @@ export async function executeToolPhase(
|
||||
SET tool_results = ${ctx.sql.json(sentinel as never)}
|
||||
WHERE id = ${toolMessageId}
|
||||
`;
|
||||
// v1.13.0: mirror the pending sentinel into message_parts. The
|
||||
// answer-endpoint UPDATE later (messages.ts:576) will delete and
|
||||
// re-insert this part when the user submits their answer.
|
||||
// TODO(v1.13.1): wrap the INSERT + UPDATE + insertParts triple in
|
||||
// a per-iteration sql.begin before flipping read authority.
|
||||
await insertParts(
|
||||
ctx.sql,
|
||||
partsFromToolMessage({ tool_results: sentinel }).map((p) => ({
|
||||
...p,
|
||||
message_id: toolMessageId,
|
||||
})),
|
||||
);
|
||||
return;
|
||||
}
|
||||
const tres = await executeToolCall(projectRoot, tc);
|
||||
@@ -164,6 +197,16 @@ export async function executeToolPhase(
|
||||
SET tool_results = ${ctx.sql.json(stored as never)}
|
||||
WHERE id = ${toolMessageId}
|
||||
`;
|
||||
// v1.13.0: dual-write the tool_result part.
|
||||
// TODO(v1.13.1): wrap the INSERT + UPDATE + insertParts triple in a
|
||||
// per-iteration sql.begin before flipping read authority.
|
||||
await insertParts(
|
||||
ctx.sql,
|
||||
partsFromToolMessage({ tool_results: stored }).map((p) => ({
|
||||
...p,
|
||||
message_id: toolMessageId,
|
||||
})),
|
||||
);
|
||||
ctx.publish(sessionId, {
|
||||
type: 'tool_result',
|
||||
tool_message_id: toolMessageId,
|
||||
|
||||
@@ -118,6 +118,9 @@ export interface StreamResult {
|
||||
toolCalls: ToolCall[];
|
||||
promptTokens: number | null;
|
||||
completionTokens: number | null;
|
||||
// v1.13.1-C: reasoning text accumulated across reasoning-delta parts.
|
||||
// Empty string when the model doesn't emit reasoning (most cases).
|
||||
reasoning: string;
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -527,6 +527,11 @@ export const askUserInput: ToolDef<AskUserInputInputT> = {
|
||||
},
|
||||
};
|
||||
|
||||
// v1.13.3: alpha-sorted by tool.name at module load. llama.cpp's prompt
|
||||
// cache hits on byte-identical prefixes; the tool list lives near the top
|
||||
// of the system prompt, so any order drift would invalidate every cached
|
||||
// turn. Single source of truth for ordering lives here — toolJsonSchemas()
|
||||
// and TOOLS_BY_NAME inherit it.
|
||||
export const ALL_TOOLS: ReadonlyArray<ToolDef<unknown>> = [
|
||||
viewFile as ToolDef<unknown>,
|
||||
listDir as ToolDef<unknown>,
|
||||
@@ -553,7 +558,7 @@ export const ALL_TOOLS: ReadonlyArray<ToolDef<unknown>> = [
|
||||
watchChanges as ToolDef<unknown>,
|
||||
getSemanticNeighborhoods as ToolDef<unknown>,
|
||||
getFrameworkAnalysis as ToolDef<unknown>,
|
||||
];
|
||||
].sort((a, b) => a.name.localeCompare(b.name));
|
||||
|
||||
// v1.8.2: forward-compatible read-only whitelist. An agent whose `tools` is
|
||||
// fully contained in this set gets a generous default tool budget (30);
|
||||
|
||||
@@ -186,6 +186,11 @@ export interface Message {
|
||||
// v1.8.2: per-message metadata. See MessageMetadata for the discriminated
|
||||
// shapes currently in use.
|
||||
metadata: MessageMetadata | null;
|
||||
// v1.13.1-C: reasoning content captured from the model's reasoning stream
|
||||
// (qwen3.6 etc.). Populated from message_parts via the messages_with_parts
|
||||
// view's reasoning_parts column. Optional — most rows have no reasoning
|
||||
// and the API may omit the field on legacy responses.
|
||||
reasoning_parts?: Array<{ text: string }> | null;
|
||||
// v1.11: anchored rolling compaction. Optional so consumers that SELECT
|
||||
// the pre-v1.11 column set still type-check. See compaction.ts +
|
||||
// schema.sql for semantics.
|
||||
|
||||
@@ -161,6 +161,11 @@ export interface Message {
|
||||
// v1.8.2: per-message metadata; see MessageMetadata. null for the vast
|
||||
// majority of messages.
|
||||
metadata: MessageMetadata | null;
|
||||
// v1.13.1-C: reasoning content captured from models that stream reasoning
|
||||
// tokens separately (qwen3.6 etc.). Backend populates from message_parts;
|
||||
// optional on the wire — frontend doesn't render this yet (reserved for
|
||||
// a v1.14 UI surface).
|
||||
reasoning_parts?: Array<{ text: string }> | null;
|
||||
// v1.11: anchored rolling compaction fields. Optional on the wire so that
|
||||
// older API responses (or test fixtures) parse without explicit nulls.
|
||||
// summary — true on the assistant row that holds the active
|
||||
|
||||
88
pnpm-lock.yaml
generated
88
pnpm-lock.yaml
generated
@@ -48,12 +48,18 @@ importers:
|
||||
|
||||
apps/server:
|
||||
dependencies:
|
||||
'@ai-sdk/openai-compatible':
|
||||
specifier: ^2.0.47
|
||||
version: 2.0.47(zod@3.25.76)
|
||||
'@fastify/static':
|
||||
specifier: ^7.0.4
|
||||
version: 7.0.4
|
||||
'@fastify/websocket':
|
||||
specifier: ^10.0.1
|
||||
version: 10.0.1
|
||||
ai:
|
||||
specifier: ^6.0.190
|
||||
version: 6.0.190(zod@3.25.76)
|
||||
fastify:
|
||||
specifier: ^4.28.1
|
||||
version: 4.29.1
|
||||
@@ -179,6 +185,28 @@ importers:
|
||||
|
||||
packages:
|
||||
|
||||
'@ai-sdk/gateway@3.0.119':
|
||||
resolution: {integrity: sha512-VAhfRWC+JexZakkVfmjaJKaTj00x7/UHdE8kMWL3NhuQAlf8oXtg9r4dfvFZrByXxchGRBvYE3biEUyibkg0xg==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/openai-compatible@2.0.47':
|
||||
resolution: {integrity: sha512-Enm5UlL0zUCrW3792opk5h7hRWxZOZzDe6eQYVFqX9LUOGGCe1h8MZWAGim765nwzgnjlpeYOsuzZmLtRsTPlg==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/provider-utils@4.0.27':
|
||||
resolution: {integrity: sha512-ubkAJ+xODouwtmN1tYlvTPphH1hPOBfZaEQe8U7skGvFAnIRs9PPpsq57bC2+Ky/MB4yzhd6YOsxTAx9sGpazw==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/provider@3.0.10':
|
||||
resolution: {integrity: sha512-Q3BZ27qfpYqnCYGvE3vt+Qi6LGOF9R5Nmzn+9JoM1lCRsD9mYaIhfJLkSunN48nfGXJ6n+XNV0J/XVpqGQl7Dw==}
|
||||
engines: {node: '>=18'}
|
||||
|
||||
'@alloc/quick-lru@5.2.0':
|
||||
resolution: {integrity: sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw==}
|
||||
engines: {node: '>=10'}
|
||||
@@ -789,6 +817,10 @@ packages:
|
||||
'@open-draft/until@2.1.0':
|
||||
resolution: {integrity: sha512-U69T3ItWHvLwGg5eJ0n3I62nWuE6ilHlmz7zM0npLBRvPRd7e6NYmg54vvRtP5mZG7kZqZCFVdsTWo7BPtBujg==}
|
||||
|
||||
'@opentelemetry/api@1.9.1':
|
||||
resolution: {integrity: sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==}
|
||||
engines: {node: '>=8.0.0'}
|
||||
|
||||
'@pinojs/redact@0.4.0':
|
||||
resolution: {integrity: sha512-k2ENnmBugE/rzQfEcdWHcCY+/FM3VLzH9cYEsbdsoqrvzAKRhUZeRNhAZvB8OitQJ1TBed3yqWtdjzS6wJKBwg==}
|
||||
|
||||
@@ -1646,6 +1678,9 @@ packages:
|
||||
resolution: {integrity: sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ==}
|
||||
engines: {node: '>=18'}
|
||||
|
||||
'@standard-schema/spec@1.1.0':
|
||||
resolution: {integrity: sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==}
|
||||
|
||||
'@tailwindcss/node@4.3.0':
|
||||
resolution: {integrity: sha512-aFb4gUhFOgdh9AXo4IzBEOzBkkAxm9VigwDJnMIYv3lcfXCJVesNfbEaBl4BNgVRyid92AmdviqwBUBRKSeY3g==}
|
||||
|
||||
@@ -1811,6 +1846,10 @@ packages:
|
||||
'@ungap/structured-clone@1.3.1':
|
||||
resolution: {integrity: sha512-mUFwbeTqrVgDQxFveS+df2yfap6iuP20NAKAsBt5jDEoOTDew+zwLAOilHCeQJOVSvmgCX4ogqIrA0mnyr08yQ==}
|
||||
|
||||
'@vercel/oidc@3.2.0':
|
||||
resolution: {integrity: sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==}
|
||||
engines: {node: '>= 20'}
|
||||
|
||||
'@vitejs/plugin-react@4.7.0':
|
||||
resolution: {integrity: sha512-gUu9hwfWvvEDBBmgtAowQCojwZmJ5mcLn3aufeCsitijs3+f2NsrPtlAWIR6OPiqljl96GVCUbLe0HyqIpVaoA==}
|
||||
engines: {node: ^14.18.0 || >=16.0.0}
|
||||
@@ -1878,6 +1917,12 @@ packages:
|
||||
resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==}
|
||||
engines: {node: '>= 14'}
|
||||
|
||||
ai@6.0.190:
|
||||
resolution: {integrity: sha512-T+ixHbWZ6jmHRREpVVJTkFyWJeCekCdzLPan7lp1F32jG5OUw4+odlVYjtMRXVzogU+pWzpMmXdRiHUmdL/q0w==}
|
||||
engines: {node: '>=18'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
ajv-formats@2.1.1:
|
||||
resolution: {integrity: sha512-Wx0Kx52hxE7C18hkMEggYlEifqWZtYaRgouJor+WMdPnQyEK13vgEWyVNup7SoeeoLMsr4kf5h6dOW11I15MUA==}
|
||||
peerDependencies:
|
||||
@@ -2694,6 +2739,9 @@ packages:
|
||||
json-schema-typed@8.0.2:
|
||||
resolution: {integrity: sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA==}
|
||||
|
||||
json-schema@0.4.0:
|
||||
resolution: {integrity: sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA==}
|
||||
|
||||
json5@2.2.3:
|
||||
resolution: {integrity: sha512-XmOWe7eyHYH14cLdVPoyg+GOH3rYX++KpzrylJwSW98t3Nk+U8XOl8FWKOgwtzdb8lXGf6zYwDUzeHMWfxasyg==}
|
||||
engines: {node: '>=6'}
|
||||
@@ -3966,6 +4014,30 @@ packages:
|
||||
|
||||
snapshots:
|
||||
|
||||
'@ai-sdk/gateway@3.0.119(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.10
|
||||
'@ai-sdk/provider-utils': 4.0.27(zod@3.25.76)
|
||||
'@vercel/oidc': 3.2.0
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/openai-compatible@2.0.47(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.10
|
||||
'@ai-sdk/provider-utils': 4.0.27(zod@3.25.76)
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/provider-utils@4.0.27(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.10
|
||||
'@standard-schema/spec': 1.1.0
|
||||
eventsource-parser: 3.0.8
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/provider@3.0.10':
|
||||
dependencies:
|
||||
json-schema: 0.4.0
|
||||
|
||||
'@alloc/quick-lru@5.2.0': {}
|
||||
|
||||
'@babel/code-frame@7.29.0':
|
||||
@@ -4516,6 +4588,8 @@ snapshots:
|
||||
|
||||
'@open-draft/until@2.1.0': {}
|
||||
|
||||
'@opentelemetry/api@1.9.1': {}
|
||||
|
||||
'@pinojs/redact@0.4.0': {}
|
||||
|
||||
'@pkgjs/parseargs@0.11.0':
|
||||
@@ -5386,6 +5460,8 @@ snapshots:
|
||||
|
||||
'@sindresorhus/merge-streams@4.0.0': {}
|
||||
|
||||
'@standard-schema/spec@1.1.0': {}
|
||||
|
||||
'@tailwindcss/node@4.3.0':
|
||||
dependencies:
|
||||
'@jridgewell/remapping': 2.3.5
|
||||
@@ -5548,6 +5624,8 @@ snapshots:
|
||||
|
||||
'@ungap/structured-clone@1.3.1': {}
|
||||
|
||||
'@vercel/oidc@3.2.0': {}
|
||||
|
||||
'@vitejs/plugin-react@4.7.0(vite@5.4.21(@types/node@20.19.41)(lightningcss@1.32.0))':
|
||||
dependencies:
|
||||
'@babel/core': 7.29.0
|
||||
@@ -5628,6 +5706,14 @@ snapshots:
|
||||
|
||||
agent-base@7.1.4: {}
|
||||
|
||||
ai@6.0.190(zod@3.25.76):
|
||||
dependencies:
|
||||
'@ai-sdk/gateway': 3.0.119(zod@3.25.76)
|
||||
'@ai-sdk/provider': 3.0.10
|
||||
'@ai-sdk/provider-utils': 4.0.27(zod@3.25.76)
|
||||
'@opentelemetry/api': 1.9.1
|
||||
zod: 3.25.76
|
||||
|
||||
ajv-formats@2.1.1(ajv@8.20.0):
|
||||
optionalDependencies:
|
||||
ajv: 8.20.0
|
||||
@@ -6453,6 +6539,8 @@ snapshots:
|
||||
|
||||
json-schema-typed@8.0.2: {}
|
||||
|
||||
json-schema@0.4.0: {}
|
||||
|
||||
json5@2.2.3: {}
|
||||
|
||||
jsonfile@6.2.1:
|
||||
|
||||
Reference in New Issue
Block a user