Compare commits

...

6 Commits

Author SHA1 Message Date
a584dd16b0 feat: sampling knobs + live PTY stream-json + token UI (v2.7.3)
Three small wins from boocode_code_review_v2 §1 #11/#7/#8.

#11 sampling knobs: top_n_sigma + dry_* family as first-class Agent fields,
threaded into the request body via providerOptions.openaiCompatible. Fixes a
latent bug — top_k (rejected by the AI-SDK provider) and min_p (never passed to
streamText) were dead on the wire; both now route through the same channel.
--reasoning-budget documented in data/AGENTS.md.

#7 live PTY stream-json: new stream-json-parser.ts line-buffers qwen/claude
NDJSON and emits text/reasoning/tool frames live + persists, with a fallback to
the old opaque slice. claude gets --output-format stream-json --verbose.

#8 token UI: agent_sessions input/output_tokens/cost now flow through the route
+ type and render beside the AgentComposerBar session chip.

Built by 3 parallel agents. Server 523 + coder 245 tests passing; builds + web
tsc clean. Builds on v2.7.2. openspec sampling-streamjson-tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:47:17 +00:00
5651f56039 Merge checkpoint-idor-fix: v2.7.2 close 2 checkpoint IDOR holes 2026-06-01 12:16:08 +00:00
9c7d80e2d8 fix(security): scope checkpoint routes to session — close 2 IDORs (v2.7.2)
Flagged by the automated push security review on v2.7.1.

- GET /checkpoints?chat_id= : the chat_id branch filtered by chat_id alone
  (any session's chat_id read its checkpoints). Now joins chats and gates on
  chats.session_id.
- restoreCheckpoint scope guard was fail-open: `cp.session_id && cp.session_id
  !== sessionId` fell through on a null denormalized session_id, allowing a
  cross-session restore (worktree reset + transcript trim). Now resolves the
  owning session via the checkpoint's chat and denies on missing/mismatch.
- Adds a DB-integration regression for the null-session_id cross-session case.

Both scope authoritatively through chats.session_id (checkpoints.session_id is
a nullable hint). Coder suite 234 passing; 7/7 checkpoint tests (incl. the
regression) against live postgres+git; typecheck clean. Hotfix on v2.7.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:15:54 +00:00
a41a02a62b Merge fuzzy-checkpoints: v2.7.1 write/edit robustness (fuzzy applier + worktree checkpoints) 2026-06-01 12:02:06 +00:00
59f07e8cb8 feat: write/edit robustness — fuzzy patch applier + worktree checkpoints (v2.7.1)
#3 Fuzzy patch applier: new pure fuzzy-match.ts (locateMatch, exact→trim→
unicode-canon→Levenshtein≥0.66, refuse-on-ambiguous) wired into pending_changes
applyOne/rewindOne so local-model whitespace/unicode drift in old_string no
longer loses the edit.

#4 Worktree checkpoint + conversation-trim: checkpoints table + checkpoints.ts
(shadow-commit of tracked+untracked into refs/boocode/checkpoints, hooked into
the 3 external-agent dispatcher paths) + POST restore route (reset --hard +
clean -fd -> transcript trim -> backend-session reset) + "Restore to here" UI.

Built by 3 parallel agents; DB-integration testing caught a created_at
self-deletion bug. Coder suite 234 passing; server+coder build + web tsc clean.
Builds on v2.7.0-mit. openspec write-edit-robustness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:01:57 +00:00
1108d07fb2 Merge relicense-agpl-to-mit: v2.7.0 AGPL-3.0 → MIT relicense 2026-06-01 08:16:25 +00:00
27 changed files with 2423 additions and 33 deletions

View File

@@ -2,6 +2,18 @@
All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch. All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch.
## v2.7.3-sampling-streamjson-tokens — 2026-06-01
Three small BooCode wins from `boocode_code_review_v2.md` §1 #11/#7/#8. **Sampling knobs:** per-agent `top_n_sigma` + the `dry_*` repetition family (`dry_multiplier`/`dry_base`/`dry_allowed_length`/`dry_penalty_last_n`) are now first-class Agent frontmatter fields, parsed in `agents.ts` and threaded into the llama-swap chat-completion body via `providerOptions.openaiCompatible` (the `@ai-sdk/openai-compatible` extra-body channel). This surfaced and fixed a **latent bug**: `top_k` (rejected by the AI-SDK provider as unsupported) and `min_p` (never passed to `streamText` at all) had been dead on the wire — no agent's `top_k`/`min_p` ever affected sampling; both now route through the same channel, so agents that set them will start using them. `--reasoning-budget` is documented in `data/AGENTS.md` (already works via `llama_extra_args`, permitted by the deny-list validator). **Live PTY stream-json:** qwen/claude PTY dispatch sliced stdout opaque; a new `stream-json-parser.ts` line-buffers the Claude-Code-compatible NDJSON and emits text/reasoning/tool frames live as they arrive (mirroring the ACP/opencode paths) + persists the structured parts, with a clean fallback to the old opaque slice when output isn't NDJSON (claude now runs `--output-format stream-json --verbose`). **Token UI:** the per-`(chat,agent)` `agent_sessions.input_tokens`/`output_tokens`/`cost` columns (accumulated since `v2.6.8` but dropped by the read route + wire type) now flow through and render condensed beside the AgentComposerBar session chip. Built by three parallel agents over disjoint subsystems; server 523 + coder 245 tests passing (incl. 11 new stream-json-parser + new agent-parse tests), all builds + web tsc clean. Builds on `v2.7.2-checkpoint-idor`; openspec `sampling-streamjson-tokens`. The qwen-vs-claude `usage` field names in #7 are best-guess pending a live smoke.
## v2.7.2-checkpoint-idor — 2026-06-01
Closes two IDOR authorization holes in the `v2.7.1-write-edit-robustness` checkpoint routes, flagged by the automated push security review. The `GET /api/sessions/:id/checkpoints?chat_id=` list route scoped its `chat_id` branch by `chat_id` alone — any session's `chat_id` would read its checkpoints; it now joins through `chats` and gates on `chats.session_id` (authoritative; `checkpoints.session_id` is a nullable denormalized hint). The `restoreCheckpoint` scope guard was fail-open — `cp.session_id && cp.session_id !== sessionId` fell through whenever the checkpoint's denormalized `session_id` was null, allowing a cross-session restore (worktree reset + transcript trim) — it now resolves the owning session via the checkpoint's chat and denies on any missing-or-mismatched row. A DB-integration regression covers the exact null-`session_id` cross-session case. Real-world blast radius is small (BooCoder is single-user behind Authelia on loopback), but both are genuine authorization bugs. Coder suite 234 passing (7/7 checkpoint tests incl. the regression against live postgres+git), typecheck clean. Hotfix on `v2.7.1-write-edit-robustness`.
## v2.7.1-write-edit-robustness — 2026-06-01
Two BooCoder hardening features for local quantized models, algorithm-reimplemented (not vendored) from the cline findings in `boocode_code_review_v2.md` §1 #3/#4. **Fuzzy patch applier:** `edit_file`'s apply path was exact-`.includes`-or-throw + first-occurrence `.replace` (`pending_changes.ts`), so a qwen3.6 whitespace/indentation/unicode drift in `old_string` lost the edit; a new pure `fuzzy-match.ts` (`locateMatch`) now runs an exact → per-line-trim → unicode-canon (curly quotes/dashes/nbsp) → Levenshtein-≥0.66 ladder and returns the real file span, refusing multi-exact matches as ambiguous rather than silently editing the first. `applyOne`/`rewindOne` both use it. **Worktree checkpoints + conversation-trim:** `rewind` only reversed BooCode's own `pending_changes`, blind to what external agents (opencode/goose/qwen/claude) write directly into the session worktree — so a new `checkpoints` table + `checkpoints.ts` shadow-commit (tracked **and** untracked, captured via a temp-index `read-tree`/`add`/`write-tree`/`commit-tree` into a GC-safe `refs/boocode/checkpoints/<id>`) snapshots the worktree before each external-agent turn (hooked into all three dispatcher paths), anchored to the turn's assistant message. A new `POST /api/sessions/:id/checkpoints/:cid/restore` resets the worktree (`reset --hard` + `clean -fd`), trims the transcript past that message, and resets the `(chat,agent)` backend session so files, transcript, and agent context land consistent at the restore point; a per-message "Restore to here" affordance in `CoderMessageList` drives it. Built by three parallel agents over disjoint files; DB-integration testing caught a microsecond-`created_at` self-deletion bug in the later-checkpoint cleanup. Full coder suite 234 passing (incl. 17 fuzzy-match + 6 checkpoint tests), server+coder build + web tsc clean. Builds on `v2.7.0-mit`; openspec `write-edit-robustness`. Live host smoke (dispatcher hook + restore UI end-to-end) still to run.
## v2.7.0-mit — 2026-06-01 ## v2.7.0-mit — 2026-06-01
Relicenses BooCode from AGPL-3.0 back to MIT by clearing the three Unsloth-Studio-derived files the `v2.4.0`/`v2.4.1` lifts pulled in — the root `LICENSE` and all five `package.json` had been `AGPL-3.0-only`, making the network-served work AGPL §13-encumbered. The enabling finding decoupled the relicense from the long-planned native-llama-server-parsing retirement: `tool-call-parser.ts`'s Unsloth-ported algorithm (`parseToolCallsFromText`/`scanBalancedBraces` + unused nudge constants) was **dead code** with no production import, so it was simply deleted while the load-bearing `extractToolCallBlocks`/`stripToolMarkup` (BooCode-authored streaming helpers) were kept byte-identical — no behavior change to the live tool-call path. `html-to-md.ts` was swapped to the MIT `node-html-markdown` library (`parse5` dropped; the only behavior delta is column-aligned tables, GFM hard-break `<br>`, and `<ol start>` renumbering, all feeding the LLM via `web_fetch`), and `llama-args-validator.ts` was clean-room rewritten with the managed-flag denylist re-derived from the public llama-server flag list (facts, not copyrightable). The license flip set `LICENSE` to MIT (`Copyright (c) 2026 indifferentketchup`), the five `package.json` to `MIT`, removed every AGPL SPDX header, added a README License section, and added a `license-mit` guard test that fails if AGPL provenance returns. Built by three parallel agents over the disjoint files; full server suite 519 passing (incl. 9 new guard tests), server build + coder typecheck clean. Resolves `boocode_code_review_v2.md` §1 #1 / §5k and the roadmap's `License-debt` batch (openspec `license-debt-mit`); supersedes that batch's original staged plan, which had entangled the flip with a live qwen3.6 validation window. Relicenses BooCode from AGPL-3.0 back to MIT by clearing the three Unsloth-Studio-derived files the `v2.4.0`/`v2.4.1` lifts pulled in — the root `LICENSE` and all five `package.json` had been `AGPL-3.0-only`, making the network-served work AGPL §13-encumbered. The enabling finding decoupled the relicense from the long-planned native-llama-server-parsing retirement: `tool-call-parser.ts`'s Unsloth-ported algorithm (`parseToolCallsFromText`/`scanBalancedBraces` + unused nudge constants) was **dead code** with no production import, so it was simply deleted while the load-bearing `extractToolCallBlocks`/`stripToolMarkup` (BooCode-authored streaming helpers) were kept byte-identical — no behavior change to the live tool-call path. `html-to-md.ts` was swapped to the MIT `node-html-markdown` library (`parse5` dropped; the only behavior delta is column-aligned tables, GFM hard-break `<br>`, and `<ol start>` renumbering, all feeding the LLM via `web_fetch`), and `llama-args-validator.ts` was clean-room rewritten with the managed-flag denylist re-derived from the public llama-server flag list (facts, not copyrightable). The license flip set `LICENSE` to MIT (`Copyright (c) 2026 indifferentketchup`), the five `package.json` to `MIT`, removed every AGPL SPDX header, added a README License section, and added a `license-mit` guard test that fails if AGPL provenance returns. Built by three parallel agents over the disjoint files; full server suite 519 passing (incl. 9 new guard tests), server build + coder typecheck clean. Resolves `boocode_code_review_v2.md` §1 #1 / §5k and the roadmap's `License-debt` batch (openspec `license-debt-mit`); supersedes that batch's original staged plan, which had entangled the flip with a live qwen3.6 validation window.

View File

@@ -25,6 +25,7 @@ import { setInferenceContext, clearInferenceContext } from './services/tools/inf
import { registerMessageRoutes } from './routes/messages.js'; import { registerMessageRoutes } from './routes/messages.js';
import { registerSkillRoutes } from './routes/skills.js'; import { registerSkillRoutes } from './routes/skills.js';
import { registerPendingRoutes } from './routes/pending.js'; import { registerPendingRoutes } from './routes/pending.js';
import { registerCheckpointRoutes } from './routes/checkpoints.js';
import { registerAgentSessionRoutes } from './routes/agent-sessions.js'; import { registerAgentSessionRoutes } from './routes/agent-sessions.js';
import { registerTaskRoutes } from './routes/tasks.js'; import { registerTaskRoutes } from './routes/tasks.js';
import { registerInboxRoutes } from './routes/inbox.js'; import { registerInboxRoutes } from './routes/inbox.js';
@@ -214,6 +215,7 @@ async function main() {
registerMessageRoutes(app, sql, broker, inferenceApi); registerMessageRoutes(app, sql, broker, inferenceApi);
registerSkillRoutes(app, sql, broker, inferenceApi); registerSkillRoutes(app, sql, broker, inferenceApi);
registerPendingRoutes(app, sql); registerPendingRoutes(app, sql);
registerCheckpointRoutes(app, sql);
registerAgentSessionRoutes(app, sql); registerAgentSessionRoutes(app, sql);
registerTaskRoutes(app, sql, inferenceApi); registerTaskRoutes(app, sql, inferenceApi);
registerInboxRoutes(app, sql); registerInboxRoutes(app, sql);

View File

@@ -16,6 +16,11 @@ export interface AgentSessionRow {
status: string; status: string;
has_session: boolean; has_session: boolean;
last_active_at: string | null; last_active_at: string | null;
// v2.6.8 per-(chat,agent) running token/cost totals (sampling-streamjson-tokens
// #8). BIGINT columns arrive as strings over the wire; the frontend coerces.
input_tokens: number;
output_tokens: number;
cost: number;
} }
export function registerAgentSessionRoutes(app: FastifyInstance, sql: Sql): void { export function registerAgentSessionRoutes(app: FastifyInstance, sql: Sql): void {
@@ -39,7 +44,10 @@ export function registerAgentSessionRoutes(app: FastifyInstance, sql: Sql): void
a.agent AS agent, a.agent AS agent,
a.status AS status, a.status AS status,
(a.agent_session_id IS NOT NULL) AS has_session, (a.agent_session_id IS NOT NULL) AS has_session,
a.last_active_at AS last_active_at a.last_active_at AS last_active_at,
a.input_tokens AS input_tokens,
a.output_tokens AS output_tokens,
a.cost AS cost
FROM agent_sessions a FROM agent_sessions a
JOIN chats c ON c.id = a.chat_id JOIN chats c ON c.id = a.chat_id
WHERE c.session_id = ${sessionId} WHERE c.session_id = ${sessionId}

View File

@@ -0,0 +1,73 @@
/**
* write-edit-robustness #4 — checkpoint restore + list routes (coder side).
*
* Proxied through the apps/server `/api/coder/*` blanket forwarder (no server-side
* change needed for new routes). Restore rewinds the session worktree to the
* checkpoint's shadow commit, trims the transcript from the anchor message forward,
* and resets the agent backend — see services/checkpoints.ts.
*/
import type { FastifyInstance } from 'fastify';
import type { Sql } from '../db.js';
import { restoreCheckpoint, CheckpointNotFoundError } from '../services/checkpoints.js';
export function registerCheckpointRoutes(app: FastifyInstance, sql: Sql): void {
// GET /api/sessions/:sessionId/checkpoints?chat_id= — list a chat's checkpoints
// so the frontend can mark which messages have a restore point. When chat_id is
// omitted, returns every checkpoint for the session's chats.
app.get<{ Params: { sessionId: string }; Querystring: { chat_id?: string } }>(
'/api/sessions/:sessionId/checkpoints',
async (req, reply) => {
const sessionId = req.params.sessionId;
const chatId = req.query.chat_id;
const session = await sql<{ id: string }[]>`SELECT id FROM sessions WHERE id = ${sessionId}`;
if (session.length === 0) {
reply.code(404);
return { error: 'session not found' };
}
// Scope authoritatively through chats.session_id (always set) — NOT the
// denormalized checkpoints.session_id (nullable). The chat_id branch must
// still be session-gated or it's an IDOR (any session's chat_id reads its
// checkpoints).
const rows = chatId
? await sql<{ id: string; chat_id: string; message_id: string | null; label: string | null; created_at: Date }[]>`
SELECT cp.id, cp.chat_id, cp.message_id, cp.label, cp.created_at
FROM checkpoints cp
JOIN chats c ON c.id = cp.chat_id
WHERE cp.chat_id = ${chatId} AND c.session_id = ${sessionId}
ORDER BY cp.created_at
`
: await sql<{ id: string; chat_id: string; message_id: string | null; label: string | null; created_at: Date }[]>`
SELECT cp.id, cp.chat_id, cp.message_id, cp.label, cp.created_at
FROM checkpoints cp
JOIN chats c ON c.id = cp.chat_id
WHERE c.session_id = ${sessionId}
ORDER BY cp.created_at
`;
return rows;
},
);
// POST /api/sessions/:sessionId/checkpoints/:checkpointId/restore — restore.
app.post<{ Params: { sessionId: string; checkpointId: string } }>(
'/api/sessions/:sessionId/checkpoints/:checkpointId/restore',
async (req, reply) => {
const { sessionId, checkpointId } = req.params;
try {
const result = await restoreCheckpoint(sql, checkpointId, {
sessionId,
log: app.log,
});
return result;
} catch (err) {
if (err instanceof CheckpointNotFoundError) {
reply.code(404);
return { error: err.message };
}
throw err;
}
},
);
}

View File

@@ -240,6 +240,27 @@ END $$;
-- v2.6: attribution for DiffPanel badges (Phase 1 UX reads this). -- v2.6: attribution for DiffPanel badges (Phase 1 UX reads this).
ALTER TABLE pending_changes ADD COLUMN IF NOT EXISTS agent TEXT; ALTER TABLE pending_changes ADD COLUMN IF NOT EXISTS agent TEXT;
-- write-edit-robustness #4: worktree checkpoints. A pre-turn shadow-commit of the
-- session worktree (tracked + untracked, captured without disturbing the real
-- index/working tree) stored in a private GC-safe ref refs/boocode/checkpoints/<id>.
-- Created best-effort before each external-agent turn (opencode / warm-ACP / one-shot
-- ACP+PTY); restore resets the worktree to commit_sha, trims the transcript from
-- message_id forward, and resets the backend session. chat_id CASCADEs from chats
-- (like agent_sessions); worktree_id SET NULL so a checkpoint outlives a reaped
-- worktree row. session_id / message_id are informational (no FK — message rows are
-- trimmed by a checkpoint restore and we must not block that on a dangling ref).
CREATE TABLE IF NOT EXISTS checkpoints (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
session_id UUID,
worktree_id UUID REFERENCES worktrees(id) ON DELETE SET NULL,
message_id UUID, -- anchor: the assistant turn row this checkpoint precedes
commit_sha TEXT NOT NULL, -- shadow-commit capturing the pre-turn worktree tree
label TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX IF NOT EXISTS checkpoints_chat_created_idx ON checkpoints(chat_id, created_at);
-- LISTEN/NOTIFY fast path: every tasks INSERT (from any call site — routes, -- LISTEN/NOTIFY fast path: every tasks INSERT (from any call site — routes,
-- new_task tool, arena, MCP server) fires pg_notify('tasks_new') in the same -- new_task tool, arena, MCP server) fires pg_notify('tasks_new') in the same
-- transaction, so the dispatcher reacts immediately instead of waiting for the -- transaction, so the dispatcher reacts immediately instead of waiting for the

View File

@@ -0,0 +1,252 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { readFileSync } from 'node:fs';
import { rm, mkdir } from 'node:fs/promises';
import { resolve } from 'node:path';
import postgres from 'postgres';
import {
buildShadowCommitCommand,
createCheckpoint,
restoreCheckpoint,
CheckpointNotFoundError,
} from '../checkpoints.js';
import { ensureSessionWorktree } from '../worktrees.js';
import { hostExec } from '../host-exec.js';
/**
* write-edit-robustness #4 — worktree checkpoint tests.
*
* Pure-helper coverage (no DB / no host) for the shadow-commit command builder,
* plus a DB+git integration block (DB-opt-in via DATABASE_URL, skips cleanly
* otherwise; mirrors reconnect_integration.test.ts) that exercises the real
* create → restore round trip against a worktree on the host fs.
*/
describe('buildShadowCommitCommand (pure)', () => {
it('parks the commit under refs/boocode/checkpoints/<id> and prints only the SHA', () => {
const cmd = buildShadowCommitCommand('/tmp/booworktrees/sess-abc', 'cp-id-123');
// Uses a temp index so the real working tree/index is untouched.
expect(cmd).toContain('TMP=$(mktemp)');
expect(cmd).toContain('GIT_INDEX_FILE="$TMP" git read-tree HEAD');
expect(cmd).toContain('GIT_INDEX_FILE="$TMP" git add -A');
expect(cmd).toContain('git write-tree');
expect(cmd).toContain("git commit-tree \"$TREE\" -p HEAD -m \"boocode checkpoint\"");
// Ref name matches the row id, and stdout is ONLY the SHA (printf, no newline).
expect(cmd).toContain("update-ref 'refs/boocode/checkpoints/cp-id-123'");
expect(cmd).toContain("printf '%s' \"$SHA\"");
expect(cmd).not.toContain('echo "$SHA"');
});
it('shell-escapes the worktree path and the id', () => {
const cmd = buildShadowCommitCommand("/tmp/it's a path", "id'; rm -rf /");
// Single quotes inside the path/id are escaped via the '\'' wrapping idiom — no
// bare interpolation that could break out of the quoting.
expect(cmd).toContain("cd '/tmp/it'\\''s a path'");
expect(cmd).toContain("refs/boocode/checkpoints/id'\\''; rm -rf /");
});
});
describe.runIf(!!process.env.DATABASE_URL)('checkpoint create + restore (DB + git)', () => {
let sql: ReturnType<typeof postgres>;
const stamp = Date.now();
const projectDir = `/tmp/boocode-checkpoint-proj-${stamp}`;
let projectId: string;
let sessionId: string;
let chatId: string;
let worktreePath: string;
beforeAll(async () => {
sql = postgres(process.env.DATABASE_URL!, { max: 3 });
// Server schema first (FK targets), then coder schema (worktrees + checkpoints).
const serverSchema = resolve(__dirname, '../../../../server/src/schema.sql');
const coderSchema = resolve(__dirname, '../../schema.sql');
await sql.unsafe(readFileSync(serverSchema, 'utf8'));
await sql.unsafe(readFileSync(coderSchema, 'utf8'));
await mkdir(projectDir, { recursive: true });
await hostExec(
`cd ${projectDir} && git init -q && git config user.email t@t && git config user.name t ` +
`&& echo hello > README.md && git add -A && git commit -qm init`,
{ timeoutMs: 20_000 },
);
const [project] = await sql<{ id: string }[]>`
INSERT INTO projects (name, path, status) VALUES ('checkpoint-test', ${projectDir}, 'open') RETURNING id
`;
projectId = project!.id;
const [session] = await sql<{ id: string }[]>`
INSERT INTO sessions (project_id, name, model, status)
VALUES (${projectId}, 'cp', 'm', 'open') RETURNING id
`;
sessionId = session!.id;
const [chat] = await sql<{ id: string }[]>`
INSERT INTO chats (session_id, name, status) VALUES (${sessionId}, 'tab', 'open') RETURNING id
`;
chatId = chat!.id;
const wt = await ensureSessionWorktree(sql, projectDir, sessionId);
worktreePath = wt.worktreePath;
});
afterAll(async () => {
if (sql) {
const rows = await sql<{ path: string }[]>`SELECT path FROM worktrees WHERE session_id = ${sessionId}`.catch(() => []);
for (const r of rows) {
await hostExec(`git -C ${projectDir} worktree remove ${r.path} --force`, { timeoutMs: 10_000 }).catch(() => {});
}
await sql`DELETE FROM checkpoints WHERE chat_id = ${chatId}`.catch(() => {});
await sql`DELETE FROM agent_sessions WHERE chat_id = ${chatId}`.catch(() => {});
await sql`DELETE FROM worktrees WHERE session_id = ${sessionId}`.catch(() => {});
await sql`DELETE FROM chats WHERE id = ${chatId}`.catch(() => {});
await sql`DELETE FROM sessions WHERE id = ${sessionId}`.catch(() => {});
await sql`DELETE FROM projects WHERE id = ${projectId}`.catch(() => {});
await sql.end({ timeout: 5 });
}
await rm(projectDir, { recursive: true, force: true });
});
it('createCheckpoint inserts a row + a private ref capturing tracked + untracked', async () => {
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const worktreeId = wt!.id;
// Pre-turn untracked + tracked-edit state the agent will start from.
await hostExec(`cd ${worktreePath} && echo edited >> README.md && echo new > extra.txt`, { timeoutMs: 10_000 });
const [assistantMsg] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', '', 'streaming') RETURNING id
`;
const messageId = assistantMsg!.id;
const cp = await createCheckpoint(sql, {
chatId,
sessionId,
worktreeId,
worktreePath,
messageId,
});
expect(cp).not.toBeNull();
expect(cp!.commit_sha).toMatch(/^[0-9a-f]{40}$/);
const [row] = await sql<{ commit_sha: string; worktree_id: string; message_id: string }[]>`
SELECT commit_sha, worktree_id, message_id FROM checkpoints WHERE id = ${cp!.id}
`;
expect(row!.commit_sha).toBe(cp!.commit_sha);
expect(row!.worktree_id).toBe(worktreeId);
expect(row!.message_id).toBe(messageId);
// The ref exists and the captured tree carries the untracked file (proves the
// temp-index `git add -A` snapshotted untracked content).
const refLs = await hostExec(
`git -C ${worktreePath} ls-tree -r --name-only ${cp!.commit_sha}`,
{ timeoutMs: 10_000 },
);
expect(refLs.exitCode).toBe(0);
expect(refLs.stdout).toContain('extra.txt');
// The shadow commit did NOT disturb the real working tree: extra.txt is still
// present + still untracked (status shows it).
const status = await hostExec(`git -C ${worktreePath} status --porcelain`, { timeoutMs: 10_000 });
expect(status.stdout).toContain('extra.txt');
});
it('restoreCheckpoint resets the worktree, trims the transcript, and drops later checkpoints', async () => {
// Clean slate for this test: reset the worktree to HEAD, clear prior rows.
await hostExec(`git -C ${worktreePath} reset --hard HEAD && git -C ${worktreePath} clean -fd`, { timeoutMs: 10_000 });
await sql`DELETE FROM checkpoints WHERE chat_id = ${chatId}`;
await sql`DELETE FROM messages WHERE chat_id = ${chatId}`;
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const worktreeId = wt!.id;
// Turn 1: a user msg, then the assistant turn the checkpoint anchors. The
// worktree is pristine (matches HEAD) when this checkpoint is captured.
await sql`INSERT INTO messages (session_id, chat_id, role, content, status) VALUES (${sessionId}, ${chatId}, 'user', 'do it', 'complete')`;
const [a1] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', 'turn 1', 'complete') RETURNING id
`;
const cp1 = await createCheckpoint(sql, { chatId, sessionId, worktreeId, worktreePath, messageId: a1!.id });
expect(cp1).not.toBeNull();
// The agent (turn 1) writes a file into the worktree.
await hostExec(`cd ${worktreePath} && echo agent-wrote > agent.txt`, { timeoutMs: 10_000 });
// Turn 2: another user msg + assistant turn, AND a second (later) checkpoint.
await sql`INSERT INTO messages (session_id, chat_id, role, content, status) VALUES (${sessionId}, ${chatId}, 'user', 'more', 'complete')`;
const [a2] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', 'turn 2', 'complete') RETURNING id
`;
const cp2 = await createCheckpoint(sql, { chatId, sessionId, worktreeId, worktreePath, messageId: a2!.id });
expect(cp2).not.toBeNull();
// An agent_sessions row that restore should mark 'crashed'.
await sql`
INSERT INTO agent_sessions (chat_id, session_id, worktree_id, agent, backend, agent_session_id, status, last_active_at)
VALUES (${chatId}, ${sessionId}, ${worktreeId}, 'goose', 'acp_warm', 'sess-1', 'active', clock_timestamp())
ON CONFLICT (chat_id, agent) DO UPDATE SET status = 'active'
`;
const before = await sql<{ id: string }[]>`SELECT id FROM messages WHERE chat_id = ${chatId} ORDER BY created_at`;
expect(before.length).toBe(4); // user, a1, user, a2
// Restore to cp1 (before turn 1's assistant message).
const result = await restoreCheckpoint(sql, cp1!.id, { sessionId });
expect(result.checkpoint_id).toBe(cp1!.id);
expect(result.worktree_reset).toBe(true);
expect(result.backend_reset).toBe(true);
// a1, user(turn2), a2 deleted (created_at >= a1) → 3 trimmed.
expect(result.messages_deleted).toBe(3);
// Transcript trimmed to just the first user message.
const after = await sql<{ role: string; content: string }[]>`SELECT role, content FROM messages WHERE chat_id = ${chatId} ORDER BY created_at`;
expect(after.length).toBe(1);
expect(after[0]!.role).toBe('user');
// Worktree reset: the agent's file is gone (it was written after cp1).
const ls = await hostExec(`ls ${worktreePath}/agent.txt`, { timeoutMs: 10_000 });
expect(ls.exitCode).not.toBe(0);
// The agent_sessions row was reset to 'crashed'.
const [as] = await sql<{ status: string }[]>`SELECT status FROM agent_sessions WHERE chat_id = ${chatId} AND agent = 'goose'`;
expect(as!.status).toBe('crashed');
// cp1 survives (re-restorable); cp2 (later) was dropped.
const cps = await sql<{ id: string }[]>`SELECT id FROM checkpoints WHERE chat_id = ${chatId}`;
expect(cps.map((c) => c.id)).toEqual([cp1!.id]);
});
it('restoreCheckpoint throws CheckpointNotFoundError for an unknown id', async () => {
await expect(
restoreCheckpoint(sql, '00000000-0000-0000-0000-000000000000', { sessionId }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
});
it('restoreCheckpoint throws when the checkpoint is not in the requested session', async () => {
// A checkpoint whose session_id differs from the route's sessionId.
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const cp = await createCheckpoint(sql, { chatId, sessionId, worktreeId: wt!.id, worktreePath, messageId: null });
expect(cp).not.toBeNull();
await expect(
restoreCheckpoint(sql, cp!.id, { sessionId: '11111111-1111-1111-1111-111111111111' }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
await sql`DELETE FROM checkpoints WHERE id = ${cp!.id}`;
});
it('restoreCheckpoint denies a NULL-session_id checkpoint from another session (no fail-open IDOR)', async () => {
// Regression for the fail-open authorization bug: a checkpoint row whose
// denormalized session_id is NULL must STILL be scoped via its chat's owning
// session (chats.session_id), not skipped. The old guard `cp.session_id &&
// cp.session_id !== sessionId` fell through on NULL → cross-session restore.
const [row] = await sql<{ id: string }[]>`
INSERT INTO checkpoints (chat_id, session_id, message_id, commit_sha)
VALUES (${chatId}, NULL, NULL, 'deadbeef')
RETURNING id
`;
await expect(
restoreCheckpoint(sql, row!.id, { sessionId: '22222222-2222-2222-2222-222222222222' }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
await sql`DELETE FROM checkpoints WHERE id = ${row!.id}`;
});
});

View File

@@ -0,0 +1,173 @@
import { describe, it, expect } from 'vitest';
import { locateMatch, SIMILARITY_THRESHOLD } from '../fuzzy-match.js';
// Helper: assert a resolved span and slice it back out of the content so the
// test pins the EXACT file text the caller would replace.
function span(result: ReturnType<typeof locateMatch>): { start: number; end: number } {
if (result.kind !== 'exact' && result.kind !== 'fuzzy') {
throw new Error(`expected a located span, got ${result.kind}`);
}
return { start: result.start, end: result.end };
}
describe('locateMatch — strategy 1: exact', () => {
it('returns an exact unique span', () => {
const content = 'alpha\nbeta\ngamma\n';
const result = locateMatch(content, 'beta');
expect(result.kind).toBe('exact');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('beta');
});
it('returns the right offsets for a multi-line exact needle', () => {
const content = 'one\ntwo\nthree\nfour\n';
const needle = 'two\nthree';
const result = locateMatch(content, needle);
expect(result.kind).toBe('exact');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(needle);
});
it('refuses when the exact needle occurs more than once', () => {
const content = 'foo\nbar\nfoo\nbar\nfoo\n';
const result = locateMatch(content, 'foo');
expect(result).toEqual({ kind: 'ambiguous', count: 3 });
});
});
describe('locateMatch — strategy 2: per-line whitespace', () => {
it('matches across trailing-whitespace drift at the real span', () => {
// File has trailing spaces the model dropped from a TWO-line copy. A
// single-line needle would be located by exact indexOf (it's a substring),
// so use two lines where line 1's trailing ws breaks an exact substring run.
const content = 'function f() {\n setup(); \n return 1;\n}\n';
const needle = ' setup();\n return 1;'; // line 1 missing trailing spaces
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// The returned span covers the ORIGINAL lines including the trailing spaces.
expect(content.slice(start, end)).toBe(' setup(); \n return 1;');
});
it('matches across indentation drift (multi-line block)', () => {
// File indents with 4 spaces; model emitted 2-space indentation. trimEnd
// alone does not normalize LEADING whitespace, so this exercises... actually
// leading-indent drift is a Levenshtein-tier fallback. Here we keep the
// leading indent identical and drift only trailing whitespace per line.
const content = ['if (x) {', ' doThing(); ', ' doOther();', '}'].join('\n');
const needle = [' doThing();', ' doOther();'].join('\n');
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(' doThing(); \n doOther();');
});
it('ignores leading/trailing blank needle lines', () => {
const content = 'header\nbody line\nfooter\n';
const needle = '\n\nbody line\n\n';
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('body line');
});
it('reports ambiguous when a whitespace-window matches twice', () => {
// Both line 1 and line 4 differ from the needle only by trailing whitespace,
// so exact indexOf fails (no exact substring) and the whitespace tier finds
// two equivalent windows → ambiguous.
const content = 'x = 1; \ny = 2;\nz = 3;\nx = 1;\t\n';
const needle = 'x = 1;'; // no trailing ws → not an exact substring of either line
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'ambiguous', count: 2 });
});
});
describe('locateMatch — strategy 3: unicode canonicalization', () => {
it('matches across curly quotes', () => {
const content = "const s = 'hello';\n";
const needle = 'const s = hello;'; // hello
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// Span maps back to ORIGINAL (straight-quote) text.
expect(content.slice(start, end)).toBe("const s = 'hello';");
});
it('matches across curly double-quotes', () => {
const content = 'log("done");\n';
const needle = 'log(“done”);'; // “done”
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('log("done");');
});
it('matches across an em-dash drift', () => {
const content = 'range 1-10 inclusive\n';
const needle = 'range 1—10 inclusive'; // em-dash
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('range 1-10 inclusive');
});
it('matches across a non-breaking space drift', () => {
const content = 'a b c\n'; // plain spaces
const needle = 'a b c'; // nbsp between words
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('a b c');
});
});
describe('locateMatch — strategy 4: Levenshtein', () => {
it('matches a >= threshold near-miss (small typo drift)', () => {
// Needle has a one-char typo ('totals' vs 'total') so it is NOT an exact
// substring and the whitespace/canonical tiers (which require equality) both
// miss; Levenshtein similarity stays well above the 0.66 floor.
const content = 'const total = sum + tax;\n';
const needle = 'const totals = sum + tax;';
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// Span maps to the real (correctly-spelled) file line.
expect(content.slice(start, end)).toBe('const total = sum + tax;');
});
it('matches a multi-line block with indentation drift via Levenshtein', () => {
const content = ['function g() {', ' return compute(a, b);', '}'].join('\n');
// 6-space indent vs file's 2-space; trimEnd does not fix leading indent, so
// this lands on the Levenshtein tier (joined-trim makes it identical → ~1.0).
const needle = [' return compute(a, b);'].join('\n');
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(' return compute(a, b);');
});
it('returns not_found for a below-threshold miss', () => {
const content = 'the quick brown fox jumps over the lazy dog\n';
const needle = 'completely unrelated string of text here xyz';
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'not_found' });
});
it('returns not_found for a genuinely-absent needle', () => {
const content = 'alpha\nbeta\ngamma\n';
const needle = 'this content does not exist anywhere at all';
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'not_found' });
});
});
describe('locateMatch — edge cases', () => {
it('returns not_found for an empty needle', () => {
expect(locateMatch('anything', '')).toEqual({ kind: 'not_found' });
});
it('exposes a sane similarity threshold', () => {
expect(SIMILARITY_THRESHOLD).toBeGreaterThan(0);
expect(SIMILARITY_THRESHOLD).toBeLessThanOrEqual(1);
});
});

View File

@@ -0,0 +1,189 @@
import { describe, it, expect } from 'vitest';
import {
makeStreamJsonParser,
makeStreamJsonState,
parseStreamJsonLine,
type AgentEventList,
} from '../stream-json-parser.js';
import type { AgentEvent } from '../agent-backend.js';
import type { AcpToolSnapshot } from '../acp-tool-snapshot.js';
// Helpers to JSON-encode the representative Claude-Code stream-json lines.
const sys = (sessionId: string) =>
JSON.stringify({ type: 'system', subtype: 'init', session_id: sessionId, tools: ['read', 'edit'] });
const streamEvent = (event: unknown) => JSON.stringify({ type: 'stream_event', event });
const textDelta = (index: number, text: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'text_delta', text } });
const thinkingDelta = (index: number, thinking: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'thinking_delta', thinking } });
const toolStart = (index: number, id: string, name: string) =>
streamEvent({ type: 'content_block_start', index, content_block: { type: 'tool_use', id, name } });
const inputJsonDelta = (index: number, partial: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'input_json_delta', partial_json: partial } });
const blockStop = (index: number) => streamEvent({ type: 'content_block_stop', index });
const resultLine = (input: number, output: number, sessionId?: string) =>
JSON.stringify({ type: 'result', subtype: 'success', session_id: sessionId, usage: { input_tokens: input, output_tokens: output } });
describe('parseStreamJsonLine (pure per-line mapping)', () => {
it('captures session_id from the system init line and emits no events', () => {
const state = makeStreamJsonState();
const events = parseStreamJsonLine(sys('sess-abc'), state);
expect(events).toEqual([]);
expect(state.sessionId).toBe('sess-abc');
});
it('maps a text_delta stream_event → a text event', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(textDelta(0, 'Hello'), state)).toEqual([{ type: 'text', text: 'Hello' }]);
});
it('maps a thinking_delta stream_event → a reasoning event', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(thinkingDelta(0, 'pondering'), state)).toEqual([
{ type: 'reasoning', text: 'pondering' },
]);
});
it('tolerates a garbage / non-JSON line (returns [], no throw)', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine('not json at all {{{', state)).toEqual([]);
expect(parseStreamJsonLine('', state)).toEqual([]);
expect(parseStreamJsonLine(' ', state)).toEqual([]);
// A truncated/partial JSON object also yields [] rather than throwing.
expect(parseStreamJsonLine('{"type":"stream_event","eve', state)).toEqual([]);
});
it('ignores unknown top-level line types and the user (tool-result) line', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(JSON.stringify({ type: 'user', message: {} }), state)).toEqual([]);
expect(parseStreamJsonLine(JSON.stringify({ type: 'whatever' }), state)).toEqual([]);
});
it('assembles a tool call across input_json_delta chunks (split across lines)', () => {
const state = makeStreamJsonState();
// start → tool_call (running, empty args)
const start = parseStreamJsonLine(toolStart(1, 'toolu_1', 'edit_file'), state);
expect(start).toHaveLength(1);
expect(start[0]!.type).toBe('tool_call');
const startSnap = (start[0] as { type: 'tool_call'; toolCall: AcpToolSnapshot }).toolCall;
expect(startSnap.toolCallId).toBe('toolu_1');
expect(startSnap.title).toBe('edit_file');
expect(startSnap.status).toBe('in_progress');
expect(startSnap.rawInput).toEqual({});
// args streamed in fragments — no events until stop
expect(parseStreamJsonLine(inputJsonDelta(1, '{"path":"a'), state)).toEqual([]);
expect(parseStreamJsonLine(inputJsonDelta(1, '.ts","content":'), state)).toEqual([]);
expect(parseStreamJsonLine(inputJsonDelta(1, '"hi"}'), state)).toEqual([]);
// stop → tool_update with the parsed, fully-assembled input
const stop = parseStreamJsonLine(blockStop(1), state);
expect(stop).toHaveLength(1);
expect(stop[0]!.type).toBe('tool_update');
const stopSnap = (stop[0] as { type: 'tool_update'; toolCall: AcpToolSnapshot }).toolCall;
expect(stopSnap.toolCallId).toBe('toolu_1');
expect(stopSnap.status).toBe('completed');
expect(stopSnap.rawInput).toEqual({ path: 'a.ts', content: 'hi' });
});
it('falls back to {_raw} when accumulated tool args are not valid JSON', () => {
const state = makeStreamJsonState();
parseStreamJsonLine(toolStart(0, 'toolu_x', 'run'), state);
parseStreamJsonLine(inputJsonDelta(0, '{"broken'), state);
const stop = parseStreamJsonLine(blockStop(0), state);
const snap = (stop[0] as { type: 'tool_update'; toolCall: AcpToolSnapshot }).toolCall;
expect(snap.rawInput).toEqual({ _raw: '{"broken' });
});
it('captures usage from message_delta and result lines', () => {
const state = makeStreamJsonState();
parseStreamJsonLine(streamEvent({ type: 'message_delta', usage: { output_tokens: 42 } }), state);
expect(state.usage.outputTokens).toBe(42);
parseStreamJsonLine(resultLine(100, 250, 'sess-z'), state);
expect(state.usage.inputTokens).toBe(100);
expect(state.usage.outputTokens).toBe(250);
expect(state.sessionId).toBe('sess-z');
});
it('maps a terminal assistant message (fallback) → text + reasoning + tool events', () => {
const state = makeStreamJsonState();
const line = JSON.stringify({
type: 'assistant',
session_id: 'sess-asst',
message: {
content: [
{ type: 'thinking', thinking: 'let me think' },
{ type: 'text', text: 'Here is the answer' },
{ type: 'tool_use', id: 'toolu_9', name: 'view_file', input: { path: 'x.ts' } },
],
usage: { input_tokens: 5, output_tokens: 7 },
},
});
const events = parseStreamJsonLine(line, state);
expect(events).toEqual([
{ type: 'reasoning', text: 'let me think' },
{ type: 'text', text: 'Here is the answer' },
{
type: 'tool_update',
toolCall: { toolCallId: 'toolu_9', title: 'view_file', kind: null, status: 'completed', rawInput: { path: 'x.ts' } },
},
]);
expect(state.usage).toEqual({ inputTokens: 5, outputTokens: 7 });
expect(state.sessionId).toBe('sess-asst');
});
});
describe('makeStreamJsonParser (stateful wrapper over a full turn)', () => {
it('streams a representative turn: init → text → thinking → tool → result', () => {
const parser = makeStreamJsonParser();
const all: AgentEvent[] = [];
const feed = (line: string): AgentEventList => {
const evs = parser.push(line);
all.push(...evs);
return evs;
};
feed(sys('sess-1'));
feed(textDelta(0, 'Reading '));
feed(textDelta(0, 'the file. '));
feed(thinkingDelta(0, 'I should edit it'));
feed(toolStart(1, 'toolu_a', 'edit_file'));
feed(inputJsonDelta(1, '{"path":'));
feed(inputJsonDelta(1, '"main.ts"}'));
feed(blockStop(1));
feed(textDelta(0, 'Done.'));
feed(resultLine(120, 80, 'sess-1'));
expect(all).toEqual([
{ type: 'text', text: 'Reading ' },
{ type: 'text', text: 'the file. ' },
{ type: 'reasoning', text: 'I should edit it' },
{
type: 'tool_call',
toolCall: { toolCallId: 'toolu_a', title: 'edit_file', kind: null, status: 'in_progress', rawInput: {} },
},
{
type: 'tool_update',
toolCall: { toolCallId: 'toolu_a', title: 'edit_file', kind: null, status: 'completed', rawInput: { path: 'main.ts' } },
},
{ type: 'text', text: 'Done.' },
]);
expect(parser.usage()).toEqual({ inputTokens: 120, outputTokens: 80 });
expect(parser.sessionId()).toBe('sess-1');
});
it('a garbage line interleaved mid-turn does not derail subsequent parsing', () => {
const parser = makeStreamJsonParser();
expect(parser.push(textDelta(0, 'a'))).toEqual([{ type: 'text', text: 'a' }]);
expect(parser.push('>>> not json <<<')).toEqual([]);
expect(parser.push(textDelta(0, 'b'))).toEqual([{ type: 'text', text: 'b' }]);
});
});

View File

@@ -0,0 +1,306 @@
/**
* write-edit-robustness #4 — worktree checkpoints.
*
* External agents (opencode / goose / qwen / claude) write DIRECTLY into the
* shared session worktree (`/tmp/booworktrees/sess-<id>`); BooCode's own `rewind`
* only reverses `pending_changes` against the project root, so it has zero coverage
* there. A checkpoint is a pre-turn shadow-commit of the worktree tree (tracked +
* untracked) captured WITHOUT touching the real index/working tree, stored in a
* private GC-safe ref. `restoreCheckpoint` rewinds the worktree to that commit,
* trims the transcript from the anchor message forward, and resets the agent
* backend so the next turn re-establishes a fresh context consistent with the
* restored files.
*
* All git goes through hostExec + shellEscape (BooCoder runs on the host; the
* worktrees live on the host fs). Checkpoint CREATION is best-effort: a failure
* logs and returns null — it must NEVER throw into the dispatch turn.
*/
import { randomUUID } from 'node:crypto';
import type { FastifyBaseLogger } from 'fastify';
import type { Sql } from '../db.js';
import { hostExec } from './host-exec.js';
import { agentPool, OPENCODE_POOL_KEY } from './agent-pool.js';
import type { AgentSessionHandle } from './agent-backend.js';
/** Minimal shell escape for paths/refs (single-quote wrapping). Mirrors worktrees.ts. */
function shellEscape(s: string): string {
return "'" + s.replace(/'/g, "'\\''") + "'";
}
/**
* Pure builder for the shadow-commit command. Captures tracked + untracked files
* in the worktree into a temp index (so the real index/working tree is untouched),
* writes a tree, commits it parented on HEAD, and parks the commit under a private
* ref `refs/boocode/checkpoints/<id>` so git's GC never reclaims it. Prints ONLY
* the resulting SHA on stdout (the trailing `printf '%s'`), so the caller parses
* stdout.trim() directly.
*
* `id` is the row UUID (minted before the ref so the ref name matches the row).
* Both the worktree path and the id are shell-escaped.
*/
export function buildShadowCommitCommand(worktreePath: string, id: string): string {
const wt = shellEscape(worktreePath);
const ref = shellEscape(`refs/boocode/checkpoints/${id}`);
return (
`cd ${wt} && TMP=$(mktemp) && GIT_INDEX_FILE="$TMP" git read-tree HEAD ` +
`&& GIT_INDEX_FILE="$TMP" git add -A ` +
`&& TREE=$(GIT_INDEX_FILE="$TMP" git write-tree) ` +
`&& SHA=$(git commit-tree "$TREE" -p HEAD -m "boocode checkpoint") ` +
`&& git update-ref ${ref} "$SHA" && rm -f "$TMP" && printf '%s' "$SHA"`
);
}
export interface CreateCheckpointArgs {
chatId: string;
sessionId: string | null;
worktreeId: string | null;
worktreePath: string;
messageId: string | null;
label?: string | null;
}
/**
* Capture a pre-turn checkpoint of the session worktree. Best-effort: returns the
* inserted row's { id, commit_sha } on success, or null on any failure (the turn
* proceeds either way — a missing checkpoint just means no restore point for that
* turn). NEVER throws.
*
* The id is minted up front so the git ref name (`refs/boocode/checkpoints/<id>`)
* matches the DB row id, keeping ref and row in lockstep.
*/
export async function createCheckpoint(
sql: Sql,
args: CreateCheckpointArgs,
opts?: { signal?: AbortSignal; log?: FastifyBaseLogger },
): Promise<{ id: string; commit_sha: string } | null> {
const id = randomUUID();
try {
const cmd = buildShadowCommitCommand(args.worktreePath, id);
const res = await hostExec(cmd, { signal: opts?.signal, timeoutMs: 30_000 });
if (res.exitCode !== 0) {
opts?.log?.warn(
{ chatId: args.chatId, worktreePath: args.worktreePath, stderr: res.stderr.trim().slice(0, 500) },
'checkpoint: shadow-commit failed (turn proceeds without a checkpoint)',
);
return null;
}
const commitSha = res.stdout.trim();
if (!commitSha) {
opts?.log?.warn(
{ chatId: args.chatId, worktreePath: args.worktreePath },
'checkpoint: shadow-commit produced no SHA (turn proceeds)',
);
return null;
}
await sql`
INSERT INTO checkpoints (id, chat_id, session_id, worktree_id, message_id, commit_sha, label)
VALUES (${id}, ${args.chatId}, ${args.sessionId}, ${args.worktreeId}, ${args.messageId}, ${commitSha}, ${args.label ?? null})
`;
opts?.log?.info({ checkpointId: id, chatId: args.chatId, commitSha }, 'checkpoint: created');
return { id, commit_sha: commitSha };
} catch (err) {
opts?.log?.warn(
{ chatId: args.chatId, err: err instanceof Error ? err.message : String(err) },
'checkpoint: create threw (turn proceeds without a checkpoint)',
);
return null;
}
}
/** Error the route maps to a 404 when the checkpoint can't be resolved / scoped. */
export class CheckpointNotFoundError extends Error {
constructor(message: string) {
super(message);
this.name = 'CheckpointNotFoundError';
}
}
export interface RestoreCheckpointResult {
checkpoint_id: string;
messages_deleted: number;
worktree_reset: boolean;
backend_reset: boolean;
}
export interface RestoreCheckpointOpts {
signal?: AbortSignal;
log?: FastifyBaseLogger;
/** If set, the checkpoint MUST belong to this session (route scope guard). */
sessionId?: string;
}
interface CheckpointRow {
id: string;
chat_id: string;
session_id: string | null;
worktree_id: string | null;
message_id: string | null;
commit_sha: string;
created_at: Date;
}
/**
* Restore a checkpoint: rewind its worktree to the shadow commit, trim the
* transcript from the anchor message forward, reset the backend session, and drop
* now-orphaned later checkpoints. Throws CheckpointNotFoundError when the
* checkpoint is missing or not in the requested session (route → 404).
*/
export async function restoreCheckpoint(
sql: Sql,
checkpointId: string,
opts?: RestoreCheckpointOpts,
): Promise<RestoreCheckpointResult> {
// 1. Resolve the checkpoint.
const [cp] = await sql<CheckpointRow[]>`
SELECT id, chat_id, session_id, worktree_id, message_id, commit_sha, created_at
FROM checkpoints WHERE id = ${checkpointId}
`;
if (!cp) {
throw new CheckpointNotFoundError('checkpoint not found');
}
// Authorization scope (fail-safe): the checkpoint's chat must belong to the
// requested session. cp.session_id is a denormalized hint that may be null, so
// gating on it directly fails open — resolve the owning session via chats
// (authoritative; chat_id is NOT NULL) and deny on any mismatch or missing row.
if (opts?.sessionId) {
const [owner] = await sql<{ session_id: string | null }[]>`
SELECT session_id FROM chats WHERE id = ${cp.chat_id}
`;
if (!owner || owner.session_id !== opts.sessionId) {
throw new CheckpointNotFoundError('checkpoint not in session');
}
}
// 2. Resolve the worktree path (by worktree_id, else the session's active one).
let worktreePath: string | null = null;
if (cp.worktree_id) {
const [wt] = await sql<{ path: string }[]>`
SELECT path FROM worktrees WHERE id = ${cp.worktree_id}
`;
worktreePath = wt?.path ?? null;
}
if (!worktreePath) {
const sid = cp.session_id ?? opts?.sessionId ?? null;
if (sid) {
const [wt] = await sql<{ path: string }[]>`
SELECT path FROM worktrees WHERE session_id = ${sid} AND status = 'active' LIMIT 1
`;
worktreePath = wt?.path ?? null;
}
}
// 3. Worktree reset — hard-reset to the shadow commit, then clean untracked.
let worktreeReset = false;
if (worktreePath) {
const resetRes = await hostExec(
`git -C ${shellEscape(worktreePath)} reset --hard ${shellEscape(cp.commit_sha)}`,
{ signal: opts?.signal, timeoutMs: 30_000 },
).catch((err) => {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: reset --hard threw',
);
return null;
});
if (resetRes && resetRes.exitCode === 0) {
const cleanRes = await hostExec(
`git -C ${shellEscape(worktreePath)} clean -fd`,
{ signal: opts?.signal, timeoutMs: 30_000 },
).catch(() => null);
worktreeReset = cleanRes != null && cleanRes.exitCode === 0;
if (!worktreeReset) {
opts?.log?.warn({ checkpointId, worktreePath }, 'checkpoint restore: clean -fd did not succeed');
}
} else {
opts?.log?.warn(
{ checkpointId, worktreePath, stderr: resetRes?.stderr?.trim()?.slice(0, 500) },
'checkpoint restore: reset --hard did not succeed',
);
}
} else {
opts?.log?.warn({ checkpointId }, 'checkpoint restore: no worktree path resolved (files not reset)');
}
// 4. Trim the transcript from the anchor message forward. message_parts FK to
// messages is ON DELETE CASCADE (apps/server schema.sql:49), so parts are
// removed with their messages — no explicit parts delete needed.
let messagesDeleted = 0;
if (cp.message_id) {
const deleted = await sql<{ id: string }[]>`
DELETE FROM messages
WHERE chat_id = ${cp.chat_id}
AND created_at >= (SELECT created_at FROM messages WHERE id = ${cp.message_id})
RETURNING id
`;
messagesDeleted = deleted.length;
}
// 5. Backend reset — mark the chat's agent sessions crashed so the next turn
// re-establishes a fresh backend, and evict the live pool session(s) for this
// (chat, agent). Warm backends hold context server-side with no partial
// rewind, so a full reset is the only consistent option (proposal §4).
const agentRows = await sql<{ agent: string; backend: string; agent_session_id: string | null; session_id: string | null; worktree_id: string | null }[]>`
SELECT agent, backend, agent_session_id, session_id, worktree_id
FROM agent_sessions WHERE chat_id = ${cp.chat_id}
`;
await sql`
UPDATE agent_sessions SET status = 'crashed' WHERE chat_id = ${cp.chat_id}
`.catch(() => {});
let backendReset = false;
try {
// opencode runs on the SHARED server (keyed on a sentinel, not the chat) — close
// just this chat's session(s) on it, mirroring the lifecycle close-hook.
const ocBackend = agentPool.peek(OPENCODE_POOL_KEY, 'opencode');
if (ocBackend) {
for (const row of agentRows) {
if (row.backend !== 'opencode_server' || !row.agent_session_id) continue;
const handle: AgentSessionHandle = {
sessionId: row.session_id ?? '',
agent: row.agent,
backend: 'opencode_server',
chatId: cp.chat_id,
worktreeId: row.worktree_id ?? '',
agentSessionId: row.agent_session_id,
serverPort: null,
};
await ocBackend.closeSession(handle).catch((err) => {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: opencode closeSession threw',
);
});
}
}
// Warm-ACP backends are pooled under the chat id — dispose them (kills the
// goose/qwen child). closeChat skips busy backends (a live turn isn't torn down).
const disposed = await agentPool.closeChat(cp.chat_id);
backendReset = true;
opts?.log?.info({ checkpointId, chatId: cp.chat_id, disposed }, 'checkpoint restore: backend reset');
} catch (err) {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: backend reset threw',
);
}
// 6. Drop now-orphaned later checkpoints for this chat (their anchor messages were
// just trimmed). Compare `created_at` SERVER-SIDE via a subquery (NOT the JS
// Date round-trip, which truncates the stored microsecond precision to ms and
// would make this checkpoint delete ITSELF), and exclude this checkpoint's own
// id so it always survives — letting the user re-restore to it.
await sql`
DELETE FROM checkpoints
WHERE chat_id = ${cp.chat_id}
AND id <> ${cp.id}
AND created_at > (SELECT created_at FROM checkpoints WHERE id = ${cp.id})
`.catch(() => {});
return {
checkpoint_id: checkpointId,
messages_deleted: messagesDeleted,
worktree_reset: worktreeReset,
backend_reset: backendReset,
};
}

View File

@@ -4,6 +4,7 @@ import type { Broker } from '@boocode/server/broker';
import type { WsFrame } from '@boocode/server/ws-frames'; import type { WsFrame } from '@boocode/server/ws-frames';
import type { Config } from '../config.js'; import type { Config } from '../config.js';
import { createWorktree, diffWorktree, cleanupWorktree, ensureSessionWorktree } from './worktrees.js'; import { createWorktree, diffWorktree, cleanupWorktree, ensureSessionWorktree } from './worktrees.js';
import { createCheckpoint } from './checkpoints.js';
import { makeDcpStreamStripper } from './dcp-strip.js'; import { makeDcpStreamStripper } from './dcp-strip.js';
import { dispatchViaAcp } from './acp-dispatch.js'; import { dispatchViaAcp } from './acp-dispatch.js';
import { getResolvedRegistry } from './provider-config-registry.js'; import { getResolvedRegistry } from './provider-config-registry.js';
@@ -358,6 +359,16 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`; `;
const assistantId = assistantMsg!.id; const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn worktree checkpoint (best-effort; a
// failure logs and never breaks dispatch). This path uses a per-task worktree
// (createWorktree, not the session worktree), so there's no worktrees-table id
// — pass null for worktreeId, the path is enough for restore's reset.
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId: null, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, { broker.publishFrame(sessionId, {
type: 'message_started', type: 'message_started',
message_id: assistantId, message_id: assistantId,
@@ -399,6 +410,52 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
outputSummary = result.output.slice(0, 500); outputSummary = result.output.slice(0, 500);
await persistExternalAgentTurn(sql, assistantId, result.toolSnapshots, acpReasoning); await persistExternalAgentTurn(sql, assistantId, result.toolSnapshots, acpReasoning);
} else { } else {
// v#7 (stream-json): claude + qwen run with --output-format stream-json.
// Parse the NDJSON live in pty-dispatch and forward AgentEvents here so we
// publish the SAME live frames the warm-ACP / opencode paths emit (text,
// reasoning, tool) and persist structured parts. Accumulate for the final
// message content + persistence; fall back to the opaque stdout slice when
// nothing parsed (agent ran without the flag, or crashed before emitting).
const ptyTextChunks: string[] = [];
const ptyReasoningChunks: string[] = [];
const ptyToolSnaps = new Map<string, AcpToolSnapshot>();
const onPtyEvent = (e: AgentEvent): void => {
switch (e.type) {
case 'text':
ptyTextChunks.push(e.text);
broker.publishFrame(sessionId, {
type: 'delta',
message_id: assistantId,
chat_id: chatId,
content: e.text,
} as WsFrame);
break;
case 'reasoning':
ptyReasoningChunks.push(e.text);
broker.publishFrame(sessionId, {
type: 'reasoning_delta',
message_id: assistantId,
chat_id: chatId,
content: e.text,
} as WsFrame);
break;
case 'tool_call':
case 'tool_update':
ptyToolSnaps.set(e.toolCall.toolCallId, e.toolCall);
broker.publishFrame(sessionId, {
type: 'tool_call',
message_id: assistantId,
chat_id: chatId,
tool_call: snapshotToWireToolCall(e.toolCall),
} as WsFrame);
break;
case 'commands':
// stream-json carries no commands today; ignore if it ever does.
break;
}
};
const result = await dispatchViaPty({ const result = await dispatchViaPty({
agent, agent,
task: task.input, task: task.input,
@@ -409,7 +466,22 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
thinkingOptionId: task.thinking_option_id ?? undefined, thinkingOptionId: task.thinking_option_id ?? undefined,
signal: ac.signal, signal: ac.signal,
log, log,
onEvent: onPtyEvent,
}); });
if (result.streamed) {
assistantContent = ptyTextChunks.join('').slice(0, 50_000);
// stream-json text can be empty for a tool-only turn — surface stderr or a
// placeholder so the message row isn't blank.
if (!assistantContent) {
assistantContent = (result.stderr || '(no text output)').slice(0, 50_000);
}
outputSummary = (ptyTextChunks.join('') || result.stderr).slice(0, 500);
acpReasoning = ptyReasoningChunks.join('').slice(0, 200_000);
await persistExternalAgentTurn(sql, assistantId, [...ptyToolSnaps.values()], acpReasoning);
} else {
// Fallback: agent produced no parseable NDJSON (ran without the flag, or
// crashed). Preserve today's opaque stdout-slice + single delta behavior.
assistantContent = (result.stdout || result.stderr || '(no output)').slice(0, 50_000); assistantContent = (result.stdout || result.stderr || '(no output)').slice(0, 50_000);
outputSummary = (result.stdout || result.stderr).slice(0, 500); outputSummary = (result.stdout || result.stderr).slice(0, 500);
@@ -422,6 +494,7 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
} as WsFrame); } as WsFrame);
} }
} }
}
await sql` await sql`
UPDATE messages UPDATE messages
@@ -617,6 +690,15 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`; `;
const assistantId = assistantMsg!.id; const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn checkpoint of the persistent session
// worktree (best-effort; never breaks dispatch). worktreeId comes from the
// worktrees table (ensureSessionWorktree above).
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, { broker.publishFrame(sessionId, {
type: 'message_started', type: 'message_started',
message_id: assistantId, message_id: assistantId,
@@ -876,6 +958,15 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`; `;
const assistantId = assistantMsg!.id; const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn checkpoint of the persistent session
// worktree (best-effort; never breaks dispatch). Same worktree the opencode
// path uses — a chat that switches opencode↔goose↔qwen shares one worktree.
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, { broker.publishFrame(sessionId, {
type: 'message_started', type: 'message_started',
message_id: assistantId, message_id: assistantId,

View File

@@ -0,0 +1,271 @@
// Fuzzy patch locator for staged edits.
//
// Local quantized models (qwen3.6 and friends) frequently reproduce an
// `old_string` with small, semantically-irrelevant drift: trailing whitespace,
// a different indent width, or "smart" unicode punctuation (curly quotes, an
// en/em-dash, a non-breaking space) where the source has the plain ASCII form.
// An exact `String.includes` then fails and the queued edit is lost even though
// a human would say it obviously matches.
//
// `locateMatch` walks a ladder of progressively looser strategies and returns
// the real `[start, end)` byte-offset span in the ORIGINAL content so the caller
// can splice in `new_string` over the true file text (preserving the file's own
// whitespace/unicode, not the model's drifted copy). The ladder stops at the
// first strategy that resolves to a single span:
//
// 1. exact — indexOf; >1 hit is reported `ambiguous` (we refuse to
// guess which occurrence the model meant).
// 2. per-line ws — line-window compare ignoring per-line trailing
// whitespace and leading/trailing blank needle lines.
// 3. unicode canon — same line-window compare after folding smart
// punctuation to ASCII on both sides; the match is
// mapped back to original offsets.
// 4. levenshtein — best line-window by normalized edit-distance
// similarity; accepted only at >= SIMILARITY_THRESHOLD.
//
// Pure and dependency-free (Levenshtein is the standard iterative two-row DP),
// reimplemented from the general technique — no vendored source.
export type MatchResult =
| { kind: 'exact' | 'fuzzy'; start: number; end: number } // [start,end) offsets into content
| { kind: 'ambiguous'; count: number }
| { kind: 'not_found' };
/** Levenshtein similarity floor for the final fuzzy fallback (strategy 4). */
export const SIMILARITY_THRESHOLD = 0.66;
export function locateMatch(content: string, needle: string): MatchResult {
// Empty needle has no meaningful match.
if (needle.length === 0) return { kind: 'not_found' };
// --- 1. Exact ----------------------------------------------------------------
const exact = locateExact(content, needle);
if (exact) return exact;
// --- 2. Per-line whitespace-insensitive -------------------------------------
const ws = locateByLineWindow(content, needle);
if (ws) return ws;
// --- 3. Unicode-canonicalized whitespace pass -------------------------------
const canon = locateCanonical(content, needle);
if (canon) return canon;
// --- 4. Levenshtein similarity ----------------------------------------------
const lev = locateByLevenshtein(content, needle);
if (lev) return lev;
return { kind: 'not_found' };
}
// --- Strategy 1: exact -------------------------------------------------------
function locateExact(content: string, needle: string): MatchResult | null {
const first = content.indexOf(needle);
if (first === -1) return null;
const second = content.indexOf(needle, first + 1);
if (second === -1) {
return { kind: 'exact', start: first, end: first + needle.length };
}
// Count all occurrences so the caller can report a useful number.
let count = 2;
let idx = content.indexOf(needle, second + 1);
while (idx !== -1) {
count++;
idx = content.indexOf(needle, idx + 1);
}
return { kind: 'ambiguous', count };
}
// --- Line-window machinery ---------------------------------------------------
interface Line {
/** Raw line text (no trailing newline). */
text: string;
/** Offset of the first char of this line in the original content. */
start: number;
/** Offset one past the last char of this line (before its newline, if any). */
end: number;
}
/**
* Split content into lines, tracking each line's real offset span. The span
* EXCLUDES the trailing newline so consecutive line spans plus their newlines
* exactly reconstruct the content; the match span we hand back covers from the
* first matched line's start through the last matched line's end (i.e. without a
* trailing newline), which is what an in-place splice wants.
*/
function splitLines(content: string): Line[] {
const lines: Line[] = [];
let start = 0;
for (let i = 0; i <= content.length; i++) {
if (i === content.length || content[i] === '\n') {
lines.push({ text: content.slice(start, i), start, end: i });
start = i + 1;
}
}
return lines;
}
/** Strip leading/trailing all-blank lines; returns the trimmed slice. */
function trimBlankLines(lines: string[]): string[] {
let lo = 0;
let hi = lines.length;
while (lo < hi && lines[lo]!.trim() === '') lo++;
while (hi > lo && lines[hi - 1]!.trim() === '') hi--;
return lines.slice(lo, hi);
}
/**
* Find a contiguous window of content lines whose trailing-whitespace-trimmed
* text equals the needle's (blank-trimmed) lines. Returns the real offset span
* over the matched content lines, or null if zero match. Multiple matches →
* ambiguous. `normalize` lets the caller fold unicode before comparing.
*/
function locateByLineWindow(
content: string,
needle: string,
normalize: (s: string) => string = (s) => s,
): MatchResult | null {
const contentLines = splitLines(content);
const needleLines = trimBlankLines(needle.split('\n'));
const n = needleLines.length;
if (n === 0) return null;
// A single needle line that is itself blank can't be located meaningfully.
if (n === 1 && needleLines[0]!.trim() === '') return null;
const needleKey = needleLines.map((l) => normalize(l.trimEnd())).join('\n');
const hits: Array<{ start: number; end: number }> = [];
for (let i = 0; i + n <= contentLines.length; i++) {
const windowKey = contentLines
.slice(i, i + n)
.map((l) => normalize(l.text.trimEnd()))
.join('\n');
if (windowKey === needleKey) {
hits.push({ start: contentLines[i]!.start, end: contentLines[i + n - 1]!.end });
}
}
if (hits.length === 0) return null;
if (hits.length > 1) return { kind: 'ambiguous', count: hits.length };
return { kind: 'fuzzy', start: hits[0]!.start, end: hits[0]!.end };
}
// --- Strategy 3: unicode canonicalization ------------------------------------
/**
* Fold smart punctuation to its ASCII equivalent. Crucially this is a
* length-PRESERVING, per-character map (every replacement is one char → one
* char), so an offset into the canonical string is also a valid offset into the
* original — letting strategy 3 reuse the line-window matcher and still hand
* back true original-content offsets.
*/
function canonicalizeChar(ch: string): string {
switch (ch) {
// single quotes / apostrophes
case '': // '
case '': // '
case '': //
case '': //
return "'";
// double quotes
case '“': // "
case '”': // "
case '„': // „
case '‟': // ‟
return '"';
// dashes
case '': // en dash
case '—': // — em dash
case '': // figure dash
case '―': // ― horizontal bar
case '': // minus sign
return '-';
// spaces
case ' ': // nbsp
case '': // figure space
case '': // narrow nbsp
return ' ';
default:
return ch;
}
}
function canonicalize(s: string): string {
let out = '';
for (const ch of s) out += canonicalizeChar(ch);
return out;
}
function locateCanonical(content: string, needle: string): MatchResult | null {
// Only worth running if canonicalization actually changes something on either
// side — otherwise it's identical to strategy 2 which already failed.
const canonContent = canonicalize(content);
const canonNeedle = canonicalize(needle);
if (canonContent === content && canonNeedle === needle) return null;
// Offsets are preserved (length-preserving fold), so a match on the canonical
// content maps directly back to the original.
return locateByLineWindow(canonContent, canonNeedle);
}
// --- Strategy 4: Levenshtein similarity --------------------------------------
/** Standard iterative two-row Levenshtein edit distance. */
function levenshtein(a: string, b: string): number {
if (a === b) return 0;
if (a.length === 0) return b.length;
if (b.length === 0) return a.length;
let prev = new Array<number>(b.length + 1);
let curr = new Array<number>(b.length + 1);
for (let j = 0; j <= b.length; j++) prev[j] = j;
for (let i = 1; i <= a.length; i++) {
curr[0] = i;
const ac = a.charCodeAt(i - 1);
for (let j = 1; j <= b.length; j++) {
const cost = ac === b.charCodeAt(j - 1) ? 0 : 1;
curr[j] = Math.min(
prev[j]! + 1, // deletion
curr[j - 1]! + 1, // insertion
prev[j - 1]! + cost, // substitution
);
}
[prev, curr] = [curr, prev];
}
return prev[b.length]!;
}
/** Normalized similarity in [0,1]: 1 - dist / max(len). */
function similarity(a: string, b: string): number {
const maxLen = Math.max(a.length, b.length);
if (maxLen === 0) return 1;
return 1 - levenshtein(a, b) / maxLen;
}
function locateByLevenshtein(content: string, needle: string): MatchResult | null {
const contentLines = splitLines(content);
const needleLines = trimBlankLines(needle.split('\n'));
const n = needleLines.length;
if (n === 0) return null;
if (contentLines.length < n) return null;
const needleJoined = needleLines.map((l) => l.trim()).join('\n');
let best = -1;
let bestSpan: { start: number; end: number } | null = null;
for (let i = 0; i + n <= contentLines.length; i++) {
const window = contentLines.slice(i, i + n);
const windowJoined = window.map((l) => l.text.trim()).join('\n');
const score = similarity(windowJoined, needleJoined);
if (score > best) {
best = score;
bestSpan = { start: window[0]!.start, end: window[n - 1]!.end };
}
}
if (bestSpan && best >= SIMILARITY_THRESHOLD) {
return { kind: 'fuzzy', start: bestSpan.start, end: bestSpan.end };
}
return null;
}

View File

@@ -2,6 +2,7 @@ import { readFile, writeFile, unlink, mkdir } from 'node:fs/promises';
import { dirname } from 'node:path'; import { dirname } from 'node:path';
import type { Sql } from '../db.js'; import type { Sql } from '../db.js';
import { resolveWritePath } from './write_guard.js'; import { resolveWritePath } from './write_guard.js';
import { locateMatch } from './fuzzy-match.js';
// --- Types ------------------------------------------------------------------- // --- Types -------------------------------------------------------------------
@@ -121,10 +122,18 @@ export async function applyOne(
case 'edit': { case 'edit': {
const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string }; const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string };
const content = await readFile(change.file_path, 'utf8'); const content = await readFile(change.file_path, 'utf8');
if (!content.includes(oldStr)) { const match = locateMatch(content, oldStr);
throw new Error('old_string not found in file — file may have changed since the edit was queued'); if (match.kind === 'ambiguous') {
throw new Error(
`old_string matches ${match.count} locations — add surrounding context to disambiguate`,
);
} }
const updated = content.replace(oldStr, newStr); if (match.kind === 'not_found') {
throw new Error(
'old_string not found in file (even fuzzily) — file may have changed since the edit was queued',
);
}
const updated = content.slice(0, match.start) + newStr + content.slice(match.end);
await writeFile(change.file_path, updated, 'utf8'); await writeFile(change.file_path, updated, 'utf8');
break; break;
} }
@@ -203,10 +212,18 @@ export async function rewindOne(
// Reverse an edit: swap old and new // Reverse an edit: swap old and new
const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string }; const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string };
const content = await readFile(change.file_path, 'utf8'); const content = await readFile(change.file_path, 'utf8');
if (!content.includes(newStr)) { const match = locateMatch(content, newStr);
throw new Error('new_string not found in file — cannot rewind; file may have been modified since apply'); if (match.kind === 'ambiguous') {
throw new Error(
`new_string matches ${match.count} locations — cannot rewind; add surrounding context to disambiguate`,
);
} }
const reverted = content.replace(newStr, oldStr); if (match.kind === 'not_found') {
throw new Error(
'new_string not found in file (even fuzzily) — cannot rewind; file may have been modified since apply',
);
}
const reverted = content.slice(0, match.start) + oldStr + content.slice(match.end);
await writeFile(change.file_path, reverted, 'utf8'); await writeFile(change.file_path, reverted, 'utf8');
break; break;
} }

View File

@@ -1,13 +1,29 @@
/** /**
* PTY dispatch — runs external agents directly on the host. * PTY dispatch — runs external agents directly on the host.
*
* claude + qwen run with `--output-format stream-json` and emit Claude-Code's
* stream-json NDJSON on stdout. When an `onEvent` callback is supplied we
* line-buffer that stdout (split on `\n`, hold the partial tail) and feed complete
* lines to `makeStreamJsonParser` so deltas surface live as AgentEvents. The raw
* stdout is still accumulated + returned for back-compat (and the dispatcher's
* fallback when nothing parsed). See `stream-json-parser.ts`.
*/ */
import type { FastifyBaseLogger } from 'fastify'; import type { FastifyBaseLogger } from 'fastify';
import { spawn } from 'node:child_process'; import { spawn } from 'node:child_process';
import type { AgentEvent } from './agent-backend.js';
import { makeStreamJsonParser, type StreamJsonUsage } from './stream-json-parser.js';
export interface DispatchResult { export interface DispatchResult {
exitCode: number; exitCode: number;
stdout: string; stdout: string;
stderr: string; stderr: string;
/** True iff at least one NDJSON AgentEvent was parsed from stdout (v#7). When
* false the dispatcher falls back to slicing stdout as the assistant content. */
streamed: boolean;
/** Final usage parsed from the stream-json `result` / `message_delta`, if any. */
usage?: StreamJsonUsage;
/** Provider session id from the stream-json `system` init line, if any. */
agentSessionId?: string | null;
} }
export interface PtyDispatchOpts { export interface PtyDispatchOpts {
@@ -20,6 +36,10 @@ export interface PtyDispatchOpts {
installPath?: string; installPath?: string;
signal?: AbortSignal; signal?: AbortSignal;
log: FastifyBaseLogger; log: FastifyBaseLogger;
/** Optional live event sink. When set, stdout is line-buffered + NDJSON-parsed
* and each AgentEvent is forwarded here as it arrives. Absent → opaque (old)
* behavior: stdout is accumulated and returned, no parsing. */
onEvent?: (e: AgentEvent) => void;
} }
interface PtySpawnSpec { interface PtySpawnSpec {
@@ -40,7 +60,9 @@ function buildPtySpawnSpec(
switch (agent) { switch (agent) {
case 'claude': { case 'claude': {
const args = ['-p']; // stream-json on -p requires --verbose (Claude Code rejects stream-json
// print mode without it). qwen needs no such flag.
const args = ['-p', '--output-format', 'stream-json', '--verbose'];
if (model) args.push('--model', model); if (model) args.push('--model', model);
if (modeId) args.push('--permission-mode', modeId); if (modeId) args.push('--permission-mode', modeId);
if (thinkingOptionId) args.push('--effort', thinkingOptionId); if (thinkingOptionId) args.push('--effort', thinkingOptionId);
@@ -73,7 +95,7 @@ function buildPtySpawnSpec(
} }
export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchResult> { export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchResult> {
const { agent, task, worktreePath, model, modeId, thinkingOptionId, installPath, signal, log } = opts; const { agent, task, worktreePath, model, modeId, thinkingOptionId, installPath, signal, log, onEvent } = opts;
const cmd = buildPtySpawnSpec(agent, task, model, modeId, thinkingOptionId, installPath); const cmd = buildPtySpawnSpec(agent, task, model, modeId, thinkingOptionId, installPath);
if (!cmd) { if (!cmd) {
@@ -81,6 +103,7 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
exitCode: 1, exitCode: 1,
stdout: '', stdout: '',
stderr: `Agent '${agent}' is not yet supported for PTY dispatch.`, stderr: `Agent '${agent}' is not yet supported for PTY dispatch.`,
streamed: false,
}; };
} }
@@ -102,7 +125,32 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
let stderr = ''; let stderr = '';
let killed = false; let killed = false;
child.stdout!.on('data', (chunk: Buffer) => { stdout += chunk.toString(); }); // Live NDJSON parsing (only when a sink is supplied). Line-buffer: split on
// '\n', dispatch complete lines, hold the partial tail until the next chunk.
const parser = onEvent ? makeStreamJsonParser() : null;
let lineBuf = '';
let streamed = false;
const feedLine = (line: string): void => {
if (!parser || !onEvent) return;
for (const e of parser.push(line)) {
streamed = true;
onEvent(e);
}
};
child.stdout!.on('data', (chunk: Buffer) => {
const text = chunk.toString();
stdout += text;
if (!parser) return;
lineBuf += text;
let nl = lineBuf.indexOf('\n');
while (nl !== -1) {
const line = lineBuf.slice(0, nl);
lineBuf = lineBuf.slice(nl + 1);
feedLine(line);
nl = lineBuf.indexOf('\n');
}
});
child.stderr!.on('data', (chunk: Buffer) => { stderr += chunk.toString(); }); child.stderr!.on('data', (chunk: Buffer) => { stderr += chunk.toString(); });
const cleanup = () => { const cleanup = () => {
@@ -116,7 +164,7 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
if (signal) { if (signal) {
if (signal.aborted) { if (signal.aborted) {
cleanup(); cleanup();
resolve({ exitCode: 130, stdout: '', stderr: 'Aborted before start' }); resolve({ exitCode: 130, stdout: '', stderr: 'Aborted before start', streamed: false });
return; return;
} }
signal.addEventListener('abort', cleanup, { once: true }); signal.addEventListener('abort', cleanup, { once: true });
@@ -124,8 +172,18 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
child.on('close', (code) => { child.on('close', (code) => {
if (signal) signal.removeEventListener('abort', cleanup); if (signal) signal.removeEventListener('abort', cleanup);
log.info({ agent, exitCode: code }, 'pty-dispatch: completed'); // Flush any final line with no trailing newline.
resolve({ exitCode: code ?? 1, stdout, stderr }); if (lineBuf.trim()) feedLine(lineBuf);
lineBuf = '';
log.info({ agent, exitCode: code, streamed }, 'pty-dispatch: completed');
resolve({
exitCode: code ?? 1,
stdout,
stderr,
streamed,
usage: parser?.usage(),
agentSessionId: parser?.sessionId() ?? null,
});
}); });
child.on('error', (err) => { child.on('error', (err) => {

View File

@@ -0,0 +1,296 @@
/**
* Claude-Code-compatible stream-json NDJSON parser (feature #7,
* openspec `sampling-streamjson-tokens`).
*
* qwen (`--output-format stream-json`) and claude (`--output-format stream-json`)
* both emit Claude-Code's stream-json NDJSON on stdout: one JSON object per line.
* This module turns that stream into the same transport-agnostic `AgentEvent`s the
* ACP / opencode-server backends emit, so the PTY dispatch path can publish live
* broker frames + persist structured parts instead of slicing stdout opaque.
*
* Two surfaces:
* - `parseStreamJsonLine(line, state)` — PURE per-line mapping (unit-testable).
* `state` is the caller-owned accumulator (open tool blocks + usage/session_id).
* - `makeStreamJsonParser()` — a thin stateful wrapper holding the state, with a
* `push(line)` that returns the events for that line and getters for the final
* `usage` / `sessionId`.
*
* Defensive by contract: a non-JSON / partial / garbage line yields `[]` and never
* throws. Tool args (`input_json_delta`) arrive fragmented across many lines; we
* accumulate the partial JSON string per content-block index and only surface the
* parsed `rawInput` once the block stops (or, as a fallback, off the terminal
* `assistant` message which carries the fully-assembled `tool_use` blocks).
*
* Schema (keyed on top-level `type`):
* - `system` — init: { session_id, tools, ... }
* - `assistant` — { message: { content: [ {type:'text'|'thinking'|'tool_use', ...} ], usage? } }
* - `user` — tool results (ignored — diffing the worktree captures effects)
* - `result` — final: { usage: { input_tokens, output_tokens }, session_id? }
* - `stream_event` — { event: { type, index?, content_block?, delta?, usage? } }
* event.type:
* content_block_start — { index, content_block: {type, id?, name?} }
* content_block_delta — { index, delta: {type, text?|thinking?|partial_json?} }
* content_block_stop — { index }
* message_delta — { usage: { output_tokens } }
* message_start — { message: { usage } }
*/
import type { AgentEvent } from './agent-backend.js';
import type { AcpToolSnapshot } from './acp-tool-snapshot.js';
/** Convenience alias for the per-line return value. */
export type AgentEventList = AgentEvent[];
export interface StreamJsonUsage {
inputTokens?: number;
outputTokens?: number;
}
/** Per-open-content-block accumulation for tool args assembled across deltas. */
interface OpenToolBlock {
toolCallId: string;
name: string;
/** Concatenated `input_json_delta.partial_json` fragments. */
partialJson: string;
}
export interface StreamJsonState {
/** content-block index → open tool block (only `tool_use` blocks are tracked). */
toolBlocks: Map<number, OpenToolBlock>;
sessionId: string | null;
usage: StreamJsonUsage;
}
export function makeStreamJsonState(): StreamJsonState {
return { toolBlocks: new Map(), sessionId: null, usage: {} };
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (value && typeof value === 'object' && !Array.isArray(value)) {
return value as Record<string, unknown>;
}
return null;
}
function asString(value: unknown): string | undefined {
return typeof value === 'string' ? value : undefined;
}
function asNumber(value: unknown): number | undefined {
return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
}
/** Pull token counts out of an Anthropic-shape `usage` object, mutating state. */
function captureUsage(usage: Record<string, unknown> | null, state: StreamJsonState): void {
if (!usage) return;
const input = asNumber(usage.input_tokens);
const output = asNumber(usage.output_tokens);
if (input !== undefined) state.usage.inputTokens = input;
// output_tokens is reported incrementally on message_delta; keep the latest.
if (output !== undefined) state.usage.outputTokens = output;
}
/** Parse the accumulated tool-arg JSON; tolerate an unparseable/partial body. */
function parseToolInput(partialJson: string): unknown {
const trimmed = partialJson.trim();
if (!trimmed) return {};
try {
return JSON.parse(trimmed);
} catch {
return { _raw: partialJson };
}
}
function toolSnapshot(block: OpenToolBlock, rawInput: unknown, status: AcpToolSnapshot['status']): AcpToolSnapshot {
return {
toolCallId: block.toolCallId,
title: block.name,
kind: null,
status,
rawInput,
};
}
/**
* Map one stream-event sub-object (the `event` field of a `stream_event` line) to
* AgentEvents, mutating `state` for open tool blocks + usage.
*/
function handleStreamEvent(event: Record<string, unknown>, state: StreamJsonState): AgentEvent[] {
const eventType = asString(event.type);
if (!eventType) return [];
switch (eventType) {
case 'content_block_start': {
const index = asNumber(event.index);
const block = asRecord(event.content_block);
if (index === undefined || !block) return [];
if (asString(block.type) !== 'tool_use') return [];
const toolCallId = asString(block.id) ?? `tool_${index}`;
const name = asString(block.name) ?? 'tool';
const open: OpenToolBlock = { toolCallId, name, partialJson: '' };
state.toolBlocks.set(index, open);
// Surface the tool start immediately (running, no args yet) so the UI shows
// the call before the args finish streaming.
return [{ type: 'tool_call', toolCall: toolSnapshot(open, {}, 'in_progress') }];
}
case 'content_block_delta': {
const index = asNumber(event.index);
const delta = asRecord(event.delta);
if (delta === null) return [];
const deltaType = asString(delta.type);
if (deltaType === 'text_delta') {
const text = asString(delta.text);
return text ? [{ type: 'text', text }] : [];
}
if (deltaType === 'thinking_delta') {
const text = asString(delta.thinking);
return text ? [{ type: 'reasoning', text }] : [];
}
if (deltaType === 'input_json_delta') {
// Accumulate tool args; no event until the block stops.
const fragment = asString(delta.partial_json);
if (index !== undefined && fragment) {
const open = state.toolBlocks.get(index);
if (open) open.partialJson += fragment;
}
return [];
}
return [];
}
case 'content_block_stop': {
const index = asNumber(event.index);
if (index === undefined) return [];
const open = state.toolBlocks.get(index);
if (!open) return [];
state.toolBlocks.delete(index);
const rawInput = parseToolInput(open.partialJson);
return [{ type: 'tool_update', toolCall: toolSnapshot(open, rawInput, 'completed') }];
}
case 'message_start': {
const message = asRecord(event.message);
captureUsage(asRecord(message?.usage), state);
return [];
}
case 'message_delta': {
captureUsage(asRecord(event.usage), state);
return [];
}
default:
return [];
}
}
/**
* Map the terminal `assistant` message (post-hoc full message) to AgentEvents. Used
* as a fallback for transports that emit only the assembled `assistant` line and no
* incremental `stream_event`s. When stream_events already streamed a block, the
* caller dedups by toolCallId, so re-emitting the assembled tool_use is harmless.
*/
function handleAssistantMessage(message: Record<string, unknown>, state: StreamJsonState): AgentEvent[] {
captureUsage(asRecord(message.usage), state);
const content = message.content;
if (!Array.isArray(content)) return [];
const out: AgentEvent[] = [];
let toolIdx = 0;
for (const rawBlock of content) {
const block = asRecord(rawBlock);
if (!block) continue;
const blockType = asString(block.type);
if (blockType === 'text') {
const text = asString(block.text);
if (text) out.push({ type: 'text', text });
} else if (blockType === 'thinking') {
const text = asString(block.thinking);
if (text) out.push({ type: 'reasoning', text });
} else if (blockType === 'tool_use') {
const toolCallId = asString(block.id) ?? `tool_${toolIdx}`;
const name = asString(block.name) ?? 'tool';
const rawInput = 'input' in block ? block.input : {};
out.push({
type: 'tool_update',
toolCall: { toolCallId, title: name, kind: null, status: 'completed', rawInput },
});
}
toolIdx++;
}
return out;
}
/**
* Pure per-line mapping. `line` is a single complete NDJSON line (no trailing
* newline required; surrounding whitespace tolerated). Returns the AgentEvents the
* line produces and mutates `state` (open tool blocks, usage, session_id). A blank,
* non-JSON, or unrecognized line yields `[]` and never throws.
*/
export function parseStreamJsonLine(line: string, state: StreamJsonState): AgentEvent[] {
const trimmed = line.trim();
if (!trimmed) return [];
let obj: Record<string, unknown> | null;
try {
const parsed: unknown = JSON.parse(trimmed);
obj = asRecord(parsed);
} catch {
return [];
}
if (!obj) return [];
const type = asString(obj.type);
switch (type) {
case 'system': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
return [];
}
case 'stream_event': {
const event = asRecord(obj.event);
return event ? handleStreamEvent(event, state) : [];
}
case 'assistant': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
const message = asRecord(obj.message);
return message ? handleAssistantMessage(message, state) : [];
}
case 'result': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
captureUsage(asRecord(obj.usage), state);
return [];
}
default:
// `user` (tool results) and any unknown line type — ignore.
return [];
}
}
export interface StreamJsonParser {
/** Feed one complete NDJSON line; returns its AgentEvents (never throws). */
push(line: string): AgentEvent[];
/** Final usage (input/output tokens) accumulated so far. */
usage(): StreamJsonUsage;
/** Provider session id from the init `system` line / `result`, if seen. */
sessionId(): string | null;
}
/**
* Stateful wrapper around `parseStreamJsonLine`. Holds per-tool-block accumulation
* + usage/session_id across the turn. Line-buffering (splitting stdout on `\n` and
* holding the partial tail) is the caller's job — see `pty-dispatch.ts`.
*/
export function makeStreamJsonParser(): StreamJsonParser {
const state = makeStreamJsonState();
return {
push: (line: string) => parseStreamJsonLine(line, state),
usage: () => ({ ...state.usage }),
sessionId: () => state.sessionId,
};
}

View File

@@ -1,4 +1,4 @@
import { describe, it, expect } from 'vitest'; import { describe, it, expect, vi, afterEach } from 'vitest';
import { isAgentRegistryMarkdown, parseAgentsMd } from '../agents.js'; import { isAgentRegistryMarkdown, parseAgentsMd } from '../agents.js';
describe('isAgentRegistryMarkdown', () => { describe('isAgentRegistryMarkdown', () => {
@@ -31,3 +31,87 @@ Start here
expect(r.errors.length).toBeGreaterThan(0); expect(r.errors.length).toBeGreaterThan(0);
}); });
}); });
// v2.6 sampling-streamjson-tokens (#11): per-agent llama.cpp sampler extensions.
describe('parseAgentsMd: v2.6 sampling knobs', () => {
afterEach(() => {
vi.restoreAllMocks();
});
const withFrontmatter = (lines: string) => `# Agents
## Sampler
---
temperature: 0.6
${lines}
tools: [view_file]
description: test
---
You sample.
`;
it('parses top_n_sigma and the dry_* family from frontmatter', () => {
const md = withFrontmatter(
[
'top_n_sigma: 1.5',
'dry_multiplier: 0.8',
'dry_base: 1.75',
'dry_allowed_length: 2',
'dry_penalty_last_n: -1',
].join('\n'),
);
const { agents, errors } = parseAgentsMd(md);
expect(errors).toHaveLength(0);
expect(agents).toHaveLength(1);
const a = agents[0]!;
expect(a.top_n_sigma).toBe(1.5);
expect(a.dry_multiplier).toBe(0.8);
expect(a.dry_base).toBe(1.75);
expect(a.dry_allowed_length).toBe(2);
expect(a.dry_penalty_last_n).toBe(-1);
});
it('defaults the new sampler fields to null when omitted', () => {
const { agents } = parseAgentsMd(withFrontmatter('top_p: 0.95'));
const a = agents[0]!;
expect(a.top_n_sigma).toBeNull();
expect(a.dry_multiplier).toBeNull();
expect(a.dry_base).toBeNull();
expect(a.dry_allowed_length).toBeNull();
expect(a.dry_penalty_last_n).toBeNull();
});
it('warns (does not error) on out-of-range top_n_sigma / dry_* values', () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const md = withFrontmatter(
[
'top_n_sigma: -1',
'dry_multiplier: -0.5',
'dry_base: -2',
'dry_allowed_length: -3',
'dry_penalty_last_n: -5',
].join('\n'),
);
const { agents, errors } = parseAgentsMd(md);
expect(errors).toHaveLength(0);
expect(agents).toHaveLength(1);
// Mirrors top_k/min_p: out-of-range still stored, with a warning.
expect(warn).toHaveBeenCalled();
const warnings = warn.mock.calls.map((c) => String(c[0])).join('\n');
expect(warnings).toContain('top_n_sigma');
expect(warnings).toContain('dry_multiplier');
expect(warnings).toContain('dry_base');
expect(warnings).toContain('dry_allowed_length');
expect(warnings).toContain('dry_penalty_last_n');
});
it('errors on non-numeric / non-integer sampler values', () => {
const md = withFrontmatter(
['top_n_sigma: high', 'dry_allowed_length: 2.5'].join('\n'),
);
const { errors } = parseAgentsMd(md);
const joined = errors.map((e) => e.reason).join('\n');
expect(joined).toContain('top_n_sigma must be a number');
expect(joined).toContain('dry_allowed_length must be an integer');
});
});

View File

@@ -88,6 +88,12 @@ interface ParsedFrontmatter {
top_k?: number; top_k?: number;
min_p?: number; min_p?: number;
presence_penalty?: number; presence_penalty?: number;
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions.
top_n_sigma?: number;
dry_multiplier?: number;
dry_base?: number;
dry_allowed_length?: number;
dry_penalty_last_n?: number;
tools?: string[]; tools?: string[];
description?: string; description?: string;
model?: string; model?: string;
@@ -178,6 +184,63 @@ function parseFrontmatter(yaml: string): { data: ParsedFrontmatter; errors: stri
} else { } else {
errors.push(`presence_penalty must be a number (got "${valueRaw}")`); errors.push(`presence_penalty must be a number (got "${valueRaw}")`);
} }
} else if (key === 'top_n_sigma') {
// v2.6 #11: llama.cpp top-n-sigma sampler. Float ≥ 0 (typical 0-3).
// Mirrors top_p/min_p: store then warn on out-of-range (non-numeric
// hard-fails the block).
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.top_n_sigma = n;
if (n < 0) {
console.warn(`agents: top_n_sigma ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`top_n_sigma must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_multiplier') {
// v2.6 #11: DRY repetition-penalty multiplier. Float ≥ 0 (0 disables DRY).
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.dry_multiplier = n;
if (n < 0) {
console.warn(`agents: dry_multiplier ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_multiplier must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_base') {
// v2.6 #11: DRY penalty growth base. Float ≥ 0.
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.dry_base = n;
if (n < 0) {
console.warn(`agents: dry_base ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_base must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_allowed_length') {
// v2.6 #11: DRY max sequence length not penalized. Integer ≥ 0.
const n = Number(valueRaw);
if (Number.isInteger(n)) {
data.dry_allowed_length = n;
if (n < 0) {
console.warn(`agents: dry_allowed_length ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_allowed_length must be an integer (got "${valueRaw}")`);
}
} else if (key === 'dry_penalty_last_n') {
// v2.6 #11: DRY lookback window. Integer ≥ -1 (-1 = whole context, 0 = off).
const n = Number(valueRaw);
if (Number.isInteger(n)) {
data.dry_penalty_last_n = n;
if (n < -1) {
console.warn(`agents: dry_penalty_last_n ${n} out of range (≥-1), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_penalty_last_n must be an integer (got "${valueRaw}")`);
}
} else if (key === 'tools') { } else if (key === 'tools') {
if (valueRaw === '') { if (valueRaw === '') {
data.tools = []; data.tools = [];
@@ -354,6 +417,11 @@ function parseAgentSection(section: RawSection): Omit<Agent, 'source'> {
top_k: typeof fm.top_k === 'number' ? fm.top_k : null, top_k: typeof fm.top_k === 'number' ? fm.top_k : null,
min_p: typeof fm.min_p === 'number' ? fm.min_p : null, min_p: typeof fm.min_p === 'number' ? fm.min_p : null,
presence_penalty: typeof fm.presence_penalty === 'number' ? fm.presence_penalty : null, presence_penalty: typeof fm.presence_penalty === 'number' ? fm.presence_penalty : null,
top_n_sigma: typeof fm.top_n_sigma === 'number' ? fm.top_n_sigma : null,
dry_multiplier: typeof fm.dry_multiplier === 'number' ? fm.dry_multiplier : null,
dry_base: typeof fm.dry_base === 'number' ? fm.dry_base : null,
dry_allowed_length: typeof fm.dry_allowed_length === 'number' ? fm.dry_allowed_length : null,
dry_penalty_last_n: typeof fm.dry_penalty_last_n === 'number' ? fm.dry_penalty_last_n : null,
tools: filteredTools, tools: filteredTools,
model: typeof fm.model === 'string' && fm.model.length > 0 ? fm.model : null, model: typeof fm.model === 'string' && fm.model.length > 0 ? fm.model : null,
max_tool_calls: typeof fm.max_tool_calls === 'number' ? fm.max_tool_calls : null, max_tool_calls: typeof fm.max_tool_calls === 'number' ? fm.max_tool_calls : null,

View File

@@ -86,7 +86,7 @@ export async function runCapHitSummary(
ctx, ctx,
session.model, session.model,
messages, messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined }, { tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => { (delta) => {
accumulated += delta; accumulated += delta;
ctx.publish(sessionId, { ctx.publish(sessionId, {
@@ -346,7 +346,7 @@ export async function runDoomLoopSummary(
ctx, ctx,
session.model, session.model,
messages, messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined }, { tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => { (delta) => {
accumulated += delta; accumulated += delta;
ctx.publish(sessionId, { ctx.publish(sessionId, {
@@ -545,7 +545,7 @@ export async function runStepCapSummary(
ctx, ctx,
session.model, session.model,
messages, messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined }, { tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => { (delta) => {
accumulated += delta; accumulated += delta;
ctx.publish(sessionId, { ctx.publish(sessionId, {

View File

@@ -33,6 +33,39 @@ interface StreamOptions {
top_k?: number | null; top_k?: number | null;
min_p?: number | null; min_p?: number | null;
presence_penalty?: number | null; presence_penalty?: number | null;
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions. These
// are NOT standard AI-SDK streamText options and are NOT serialized by the
// openai-compatible provider's standardized-settings path (topK is even
// explicitly dropped with an "unsupported feature: topK" warning). They reach
// llama-server only via providerOptions.openaiCompatible (see buildSamplerProviderOptions).
top_n_sigma?: number | null;
dry_multiplier?: number | null;
dry_base?: number | null;
dry_allowed_length?: number | null;
dry_penalty_last_n?: number | null;
}
// v2.6 #11: build the providerOptions.openaiCompatible extraBody object for the
// llama.cpp sampler extensions. @ai-sdk/openai-compatible (2.0.47) merges every
// non-reserved key under providerOptions.openaiCompatible straight into the
// chat-completion request body (see its getArgs: the Object.fromEntries spread
// filtered against openaiCompatibleLanguageModelChatOptions.shape). This is the
// ONLY working passthrough for these params:
// - top_k / min_p were latently dropped before this: top_k was passed as the
// AI-SDK `topK` setting which the openai-compatible provider rejects as
// unsupported; min_p was never passed to streamText at all.
// - top_n_sigma + the dry_* family have no AI-SDK equivalent.
// Keys use llama-server's snake_case body names so they land verbatim.
function buildSamplerProviderOptions(opts: StreamOptions): Record<string, number> | undefined {
const body: Record<string, number> = {};
if (typeof opts.top_k === 'number') body.top_k = opts.top_k;
if (typeof opts.min_p === 'number') body.min_p = opts.min_p;
if (typeof opts.top_n_sigma === 'number') body.top_n_sigma = opts.top_n_sigma;
if (typeof opts.dry_multiplier === 'number') body.dry_multiplier = opts.dry_multiplier;
if (typeof opts.dry_base === 'number') body.dry_base = opts.dry_base;
if (typeof opts.dry_allowed_length === 'number') body.dry_allowed_length = opts.dry_allowed_length;
if (typeof opts.dry_penalty_last_n === 'number') body.dry_penalty_last_n = opts.dry_penalty_last_n;
return Object.keys(body).length > 0 ? body : undefined;
} }
// v1.13.1-A: convert BooCode's OpenAI-shaped history into AI SDK // v1.13.1-A: convert BooCode's OpenAI-shaped history into AI SDK
@@ -195,6 +228,14 @@ export async function streamCompletion(
return toolCall; return toolCall;
}; };
// v2.6 #11: llama.cpp sampler extensions (top_k, min_p, top_n_sigma, dry_*)
// ride providerOptions.openaiCompatible — they are NOT standardized streamText
// settings. NB: top_k used to be passed below as the AI-SDK `topK` setting;
// the openai-compatible provider dropped it with an "unsupported feature: topK"
// warning and min_p was never wired at all, so both were dead on the wire
// before this. They now go through the same extraBody path as the new params.
const samplerBody = buildSamplerProviderOptions(opts);
const result = streamText({ const result = streamText({
model: upstreamModel(ctx.config, model, agent ?? null), model: upstreamModel(ctx.config, model, agent ?? null),
messages: aiMessages, messages: aiMessages,
@@ -203,8 +244,8 @@ export async function streamCompletion(
: {}), : {}),
...(typeof opts.temperature === 'number' ? { temperature: opts.temperature } : {}), ...(typeof opts.temperature === 'number' ? { temperature: opts.temperature } : {}),
...(typeof opts.top_p === 'number' ? { topP: opts.top_p } : {}), ...(typeof opts.top_p === 'number' ? { topP: opts.top_p } : {}),
...(typeof opts.top_k === 'number' ? { topK: opts.top_k } : {}),
...(typeof opts.presence_penalty === 'number' ? { presencePenalty: opts.presence_penalty } : {}), ...(typeof opts.presence_penalty === 'number' ? { presencePenalty: opts.presence_penalty } : {}),
...(samplerBody ? { providerOptions: { openaiCompatible: samplerBody } } : {}),
abortSignal: signal, abortSignal: signal,
}); });
@@ -398,6 +439,12 @@ export async function executeStreamPhase(
const effectiveTopK = agent?.top_k ?? undefined; const effectiveTopK = agent?.top_k ?? undefined;
const effectiveMinP = agent?.min_p ?? undefined; const effectiveMinP = agent?.min_p ?? undefined;
const effectivePresencePenalty = agent?.presence_penalty ?? undefined; const effectivePresencePenalty = agent?.presence_penalty ?? undefined;
// v2.6 #11: llama.cpp sampler extensions, threaded the same way as top_k/min_p.
const effectiveTopNSigma = agent?.top_n_sigma ?? undefined;
const effectiveDryMultiplier = agent?.dry_multiplier ?? undefined;
const effectiveDryBase = agent?.dry_base ?? undefined;
const effectiveDryAllowedLength = agent?.dry_allowed_length ?? undefined;
const effectiveDryPenaltyLastN = agent?.dry_penalty_last_n ?? undefined;
// v1.12.2: ctx_max lookup is cached after the first hit per model, so this // v1.12.2: ctx_max lookup is cached after the first hit per model, so this
// is a Map probe in steady state. We capture nCtx once at the top of the // is a Map probe in steady state. We capture nCtx once at the top of the
@@ -435,7 +482,19 @@ export async function executeStreamPhase(
ctx, ctx,
session.model, session.model,
messages, messages,
{ tools: effectiveTools, temperature: effectiveTemperature, top_p: effectiveTopP, top_k: effectiveTopK, min_p: effectiveMinP, presence_penalty: effectivePresencePenalty }, {
tools: effectiveTools,
temperature: effectiveTemperature,
top_p: effectiveTopP,
top_k: effectiveTopK,
min_p: effectiveMinP,
presence_penalty: effectivePresencePenalty,
top_n_sigma: effectiveTopNSigma,
dry_multiplier: effectiveDryMultiplier,
dry_base: effectiveDryBase,
dry_allowed_length: effectiveDryAllowedLength,
dry_penalty_last_n: effectiveDryPenaltyLastN,
},
(delta) => { (delta) => {
state.accumulated += delta; state.accumulated += delta;
ctx.publish(sessionId, { ctx.publish(sessionId, {

View File

@@ -117,6 +117,15 @@ export interface Agent {
top_k: number | null; // null means omit from request body top_k: number | null; // null means omit from request body
min_p: number | null; // null means omit from request body min_p: number | null; // null means omit from request body
presence_penalty: number | null; // null means omit from request body presence_penalty: number | null; // null means omit from request body
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions.
// null = omit from request body. top_n_sigma + the DRY repetition family
// help the doom-loop-prone local model. All travel via the same
// providerOptions.openaiCompatible extraBody channel as top_k/min_p.
top_n_sigma: number | null;
dry_multiplier: number | null;
dry_base: number | null;
dry_allowed_length: number | null;
dry_penalty_last_n: number | null;
tools: string[]; // whitelist of tool names; empty = no tools allowed tools: string[]; // whitelist of tool names; empty = no tools allowed
model: string | null; // null means "session.model wins" model: string | null; // null means "session.model wins"
source: AgentSource; source: AgentSource;

View File

@@ -34,6 +34,30 @@ export interface AgentSessionInfo {
status: string; status: string;
has_session: boolean; has_session: boolean;
last_active_at: string | null; last_active_at: string | null;
// v2.6.8 per-(chat,agent) running token/cost totals (sampling-streamjson-tokens
// #8). input_tokens/output_tokens are BIGINT and may arrive as strings; cost is
// DOUBLE. AgentComposerBar coerces with Number(...) before rendering.
input_tokens: number;
output_tokens: number;
cost: number;
}
// write-edit-robustness #4: a pre-turn worktree snapshot anchored to an
// assistant message. Returned by GET .../checkpoints; drives the per-message
// "Restore to here" affordance in CoderMessageList.
export interface CoderCheckpoint {
id: string;
message_id: string;
created_at: string;
label: string | null;
}
// write-edit-robustness #4: result of POST .../checkpoints/:id/restore.
export interface CoderRestoreResult {
checkpoint_id: string;
messages_deleted: number;
worktree_reset: boolean;
backend_reset: boolean;
} }
export class ApiError extends Error { export class ApiError extends Error {
@@ -407,6 +431,22 @@ export const api = {
...(config?.thinking_option_id ? { thinking_option_id: config.thinking_option_id } : {}), ...(config?.thinking_option_id ? { thinking_option_id: config.thinking_option_id } : {}),
}), }),
}), }),
// write-edit-robustness #4: worktree checkpoints. List which assistant
// messages in a chat have a pre-turn worktree snapshot ("Restore to here"
// is offered only on those). Proxied to boocoder.
getCheckpoints: (sessionId: string, chatId: string) =>
request<{ checkpoints: CoderCheckpoint[] }>(
`/api/coder/sessions/${sessionId}/checkpoints?chat_id=${encodeURIComponent(chatId)}`,
),
// write-edit-robustness #4: reset the worktree to a checkpoint, trim the
// transcript past its anchor message, and reset the agent backend. After it
// returns, the caller refetches messages (+ checkpoints) so the trimmed
// transcript shows.
restoreCheckpoint: (sessionId: string, checkpointId: string) =>
request<CoderRestoreResult>(
`/api/coder/sessions/${sessionId}/checkpoints/${encodeURIComponent(checkpointId)}/restore`,
{ method: 'POST' },
),
// Queue a new-file create from the RightRail browser → BooCoder // Queue a new-file create from the RightRail browser → BooCoder
// pending_changes (operation='create'). Surfaces in the CoderPane DiffPanel // pending_changes (operation='create'). Surfaces in the CoderPane DiffPanel
// for explicit apply. A WriteGuardError comes back as a 422 whose { error } // for explicit apply. A WriteGuardError comes back as a 422 whose { error }

View File

@@ -185,6 +185,14 @@ interface Props {
hasPriorTurn?: boolean; hasPriorTurn?: boolean;
} }
// Condensed token count: 950 → "950", 12_400 → "12.4K", 3_200_000 → "3.2M".
// Sub-1000 stays exact; thousands/millions get one decimal, trailing .0 trimmed.
function abbrevTokens(n: number): string {
if (!Number.isFinite(n) || n < 1000) return String(Math.max(0, Math.round(n)));
if (n < 1_000_000) return `${(n / 1000).toFixed(1).replace(/\.0$/, '')}K`;
return `${(n / 1_000_000).toFixed(1).replace(/\.0$/, '')}M`;
}
// Relative-time formatter for the resumed-chip title (e.g. "3m ago"). // Relative-time formatter for the resumed-chip title (e.g. "3m ago").
function relativeTime(iso: string | null): string { function relativeTime(iso: string | null): string {
if (!iso) return 'unknown'; if (!iso) return 'unknown';
@@ -353,6 +361,21 @@ export function AgentComposerBar({ projectPath, value, onChange, onProviderComma
: { label: 'new session', title: `${value.provider} starts a fresh session this turn` } : { label: 'new session', title: `${value.provider} starts a fresh session this turn` }
: null; : null;
// sampling-streamjson-tokens #8: condensed per-(chat,agent) token/cost readout
// beside the session chip. Coerce — input/output are BIGINT (string over wire).
// Hidden when no session row or all totals are zero (e.g. native boocode, which
// holds no agent_sessions row, or a provider that hasn't run yet).
const usageReadout = (() => {
if (!sessionChip || !sessionRow) return null;
const inTok = Number(sessionRow.input_tokens) || 0;
const outTok = Number(sessionRow.output_tokens) || 0;
const cost = Number(sessionRow.cost) || 0;
if (inTok <= 0 && outTok <= 0 && cost <= 0) return null;
const parts = [`${abbrevTokens(inTok)} in`, `${abbrevTokens(outTok)} out`];
if (cost > 0) parts.push(`$${cost.toFixed(2)}`);
return parts.join(' · ');
})();
return ( return (
<div className="flex flex-wrap items-center gap-1 px-2 py-1 border-b border-border bg-muted/20 shrink-0"> <div className="flex flex-wrap items-center gap-1 px-2 py-1 border-b border-border bg-muted/20 shrink-0">
<CompactPicker <CompactPicker
@@ -374,6 +397,14 @@ export function AgentComposerBar({ projectPath, value, onChange, onProviderComma
{sessionChip.label} {sessionChip.label}
</span> </span>
)} )}
{usageReadout && (
<span
className="text-[10px] text-muted-foreground tabular-nums whitespace-nowrap shrink-0"
title="Tokens in · out · cost for this agent session"
>
{usageReadout}
</span>
)}
<CompactPicker <CompactPicker
label="Mode" label="Mode"
value={value.modeId ?? ''} value={value.modeId ?? ''}

View File

@@ -1,6 +1,6 @@
import { useEffect, useState } from 'react'; import { useEffect, useState } from 'react';
import type { ReactNode } from 'react'; import type { ReactNode } from 'react';
import { ChevronDown, ChevronRight, Copy, RefreshCw, Check, Share2, RotateCw, GitFork, Trash2, Brain } from 'lucide-react'; import { ChevronDown, ChevronRight, Copy, RefreshCw, Check, Share2, RotateCw, GitFork, Trash2, Brain, History } from 'lucide-react';
import { toast } from 'sonner'; import { toast } from 'sonner';
import type { Chat, ErrorReason, Message } from '@/api/types'; import type { Chat, ErrorReason, Message } from '@/api/types';
import { api } from '@/api/client'; import { api } from '@/api/client';
@@ -110,6 +110,10 @@ export interface MessageActions {
onResend?: (chatId: string, content: string) => Promise<void>; onResend?: (chatId: string, content: string) => Promise<void>;
onFork?: (chatId: string, messageId: string) => Promise<void>; onFork?: (chatId: string, messageId: string) => Promise<void>;
onDelete?: (chatId: string, messageId: string) => Promise<void>; onDelete?: (chatId: string, messageId: string) => Promise<void>;
// write-edit-robustness #4 (BooCoder only): reset the worktree to this
// message's pre-turn checkpoint and trim the transcript past it. BooChat
// passes no such callback → the "Restore to here" control never renders.
onRestoreCheckpoint?: (chatId: string, messageId: string) => Promise<void>;
} }
interface Props { interface Props {
@@ -119,6 +123,17 @@ interface Props {
actions?: MessageActions; actions?: MessageActions;
/** Hide actions that don't apply (fork, delete). */ /** Hide actions that don't apply (fork, delete). */
hideActions?: ('fork' | 'delete')[]; hideActions?: ('fork' | 'delete')[];
/**
* write-edit-robustness #4: this assistant message has a worktree checkpoint
* → render "Restore to here" (only when `actions.onRestoreCheckpoint` is also
* provided). CoderMessageList sets this from the checkpoint set.
*/
hasCheckpoint?: boolean;
/**
* write-edit-robustness #4: suppress the restore control during an active
* turn (mirrors composer gating). Defaults to enabled.
*/
restoreDisabled?: boolean;
} }
function StatsLine({ message }: { message: Message }) { function StatsLine({ message }: { message: Message }) {
@@ -155,16 +170,22 @@ function ActionRow({
message, message,
actions, actions,
hiddenSet, hiddenSet,
hasCheckpoint = false,
restoreDisabled = false,
}: { }: {
message: Message; message: Message;
actions?: MessageActions; actions?: MessageActions;
hiddenSet: Set<string>; hiddenSet: Set<string>;
hasCheckpoint?: boolean;
restoreDisabled?: boolean;
}) { }) {
const [justCopied, setJustCopied] = useState(false); const [justCopied, setJustCopied] = useState(false);
const [regenerating, setRegenerating] = useState(false); const [regenerating, setRegenerating] = useState(false);
const [forking, setForking] = useState(false); const [forking, setForking] = useState(false);
const [deleteOpen, setDeleteOpen] = useState(false); const [deleteOpen, setDeleteOpen] = useState(false);
const [deleting, setDeleting] = useState(false); const [deleting, setDeleting] = useState(false);
const [restoreOpen, setRestoreOpen] = useState(false);
const [restoring, setRestoring] = useState(false);
async function copy() { async function copy() {
try { try {
@@ -240,12 +261,33 @@ function ActionRow({
} }
} }
async function confirmRestore() {
if (restoring || !actions?.onRestoreCheckpoint) return;
setRestoring(true);
try {
await actions.onRestoreCheckpoint(message.chat_id, message.id);
setRestoreOpen(false);
} catch (err) {
toast.error(err instanceof Error ? err.message : 'restore failed');
} finally {
setRestoring(false);
}
}
const isAssistant = message.role === 'assistant'; const isAssistant = message.role === 'assistant';
const isUser = message.role === 'user'; const isUser = message.role === 'user';
const canRegen = isAssistant && message.status !== 'streaming'; const canRegen = isAssistant && message.status !== 'streaming';
const canResend = isUser && message.status === 'complete' && !!message.content?.trim(); const canResend = isUser && message.status === 'complete' && !!message.content?.trim();
const canFork = message.status === 'complete'; const canFork = message.status === 'complete';
const canDelete = message.status !== 'streaming'; const canDelete = message.status !== 'streaming';
// write-edit-robustness #4: show "Restore to here" only for a completed
// assistant message that has a checkpoint AND when the coder wired the
// callback. Disabled (but visible) during an active turn.
const canRestore =
isAssistant &&
hasCheckpoint &&
message.status === 'complete' &&
!!actions?.onRestoreCheckpoint;
return ( return (
<> <>
@@ -306,6 +348,18 @@ function ActionRow({
<Trash2 className="size-3" /> <Trash2 className="size-3" />
</button> </button>
)} )}
{canRestore && (
<button
type="button"
onClick={() => setRestoreOpen(true)}
disabled={restoreDisabled || restoring}
className="inline-flex items-center justify-center size-6 rounded text-muted-foreground hover:bg-muted hover:text-foreground disabled:opacity-40 disabled:cursor-not-allowed max-md:min-h-[44px] max-md:min-w-[44px]"
aria-label="Restore to here"
title="Restore worktree to this point"
>
<History className="size-3" />
</button>
)}
</div> </div>
<Dialog <Dialog
open={deleteOpen} open={deleteOpen}
@@ -338,6 +392,39 @@ function ActionRow({
</DialogFooter> </DialogFooter>
</DialogContent> </DialogContent>
</Dialog> </Dialog>
<Dialog
open={restoreOpen}
onOpenChange={(open) => {
if (!restoring) setRestoreOpen(open);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Restore to this point?</DialogTitle>
<DialogDescription>
This resets the worktree to before this turn, removes every later
message in this chat, and resets the agent's session. This cannot
be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => setRestoreOpen(false)}
disabled={restoring}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={() => void confirmRestore()}
disabled={restoring}
>
{restoring ? 'Restoring' : 'Restore'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</> </>
); );
} }
@@ -550,7 +637,15 @@ function ReasoningBlock({ text, streaming }: { text: string; streaming: boolean
); );
} }
export function MessageBubble({ message, sessionChats, capHitInfo, actions, hideActions }: Props) { export function MessageBubble({
message,
sessionChats,
capHitInfo,
actions,
hideActions,
hasCheckpoint,
restoreDisabled,
}: Props) {
const hiddenSet = new Set(hideActions ?? []); const hiddenSet = new Set(hideActions ?? []);
// v1.11: anchored rolling summary row. Checked BEFORE the kind==='compact' // v1.11: anchored rolling summary row. Checked BEFORE the kind==='compact'
// branch because summary=true never coexists with kind='compact' (new // branch because summary=true never coexists with kind='compact' (new
@@ -652,7 +747,15 @@ export function MessageBubble({ message, sessionChats, capHitInfo, actions, hide
</div> </div>
)} )}
{!isStreaming && <StatsLine message={message} />} {!isStreaming && <StatsLine message={message} />}
{!isStreaming && hasContent && <ActionRow message={message} actions={actions} hiddenSet={hiddenSet} />} {!isStreaming && hasContent && (
<ActionRow
message={message}
actions={actions}
hiddenSet={hiddenSet}
hasCheckpoint={hasCheckpoint}
restoreDisabled={restoreDisabled}
/>
)}
</div> </div>
); );
} }

View File

@@ -147,11 +147,24 @@ interface Props {
chatId?: string; chatId?: string;
footer?: ReactNode; footer?: ReactNode;
actions?: MessageActions; actions?: MessageActions;
// write-edit-robustness #4: assistant message ids that have a worktree
// checkpoint. The "Restore to here" control renders only on these.
checkpointMessageIds?: Set<string>;
// write-edit-robustness #4: suppress restore during an active turn (mirrors
// composer gating in CoderPane).
restoreDisabled?: boolean;
} }
const CODER_HIDDEN_ACTIONS: ('fork' | 'delete')[] = ['fork']; const CODER_HIDDEN_ACTIONS: ('fork' | 'delete')[] = ['fork'];
export function CoderMessageList({ messages, chatId, footer, actions }: Props) { export function CoderMessageList({
messages,
chatId,
footer,
actions,
checkpointMessageIds,
restoreDisabled,
}: Props) {
const endRef = useRef<HTMLDivElement>(null); const endRef = useRef<HTMLDivElement>(null);
const scrollRef = useRef<HTMLDivElement>(null); const scrollRef = useRef<HTMLDivElement>(null);
const isNearBottomRef = useRef(true); const isNearBottomRef = useRef(true);
@@ -189,6 +202,8 @@ export function CoderMessageList({ messages, chatId, footer, actions }: Props) {
message={item.message as unknown as Message} message={item.message as unknown as Message}
actions={actions} actions={actions}
hideActions={CODER_HIDDEN_ACTIONS} hideActions={CODER_HIDDEN_ACTIONS}
hasCheckpoint={checkpointMessageIds?.has(item.message.id) ?? false}
restoreDisabled={restoreDisabled}
/> />
); );
} }

View File

@@ -381,6 +381,29 @@ function usePendingChanges(sessionId: string) {
return { changes, loading, refresh, approve, reject }; return { changes, loading, refresh, approve, reject };
} }
// write-edit-robustness #4: which assistant messages in this chat have a
// worktree checkpoint, so CoderMessageList can offer "Restore to here" only on
// those. Refetched on message_complete (same trigger as pending changes) and
// after a successful restore.
function useCheckpoints(sessionId: string, chatId: string | undefined) {
const [messageIds, setMessageIds] = useState<Set<string>>(() => new Set());
const refresh = useCallback(() => {
if (!chatId) {
setMessageIds(new Set());
return Promise.resolve();
}
return api.coder
.getCheckpoints(sessionId, chatId)
.then((res) => setMessageIds(new Set(res.checkpoints.map((c) => c.message_id))))
.catch(() => {/* boocoder may be down / endpoint not ready */});
}, [sessionId, chatId]);
useEffect(() => { void refresh(); }, [refresh]);
return { checkpointMessageIds: messageIds, refreshCheckpoints: refresh };
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Sub-components // Sub-components
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -640,6 +663,7 @@ export function CoderPane({
}, },
}); });
const { changes, loading, refresh, approve, reject } = usePendingChanges(sessionId); const { changes, loading, refresh, approve, reject } = usePendingChanges(sessionId);
const { checkpointMessageIds, refreshCheckpoints } = useCheckpoints(sessionId, chatId);
const [input, setInput] = useState(''); const [input, setInput] = useState('');
const [sending, setSending] = useState(false); const [sending, setSending] = useState(false);
const [queue, setQueue] = useState<string[]>([]); const [queue, setQueue] = useState<string[]>([]);
@@ -652,15 +676,18 @@ export function CoderPane({
// Refresh pending changes (and agent-session state for the §9b chip) when a // Refresh pending changes (and agent-session state for the §9b chip) when a
// message_complete arrives — same trigger usePendingChanges already uses. // message_complete arrives — same trigger usePendingChanges already uses.
// write-edit-robustness #4: also refetch checkpoints so a new turn's snapshot
// surfaces its "Restore to here" control.
useEffect(() => { useEffect(() => {
const lastAssistant = [...messages].reverse().find( const lastAssistant = [...messages].reverse().find(
(m): m is CoderMessage => m.role === 'assistant', (m): m is CoderMessage => m.role === 'assistant',
); );
if (lastAssistant?.status === 'complete') { if (lastAssistant?.status === 'complete') {
refresh(); refresh();
void refreshCheckpoints();
void refreshAgentSessions(sessionId); void refreshAgentSessions(sessionId);
} }
}, [messages, refresh, sessionId]); }, [messages, refresh, refreshCheckpoints, sessionId]);
// The §9b chip only shows once the chat has ≥1 prior turn (a completed // The §9b chip only shows once the chat has ≥1 prior turn (a completed
// assistant message). Hidden on a brand-new chat. // assistant message). Hidden on a brand-new chat.
@@ -867,6 +894,38 @@ export function CoderPane({
} }
}, [activeTaskId]); }, [activeTaskId]);
// write-edit-robustness #4: reset the worktree to a message's checkpoint and
// trim the transcript past it. The confirm lives in MessageBubble's ActionRow
// (plain Cancel/Restore). The restore route is keyed by checkpoint id, so we
// resolve message→checkpoint via a fresh GET (cheap, and avoids a stale id if
// the set changed). On success, refetch messages so the trimmed transcript
// shows, plus checkpoints (later ones were deleted server-side) and pending
// changes (the worktree was reset).
const handleRestoreCheckpoint = useCallback(async (_chatId: string, messageId: string) => {
if (!chatId || generating) return;
let checkpointId: string | undefined;
try {
const res = await api.coder.getCheckpoints(sessionId, chatId);
checkpointId = res.checkpoints.find((c) => c.message_id === messageId)?.id;
} catch (err) {
toast.error(err instanceof Error ? err.message : 'failed to load checkpoint');
return;
}
if (!checkpointId) {
toast.error('No checkpoint found for this message');
return;
}
try {
await api.coder.restoreCheckpoint(sessionId, checkpointId);
await loadMessages();
await refreshCheckpoints();
refresh();
toast.success('Restored to checkpoint');
} catch (err) {
toast.error(err instanceof Error ? err.message : 'restore failed');
}
}, [chatId, generating, sessionId, loadMessages, refreshCheckpoints, refresh]);
const handleChatInputSlash = useCallback(async (skillName: string, userMessage: string) => { const handleChatInputSlash = useCallback(async (skillName: string, userMessage: string) => {
if (!chatId) return; if (!chatId) return;
// Only BooCoder skills route here; an agent's own commands (not skills) fall // Only BooCoder skills route here; an agent's own commands (not skills) fall
@@ -921,8 +980,11 @@ export function CoderPane({
<CoderMessageList <CoderMessageList
messages={messages as CoderTimelineWire[]} messages={messages as CoderTimelineWire[]}
chatId={chatId} chatId={chatId}
checkpointMessageIds={checkpointMessageIds}
restoreDisabled={generating}
actions={{ actions={{
onResend: async (_chatId, content) => { await sendOneMessage(content); }, onResend: async (_chatId, content) => { await sendOneMessage(content); },
onRestoreCheckpoint: handleRestoreCheckpoint,
}} }}
footer={ footer={
activeTaskId && !permissionPrompt && sending === false ? ( activeTaskId && !permissionPrompt && sending === false ? (

View File

@@ -6,6 +6,10 @@ Operating rules for every agent in this registry. Full procedures live in the `c
**Worktrees** — Isolate work in a worktree when it is parallel to in-progress work, risky/experimental, a hotfix interrupting other work, or splits into independent units — just create when clear, propose in one line when ambiguous, skip quick/small single-stream work. Branch from a stable base (default branch); worktrees persist (never auto-remove or auto-merge); they isolate code state, not runtime (ports/DBs/services still collide). Full heuristic: invoke `using-worktrees`. **Worktrees** — Isolate work in a worktree when it is parallel to in-progress work, risky/experimental, a hotfix interrupting other work, or splits into independent units — just create when clear, propose in one line when ambiguous, skip quick/small single-stream work. Branch from a stable base (default branch); worktrees persist (never auto-remove or auto-merge); they isolate code state, not runtime (ports/DBs/services still collide). Full heuristic: invoke `using-worktrees`.
**Sampling knobs** — Each `## Name` frontmatter block accepts these per-agent sampler fields, threaded into the llama-swap chat-completion request: `temperature`, `top_p`, `top_k`, `min_p`, `presence_penalty`, and (v2.6) `top_n_sigma`, `dry_multiplier`, `dry_base`, `dry_allowed_length`, `dry_penalty_last_n`. The `top_n_sigma` + `dry_*` repetition family curb the doom-loop-prone local model. Omit a field to leave it at the server default. Example: `top_n_sigma: 1.0`, `dry_multiplier: 0.8`, `dry_base: 1.75`, `dry_allowed_length: 2`, `dry_penalty_last_n: -1` (-1 = whole context).
**Reasoning budget** — To cap a reasoning model's thinking tokens, pass `--reasoning-budget` through `llama_extra_args` (already permitted by the deny-list validator; routes the agent to llama-sidecar). Example frontmatter line: `llama_extra_args: ["--reasoning-budget", "2048"]`. This is a sidecar process flag, not a chat-completion body param — distinct from the sampling knobs above.
## Code Reviewer ## Code Reviewer
--- ---
temperature: 0.6 temperature: 0.6

View File

@@ -0,0 +1,45 @@
# Small wins — sampling knobs + PTY stream-json + token UI
**Status:** in progress (started 2026-06-01)
**Source:** `boocode_code_review_v2.md` §1 #11 / #7 / #8 (config-adopt + qwen-code §5g + opencode §3 #4).
Three independent BooCode improvements, disjoint subsystems (apps/server / apps/coder / apps/web).
## #11 — New sampling knobs (apps/server)
Per-agent `top_n_sigma` + the `dry_*` repetition family help the doom-loop-prone local model.
Today the Agent type threads `temperature/top_p/top_k/min_p/presence_penalty` into the inference
request (`stream-phase.ts:396438`). Add `top_n_sigma`, `dry_multiplier`, `dry_base`,
`dry_allowed_length`, `dry_penalty_last_n` as first-class Agent fields (`types/api.ts`), parse them in
`agents.ts:parseFrontmatter` (same bounded per-field numeric pattern + out-of-range warn), and thread
them into the request body **via the same mechanism `top_k`/`min_p` already use** (the agent must
confirm whether that's an AI-SDK `providerOptions`/`extraBody` passthrough — these are llama.cpp
extensions, not standard OpenAI fields — and ride it; surface it if `top_k`/`min_p` turn out to be
silently dropped today). `--reasoning-budget` is a llama-server CLI flag already permitted by the
deny-list validator, so it works via `llama_extra_args: ["--reasoning-budget","N"]` now — document it
in `data/AGENTS.md`. apps/server only.
## #7 — Live PTY stream-json NDJSON parsing (apps/coder)
qwen/claude PTY dispatch slices stdout opaque (`dispatcher.ts` PTY path; qwen already runs
`--output-format stream-json`). Add a parser for the Claude-Code-compatible NDJSON
(`system`/`assistant`/`result`/`stream_event``content_block_delta` text/thinking/tool deltas +
`usage` + `session_id`) that maps to the existing `AgentEvent` union (`agent-backend.ts`). **Live
incremental** (decision 2026-06-01): line-buffer the PTY stdout `data` events, parse each complete
NDJSON line as it arrives, and emit broker frames live (text/reasoning/tool) like the ACP/opencode
paths — plus accumulate for `persistExternalAgentTurn`. claude gets `--output-format stream-json` too.
One parser serves both (same schema). apps/coder only (`pty-dispatch.ts`, `dispatcher.ts`, new
`stream-json-parser.ts` + test).
## #8 — Surface opencode token usage (apps/coder route + apps/web)
`agent_sessions.input_tokens/output_tokens/cost` are accumulated (v2.6.8) but the
`GET /api/sessions/:id/agent-sessions` SELECT + the `AgentSessionInfo` type drop them. Add the 3
columns to both, render condensed beside the existing session chip in `AgentComposerBar`
(ChatThroughput styling: `tabular-nums`, muted, e.g. "12.4K in / 3.2K out / $0.25"). MUST NOT touch
Sam's uncommitted WIP (`ChatTabBar`, `SessionLandingPage`, `Workspace`, `useWorkspacePanes`,
`PaneHeaderActions`).
## Decisions (2026-06-01)
- #7 surfacing: **live incremental** streaming (not parse-at-end).
## Verify
- `pnpm -C apps/server test` (+ new agent-parse tests); `pnpm -C apps/coder test` (+ new parser tests)
- `pnpm -C apps/server build && pnpm -C apps/coder build`; `npx tsc -p apps/web/tsconfig.app.json --noEmit`

View File

@@ -0,0 +1,101 @@
# Write/edit robustness — fuzzy patch applier + worktree checkpoints
**Status:** in progress (started 2026-06-01)
**Source:** `boocode_code_review_v2.md` §1 #3 + #4, §5b/§5d5e (cline, Apache-2.0 — algorithm clean-reimplemented, not vendored).
Two independent BooCoder hardening features for local quantized models.
## #3 — Fuzzy patch applier
**Problem:** `applyOne`'s edit case (`apps/coder/src/services/pending_changes.ts:124`) does exact
`content.includes(oldStr)` → throw, then `content.replace(oldStr, newStr)` (first occurrence).
`rewindOne` (line 206) is the same. Local models (qwen3.6) drift `old_string` by whitespace/
indentation/unicode (curly quotes, en/em-dash, nbsp), so a valid edit fails at apply with
"old_string not found" and is lost.
**Design:** new pure module `apps/coder/src/services/fuzzy-match.ts`:
`locateMatch(content: string, needle: string): { kind: 'exact'|'fuzzy'; start: number; end: number }
| { kind: 'ambiguous'; count: number } | { kind: 'not_found' }`. Match ladder:
1. **Exact** `indexOf`. If exactly one → exact span. If >1 → **ambiguous** (refuse; decision
2026-06-01: safer than silently editing the first).
2. **Per-line whitespace-insensitive** — compare `needle` lines to file line-windows ignoring per-line
`trimEnd`/leading-trailing blank lines.
3. **Unicode canonicalization** — normalize curly→straight quotes, en/em-dash→`-`, nbsp→space on both
sides, then retry the whitespace pass.
4. **Levenshtein** similarity ≥ 0.66 over line-windows sized to `needle`'s line count; best window wins.
Non-exact (fuzzy) matches return the actual file span so the caller replaces the real file text with
`new_string`. `pending_changes.ts` `applyOne`/`rewindOne` use `locateMatch`; `ambiguous`/`not_found`
return `success:false` with a clear message (no throw escaping the existing catch). Unit-tested
(`apps/coder/src/services/__tests__/fuzzy-match.test.ts`), per the `turn-guard.ts` pure-helper pattern.
## #4 — Worktree checkpoint + conversation-trim
**Problem:** `rewind` only reverses BooCode's own `pending_changes` (applied to the project root).
External agents (opencode/goose/qwen/claude) write **directly into the session worktree**
(`/tmp/booworktrees/sess-<id>`); rewind has zero coverage there.
**Schema** (`apps/coder/src/schema.sql`):
```sql
CREATE TABLE IF NOT EXISTS checkpoints (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
session_id UUID,
worktree_id UUID REFERENCES worktrees(id) ON DELETE SET NULL,
message_id UUID, -- anchor: the assistant turn row this checkpoint precedes
commit_sha TEXT NOT NULL, -- shadow-commit capturing the pre-turn worktree tree
label TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX IF NOT EXISTS checkpoints_chat_created_idx ON checkpoints(chat_id, created_at);
```
**Create** (`apps/coder/src/services/checkpoints.ts``createCheckpoint`): hooked into the three
external-agent dispatch paths in `dispatcher.ts` (`runWarmAcpTask` ~821, `runOpenCodeServerTask` ~513,
`runExternalAgent` ~255) — after `ensureSessionWorktree()` and the assistant-message insert (so the
anchor `message_id` exists), before the backend runs. Snapshot captures tracked **+ untracked** via a
temp-index shadow commit, stored in a private GC-safe ref:
```
cd <wt> && TMP=$(mktemp) && GIT_INDEX_FILE="$TMP" git read-tree HEAD \
&& GIT_INDEX_FILE="$TMP" git add -A \
&& TREE=$(GIT_INDEX_FILE="$TMP" git write-tree) \
&& SHA=$(git commit-tree "$TREE" -p HEAD -m "boocode checkpoint") \
&& git update-ref refs/boocode/checkpoints/<id> "$SHA" && rm -f "$TMP" && echo "$SHA"
```
Best-effort: a checkpoint failure logs and never breaks the turn. Native-boocode turns (project-root,
rewind-covered) get no checkpoint.
**Restore** (`POST /api/sessions/:sessionId/checkpoints/:checkpointId/restore`, proxied `/api/coder/*`):
1. Resolve + validate the checkpoint belongs to the session.
2. Reset worktree: `git -C <wt> reset --hard <commit_sha> && git -C <wt> clean -fd` (hostExec+shellEscape).
3. Trim transcript: `DELETE FROM messages WHERE chat_id = <cp.chat_id> AND created_at >=
(SELECT created_at FROM messages WHERE id = <cp.message_id>)` (+ explicit `message_parts` delete if
the FK isn't ON DELETE CASCADE — verify).
4. Reset backend (decision 2026-06-01): `UPDATE agent_sessions SET status='crashed' WHERE
chat_id=<cp.chat_id>` and evict the live pool session for `(chat,agent)` if present, so the next turn
re-establishes a fresh backend — transcript, files, and agent context all consistent at the restore
point. (Warm backends hold context server-side; no partial rewind exists.)
5. Delete now-orphaned later checkpoints: `DELETE FROM checkpoints WHERE chat_id=? AND created_at >
<cp.created_at>`.
6. Return `{ checkpoint_id, messages_deleted, worktree_reset, backend_reset }`.
**Frontend:** per-message "Restore to here" in `CoderMessageList.tsx` (via a new optional
`onRestoreCheckpoint?(chatId, messageId)` on `MessageActions` in `MessageBubble.tsx`), wired in
`CoderPane.tsx`; guarded to `status==='complete'` and to messages that have a checkpoint. After the call
returns, refetch the chat's messages (existing GET) — no new WS frame required.
## Decisions (2026-06-01)
- Multi-exact-match → **refuse as ambiguous** (#3).
- #4 **full** scope incl. conversation-trim.
- Restore **resets** the external-agent backend session (context re-established fresh).
## Parallelization
- **Unit 1 (#3)** — fully independent (`fuzzy-match.ts` + `pending_changes.ts` + test).
- **Unit 2 (#4 backend)** — schema + `checkpoints.ts` (create+restore) + 3 dispatcher hooks + restore route + backend reset. One agent owns all #4 coder backend (shared `checkpoints.ts`).
- **Unit 3 (#4 frontend)** — `CoderMessageList`/`MessageBubble`/`CoderPane`, against the pinned restore contract. Parallel with Unit 2. MUST NOT touch Sam's uncommitted WIP (`ChatTabBar`, `SessionLandingPage`, `Workspace`, `useWorkspacePanes`, `PaneHeaderActions`).
## Verify
- `pnpm -C apps/coder test` (incl. new `fuzzy-match` + any checkpoint pure-helper tests)
- `pnpm -C apps/server build` then `pnpm -C apps/coder build`
- `npx tsc -p apps/web/tsconfig.app.json --noEmit`
- Live smoke (manual, host): external-agent edit → checkpoint row; "Restore to here" → worktree reset + transcript trimmed + next turn fresh.