Compare commits

...

6 Commits

Author SHA1 Message Date
a584dd16b0 feat: sampling knobs + live PTY stream-json + token UI (v2.7.3)
Three small wins from boocode_code_review_v2 §1 #11/#7/#8.

#11 sampling knobs: top_n_sigma + dry_* family as first-class Agent fields,
threaded into the request body via providerOptions.openaiCompatible. Fixes a
latent bug — top_k (rejected by the AI-SDK provider) and min_p (never passed to
streamText) were dead on the wire; both now route through the same channel.
--reasoning-budget documented in data/AGENTS.md.

#7 live PTY stream-json: new stream-json-parser.ts line-buffers qwen/claude
NDJSON and emits text/reasoning/tool frames live + persists, with a fallback to
the old opaque slice. claude gets --output-format stream-json --verbose.

#8 token UI: agent_sessions input/output_tokens/cost now flow through the route
+ type and render beside the AgentComposerBar session chip.

Built by 3 parallel agents. Server 523 + coder 245 tests passing; builds + web
tsc clean. Builds on v2.7.2. openspec sampling-streamjson-tokens.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:47:17 +00:00
5651f56039 Merge checkpoint-idor-fix: v2.7.2 close 2 checkpoint IDOR holes 2026-06-01 12:16:08 +00:00
9c7d80e2d8 fix(security): scope checkpoint routes to session — close 2 IDORs (v2.7.2)
Flagged by the automated push security review on v2.7.1.

- GET /checkpoints?chat_id= : the chat_id branch filtered by chat_id alone
  (any session's chat_id read its checkpoints). Now joins chats and gates on
  chats.session_id.
- restoreCheckpoint scope guard was fail-open: `cp.session_id && cp.session_id
  !== sessionId` fell through on a null denormalized session_id, allowing a
  cross-session restore (worktree reset + transcript trim). Now resolves the
  owning session via the checkpoint's chat and denies on missing/mismatch.
- Adds a DB-integration regression for the null-session_id cross-session case.

Both scope authoritatively through chats.session_id (checkpoints.session_id is
a nullable hint). Coder suite 234 passing; 7/7 checkpoint tests (incl. the
regression) against live postgres+git; typecheck clean. Hotfix on v2.7.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:15:54 +00:00
a41a02a62b Merge fuzzy-checkpoints: v2.7.1 write/edit robustness (fuzzy applier + worktree checkpoints) 2026-06-01 12:02:06 +00:00
59f07e8cb8 feat: write/edit robustness — fuzzy patch applier + worktree checkpoints (v2.7.1)
#3 Fuzzy patch applier: new pure fuzzy-match.ts (locateMatch, exact→trim→
unicode-canon→Levenshtein≥0.66, refuse-on-ambiguous) wired into pending_changes
applyOne/rewindOne so local-model whitespace/unicode drift in old_string no
longer loses the edit.

#4 Worktree checkpoint + conversation-trim: checkpoints table + checkpoints.ts
(shadow-commit of tracked+untracked into refs/boocode/checkpoints, hooked into
the 3 external-agent dispatcher paths) + POST restore route (reset --hard +
clean -fd -> transcript trim -> backend-session reset) + "Restore to here" UI.

Built by 3 parallel agents; DB-integration testing caught a created_at
self-deletion bug. Coder suite 234 passing; server+coder build + web tsc clean.
Builds on v2.7.0-mit. openspec write-edit-robustness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 12:01:57 +00:00
1108d07fb2 Merge relicense-agpl-to-mit: v2.7.0 AGPL-3.0 → MIT relicense 2026-06-01 08:16:25 +00:00
27 changed files with 2423 additions and 33 deletions

View File

@@ -2,6 +2,18 @@
All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch.
## v2.7.3-sampling-streamjson-tokens — 2026-06-01
Three small BooCode wins from `boocode_code_review_v2.md` §1 #11/#7/#8. **Sampling knobs:** per-agent `top_n_sigma` + the `dry_*` repetition family (`dry_multiplier`/`dry_base`/`dry_allowed_length`/`dry_penalty_last_n`) are now first-class Agent frontmatter fields, parsed in `agents.ts` and threaded into the llama-swap chat-completion body via `providerOptions.openaiCompatible` (the `@ai-sdk/openai-compatible` extra-body channel). This surfaced and fixed a **latent bug**: `top_k` (rejected by the AI-SDK provider as unsupported) and `min_p` (never passed to `streamText` at all) had been dead on the wire — no agent's `top_k`/`min_p` ever affected sampling; both now route through the same channel, so agents that set them will start using them. `--reasoning-budget` is documented in `data/AGENTS.md` (already works via `llama_extra_args`, permitted by the deny-list validator). **Live PTY stream-json:** qwen/claude PTY dispatch sliced stdout opaque; a new `stream-json-parser.ts` line-buffers the Claude-Code-compatible NDJSON and emits text/reasoning/tool frames live as they arrive (mirroring the ACP/opencode paths) + persists the structured parts, with a clean fallback to the old opaque slice when output isn't NDJSON (claude now runs `--output-format stream-json --verbose`). **Token UI:** the per-`(chat,agent)` `agent_sessions.input_tokens`/`output_tokens`/`cost` columns (accumulated since `v2.6.8` but dropped by the read route + wire type) now flow through and render condensed beside the AgentComposerBar session chip. Built by three parallel agents over disjoint subsystems; server 523 + coder 245 tests passing (incl. 11 new stream-json-parser + new agent-parse tests), all builds + web tsc clean. Builds on `v2.7.2-checkpoint-idor`; openspec `sampling-streamjson-tokens`. The qwen-vs-claude `usage` field names in #7 are best-guess pending a live smoke.
## v2.7.2-checkpoint-idor — 2026-06-01
Closes two IDOR authorization holes in the `v2.7.1-write-edit-robustness` checkpoint routes, flagged by the automated push security review. The `GET /api/sessions/:id/checkpoints?chat_id=` list route scoped its `chat_id` branch by `chat_id` alone — any session's `chat_id` would read its checkpoints; it now joins through `chats` and gates on `chats.session_id` (authoritative; `checkpoints.session_id` is a nullable denormalized hint). The `restoreCheckpoint` scope guard was fail-open — `cp.session_id && cp.session_id !== sessionId` fell through whenever the checkpoint's denormalized `session_id` was null, allowing a cross-session restore (worktree reset + transcript trim) — it now resolves the owning session via the checkpoint's chat and denies on any missing-or-mismatched row. A DB-integration regression covers the exact null-`session_id` cross-session case. Real-world blast radius is small (BooCoder is single-user behind Authelia on loopback), but both are genuine authorization bugs. Coder suite 234 passing (7/7 checkpoint tests incl. the regression against live postgres+git), typecheck clean. Hotfix on `v2.7.1-write-edit-robustness`.
## v2.7.1-write-edit-robustness — 2026-06-01
Two BooCoder hardening features for local quantized models, algorithm-reimplemented (not vendored) from the cline findings in `boocode_code_review_v2.md` §1 #3/#4. **Fuzzy patch applier:** `edit_file`'s apply path was exact-`.includes`-or-throw + first-occurrence `.replace` (`pending_changes.ts`), so a qwen3.6 whitespace/indentation/unicode drift in `old_string` lost the edit; a new pure `fuzzy-match.ts` (`locateMatch`) now runs an exact → per-line-trim → unicode-canon (curly quotes/dashes/nbsp) → Levenshtein-≥0.66 ladder and returns the real file span, refusing multi-exact matches as ambiguous rather than silently editing the first. `applyOne`/`rewindOne` both use it. **Worktree checkpoints + conversation-trim:** `rewind` only reversed BooCode's own `pending_changes`, blind to what external agents (opencode/goose/qwen/claude) write directly into the session worktree — so a new `checkpoints` table + `checkpoints.ts` shadow-commit (tracked **and** untracked, captured via a temp-index `read-tree`/`add`/`write-tree`/`commit-tree` into a GC-safe `refs/boocode/checkpoints/<id>`) snapshots the worktree before each external-agent turn (hooked into all three dispatcher paths), anchored to the turn's assistant message. A new `POST /api/sessions/:id/checkpoints/:cid/restore` resets the worktree (`reset --hard` + `clean -fd`), trims the transcript past that message, and resets the `(chat,agent)` backend session so files, transcript, and agent context land consistent at the restore point; a per-message "Restore to here" affordance in `CoderMessageList` drives it. Built by three parallel agents over disjoint files; DB-integration testing caught a microsecond-`created_at` self-deletion bug in the later-checkpoint cleanup. Full coder suite 234 passing (incl. 17 fuzzy-match + 6 checkpoint tests), server+coder build + web tsc clean. Builds on `v2.7.0-mit`; openspec `write-edit-robustness`. Live host smoke (dispatcher hook + restore UI end-to-end) still to run.
## v2.7.0-mit — 2026-06-01
Relicenses BooCode from AGPL-3.0 back to MIT by clearing the three Unsloth-Studio-derived files the `v2.4.0`/`v2.4.1` lifts pulled in — the root `LICENSE` and all five `package.json` had been `AGPL-3.0-only`, making the network-served work AGPL §13-encumbered. The enabling finding decoupled the relicense from the long-planned native-llama-server-parsing retirement: `tool-call-parser.ts`'s Unsloth-ported algorithm (`parseToolCallsFromText`/`scanBalancedBraces` + unused nudge constants) was **dead code** with no production import, so it was simply deleted while the load-bearing `extractToolCallBlocks`/`stripToolMarkup` (BooCode-authored streaming helpers) were kept byte-identical — no behavior change to the live tool-call path. `html-to-md.ts` was swapped to the MIT `node-html-markdown` library (`parse5` dropped; the only behavior delta is column-aligned tables, GFM hard-break `<br>`, and `<ol start>` renumbering, all feeding the LLM via `web_fetch`), and `llama-args-validator.ts` was clean-room rewritten with the managed-flag denylist re-derived from the public llama-server flag list (facts, not copyrightable). The license flip set `LICENSE` to MIT (`Copyright (c) 2026 indifferentketchup`), the five `package.json` to `MIT`, removed every AGPL SPDX header, added a README License section, and added a `license-mit` guard test that fails if AGPL provenance returns. Built by three parallel agents over the disjoint files; full server suite 519 passing (incl. 9 new guard tests), server build + coder typecheck clean. Resolves `boocode_code_review_v2.md` §1 #1 / §5k and the roadmap's `License-debt` batch (openspec `license-debt-mit`); supersedes that batch's original staged plan, which had entangled the flip with a live qwen3.6 validation window.

View File

@@ -25,6 +25,7 @@ import { setInferenceContext, clearInferenceContext } from './services/tools/inf
import { registerMessageRoutes } from './routes/messages.js';
import { registerSkillRoutes } from './routes/skills.js';
import { registerPendingRoutes } from './routes/pending.js';
import { registerCheckpointRoutes } from './routes/checkpoints.js';
import { registerAgentSessionRoutes } from './routes/agent-sessions.js';
import { registerTaskRoutes } from './routes/tasks.js';
import { registerInboxRoutes } from './routes/inbox.js';
@@ -214,6 +215,7 @@ async function main() {
registerMessageRoutes(app, sql, broker, inferenceApi);
registerSkillRoutes(app, sql, broker, inferenceApi);
registerPendingRoutes(app, sql);
registerCheckpointRoutes(app, sql);
registerAgentSessionRoutes(app, sql);
registerTaskRoutes(app, sql, inferenceApi);
registerInboxRoutes(app, sql);

View File

@@ -16,6 +16,11 @@ export interface AgentSessionRow {
status: string;
has_session: boolean;
last_active_at: string | null;
// v2.6.8 per-(chat,agent) running token/cost totals (sampling-streamjson-tokens
// #8). BIGINT columns arrive as strings over the wire; the frontend coerces.
input_tokens: number;
output_tokens: number;
cost: number;
}
export function registerAgentSessionRoutes(app: FastifyInstance, sql: Sql): void {
@@ -39,7 +44,10 @@ export function registerAgentSessionRoutes(app: FastifyInstance, sql: Sql): void
a.agent AS agent,
a.status AS status,
(a.agent_session_id IS NOT NULL) AS has_session,
a.last_active_at AS last_active_at
a.last_active_at AS last_active_at,
a.input_tokens AS input_tokens,
a.output_tokens AS output_tokens,
a.cost AS cost
FROM agent_sessions a
JOIN chats c ON c.id = a.chat_id
WHERE c.session_id = ${sessionId}

View File

@@ -0,0 +1,73 @@
/**
* write-edit-robustness #4 — checkpoint restore + list routes (coder side).
*
* Proxied through the apps/server `/api/coder/*` blanket forwarder (no server-side
* change needed for new routes). Restore rewinds the session worktree to the
* checkpoint's shadow commit, trims the transcript from the anchor message forward,
* and resets the agent backend — see services/checkpoints.ts.
*/
import type { FastifyInstance } from 'fastify';
import type { Sql } from '../db.js';
import { restoreCheckpoint, CheckpointNotFoundError } from '../services/checkpoints.js';
export function registerCheckpointRoutes(app: FastifyInstance, sql: Sql): void {
// GET /api/sessions/:sessionId/checkpoints?chat_id= — list a chat's checkpoints
// so the frontend can mark which messages have a restore point. When chat_id is
// omitted, returns every checkpoint for the session's chats.
app.get<{ Params: { sessionId: string }; Querystring: { chat_id?: string } }>(
'/api/sessions/:sessionId/checkpoints',
async (req, reply) => {
const sessionId = req.params.sessionId;
const chatId = req.query.chat_id;
const session = await sql<{ id: string }[]>`SELECT id FROM sessions WHERE id = ${sessionId}`;
if (session.length === 0) {
reply.code(404);
return { error: 'session not found' };
}
// Scope authoritatively through chats.session_id (always set) — NOT the
// denormalized checkpoints.session_id (nullable). The chat_id branch must
// still be session-gated or it's an IDOR (any session's chat_id reads its
// checkpoints).
const rows = chatId
? await sql<{ id: string; chat_id: string; message_id: string | null; label: string | null; created_at: Date }[]>`
SELECT cp.id, cp.chat_id, cp.message_id, cp.label, cp.created_at
FROM checkpoints cp
JOIN chats c ON c.id = cp.chat_id
WHERE cp.chat_id = ${chatId} AND c.session_id = ${sessionId}
ORDER BY cp.created_at
`
: await sql<{ id: string; chat_id: string; message_id: string | null; label: string | null; created_at: Date }[]>`
SELECT cp.id, cp.chat_id, cp.message_id, cp.label, cp.created_at
FROM checkpoints cp
JOIN chats c ON c.id = cp.chat_id
WHERE c.session_id = ${sessionId}
ORDER BY cp.created_at
`;
return rows;
},
);
// POST /api/sessions/:sessionId/checkpoints/:checkpointId/restore — restore.
app.post<{ Params: { sessionId: string; checkpointId: string } }>(
'/api/sessions/:sessionId/checkpoints/:checkpointId/restore',
async (req, reply) => {
const { sessionId, checkpointId } = req.params;
try {
const result = await restoreCheckpoint(sql, checkpointId, {
sessionId,
log: app.log,
});
return result;
} catch (err) {
if (err instanceof CheckpointNotFoundError) {
reply.code(404);
return { error: err.message };
}
throw err;
}
},
);
}

View File

@@ -240,6 +240,27 @@ END $$;
-- v2.6: attribution for DiffPanel badges (Phase 1 UX reads this).
ALTER TABLE pending_changes ADD COLUMN IF NOT EXISTS agent TEXT;
-- write-edit-robustness #4: worktree checkpoints. A pre-turn shadow-commit of the
-- session worktree (tracked + untracked, captured without disturbing the real
-- index/working tree) stored in a private GC-safe ref refs/boocode/checkpoints/<id>.
-- Created best-effort before each external-agent turn (opencode / warm-ACP / one-shot
-- ACP+PTY); restore resets the worktree to commit_sha, trims the transcript from
-- message_id forward, and resets the backend session. chat_id CASCADEs from chats
-- (like agent_sessions); worktree_id SET NULL so a checkpoint outlives a reaped
-- worktree row. session_id / message_id are informational (no FK — message rows are
-- trimmed by a checkpoint restore and we must not block that on a dangling ref).
CREATE TABLE IF NOT EXISTS checkpoints (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
session_id UUID,
worktree_id UUID REFERENCES worktrees(id) ON DELETE SET NULL,
message_id UUID, -- anchor: the assistant turn row this checkpoint precedes
commit_sha TEXT NOT NULL, -- shadow-commit capturing the pre-turn worktree tree
label TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX IF NOT EXISTS checkpoints_chat_created_idx ON checkpoints(chat_id, created_at);
-- LISTEN/NOTIFY fast path: every tasks INSERT (from any call site — routes,
-- new_task tool, arena, MCP server) fires pg_notify('tasks_new') in the same
-- transaction, so the dispatcher reacts immediately instead of waiting for the

View File

@@ -0,0 +1,252 @@
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { readFileSync } from 'node:fs';
import { rm, mkdir } from 'node:fs/promises';
import { resolve } from 'node:path';
import postgres from 'postgres';
import {
buildShadowCommitCommand,
createCheckpoint,
restoreCheckpoint,
CheckpointNotFoundError,
} from '../checkpoints.js';
import { ensureSessionWorktree } from '../worktrees.js';
import { hostExec } from '../host-exec.js';
/**
* write-edit-robustness #4 — worktree checkpoint tests.
*
* Pure-helper coverage (no DB / no host) for the shadow-commit command builder,
* plus a DB+git integration block (DB-opt-in via DATABASE_URL, skips cleanly
* otherwise; mirrors reconnect_integration.test.ts) that exercises the real
* create → restore round trip against a worktree on the host fs.
*/
describe('buildShadowCommitCommand (pure)', () => {
it('parks the commit under refs/boocode/checkpoints/<id> and prints only the SHA', () => {
const cmd = buildShadowCommitCommand('/tmp/booworktrees/sess-abc', 'cp-id-123');
// Uses a temp index so the real working tree/index is untouched.
expect(cmd).toContain('TMP=$(mktemp)');
expect(cmd).toContain('GIT_INDEX_FILE="$TMP" git read-tree HEAD');
expect(cmd).toContain('GIT_INDEX_FILE="$TMP" git add -A');
expect(cmd).toContain('git write-tree');
expect(cmd).toContain("git commit-tree \"$TREE\" -p HEAD -m \"boocode checkpoint\"");
// Ref name matches the row id, and stdout is ONLY the SHA (printf, no newline).
expect(cmd).toContain("update-ref 'refs/boocode/checkpoints/cp-id-123'");
expect(cmd).toContain("printf '%s' \"$SHA\"");
expect(cmd).not.toContain('echo "$SHA"');
});
it('shell-escapes the worktree path and the id', () => {
const cmd = buildShadowCommitCommand("/tmp/it's a path", "id'; rm -rf /");
// Single quotes inside the path/id are escaped via the '\'' wrapping idiom — no
// bare interpolation that could break out of the quoting.
expect(cmd).toContain("cd '/tmp/it'\\''s a path'");
expect(cmd).toContain("refs/boocode/checkpoints/id'\\''; rm -rf /");
});
});
describe.runIf(!!process.env.DATABASE_URL)('checkpoint create + restore (DB + git)', () => {
let sql: ReturnType<typeof postgres>;
const stamp = Date.now();
const projectDir = `/tmp/boocode-checkpoint-proj-${stamp}`;
let projectId: string;
let sessionId: string;
let chatId: string;
let worktreePath: string;
beforeAll(async () => {
sql = postgres(process.env.DATABASE_URL!, { max: 3 });
// Server schema first (FK targets), then coder schema (worktrees + checkpoints).
const serverSchema = resolve(__dirname, '../../../../server/src/schema.sql');
const coderSchema = resolve(__dirname, '../../schema.sql');
await sql.unsafe(readFileSync(serverSchema, 'utf8'));
await sql.unsafe(readFileSync(coderSchema, 'utf8'));
await mkdir(projectDir, { recursive: true });
await hostExec(
`cd ${projectDir} && git init -q && git config user.email t@t && git config user.name t ` +
`&& echo hello > README.md && git add -A && git commit -qm init`,
{ timeoutMs: 20_000 },
);
const [project] = await sql<{ id: string }[]>`
INSERT INTO projects (name, path, status) VALUES ('checkpoint-test', ${projectDir}, 'open') RETURNING id
`;
projectId = project!.id;
const [session] = await sql<{ id: string }[]>`
INSERT INTO sessions (project_id, name, model, status)
VALUES (${projectId}, 'cp', 'm', 'open') RETURNING id
`;
sessionId = session!.id;
const [chat] = await sql<{ id: string }[]>`
INSERT INTO chats (session_id, name, status) VALUES (${sessionId}, 'tab', 'open') RETURNING id
`;
chatId = chat!.id;
const wt = await ensureSessionWorktree(sql, projectDir, sessionId);
worktreePath = wt.worktreePath;
});
afterAll(async () => {
if (sql) {
const rows = await sql<{ path: string }[]>`SELECT path FROM worktrees WHERE session_id = ${sessionId}`.catch(() => []);
for (const r of rows) {
await hostExec(`git -C ${projectDir} worktree remove ${r.path} --force`, { timeoutMs: 10_000 }).catch(() => {});
}
await sql`DELETE FROM checkpoints WHERE chat_id = ${chatId}`.catch(() => {});
await sql`DELETE FROM agent_sessions WHERE chat_id = ${chatId}`.catch(() => {});
await sql`DELETE FROM worktrees WHERE session_id = ${sessionId}`.catch(() => {});
await sql`DELETE FROM chats WHERE id = ${chatId}`.catch(() => {});
await sql`DELETE FROM sessions WHERE id = ${sessionId}`.catch(() => {});
await sql`DELETE FROM projects WHERE id = ${projectId}`.catch(() => {});
await sql.end({ timeout: 5 });
}
await rm(projectDir, { recursive: true, force: true });
});
it('createCheckpoint inserts a row + a private ref capturing tracked + untracked', async () => {
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const worktreeId = wt!.id;
// Pre-turn untracked + tracked-edit state the agent will start from.
await hostExec(`cd ${worktreePath} && echo edited >> README.md && echo new > extra.txt`, { timeoutMs: 10_000 });
const [assistantMsg] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', '', 'streaming') RETURNING id
`;
const messageId = assistantMsg!.id;
const cp = await createCheckpoint(sql, {
chatId,
sessionId,
worktreeId,
worktreePath,
messageId,
});
expect(cp).not.toBeNull();
expect(cp!.commit_sha).toMatch(/^[0-9a-f]{40}$/);
const [row] = await sql<{ commit_sha: string; worktree_id: string; message_id: string }[]>`
SELECT commit_sha, worktree_id, message_id FROM checkpoints WHERE id = ${cp!.id}
`;
expect(row!.commit_sha).toBe(cp!.commit_sha);
expect(row!.worktree_id).toBe(worktreeId);
expect(row!.message_id).toBe(messageId);
// The ref exists and the captured tree carries the untracked file (proves the
// temp-index `git add -A` snapshotted untracked content).
const refLs = await hostExec(
`git -C ${worktreePath} ls-tree -r --name-only ${cp!.commit_sha}`,
{ timeoutMs: 10_000 },
);
expect(refLs.exitCode).toBe(0);
expect(refLs.stdout).toContain('extra.txt');
// The shadow commit did NOT disturb the real working tree: extra.txt is still
// present + still untracked (status shows it).
const status = await hostExec(`git -C ${worktreePath} status --porcelain`, { timeoutMs: 10_000 });
expect(status.stdout).toContain('extra.txt');
});
it('restoreCheckpoint resets the worktree, trims the transcript, and drops later checkpoints', async () => {
// Clean slate for this test: reset the worktree to HEAD, clear prior rows.
await hostExec(`git -C ${worktreePath} reset --hard HEAD && git -C ${worktreePath} clean -fd`, { timeoutMs: 10_000 });
await sql`DELETE FROM checkpoints WHERE chat_id = ${chatId}`;
await sql`DELETE FROM messages WHERE chat_id = ${chatId}`;
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const worktreeId = wt!.id;
// Turn 1: a user msg, then the assistant turn the checkpoint anchors. The
// worktree is pristine (matches HEAD) when this checkpoint is captured.
await sql`INSERT INTO messages (session_id, chat_id, role, content, status) VALUES (${sessionId}, ${chatId}, 'user', 'do it', 'complete')`;
const [a1] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', 'turn 1', 'complete') RETURNING id
`;
const cp1 = await createCheckpoint(sql, { chatId, sessionId, worktreeId, worktreePath, messageId: a1!.id });
expect(cp1).not.toBeNull();
// The agent (turn 1) writes a file into the worktree.
await hostExec(`cd ${worktreePath} && echo agent-wrote > agent.txt`, { timeoutMs: 10_000 });
// Turn 2: another user msg + assistant turn, AND a second (later) checkpoint.
await sql`INSERT INTO messages (session_id, chat_id, role, content, status) VALUES (${sessionId}, ${chatId}, 'user', 'more', 'complete')`;
const [a2] = await sql<{ id: string }[]>`
INSERT INTO messages (session_id, chat_id, role, content, status)
VALUES (${sessionId}, ${chatId}, 'assistant', 'turn 2', 'complete') RETURNING id
`;
const cp2 = await createCheckpoint(sql, { chatId, sessionId, worktreeId, worktreePath, messageId: a2!.id });
expect(cp2).not.toBeNull();
// An agent_sessions row that restore should mark 'crashed'.
await sql`
INSERT INTO agent_sessions (chat_id, session_id, worktree_id, agent, backend, agent_session_id, status, last_active_at)
VALUES (${chatId}, ${sessionId}, ${worktreeId}, 'goose', 'acp_warm', 'sess-1', 'active', clock_timestamp())
ON CONFLICT (chat_id, agent) DO UPDATE SET status = 'active'
`;
const before = await sql<{ id: string }[]>`SELECT id FROM messages WHERE chat_id = ${chatId} ORDER BY created_at`;
expect(before.length).toBe(4); // user, a1, user, a2
// Restore to cp1 (before turn 1's assistant message).
const result = await restoreCheckpoint(sql, cp1!.id, { sessionId });
expect(result.checkpoint_id).toBe(cp1!.id);
expect(result.worktree_reset).toBe(true);
expect(result.backend_reset).toBe(true);
// a1, user(turn2), a2 deleted (created_at >= a1) → 3 trimmed.
expect(result.messages_deleted).toBe(3);
// Transcript trimmed to just the first user message.
const after = await sql<{ role: string; content: string }[]>`SELECT role, content FROM messages WHERE chat_id = ${chatId} ORDER BY created_at`;
expect(after.length).toBe(1);
expect(after[0]!.role).toBe('user');
// Worktree reset: the agent's file is gone (it was written after cp1).
const ls = await hostExec(`ls ${worktreePath}/agent.txt`, { timeoutMs: 10_000 });
expect(ls.exitCode).not.toBe(0);
// The agent_sessions row was reset to 'crashed'.
const [as] = await sql<{ status: string }[]>`SELECT status FROM agent_sessions WHERE chat_id = ${chatId} AND agent = 'goose'`;
expect(as!.status).toBe('crashed');
// cp1 survives (re-restorable); cp2 (later) was dropped.
const cps = await sql<{ id: string }[]>`SELECT id FROM checkpoints WHERE chat_id = ${chatId}`;
expect(cps.map((c) => c.id)).toEqual([cp1!.id]);
});
it('restoreCheckpoint throws CheckpointNotFoundError for an unknown id', async () => {
await expect(
restoreCheckpoint(sql, '00000000-0000-0000-0000-000000000000', { sessionId }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
});
it('restoreCheckpoint throws when the checkpoint is not in the requested session', async () => {
// A checkpoint whose session_id differs from the route's sessionId.
const [wt] = await sql<{ id: string }[]>`SELECT id FROM worktrees WHERE session_id = ${sessionId} AND status = 'active'`;
const cp = await createCheckpoint(sql, { chatId, sessionId, worktreeId: wt!.id, worktreePath, messageId: null });
expect(cp).not.toBeNull();
await expect(
restoreCheckpoint(sql, cp!.id, { sessionId: '11111111-1111-1111-1111-111111111111' }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
await sql`DELETE FROM checkpoints WHERE id = ${cp!.id}`;
});
it('restoreCheckpoint denies a NULL-session_id checkpoint from another session (no fail-open IDOR)', async () => {
// Regression for the fail-open authorization bug: a checkpoint row whose
// denormalized session_id is NULL must STILL be scoped via its chat's owning
// session (chats.session_id), not skipped. The old guard `cp.session_id &&
// cp.session_id !== sessionId` fell through on NULL → cross-session restore.
const [row] = await sql<{ id: string }[]>`
INSERT INTO checkpoints (chat_id, session_id, message_id, commit_sha)
VALUES (${chatId}, NULL, NULL, 'deadbeef')
RETURNING id
`;
await expect(
restoreCheckpoint(sql, row!.id, { sessionId: '22222222-2222-2222-2222-222222222222' }),
).rejects.toBeInstanceOf(CheckpointNotFoundError);
await sql`DELETE FROM checkpoints WHERE id = ${row!.id}`;
});
});

View File

@@ -0,0 +1,173 @@
import { describe, it, expect } from 'vitest';
import { locateMatch, SIMILARITY_THRESHOLD } from '../fuzzy-match.js';
// Helper: assert a resolved span and slice it back out of the content so the
// test pins the EXACT file text the caller would replace.
function span(result: ReturnType<typeof locateMatch>): { start: number; end: number } {
if (result.kind !== 'exact' && result.kind !== 'fuzzy') {
throw new Error(`expected a located span, got ${result.kind}`);
}
return { start: result.start, end: result.end };
}
describe('locateMatch — strategy 1: exact', () => {
it('returns an exact unique span', () => {
const content = 'alpha\nbeta\ngamma\n';
const result = locateMatch(content, 'beta');
expect(result.kind).toBe('exact');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('beta');
});
it('returns the right offsets for a multi-line exact needle', () => {
const content = 'one\ntwo\nthree\nfour\n';
const needle = 'two\nthree';
const result = locateMatch(content, needle);
expect(result.kind).toBe('exact');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(needle);
});
it('refuses when the exact needle occurs more than once', () => {
const content = 'foo\nbar\nfoo\nbar\nfoo\n';
const result = locateMatch(content, 'foo');
expect(result).toEqual({ kind: 'ambiguous', count: 3 });
});
});
describe('locateMatch — strategy 2: per-line whitespace', () => {
it('matches across trailing-whitespace drift at the real span', () => {
// File has trailing spaces the model dropped from a TWO-line copy. A
// single-line needle would be located by exact indexOf (it's a substring),
// so use two lines where line 1's trailing ws breaks an exact substring run.
const content = 'function f() {\n setup(); \n return 1;\n}\n';
const needle = ' setup();\n return 1;'; // line 1 missing trailing spaces
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// The returned span covers the ORIGINAL lines including the trailing spaces.
expect(content.slice(start, end)).toBe(' setup(); \n return 1;');
});
it('matches across indentation drift (multi-line block)', () => {
// File indents with 4 spaces; model emitted 2-space indentation. trimEnd
// alone does not normalize LEADING whitespace, so this exercises... actually
// leading-indent drift is a Levenshtein-tier fallback. Here we keep the
// leading indent identical and drift only trailing whitespace per line.
const content = ['if (x) {', ' doThing(); ', ' doOther();', '}'].join('\n');
const needle = [' doThing();', ' doOther();'].join('\n');
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(' doThing(); \n doOther();');
});
it('ignores leading/trailing blank needle lines', () => {
const content = 'header\nbody line\nfooter\n';
const needle = '\n\nbody line\n\n';
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('body line');
});
it('reports ambiguous when a whitespace-window matches twice', () => {
// Both line 1 and line 4 differ from the needle only by trailing whitespace,
// so exact indexOf fails (no exact substring) and the whitespace tier finds
// two equivalent windows → ambiguous.
const content = 'x = 1; \ny = 2;\nz = 3;\nx = 1;\t\n';
const needle = 'x = 1;'; // no trailing ws → not an exact substring of either line
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'ambiguous', count: 2 });
});
});
describe('locateMatch — strategy 3: unicode canonicalization', () => {
it('matches across curly quotes', () => {
const content = "const s = 'hello';\n";
const needle = 'const s = hello;'; // hello
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// Span maps back to ORIGINAL (straight-quote) text.
expect(content.slice(start, end)).toBe("const s = 'hello';");
});
it('matches across curly double-quotes', () => {
const content = 'log("done");\n';
const needle = 'log(“done”);'; // “done”
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('log("done");');
});
it('matches across an em-dash drift', () => {
const content = 'range 1-10 inclusive\n';
const needle = 'range 1—10 inclusive'; // em-dash
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('range 1-10 inclusive');
});
it('matches across a non-breaking space drift', () => {
const content = 'a b c\n'; // plain spaces
const needle = 'a b c'; // nbsp between words
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe('a b c');
});
});
describe('locateMatch — strategy 4: Levenshtein', () => {
it('matches a >= threshold near-miss (small typo drift)', () => {
// Needle has a one-char typo ('totals' vs 'total') so it is NOT an exact
// substring and the whitespace/canonical tiers (which require equality) both
// miss; Levenshtein similarity stays well above the 0.66 floor.
const content = 'const total = sum + tax;\n';
const needle = 'const totals = sum + tax;';
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
// Span maps to the real (correctly-spelled) file line.
expect(content.slice(start, end)).toBe('const total = sum + tax;');
});
it('matches a multi-line block with indentation drift via Levenshtein', () => {
const content = ['function g() {', ' return compute(a, b);', '}'].join('\n');
// 6-space indent vs file's 2-space; trimEnd does not fix leading indent, so
// this lands on the Levenshtein tier (joined-trim makes it identical → ~1.0).
const needle = [' return compute(a, b);'].join('\n');
const result = locateMatch(content, needle);
expect(result.kind).toBe('fuzzy');
const { start, end } = span(result);
expect(content.slice(start, end)).toBe(' return compute(a, b);');
});
it('returns not_found for a below-threshold miss', () => {
const content = 'the quick brown fox jumps over the lazy dog\n';
const needle = 'completely unrelated string of text here xyz';
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'not_found' });
});
it('returns not_found for a genuinely-absent needle', () => {
const content = 'alpha\nbeta\ngamma\n';
const needle = 'this content does not exist anywhere at all';
const result = locateMatch(content, needle);
expect(result).toEqual({ kind: 'not_found' });
});
});
describe('locateMatch — edge cases', () => {
it('returns not_found for an empty needle', () => {
expect(locateMatch('anything', '')).toEqual({ kind: 'not_found' });
});
it('exposes a sane similarity threshold', () => {
expect(SIMILARITY_THRESHOLD).toBeGreaterThan(0);
expect(SIMILARITY_THRESHOLD).toBeLessThanOrEqual(1);
});
});

View File

@@ -0,0 +1,189 @@
import { describe, it, expect } from 'vitest';
import {
makeStreamJsonParser,
makeStreamJsonState,
parseStreamJsonLine,
type AgentEventList,
} from '../stream-json-parser.js';
import type { AgentEvent } from '../agent-backend.js';
import type { AcpToolSnapshot } from '../acp-tool-snapshot.js';
// Helpers to JSON-encode the representative Claude-Code stream-json lines.
const sys = (sessionId: string) =>
JSON.stringify({ type: 'system', subtype: 'init', session_id: sessionId, tools: ['read', 'edit'] });
const streamEvent = (event: unknown) => JSON.stringify({ type: 'stream_event', event });
const textDelta = (index: number, text: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'text_delta', text } });
const thinkingDelta = (index: number, thinking: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'thinking_delta', thinking } });
const toolStart = (index: number, id: string, name: string) =>
streamEvent({ type: 'content_block_start', index, content_block: { type: 'tool_use', id, name } });
const inputJsonDelta = (index: number, partial: string) =>
streamEvent({ type: 'content_block_delta', index, delta: { type: 'input_json_delta', partial_json: partial } });
const blockStop = (index: number) => streamEvent({ type: 'content_block_stop', index });
const resultLine = (input: number, output: number, sessionId?: string) =>
JSON.stringify({ type: 'result', subtype: 'success', session_id: sessionId, usage: { input_tokens: input, output_tokens: output } });
describe('parseStreamJsonLine (pure per-line mapping)', () => {
it('captures session_id from the system init line and emits no events', () => {
const state = makeStreamJsonState();
const events = parseStreamJsonLine(sys('sess-abc'), state);
expect(events).toEqual([]);
expect(state.sessionId).toBe('sess-abc');
});
it('maps a text_delta stream_event → a text event', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(textDelta(0, 'Hello'), state)).toEqual([{ type: 'text', text: 'Hello' }]);
});
it('maps a thinking_delta stream_event → a reasoning event', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(thinkingDelta(0, 'pondering'), state)).toEqual([
{ type: 'reasoning', text: 'pondering' },
]);
});
it('tolerates a garbage / non-JSON line (returns [], no throw)', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine('not json at all {{{', state)).toEqual([]);
expect(parseStreamJsonLine('', state)).toEqual([]);
expect(parseStreamJsonLine(' ', state)).toEqual([]);
// A truncated/partial JSON object also yields [] rather than throwing.
expect(parseStreamJsonLine('{"type":"stream_event","eve', state)).toEqual([]);
});
it('ignores unknown top-level line types and the user (tool-result) line', () => {
const state = makeStreamJsonState();
expect(parseStreamJsonLine(JSON.stringify({ type: 'user', message: {} }), state)).toEqual([]);
expect(parseStreamJsonLine(JSON.stringify({ type: 'whatever' }), state)).toEqual([]);
});
it('assembles a tool call across input_json_delta chunks (split across lines)', () => {
const state = makeStreamJsonState();
// start → tool_call (running, empty args)
const start = parseStreamJsonLine(toolStart(1, 'toolu_1', 'edit_file'), state);
expect(start).toHaveLength(1);
expect(start[0]!.type).toBe('tool_call');
const startSnap = (start[0] as { type: 'tool_call'; toolCall: AcpToolSnapshot }).toolCall;
expect(startSnap.toolCallId).toBe('toolu_1');
expect(startSnap.title).toBe('edit_file');
expect(startSnap.status).toBe('in_progress');
expect(startSnap.rawInput).toEqual({});
// args streamed in fragments — no events until stop
expect(parseStreamJsonLine(inputJsonDelta(1, '{"path":"a'), state)).toEqual([]);
expect(parseStreamJsonLine(inputJsonDelta(1, '.ts","content":'), state)).toEqual([]);
expect(parseStreamJsonLine(inputJsonDelta(1, '"hi"}'), state)).toEqual([]);
// stop → tool_update with the parsed, fully-assembled input
const stop = parseStreamJsonLine(blockStop(1), state);
expect(stop).toHaveLength(1);
expect(stop[0]!.type).toBe('tool_update');
const stopSnap = (stop[0] as { type: 'tool_update'; toolCall: AcpToolSnapshot }).toolCall;
expect(stopSnap.toolCallId).toBe('toolu_1');
expect(stopSnap.status).toBe('completed');
expect(stopSnap.rawInput).toEqual({ path: 'a.ts', content: 'hi' });
});
it('falls back to {_raw} when accumulated tool args are not valid JSON', () => {
const state = makeStreamJsonState();
parseStreamJsonLine(toolStart(0, 'toolu_x', 'run'), state);
parseStreamJsonLine(inputJsonDelta(0, '{"broken'), state);
const stop = parseStreamJsonLine(blockStop(0), state);
const snap = (stop[0] as { type: 'tool_update'; toolCall: AcpToolSnapshot }).toolCall;
expect(snap.rawInput).toEqual({ _raw: '{"broken' });
});
it('captures usage from message_delta and result lines', () => {
const state = makeStreamJsonState();
parseStreamJsonLine(streamEvent({ type: 'message_delta', usage: { output_tokens: 42 } }), state);
expect(state.usage.outputTokens).toBe(42);
parseStreamJsonLine(resultLine(100, 250, 'sess-z'), state);
expect(state.usage.inputTokens).toBe(100);
expect(state.usage.outputTokens).toBe(250);
expect(state.sessionId).toBe('sess-z');
});
it('maps a terminal assistant message (fallback) → text + reasoning + tool events', () => {
const state = makeStreamJsonState();
const line = JSON.stringify({
type: 'assistant',
session_id: 'sess-asst',
message: {
content: [
{ type: 'thinking', thinking: 'let me think' },
{ type: 'text', text: 'Here is the answer' },
{ type: 'tool_use', id: 'toolu_9', name: 'view_file', input: { path: 'x.ts' } },
],
usage: { input_tokens: 5, output_tokens: 7 },
},
});
const events = parseStreamJsonLine(line, state);
expect(events).toEqual([
{ type: 'reasoning', text: 'let me think' },
{ type: 'text', text: 'Here is the answer' },
{
type: 'tool_update',
toolCall: { toolCallId: 'toolu_9', title: 'view_file', kind: null, status: 'completed', rawInput: { path: 'x.ts' } },
},
]);
expect(state.usage).toEqual({ inputTokens: 5, outputTokens: 7 });
expect(state.sessionId).toBe('sess-asst');
});
});
describe('makeStreamJsonParser (stateful wrapper over a full turn)', () => {
it('streams a representative turn: init → text → thinking → tool → result', () => {
const parser = makeStreamJsonParser();
const all: AgentEvent[] = [];
const feed = (line: string): AgentEventList => {
const evs = parser.push(line);
all.push(...evs);
return evs;
};
feed(sys('sess-1'));
feed(textDelta(0, 'Reading '));
feed(textDelta(0, 'the file. '));
feed(thinkingDelta(0, 'I should edit it'));
feed(toolStart(1, 'toolu_a', 'edit_file'));
feed(inputJsonDelta(1, '{"path":'));
feed(inputJsonDelta(1, '"main.ts"}'));
feed(blockStop(1));
feed(textDelta(0, 'Done.'));
feed(resultLine(120, 80, 'sess-1'));
expect(all).toEqual([
{ type: 'text', text: 'Reading ' },
{ type: 'text', text: 'the file. ' },
{ type: 'reasoning', text: 'I should edit it' },
{
type: 'tool_call',
toolCall: { toolCallId: 'toolu_a', title: 'edit_file', kind: null, status: 'in_progress', rawInput: {} },
},
{
type: 'tool_update',
toolCall: { toolCallId: 'toolu_a', title: 'edit_file', kind: null, status: 'completed', rawInput: { path: 'main.ts' } },
},
{ type: 'text', text: 'Done.' },
]);
expect(parser.usage()).toEqual({ inputTokens: 120, outputTokens: 80 });
expect(parser.sessionId()).toBe('sess-1');
});
it('a garbage line interleaved mid-turn does not derail subsequent parsing', () => {
const parser = makeStreamJsonParser();
expect(parser.push(textDelta(0, 'a'))).toEqual([{ type: 'text', text: 'a' }]);
expect(parser.push('>>> not json <<<')).toEqual([]);
expect(parser.push(textDelta(0, 'b'))).toEqual([{ type: 'text', text: 'b' }]);
});
});

View File

@@ -0,0 +1,306 @@
/**
* write-edit-robustness #4 — worktree checkpoints.
*
* External agents (opencode / goose / qwen / claude) write DIRECTLY into the
* shared session worktree (`/tmp/booworktrees/sess-<id>`); BooCode's own `rewind`
* only reverses `pending_changes` against the project root, so it has zero coverage
* there. A checkpoint is a pre-turn shadow-commit of the worktree tree (tracked +
* untracked) captured WITHOUT touching the real index/working tree, stored in a
* private GC-safe ref. `restoreCheckpoint` rewinds the worktree to that commit,
* trims the transcript from the anchor message forward, and resets the agent
* backend so the next turn re-establishes a fresh context consistent with the
* restored files.
*
* All git goes through hostExec + shellEscape (BooCoder runs on the host; the
* worktrees live on the host fs). Checkpoint CREATION is best-effort: a failure
* logs and returns null — it must NEVER throw into the dispatch turn.
*/
import { randomUUID } from 'node:crypto';
import type { FastifyBaseLogger } from 'fastify';
import type { Sql } from '../db.js';
import { hostExec } from './host-exec.js';
import { agentPool, OPENCODE_POOL_KEY } from './agent-pool.js';
import type { AgentSessionHandle } from './agent-backend.js';
/** Minimal shell escape for paths/refs (single-quote wrapping). Mirrors worktrees.ts. */
function shellEscape(s: string): string {
return "'" + s.replace(/'/g, "'\\''") + "'";
}
/**
* Pure builder for the shadow-commit command. Captures tracked + untracked files
* in the worktree into a temp index (so the real index/working tree is untouched),
* writes a tree, commits it parented on HEAD, and parks the commit under a private
* ref `refs/boocode/checkpoints/<id>` so git's GC never reclaims it. Prints ONLY
* the resulting SHA on stdout (the trailing `printf '%s'`), so the caller parses
* stdout.trim() directly.
*
* `id` is the row UUID (minted before the ref so the ref name matches the row).
* Both the worktree path and the id are shell-escaped.
*/
export function buildShadowCommitCommand(worktreePath: string, id: string): string {
const wt = shellEscape(worktreePath);
const ref = shellEscape(`refs/boocode/checkpoints/${id}`);
return (
`cd ${wt} && TMP=$(mktemp) && GIT_INDEX_FILE="$TMP" git read-tree HEAD ` +
`&& GIT_INDEX_FILE="$TMP" git add -A ` +
`&& TREE=$(GIT_INDEX_FILE="$TMP" git write-tree) ` +
`&& SHA=$(git commit-tree "$TREE" -p HEAD -m "boocode checkpoint") ` +
`&& git update-ref ${ref} "$SHA" && rm -f "$TMP" && printf '%s' "$SHA"`
);
}
export interface CreateCheckpointArgs {
chatId: string;
sessionId: string | null;
worktreeId: string | null;
worktreePath: string;
messageId: string | null;
label?: string | null;
}
/**
* Capture a pre-turn checkpoint of the session worktree. Best-effort: returns the
* inserted row's { id, commit_sha } on success, or null on any failure (the turn
* proceeds either way — a missing checkpoint just means no restore point for that
* turn). NEVER throws.
*
* The id is minted up front so the git ref name (`refs/boocode/checkpoints/<id>`)
* matches the DB row id, keeping ref and row in lockstep.
*/
export async function createCheckpoint(
sql: Sql,
args: CreateCheckpointArgs,
opts?: { signal?: AbortSignal; log?: FastifyBaseLogger },
): Promise<{ id: string; commit_sha: string } | null> {
const id = randomUUID();
try {
const cmd = buildShadowCommitCommand(args.worktreePath, id);
const res = await hostExec(cmd, { signal: opts?.signal, timeoutMs: 30_000 });
if (res.exitCode !== 0) {
opts?.log?.warn(
{ chatId: args.chatId, worktreePath: args.worktreePath, stderr: res.stderr.trim().slice(0, 500) },
'checkpoint: shadow-commit failed (turn proceeds without a checkpoint)',
);
return null;
}
const commitSha = res.stdout.trim();
if (!commitSha) {
opts?.log?.warn(
{ chatId: args.chatId, worktreePath: args.worktreePath },
'checkpoint: shadow-commit produced no SHA (turn proceeds)',
);
return null;
}
await sql`
INSERT INTO checkpoints (id, chat_id, session_id, worktree_id, message_id, commit_sha, label)
VALUES (${id}, ${args.chatId}, ${args.sessionId}, ${args.worktreeId}, ${args.messageId}, ${commitSha}, ${args.label ?? null})
`;
opts?.log?.info({ checkpointId: id, chatId: args.chatId, commitSha }, 'checkpoint: created');
return { id, commit_sha: commitSha };
} catch (err) {
opts?.log?.warn(
{ chatId: args.chatId, err: err instanceof Error ? err.message : String(err) },
'checkpoint: create threw (turn proceeds without a checkpoint)',
);
return null;
}
}
/** Error the route maps to a 404 when the checkpoint can't be resolved / scoped. */
export class CheckpointNotFoundError extends Error {
constructor(message: string) {
super(message);
this.name = 'CheckpointNotFoundError';
}
}
export interface RestoreCheckpointResult {
checkpoint_id: string;
messages_deleted: number;
worktree_reset: boolean;
backend_reset: boolean;
}
export interface RestoreCheckpointOpts {
signal?: AbortSignal;
log?: FastifyBaseLogger;
/** If set, the checkpoint MUST belong to this session (route scope guard). */
sessionId?: string;
}
interface CheckpointRow {
id: string;
chat_id: string;
session_id: string | null;
worktree_id: string | null;
message_id: string | null;
commit_sha: string;
created_at: Date;
}
/**
* Restore a checkpoint: rewind its worktree to the shadow commit, trim the
* transcript from the anchor message forward, reset the backend session, and drop
* now-orphaned later checkpoints. Throws CheckpointNotFoundError when the
* checkpoint is missing or not in the requested session (route → 404).
*/
export async function restoreCheckpoint(
sql: Sql,
checkpointId: string,
opts?: RestoreCheckpointOpts,
): Promise<RestoreCheckpointResult> {
// 1. Resolve the checkpoint.
const [cp] = await sql<CheckpointRow[]>`
SELECT id, chat_id, session_id, worktree_id, message_id, commit_sha, created_at
FROM checkpoints WHERE id = ${checkpointId}
`;
if (!cp) {
throw new CheckpointNotFoundError('checkpoint not found');
}
// Authorization scope (fail-safe): the checkpoint's chat must belong to the
// requested session. cp.session_id is a denormalized hint that may be null, so
// gating on it directly fails open — resolve the owning session via chats
// (authoritative; chat_id is NOT NULL) and deny on any mismatch or missing row.
if (opts?.sessionId) {
const [owner] = await sql<{ session_id: string | null }[]>`
SELECT session_id FROM chats WHERE id = ${cp.chat_id}
`;
if (!owner || owner.session_id !== opts.sessionId) {
throw new CheckpointNotFoundError('checkpoint not in session');
}
}
// 2. Resolve the worktree path (by worktree_id, else the session's active one).
let worktreePath: string | null = null;
if (cp.worktree_id) {
const [wt] = await sql<{ path: string }[]>`
SELECT path FROM worktrees WHERE id = ${cp.worktree_id}
`;
worktreePath = wt?.path ?? null;
}
if (!worktreePath) {
const sid = cp.session_id ?? opts?.sessionId ?? null;
if (sid) {
const [wt] = await sql<{ path: string }[]>`
SELECT path FROM worktrees WHERE session_id = ${sid} AND status = 'active' LIMIT 1
`;
worktreePath = wt?.path ?? null;
}
}
// 3. Worktree reset — hard-reset to the shadow commit, then clean untracked.
let worktreeReset = false;
if (worktreePath) {
const resetRes = await hostExec(
`git -C ${shellEscape(worktreePath)} reset --hard ${shellEscape(cp.commit_sha)}`,
{ signal: opts?.signal, timeoutMs: 30_000 },
).catch((err) => {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: reset --hard threw',
);
return null;
});
if (resetRes && resetRes.exitCode === 0) {
const cleanRes = await hostExec(
`git -C ${shellEscape(worktreePath)} clean -fd`,
{ signal: opts?.signal, timeoutMs: 30_000 },
).catch(() => null);
worktreeReset = cleanRes != null && cleanRes.exitCode === 0;
if (!worktreeReset) {
opts?.log?.warn({ checkpointId, worktreePath }, 'checkpoint restore: clean -fd did not succeed');
}
} else {
opts?.log?.warn(
{ checkpointId, worktreePath, stderr: resetRes?.stderr?.trim()?.slice(0, 500) },
'checkpoint restore: reset --hard did not succeed',
);
}
} else {
opts?.log?.warn({ checkpointId }, 'checkpoint restore: no worktree path resolved (files not reset)');
}
// 4. Trim the transcript from the anchor message forward. message_parts FK to
// messages is ON DELETE CASCADE (apps/server schema.sql:49), so parts are
// removed with their messages — no explicit parts delete needed.
let messagesDeleted = 0;
if (cp.message_id) {
const deleted = await sql<{ id: string }[]>`
DELETE FROM messages
WHERE chat_id = ${cp.chat_id}
AND created_at >= (SELECT created_at FROM messages WHERE id = ${cp.message_id})
RETURNING id
`;
messagesDeleted = deleted.length;
}
// 5. Backend reset — mark the chat's agent sessions crashed so the next turn
// re-establishes a fresh backend, and evict the live pool session(s) for this
// (chat, agent). Warm backends hold context server-side with no partial
// rewind, so a full reset is the only consistent option (proposal §4).
const agentRows = await sql<{ agent: string; backend: string; agent_session_id: string | null; session_id: string | null; worktree_id: string | null }[]>`
SELECT agent, backend, agent_session_id, session_id, worktree_id
FROM agent_sessions WHERE chat_id = ${cp.chat_id}
`;
await sql`
UPDATE agent_sessions SET status = 'crashed' WHERE chat_id = ${cp.chat_id}
`.catch(() => {});
let backendReset = false;
try {
// opencode runs on the SHARED server (keyed on a sentinel, not the chat) — close
// just this chat's session(s) on it, mirroring the lifecycle close-hook.
const ocBackend = agentPool.peek(OPENCODE_POOL_KEY, 'opencode');
if (ocBackend) {
for (const row of agentRows) {
if (row.backend !== 'opencode_server' || !row.agent_session_id) continue;
const handle: AgentSessionHandle = {
sessionId: row.session_id ?? '',
agent: row.agent,
backend: 'opencode_server',
chatId: cp.chat_id,
worktreeId: row.worktree_id ?? '',
agentSessionId: row.agent_session_id,
serverPort: null,
};
await ocBackend.closeSession(handle).catch((err) => {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: opencode closeSession threw',
);
});
}
}
// Warm-ACP backends are pooled under the chat id — dispose them (kills the
// goose/qwen child). closeChat skips busy backends (a live turn isn't torn down).
const disposed = await agentPool.closeChat(cp.chat_id);
backendReset = true;
opts?.log?.info({ checkpointId, chatId: cp.chat_id, disposed }, 'checkpoint restore: backend reset');
} catch (err) {
opts?.log?.warn(
{ checkpointId, err: err instanceof Error ? err.message : String(err) },
'checkpoint restore: backend reset threw',
);
}
// 6. Drop now-orphaned later checkpoints for this chat (their anchor messages were
// just trimmed). Compare `created_at` SERVER-SIDE via a subquery (NOT the JS
// Date round-trip, which truncates the stored microsecond precision to ms and
// would make this checkpoint delete ITSELF), and exclude this checkpoint's own
// id so it always survives — letting the user re-restore to it.
await sql`
DELETE FROM checkpoints
WHERE chat_id = ${cp.chat_id}
AND id <> ${cp.id}
AND created_at > (SELECT created_at FROM checkpoints WHERE id = ${cp.id})
`.catch(() => {});
return {
checkpoint_id: checkpointId,
messages_deleted: messagesDeleted,
worktree_reset: worktreeReset,
backend_reset: backendReset,
};
}

View File

@@ -4,6 +4,7 @@ import type { Broker } from '@boocode/server/broker';
import type { WsFrame } from '@boocode/server/ws-frames';
import type { Config } from '../config.js';
import { createWorktree, diffWorktree, cleanupWorktree, ensureSessionWorktree } from './worktrees.js';
import { createCheckpoint } from './checkpoints.js';
import { makeDcpStreamStripper } from './dcp-strip.js';
import { dispatchViaAcp } from './acp-dispatch.js';
import { getResolvedRegistry } from './provider-config-registry.js';
@@ -358,6 +359,16 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`;
const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn worktree checkpoint (best-effort; a
// failure logs and never breaks dispatch). This path uses a per-task worktree
// (createWorktree, not the session worktree), so there's no worktrees-table id
// — pass null for worktreeId, the path is enough for restore's reset.
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId: null, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, {
type: 'message_started',
message_id: assistantId,
@@ -399,6 +410,52 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
outputSummary = result.output.slice(0, 500);
await persistExternalAgentTurn(sql, assistantId, result.toolSnapshots, acpReasoning);
} else {
// v#7 (stream-json): claude + qwen run with --output-format stream-json.
// Parse the NDJSON live in pty-dispatch and forward AgentEvents here so we
// publish the SAME live frames the warm-ACP / opencode paths emit (text,
// reasoning, tool) and persist structured parts. Accumulate for the final
// message content + persistence; fall back to the opaque stdout slice when
// nothing parsed (agent ran without the flag, or crashed before emitting).
const ptyTextChunks: string[] = [];
const ptyReasoningChunks: string[] = [];
const ptyToolSnaps = new Map<string, AcpToolSnapshot>();
const onPtyEvent = (e: AgentEvent): void => {
switch (e.type) {
case 'text':
ptyTextChunks.push(e.text);
broker.publishFrame(sessionId, {
type: 'delta',
message_id: assistantId,
chat_id: chatId,
content: e.text,
} as WsFrame);
break;
case 'reasoning':
ptyReasoningChunks.push(e.text);
broker.publishFrame(sessionId, {
type: 'reasoning_delta',
message_id: assistantId,
chat_id: chatId,
content: e.text,
} as WsFrame);
break;
case 'tool_call':
case 'tool_update':
ptyToolSnaps.set(e.toolCall.toolCallId, e.toolCall);
broker.publishFrame(sessionId, {
type: 'tool_call',
message_id: assistantId,
chat_id: chatId,
tool_call: snapshotToWireToolCall(e.toolCall),
} as WsFrame);
break;
case 'commands':
// stream-json carries no commands today; ignore if it ever does.
break;
}
};
const result = await dispatchViaPty({
agent,
task: task.input,
@@ -409,17 +466,33 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
thinkingOptionId: task.thinking_option_id ?? undefined,
signal: ac.signal,
log,
onEvent: onPtyEvent,
});
assistantContent = (result.stdout || result.stderr || '(no output)').slice(0, 50_000);
outputSummary = (result.stdout || result.stderr).slice(0, 500);
if (assistantContent) {
broker.publishFrame(sessionId, {
type: 'delta',
message_id: assistantId,
chat_id: chatId,
content: assistantContent,
} as WsFrame);
if (result.streamed) {
assistantContent = ptyTextChunks.join('').slice(0, 50_000);
// stream-json text can be empty for a tool-only turn — surface stderr or a
// placeholder so the message row isn't blank.
if (!assistantContent) {
assistantContent = (result.stderr || '(no text output)').slice(0, 50_000);
}
outputSummary = (ptyTextChunks.join('') || result.stderr).slice(0, 500);
acpReasoning = ptyReasoningChunks.join('').slice(0, 200_000);
await persistExternalAgentTurn(sql, assistantId, [...ptyToolSnaps.values()], acpReasoning);
} else {
// Fallback: agent produced no parseable NDJSON (ran without the flag, or
// crashed). Preserve today's opaque stdout-slice + single delta behavior.
assistantContent = (result.stdout || result.stderr || '(no output)').slice(0, 50_000);
outputSummary = (result.stdout || result.stderr).slice(0, 500);
if (assistantContent) {
broker.publishFrame(sessionId, {
type: 'delta',
message_id: assistantId,
chat_id: chatId,
content: assistantContent,
} as WsFrame);
}
}
}
@@ -617,6 +690,15 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`;
const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn checkpoint of the persistent session
// worktree (best-effort; never breaks dispatch). worktreeId comes from the
// worktrees table (ensureSessionWorktree above).
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, {
type: 'message_started',
message_id: assistantId,
@@ -876,6 +958,15 @@ export function createDispatcher(deps: Deps): { start(): void; stop(): Promise<v
`;
const assistantId = assistantMsg!.id;
// write-edit-robustness #4: pre-turn checkpoint of the persistent session
// worktree (best-effort; never breaks dispatch). Same worktree the opencode
// path uses — a chat that switches opencode↔goose↔qwen shares one worktree.
await createCheckpoint(
sql,
{ chatId, sessionId, worktreeId, worktreePath, messageId: assistantId },
{ signal: ac.signal, log },
).catch(() => null);
broker.publishFrame(sessionId, {
type: 'message_started',
message_id: assistantId,

View File

@@ -0,0 +1,271 @@
// Fuzzy patch locator for staged edits.
//
// Local quantized models (qwen3.6 and friends) frequently reproduce an
// `old_string` with small, semantically-irrelevant drift: trailing whitespace,
// a different indent width, or "smart" unicode punctuation (curly quotes, an
// en/em-dash, a non-breaking space) where the source has the plain ASCII form.
// An exact `String.includes` then fails and the queued edit is lost even though
// a human would say it obviously matches.
//
// `locateMatch` walks a ladder of progressively looser strategies and returns
// the real `[start, end)` byte-offset span in the ORIGINAL content so the caller
// can splice in `new_string` over the true file text (preserving the file's own
// whitespace/unicode, not the model's drifted copy). The ladder stops at the
// first strategy that resolves to a single span:
//
// 1. exact — indexOf; >1 hit is reported `ambiguous` (we refuse to
// guess which occurrence the model meant).
// 2. per-line ws — line-window compare ignoring per-line trailing
// whitespace and leading/trailing blank needle lines.
// 3. unicode canon — same line-window compare after folding smart
// punctuation to ASCII on both sides; the match is
// mapped back to original offsets.
// 4. levenshtein — best line-window by normalized edit-distance
// similarity; accepted only at >= SIMILARITY_THRESHOLD.
//
// Pure and dependency-free (Levenshtein is the standard iterative two-row DP),
// reimplemented from the general technique — no vendored source.
export type MatchResult =
| { kind: 'exact' | 'fuzzy'; start: number; end: number } // [start,end) offsets into content
| { kind: 'ambiguous'; count: number }
| { kind: 'not_found' };
/** Levenshtein similarity floor for the final fuzzy fallback (strategy 4). */
export const SIMILARITY_THRESHOLD = 0.66;
export function locateMatch(content: string, needle: string): MatchResult {
// Empty needle has no meaningful match.
if (needle.length === 0) return { kind: 'not_found' };
// --- 1. Exact ----------------------------------------------------------------
const exact = locateExact(content, needle);
if (exact) return exact;
// --- 2. Per-line whitespace-insensitive -------------------------------------
const ws = locateByLineWindow(content, needle);
if (ws) return ws;
// --- 3. Unicode-canonicalized whitespace pass -------------------------------
const canon = locateCanonical(content, needle);
if (canon) return canon;
// --- 4. Levenshtein similarity ----------------------------------------------
const lev = locateByLevenshtein(content, needle);
if (lev) return lev;
return { kind: 'not_found' };
}
// --- Strategy 1: exact -------------------------------------------------------
function locateExact(content: string, needle: string): MatchResult | null {
const first = content.indexOf(needle);
if (first === -1) return null;
const second = content.indexOf(needle, first + 1);
if (second === -1) {
return { kind: 'exact', start: first, end: first + needle.length };
}
// Count all occurrences so the caller can report a useful number.
let count = 2;
let idx = content.indexOf(needle, second + 1);
while (idx !== -1) {
count++;
idx = content.indexOf(needle, idx + 1);
}
return { kind: 'ambiguous', count };
}
// --- Line-window machinery ---------------------------------------------------
interface Line {
/** Raw line text (no trailing newline). */
text: string;
/** Offset of the first char of this line in the original content. */
start: number;
/** Offset one past the last char of this line (before its newline, if any). */
end: number;
}
/**
* Split content into lines, tracking each line's real offset span. The span
* EXCLUDES the trailing newline so consecutive line spans plus their newlines
* exactly reconstruct the content; the match span we hand back covers from the
* first matched line's start through the last matched line's end (i.e. without a
* trailing newline), which is what an in-place splice wants.
*/
function splitLines(content: string): Line[] {
const lines: Line[] = [];
let start = 0;
for (let i = 0; i <= content.length; i++) {
if (i === content.length || content[i] === '\n') {
lines.push({ text: content.slice(start, i), start, end: i });
start = i + 1;
}
}
return lines;
}
/** Strip leading/trailing all-blank lines; returns the trimmed slice. */
function trimBlankLines(lines: string[]): string[] {
let lo = 0;
let hi = lines.length;
while (lo < hi && lines[lo]!.trim() === '') lo++;
while (hi > lo && lines[hi - 1]!.trim() === '') hi--;
return lines.slice(lo, hi);
}
/**
* Find a contiguous window of content lines whose trailing-whitespace-trimmed
* text equals the needle's (blank-trimmed) lines. Returns the real offset span
* over the matched content lines, or null if zero match. Multiple matches →
* ambiguous. `normalize` lets the caller fold unicode before comparing.
*/
function locateByLineWindow(
content: string,
needle: string,
normalize: (s: string) => string = (s) => s,
): MatchResult | null {
const contentLines = splitLines(content);
const needleLines = trimBlankLines(needle.split('\n'));
const n = needleLines.length;
if (n === 0) return null;
// A single needle line that is itself blank can't be located meaningfully.
if (n === 1 && needleLines[0]!.trim() === '') return null;
const needleKey = needleLines.map((l) => normalize(l.trimEnd())).join('\n');
const hits: Array<{ start: number; end: number }> = [];
for (let i = 0; i + n <= contentLines.length; i++) {
const windowKey = contentLines
.slice(i, i + n)
.map((l) => normalize(l.text.trimEnd()))
.join('\n');
if (windowKey === needleKey) {
hits.push({ start: contentLines[i]!.start, end: contentLines[i + n - 1]!.end });
}
}
if (hits.length === 0) return null;
if (hits.length > 1) return { kind: 'ambiguous', count: hits.length };
return { kind: 'fuzzy', start: hits[0]!.start, end: hits[0]!.end };
}
// --- Strategy 3: unicode canonicalization ------------------------------------
/**
* Fold smart punctuation to its ASCII equivalent. Crucially this is a
* length-PRESERVING, per-character map (every replacement is one char → one
* char), so an offset into the canonical string is also a valid offset into the
* original — letting strategy 3 reuse the line-window matcher and still hand
* back true original-content offsets.
*/
function canonicalizeChar(ch: string): string {
switch (ch) {
// single quotes / apostrophes
case '': // '
case '': // '
case '': //
case '': //
return "'";
// double quotes
case '“': // "
case '”': // "
case '„': // „
case '‟': // ‟
return '"';
// dashes
case '': // en dash
case '—': // — em dash
case '': // figure dash
case '―': // ― horizontal bar
case '': // minus sign
return '-';
// spaces
case ' ': // nbsp
case '': // figure space
case '': // narrow nbsp
return ' ';
default:
return ch;
}
}
function canonicalize(s: string): string {
let out = '';
for (const ch of s) out += canonicalizeChar(ch);
return out;
}
function locateCanonical(content: string, needle: string): MatchResult | null {
// Only worth running if canonicalization actually changes something on either
// side — otherwise it's identical to strategy 2 which already failed.
const canonContent = canonicalize(content);
const canonNeedle = canonicalize(needle);
if (canonContent === content && canonNeedle === needle) return null;
// Offsets are preserved (length-preserving fold), so a match on the canonical
// content maps directly back to the original.
return locateByLineWindow(canonContent, canonNeedle);
}
// --- Strategy 4: Levenshtein similarity --------------------------------------
/** Standard iterative two-row Levenshtein edit distance. */
function levenshtein(a: string, b: string): number {
if (a === b) return 0;
if (a.length === 0) return b.length;
if (b.length === 0) return a.length;
let prev = new Array<number>(b.length + 1);
let curr = new Array<number>(b.length + 1);
for (let j = 0; j <= b.length; j++) prev[j] = j;
for (let i = 1; i <= a.length; i++) {
curr[0] = i;
const ac = a.charCodeAt(i - 1);
for (let j = 1; j <= b.length; j++) {
const cost = ac === b.charCodeAt(j - 1) ? 0 : 1;
curr[j] = Math.min(
prev[j]! + 1, // deletion
curr[j - 1]! + 1, // insertion
prev[j - 1]! + cost, // substitution
);
}
[prev, curr] = [curr, prev];
}
return prev[b.length]!;
}
/** Normalized similarity in [0,1]: 1 - dist / max(len). */
function similarity(a: string, b: string): number {
const maxLen = Math.max(a.length, b.length);
if (maxLen === 0) return 1;
return 1 - levenshtein(a, b) / maxLen;
}
function locateByLevenshtein(content: string, needle: string): MatchResult | null {
const contentLines = splitLines(content);
const needleLines = trimBlankLines(needle.split('\n'));
const n = needleLines.length;
if (n === 0) return null;
if (contentLines.length < n) return null;
const needleJoined = needleLines.map((l) => l.trim()).join('\n');
let best = -1;
let bestSpan: { start: number; end: number } | null = null;
for (let i = 0; i + n <= contentLines.length; i++) {
const window = contentLines.slice(i, i + n);
const windowJoined = window.map((l) => l.text.trim()).join('\n');
const score = similarity(windowJoined, needleJoined);
if (score > best) {
best = score;
bestSpan = { start: window[0]!.start, end: window[n - 1]!.end };
}
}
if (bestSpan && best >= SIMILARITY_THRESHOLD) {
return { kind: 'fuzzy', start: bestSpan.start, end: bestSpan.end };
}
return null;
}

View File

@@ -2,6 +2,7 @@ import { readFile, writeFile, unlink, mkdir } from 'node:fs/promises';
import { dirname } from 'node:path';
import type { Sql } from '../db.js';
import { resolveWritePath } from './write_guard.js';
import { locateMatch } from './fuzzy-match.js';
// --- Types -------------------------------------------------------------------
@@ -121,10 +122,18 @@ export async function applyOne(
case 'edit': {
const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string };
const content = await readFile(change.file_path, 'utf8');
if (!content.includes(oldStr)) {
throw new Error('old_string not found in file — file may have changed since the edit was queued');
const match = locateMatch(content, oldStr);
if (match.kind === 'ambiguous') {
throw new Error(
`old_string matches ${match.count} locations — add surrounding context to disambiguate`,
);
}
const updated = content.replace(oldStr, newStr);
if (match.kind === 'not_found') {
throw new Error(
'old_string not found in file (even fuzzily) — file may have changed since the edit was queued',
);
}
const updated = content.slice(0, match.start) + newStr + content.slice(match.end);
await writeFile(change.file_path, updated, 'utf8');
break;
}
@@ -203,10 +212,18 @@ export async function rewindOne(
// Reverse an edit: swap old and new
const { old: oldStr, new: newStr } = JSON.parse(change.diff) as { old: string; new: string };
const content = await readFile(change.file_path, 'utf8');
if (!content.includes(newStr)) {
throw new Error('new_string not found in file — cannot rewind; file may have been modified since apply');
const match = locateMatch(content, newStr);
if (match.kind === 'ambiguous') {
throw new Error(
`new_string matches ${match.count} locations — cannot rewind; add surrounding context to disambiguate`,
);
}
const reverted = content.replace(newStr, oldStr);
if (match.kind === 'not_found') {
throw new Error(
'new_string not found in file (even fuzzily) — cannot rewind; file may have been modified since apply',
);
}
const reverted = content.slice(0, match.start) + oldStr + content.slice(match.end);
await writeFile(change.file_path, reverted, 'utf8');
break;
}

View File

@@ -1,13 +1,29 @@
/**
* PTY dispatch — runs external agents directly on the host.
*
* claude + qwen run with `--output-format stream-json` and emit Claude-Code's
* stream-json NDJSON on stdout. When an `onEvent` callback is supplied we
* line-buffer that stdout (split on `\n`, hold the partial tail) and feed complete
* lines to `makeStreamJsonParser` so deltas surface live as AgentEvents. The raw
* stdout is still accumulated + returned for back-compat (and the dispatcher's
* fallback when nothing parsed). See `stream-json-parser.ts`.
*/
import type { FastifyBaseLogger } from 'fastify';
import { spawn } from 'node:child_process';
import type { AgentEvent } from './agent-backend.js';
import { makeStreamJsonParser, type StreamJsonUsage } from './stream-json-parser.js';
export interface DispatchResult {
exitCode: number;
stdout: string;
stderr: string;
/** True iff at least one NDJSON AgentEvent was parsed from stdout (v#7). When
* false the dispatcher falls back to slicing stdout as the assistant content. */
streamed: boolean;
/** Final usage parsed from the stream-json `result` / `message_delta`, if any. */
usage?: StreamJsonUsage;
/** Provider session id from the stream-json `system` init line, if any. */
agentSessionId?: string | null;
}
export interface PtyDispatchOpts {
@@ -20,6 +36,10 @@ export interface PtyDispatchOpts {
installPath?: string;
signal?: AbortSignal;
log: FastifyBaseLogger;
/** Optional live event sink. When set, stdout is line-buffered + NDJSON-parsed
* and each AgentEvent is forwarded here as it arrives. Absent → opaque (old)
* behavior: stdout is accumulated and returned, no parsing. */
onEvent?: (e: AgentEvent) => void;
}
interface PtySpawnSpec {
@@ -40,7 +60,9 @@ function buildPtySpawnSpec(
switch (agent) {
case 'claude': {
const args = ['-p'];
// stream-json on -p requires --verbose (Claude Code rejects stream-json
// print mode without it). qwen needs no such flag.
const args = ['-p', '--output-format', 'stream-json', '--verbose'];
if (model) args.push('--model', model);
if (modeId) args.push('--permission-mode', modeId);
if (thinkingOptionId) args.push('--effort', thinkingOptionId);
@@ -73,7 +95,7 @@ function buildPtySpawnSpec(
}
export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchResult> {
const { agent, task, worktreePath, model, modeId, thinkingOptionId, installPath, signal, log } = opts;
const { agent, task, worktreePath, model, modeId, thinkingOptionId, installPath, signal, log, onEvent } = opts;
const cmd = buildPtySpawnSpec(agent, task, model, modeId, thinkingOptionId, installPath);
if (!cmd) {
@@ -81,6 +103,7 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
exitCode: 1,
stdout: '',
stderr: `Agent '${agent}' is not yet supported for PTY dispatch.`,
streamed: false,
};
}
@@ -102,7 +125,32 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
let stderr = '';
let killed = false;
child.stdout!.on('data', (chunk: Buffer) => { stdout += chunk.toString(); });
// Live NDJSON parsing (only when a sink is supplied). Line-buffer: split on
// '\n', dispatch complete lines, hold the partial tail until the next chunk.
const parser = onEvent ? makeStreamJsonParser() : null;
let lineBuf = '';
let streamed = false;
const feedLine = (line: string): void => {
if (!parser || !onEvent) return;
for (const e of parser.push(line)) {
streamed = true;
onEvent(e);
}
};
child.stdout!.on('data', (chunk: Buffer) => {
const text = chunk.toString();
stdout += text;
if (!parser) return;
lineBuf += text;
let nl = lineBuf.indexOf('\n');
while (nl !== -1) {
const line = lineBuf.slice(0, nl);
lineBuf = lineBuf.slice(nl + 1);
feedLine(line);
nl = lineBuf.indexOf('\n');
}
});
child.stderr!.on('data', (chunk: Buffer) => { stderr += chunk.toString(); });
const cleanup = () => {
@@ -116,7 +164,7 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
if (signal) {
if (signal.aborted) {
cleanup();
resolve({ exitCode: 130, stdout: '', stderr: 'Aborted before start' });
resolve({ exitCode: 130, stdout: '', stderr: 'Aborted before start', streamed: false });
return;
}
signal.addEventListener('abort', cleanup, { once: true });
@@ -124,8 +172,18 @@ export async function dispatchViaPty(opts: PtyDispatchOpts): Promise<DispatchRes
child.on('close', (code) => {
if (signal) signal.removeEventListener('abort', cleanup);
log.info({ agent, exitCode: code }, 'pty-dispatch: completed');
resolve({ exitCode: code ?? 1, stdout, stderr });
// Flush any final line with no trailing newline.
if (lineBuf.trim()) feedLine(lineBuf);
lineBuf = '';
log.info({ agent, exitCode: code, streamed }, 'pty-dispatch: completed');
resolve({
exitCode: code ?? 1,
stdout,
stderr,
streamed,
usage: parser?.usage(),
agentSessionId: parser?.sessionId() ?? null,
});
});
child.on('error', (err) => {

View File

@@ -0,0 +1,296 @@
/**
* Claude-Code-compatible stream-json NDJSON parser (feature #7,
* openspec `sampling-streamjson-tokens`).
*
* qwen (`--output-format stream-json`) and claude (`--output-format stream-json`)
* both emit Claude-Code's stream-json NDJSON on stdout: one JSON object per line.
* This module turns that stream into the same transport-agnostic `AgentEvent`s the
* ACP / opencode-server backends emit, so the PTY dispatch path can publish live
* broker frames + persist structured parts instead of slicing stdout opaque.
*
* Two surfaces:
* - `parseStreamJsonLine(line, state)` — PURE per-line mapping (unit-testable).
* `state` is the caller-owned accumulator (open tool blocks + usage/session_id).
* - `makeStreamJsonParser()` — a thin stateful wrapper holding the state, with a
* `push(line)` that returns the events for that line and getters for the final
* `usage` / `sessionId`.
*
* Defensive by contract: a non-JSON / partial / garbage line yields `[]` and never
* throws. Tool args (`input_json_delta`) arrive fragmented across many lines; we
* accumulate the partial JSON string per content-block index and only surface the
* parsed `rawInput` once the block stops (or, as a fallback, off the terminal
* `assistant` message which carries the fully-assembled `tool_use` blocks).
*
* Schema (keyed on top-level `type`):
* - `system` — init: { session_id, tools, ... }
* - `assistant` — { message: { content: [ {type:'text'|'thinking'|'tool_use', ...} ], usage? } }
* - `user` — tool results (ignored — diffing the worktree captures effects)
* - `result` — final: { usage: { input_tokens, output_tokens }, session_id? }
* - `stream_event` — { event: { type, index?, content_block?, delta?, usage? } }
* event.type:
* content_block_start — { index, content_block: {type, id?, name?} }
* content_block_delta — { index, delta: {type, text?|thinking?|partial_json?} }
* content_block_stop — { index }
* message_delta — { usage: { output_tokens } }
* message_start — { message: { usage } }
*/
import type { AgentEvent } from './agent-backend.js';
import type { AcpToolSnapshot } from './acp-tool-snapshot.js';
/** Convenience alias for the per-line return value. */
export type AgentEventList = AgentEvent[];
export interface StreamJsonUsage {
inputTokens?: number;
outputTokens?: number;
}
/** Per-open-content-block accumulation for tool args assembled across deltas. */
interface OpenToolBlock {
toolCallId: string;
name: string;
/** Concatenated `input_json_delta.partial_json` fragments. */
partialJson: string;
}
export interface StreamJsonState {
/** content-block index → open tool block (only `tool_use` blocks are tracked). */
toolBlocks: Map<number, OpenToolBlock>;
sessionId: string | null;
usage: StreamJsonUsage;
}
export function makeStreamJsonState(): StreamJsonState {
return { toolBlocks: new Map(), sessionId: null, usage: {} };
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (value && typeof value === 'object' && !Array.isArray(value)) {
return value as Record<string, unknown>;
}
return null;
}
function asString(value: unknown): string | undefined {
return typeof value === 'string' ? value : undefined;
}
function asNumber(value: unknown): number | undefined {
return typeof value === 'number' && Number.isFinite(value) ? value : undefined;
}
/** Pull token counts out of an Anthropic-shape `usage` object, mutating state. */
function captureUsage(usage: Record<string, unknown> | null, state: StreamJsonState): void {
if (!usage) return;
const input = asNumber(usage.input_tokens);
const output = asNumber(usage.output_tokens);
if (input !== undefined) state.usage.inputTokens = input;
// output_tokens is reported incrementally on message_delta; keep the latest.
if (output !== undefined) state.usage.outputTokens = output;
}
/** Parse the accumulated tool-arg JSON; tolerate an unparseable/partial body. */
function parseToolInput(partialJson: string): unknown {
const trimmed = partialJson.trim();
if (!trimmed) return {};
try {
return JSON.parse(trimmed);
} catch {
return { _raw: partialJson };
}
}
function toolSnapshot(block: OpenToolBlock, rawInput: unknown, status: AcpToolSnapshot['status']): AcpToolSnapshot {
return {
toolCallId: block.toolCallId,
title: block.name,
kind: null,
status,
rawInput,
};
}
/**
* Map one stream-event sub-object (the `event` field of a `stream_event` line) to
* AgentEvents, mutating `state` for open tool blocks + usage.
*/
function handleStreamEvent(event: Record<string, unknown>, state: StreamJsonState): AgentEvent[] {
const eventType = asString(event.type);
if (!eventType) return [];
switch (eventType) {
case 'content_block_start': {
const index = asNumber(event.index);
const block = asRecord(event.content_block);
if (index === undefined || !block) return [];
if (asString(block.type) !== 'tool_use') return [];
const toolCallId = asString(block.id) ?? `tool_${index}`;
const name = asString(block.name) ?? 'tool';
const open: OpenToolBlock = { toolCallId, name, partialJson: '' };
state.toolBlocks.set(index, open);
// Surface the tool start immediately (running, no args yet) so the UI shows
// the call before the args finish streaming.
return [{ type: 'tool_call', toolCall: toolSnapshot(open, {}, 'in_progress') }];
}
case 'content_block_delta': {
const index = asNumber(event.index);
const delta = asRecord(event.delta);
if (delta === null) return [];
const deltaType = asString(delta.type);
if (deltaType === 'text_delta') {
const text = asString(delta.text);
return text ? [{ type: 'text', text }] : [];
}
if (deltaType === 'thinking_delta') {
const text = asString(delta.thinking);
return text ? [{ type: 'reasoning', text }] : [];
}
if (deltaType === 'input_json_delta') {
// Accumulate tool args; no event until the block stops.
const fragment = asString(delta.partial_json);
if (index !== undefined && fragment) {
const open = state.toolBlocks.get(index);
if (open) open.partialJson += fragment;
}
return [];
}
return [];
}
case 'content_block_stop': {
const index = asNumber(event.index);
if (index === undefined) return [];
const open = state.toolBlocks.get(index);
if (!open) return [];
state.toolBlocks.delete(index);
const rawInput = parseToolInput(open.partialJson);
return [{ type: 'tool_update', toolCall: toolSnapshot(open, rawInput, 'completed') }];
}
case 'message_start': {
const message = asRecord(event.message);
captureUsage(asRecord(message?.usage), state);
return [];
}
case 'message_delta': {
captureUsage(asRecord(event.usage), state);
return [];
}
default:
return [];
}
}
/**
* Map the terminal `assistant` message (post-hoc full message) to AgentEvents. Used
* as a fallback for transports that emit only the assembled `assistant` line and no
* incremental `stream_event`s. When stream_events already streamed a block, the
* caller dedups by toolCallId, so re-emitting the assembled tool_use is harmless.
*/
function handleAssistantMessage(message: Record<string, unknown>, state: StreamJsonState): AgentEvent[] {
captureUsage(asRecord(message.usage), state);
const content = message.content;
if (!Array.isArray(content)) return [];
const out: AgentEvent[] = [];
let toolIdx = 0;
for (const rawBlock of content) {
const block = asRecord(rawBlock);
if (!block) continue;
const blockType = asString(block.type);
if (blockType === 'text') {
const text = asString(block.text);
if (text) out.push({ type: 'text', text });
} else if (blockType === 'thinking') {
const text = asString(block.thinking);
if (text) out.push({ type: 'reasoning', text });
} else if (blockType === 'tool_use') {
const toolCallId = asString(block.id) ?? `tool_${toolIdx}`;
const name = asString(block.name) ?? 'tool';
const rawInput = 'input' in block ? block.input : {};
out.push({
type: 'tool_update',
toolCall: { toolCallId, title: name, kind: null, status: 'completed', rawInput },
});
}
toolIdx++;
}
return out;
}
/**
* Pure per-line mapping. `line` is a single complete NDJSON line (no trailing
* newline required; surrounding whitespace tolerated). Returns the AgentEvents the
* line produces and mutates `state` (open tool blocks, usage, session_id). A blank,
* non-JSON, or unrecognized line yields `[]` and never throws.
*/
export function parseStreamJsonLine(line: string, state: StreamJsonState): AgentEvent[] {
const trimmed = line.trim();
if (!trimmed) return [];
let obj: Record<string, unknown> | null;
try {
const parsed: unknown = JSON.parse(trimmed);
obj = asRecord(parsed);
} catch {
return [];
}
if (!obj) return [];
const type = asString(obj.type);
switch (type) {
case 'system': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
return [];
}
case 'stream_event': {
const event = asRecord(obj.event);
return event ? handleStreamEvent(event, state) : [];
}
case 'assistant': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
const message = asRecord(obj.message);
return message ? handleAssistantMessage(message, state) : [];
}
case 'result': {
const sid = asString(obj.session_id);
if (sid) state.sessionId = sid;
captureUsage(asRecord(obj.usage), state);
return [];
}
default:
// `user` (tool results) and any unknown line type — ignore.
return [];
}
}
export interface StreamJsonParser {
/** Feed one complete NDJSON line; returns its AgentEvents (never throws). */
push(line: string): AgentEvent[];
/** Final usage (input/output tokens) accumulated so far. */
usage(): StreamJsonUsage;
/** Provider session id from the init `system` line / `result`, if seen. */
sessionId(): string | null;
}
/**
* Stateful wrapper around `parseStreamJsonLine`. Holds per-tool-block accumulation
* + usage/session_id across the turn. Line-buffering (splitting stdout on `\n` and
* holding the partial tail) is the caller's job — see `pty-dispatch.ts`.
*/
export function makeStreamJsonParser(): StreamJsonParser {
const state = makeStreamJsonState();
return {
push: (line: string) => parseStreamJsonLine(line, state),
usage: () => ({ ...state.usage }),
sessionId: () => state.sessionId,
};
}

View File

@@ -1,4 +1,4 @@
import { describe, it, expect } from 'vitest';
import { describe, it, expect, vi, afterEach } from 'vitest';
import { isAgentRegistryMarkdown, parseAgentsMd } from '../agents.js';
describe('isAgentRegistryMarkdown', () => {
@@ -31,3 +31,87 @@ Start here
expect(r.errors.length).toBeGreaterThan(0);
});
});
// v2.6 sampling-streamjson-tokens (#11): per-agent llama.cpp sampler extensions.
describe('parseAgentsMd: v2.6 sampling knobs', () => {
afterEach(() => {
vi.restoreAllMocks();
});
const withFrontmatter = (lines: string) => `# Agents
## Sampler
---
temperature: 0.6
${lines}
tools: [view_file]
description: test
---
You sample.
`;
it('parses top_n_sigma and the dry_* family from frontmatter', () => {
const md = withFrontmatter(
[
'top_n_sigma: 1.5',
'dry_multiplier: 0.8',
'dry_base: 1.75',
'dry_allowed_length: 2',
'dry_penalty_last_n: -1',
].join('\n'),
);
const { agents, errors } = parseAgentsMd(md);
expect(errors).toHaveLength(0);
expect(agents).toHaveLength(1);
const a = agents[0]!;
expect(a.top_n_sigma).toBe(1.5);
expect(a.dry_multiplier).toBe(0.8);
expect(a.dry_base).toBe(1.75);
expect(a.dry_allowed_length).toBe(2);
expect(a.dry_penalty_last_n).toBe(-1);
});
it('defaults the new sampler fields to null when omitted', () => {
const { agents } = parseAgentsMd(withFrontmatter('top_p: 0.95'));
const a = agents[0]!;
expect(a.top_n_sigma).toBeNull();
expect(a.dry_multiplier).toBeNull();
expect(a.dry_base).toBeNull();
expect(a.dry_allowed_length).toBeNull();
expect(a.dry_penalty_last_n).toBeNull();
});
it('warns (does not error) on out-of-range top_n_sigma / dry_* values', () => {
const warn = vi.spyOn(console, 'warn').mockImplementation(() => {});
const md = withFrontmatter(
[
'top_n_sigma: -1',
'dry_multiplier: -0.5',
'dry_base: -2',
'dry_allowed_length: -3',
'dry_penalty_last_n: -5',
].join('\n'),
);
const { agents, errors } = parseAgentsMd(md);
expect(errors).toHaveLength(0);
expect(agents).toHaveLength(1);
// Mirrors top_k/min_p: out-of-range still stored, with a warning.
expect(warn).toHaveBeenCalled();
const warnings = warn.mock.calls.map((c) => String(c[0])).join('\n');
expect(warnings).toContain('top_n_sigma');
expect(warnings).toContain('dry_multiplier');
expect(warnings).toContain('dry_base');
expect(warnings).toContain('dry_allowed_length');
expect(warnings).toContain('dry_penalty_last_n');
});
it('errors on non-numeric / non-integer sampler values', () => {
const md = withFrontmatter(
['top_n_sigma: high', 'dry_allowed_length: 2.5'].join('\n'),
);
const { errors } = parseAgentsMd(md);
const joined = errors.map((e) => e.reason).join('\n');
expect(joined).toContain('top_n_sigma must be a number');
expect(joined).toContain('dry_allowed_length must be an integer');
});
});

View File

@@ -88,6 +88,12 @@ interface ParsedFrontmatter {
top_k?: number;
min_p?: number;
presence_penalty?: number;
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions.
top_n_sigma?: number;
dry_multiplier?: number;
dry_base?: number;
dry_allowed_length?: number;
dry_penalty_last_n?: number;
tools?: string[];
description?: string;
model?: string;
@@ -178,6 +184,63 @@ function parseFrontmatter(yaml: string): { data: ParsedFrontmatter; errors: stri
} else {
errors.push(`presence_penalty must be a number (got "${valueRaw}")`);
}
} else if (key === 'top_n_sigma') {
// v2.6 #11: llama.cpp top-n-sigma sampler. Float ≥ 0 (typical 0-3).
// Mirrors top_p/min_p: store then warn on out-of-range (non-numeric
// hard-fails the block).
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.top_n_sigma = n;
if (n < 0) {
console.warn(`agents: top_n_sigma ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`top_n_sigma must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_multiplier') {
// v2.6 #11: DRY repetition-penalty multiplier. Float ≥ 0 (0 disables DRY).
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.dry_multiplier = n;
if (n < 0) {
console.warn(`agents: dry_multiplier ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_multiplier must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_base') {
// v2.6 #11: DRY penalty growth base. Float ≥ 0.
const n = Number(valueRaw);
if (Number.isFinite(n)) {
data.dry_base = n;
if (n < 0) {
console.warn(`agents: dry_base ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_base must be a number (got "${valueRaw}")`);
}
} else if (key === 'dry_allowed_length') {
// v2.6 #11: DRY max sequence length not penalized. Integer ≥ 0.
const n = Number(valueRaw);
if (Number.isInteger(n)) {
data.dry_allowed_length = n;
if (n < 0) {
console.warn(`agents: dry_allowed_length ${n} out of range (≥0), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_allowed_length must be an integer (got "${valueRaw}")`);
}
} else if (key === 'dry_penalty_last_n') {
// v2.6 #11: DRY lookback window. Integer ≥ -1 (-1 = whole context, 0 = off).
const n = Number(valueRaw);
if (Number.isInteger(n)) {
data.dry_penalty_last_n = n;
if (n < -1) {
console.warn(`agents: dry_penalty_last_n ${n} out of range (≥-1), ignoring (falling back to default)`);
}
} else {
errors.push(`dry_penalty_last_n must be an integer (got "${valueRaw}")`);
}
} else if (key === 'tools') {
if (valueRaw === '') {
data.tools = [];
@@ -354,6 +417,11 @@ function parseAgentSection(section: RawSection): Omit<Agent, 'source'> {
top_k: typeof fm.top_k === 'number' ? fm.top_k : null,
min_p: typeof fm.min_p === 'number' ? fm.min_p : null,
presence_penalty: typeof fm.presence_penalty === 'number' ? fm.presence_penalty : null,
top_n_sigma: typeof fm.top_n_sigma === 'number' ? fm.top_n_sigma : null,
dry_multiplier: typeof fm.dry_multiplier === 'number' ? fm.dry_multiplier : null,
dry_base: typeof fm.dry_base === 'number' ? fm.dry_base : null,
dry_allowed_length: typeof fm.dry_allowed_length === 'number' ? fm.dry_allowed_length : null,
dry_penalty_last_n: typeof fm.dry_penalty_last_n === 'number' ? fm.dry_penalty_last_n : null,
tools: filteredTools,
model: typeof fm.model === 'string' && fm.model.length > 0 ? fm.model : null,
max_tool_calls: typeof fm.max_tool_calls === 'number' ? fm.max_tool_calls : null,

View File

@@ -86,7 +86,7 @@ export async function runCapHitSummary(
ctx,
session.model,
messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined },
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => {
accumulated += delta;
ctx.publish(sessionId, {
@@ -346,7 +346,7 @@ export async function runDoomLoopSummary(
ctx,
session.model,
messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined },
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => {
accumulated += delta;
ctx.publish(sessionId, {
@@ -545,7 +545,7 @@ export async function runStepCapSummary(
ctx,
session.model,
messages,
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined },
{ tools: null, temperature: agent?.temperature, top_p: agent?.top_p ?? undefined, top_k: agent?.top_k ?? undefined, min_p: agent?.min_p ?? undefined, presence_penalty: agent?.presence_penalty ?? undefined, top_n_sigma: agent?.top_n_sigma ?? undefined, dry_multiplier: agent?.dry_multiplier ?? undefined, dry_base: agent?.dry_base ?? undefined, dry_allowed_length: agent?.dry_allowed_length ?? undefined, dry_penalty_last_n: agent?.dry_penalty_last_n ?? undefined },
(delta) => {
accumulated += delta;
ctx.publish(sessionId, {

View File

@@ -33,6 +33,39 @@ interface StreamOptions {
top_k?: number | null;
min_p?: number | null;
presence_penalty?: number | null;
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions. These
// are NOT standard AI-SDK streamText options and are NOT serialized by the
// openai-compatible provider's standardized-settings path (topK is even
// explicitly dropped with an "unsupported feature: topK" warning). They reach
// llama-server only via providerOptions.openaiCompatible (see buildSamplerProviderOptions).
top_n_sigma?: number | null;
dry_multiplier?: number | null;
dry_base?: number | null;
dry_allowed_length?: number | null;
dry_penalty_last_n?: number | null;
}
// v2.6 #11: build the providerOptions.openaiCompatible extraBody object for the
// llama.cpp sampler extensions. @ai-sdk/openai-compatible (2.0.47) merges every
// non-reserved key under providerOptions.openaiCompatible straight into the
// chat-completion request body (see its getArgs: the Object.fromEntries spread
// filtered against openaiCompatibleLanguageModelChatOptions.shape). This is the
// ONLY working passthrough for these params:
// - top_k / min_p were latently dropped before this: top_k was passed as the
// AI-SDK `topK` setting which the openai-compatible provider rejects as
// unsupported; min_p was never passed to streamText at all.
// - top_n_sigma + the dry_* family have no AI-SDK equivalent.
// Keys use llama-server's snake_case body names so they land verbatim.
function buildSamplerProviderOptions(opts: StreamOptions): Record<string, number> | undefined {
const body: Record<string, number> = {};
if (typeof opts.top_k === 'number') body.top_k = opts.top_k;
if (typeof opts.min_p === 'number') body.min_p = opts.min_p;
if (typeof opts.top_n_sigma === 'number') body.top_n_sigma = opts.top_n_sigma;
if (typeof opts.dry_multiplier === 'number') body.dry_multiplier = opts.dry_multiplier;
if (typeof opts.dry_base === 'number') body.dry_base = opts.dry_base;
if (typeof opts.dry_allowed_length === 'number') body.dry_allowed_length = opts.dry_allowed_length;
if (typeof opts.dry_penalty_last_n === 'number') body.dry_penalty_last_n = opts.dry_penalty_last_n;
return Object.keys(body).length > 0 ? body : undefined;
}
// v1.13.1-A: convert BooCode's OpenAI-shaped history into AI SDK
@@ -195,6 +228,14 @@ export async function streamCompletion(
return toolCall;
};
// v2.6 #11: llama.cpp sampler extensions (top_k, min_p, top_n_sigma, dry_*)
// ride providerOptions.openaiCompatible — they are NOT standardized streamText
// settings. NB: top_k used to be passed below as the AI-SDK `topK` setting;
// the openai-compatible provider dropped it with an "unsupported feature: topK"
// warning and min_p was never wired at all, so both were dead on the wire
// before this. They now go through the same extraBody path as the new params.
const samplerBody = buildSamplerProviderOptions(opts);
const result = streamText({
model: upstreamModel(ctx.config, model, agent ?? null),
messages: aiMessages,
@@ -203,8 +244,8 @@ export async function streamCompletion(
: {}),
...(typeof opts.temperature === 'number' ? { temperature: opts.temperature } : {}),
...(typeof opts.top_p === 'number' ? { topP: opts.top_p } : {}),
...(typeof opts.top_k === 'number' ? { topK: opts.top_k } : {}),
...(typeof opts.presence_penalty === 'number' ? { presencePenalty: opts.presence_penalty } : {}),
...(samplerBody ? { providerOptions: { openaiCompatible: samplerBody } } : {}),
abortSignal: signal,
});
@@ -398,6 +439,12 @@ export async function executeStreamPhase(
const effectiveTopK = agent?.top_k ?? undefined;
const effectiveMinP = agent?.min_p ?? undefined;
const effectivePresencePenalty = agent?.presence_penalty ?? undefined;
// v2.6 #11: llama.cpp sampler extensions, threaded the same way as top_k/min_p.
const effectiveTopNSigma = agent?.top_n_sigma ?? undefined;
const effectiveDryMultiplier = agent?.dry_multiplier ?? undefined;
const effectiveDryBase = agent?.dry_base ?? undefined;
const effectiveDryAllowedLength = agent?.dry_allowed_length ?? undefined;
const effectiveDryPenaltyLastN = agent?.dry_penalty_last_n ?? undefined;
// v1.12.2: ctx_max lookup is cached after the first hit per model, so this
// is a Map probe in steady state. We capture nCtx once at the top of the
@@ -435,7 +482,19 @@ export async function executeStreamPhase(
ctx,
session.model,
messages,
{ tools: effectiveTools, temperature: effectiveTemperature, top_p: effectiveTopP, top_k: effectiveTopK, min_p: effectiveMinP, presence_penalty: effectivePresencePenalty },
{
tools: effectiveTools,
temperature: effectiveTemperature,
top_p: effectiveTopP,
top_k: effectiveTopK,
min_p: effectiveMinP,
presence_penalty: effectivePresencePenalty,
top_n_sigma: effectiveTopNSigma,
dry_multiplier: effectiveDryMultiplier,
dry_base: effectiveDryBase,
dry_allowed_length: effectiveDryAllowedLength,
dry_penalty_last_n: effectiveDryPenaltyLastN,
},
(delta) => {
state.accumulated += delta;
ctx.publish(sessionId, {

View File

@@ -117,6 +117,15 @@ export interface Agent {
top_k: number | null; // null means omit from request body
min_p: number | null; // null means omit from request body
presence_penalty: number | null; // null means omit from request body
// v2.6 sampling-streamjson-tokens (#11): llama.cpp sampler extensions.
// null = omit from request body. top_n_sigma + the DRY repetition family
// help the doom-loop-prone local model. All travel via the same
// providerOptions.openaiCompatible extraBody channel as top_k/min_p.
top_n_sigma: number | null;
dry_multiplier: number | null;
dry_base: number | null;
dry_allowed_length: number | null;
dry_penalty_last_n: number | null;
tools: string[]; // whitelist of tool names; empty = no tools allowed
model: string | null; // null means "session.model wins"
source: AgentSource;

View File

@@ -34,6 +34,30 @@ export interface AgentSessionInfo {
status: string;
has_session: boolean;
last_active_at: string | null;
// v2.6.8 per-(chat,agent) running token/cost totals (sampling-streamjson-tokens
// #8). input_tokens/output_tokens are BIGINT and may arrive as strings; cost is
// DOUBLE. AgentComposerBar coerces with Number(...) before rendering.
input_tokens: number;
output_tokens: number;
cost: number;
}
// write-edit-robustness #4: a pre-turn worktree snapshot anchored to an
// assistant message. Returned by GET .../checkpoints; drives the per-message
// "Restore to here" affordance in CoderMessageList.
export interface CoderCheckpoint {
id: string;
message_id: string;
created_at: string;
label: string | null;
}
// write-edit-robustness #4: result of POST .../checkpoints/:id/restore.
export interface CoderRestoreResult {
checkpoint_id: string;
messages_deleted: number;
worktree_reset: boolean;
backend_reset: boolean;
}
export class ApiError extends Error {
@@ -407,6 +431,22 @@ export const api = {
...(config?.thinking_option_id ? { thinking_option_id: config.thinking_option_id } : {}),
}),
}),
// write-edit-robustness #4: worktree checkpoints. List which assistant
// messages in a chat have a pre-turn worktree snapshot ("Restore to here"
// is offered only on those). Proxied to boocoder.
getCheckpoints: (sessionId: string, chatId: string) =>
request<{ checkpoints: CoderCheckpoint[] }>(
`/api/coder/sessions/${sessionId}/checkpoints?chat_id=${encodeURIComponent(chatId)}`,
),
// write-edit-robustness #4: reset the worktree to a checkpoint, trim the
// transcript past its anchor message, and reset the agent backend. After it
// returns, the caller refetches messages (+ checkpoints) so the trimmed
// transcript shows.
restoreCheckpoint: (sessionId: string, checkpointId: string) =>
request<CoderRestoreResult>(
`/api/coder/sessions/${sessionId}/checkpoints/${encodeURIComponent(checkpointId)}/restore`,
{ method: 'POST' },
),
// Queue a new-file create from the RightRail browser → BooCoder
// pending_changes (operation='create'). Surfaces in the CoderPane DiffPanel
// for explicit apply. A WriteGuardError comes back as a 422 whose { error }

View File

@@ -185,6 +185,14 @@ interface Props {
hasPriorTurn?: boolean;
}
// Condensed token count: 950 → "950", 12_400 → "12.4K", 3_200_000 → "3.2M".
// Sub-1000 stays exact; thousands/millions get one decimal, trailing .0 trimmed.
function abbrevTokens(n: number): string {
if (!Number.isFinite(n) || n < 1000) return String(Math.max(0, Math.round(n)));
if (n < 1_000_000) return `${(n / 1000).toFixed(1).replace(/\.0$/, '')}K`;
return `${(n / 1_000_000).toFixed(1).replace(/\.0$/, '')}M`;
}
// Relative-time formatter for the resumed-chip title (e.g. "3m ago").
function relativeTime(iso: string | null): string {
if (!iso) return 'unknown';
@@ -353,6 +361,21 @@ export function AgentComposerBar({ projectPath, value, onChange, onProviderComma
: { label: 'new session', title: `${value.provider} starts a fresh session this turn` }
: null;
// sampling-streamjson-tokens #8: condensed per-(chat,agent) token/cost readout
// beside the session chip. Coerce — input/output are BIGINT (string over wire).
// Hidden when no session row or all totals are zero (e.g. native boocode, which
// holds no agent_sessions row, or a provider that hasn't run yet).
const usageReadout = (() => {
if (!sessionChip || !sessionRow) return null;
const inTok = Number(sessionRow.input_tokens) || 0;
const outTok = Number(sessionRow.output_tokens) || 0;
const cost = Number(sessionRow.cost) || 0;
if (inTok <= 0 && outTok <= 0 && cost <= 0) return null;
const parts = [`${abbrevTokens(inTok)} in`, `${abbrevTokens(outTok)} out`];
if (cost > 0) parts.push(`$${cost.toFixed(2)}`);
return parts.join(' · ');
})();
return (
<div className="flex flex-wrap items-center gap-1 px-2 py-1 border-b border-border bg-muted/20 shrink-0">
<CompactPicker
@@ -374,6 +397,14 @@ export function AgentComposerBar({ projectPath, value, onChange, onProviderComma
{sessionChip.label}
</span>
)}
{usageReadout && (
<span
className="text-[10px] text-muted-foreground tabular-nums whitespace-nowrap shrink-0"
title="Tokens in · out · cost for this agent session"
>
{usageReadout}
</span>
)}
<CompactPicker
label="Mode"
value={value.modeId ?? ''}

View File

@@ -1,6 +1,6 @@
import { useEffect, useState } from 'react';
import type { ReactNode } from 'react';
import { ChevronDown, ChevronRight, Copy, RefreshCw, Check, Share2, RotateCw, GitFork, Trash2, Brain } from 'lucide-react';
import { ChevronDown, ChevronRight, Copy, RefreshCw, Check, Share2, RotateCw, GitFork, Trash2, Brain, History } from 'lucide-react';
import { toast } from 'sonner';
import type { Chat, ErrorReason, Message } from '@/api/types';
import { api } from '@/api/client';
@@ -110,6 +110,10 @@ export interface MessageActions {
onResend?: (chatId: string, content: string) => Promise<void>;
onFork?: (chatId: string, messageId: string) => Promise<void>;
onDelete?: (chatId: string, messageId: string) => Promise<void>;
// write-edit-robustness #4 (BooCoder only): reset the worktree to this
// message's pre-turn checkpoint and trim the transcript past it. BooChat
// passes no such callback → the "Restore to here" control never renders.
onRestoreCheckpoint?: (chatId: string, messageId: string) => Promise<void>;
}
interface Props {
@@ -119,6 +123,17 @@ interface Props {
actions?: MessageActions;
/** Hide actions that don't apply (fork, delete). */
hideActions?: ('fork' | 'delete')[];
/**
* write-edit-robustness #4: this assistant message has a worktree checkpoint
* → render "Restore to here" (only when `actions.onRestoreCheckpoint` is also
* provided). CoderMessageList sets this from the checkpoint set.
*/
hasCheckpoint?: boolean;
/**
* write-edit-robustness #4: suppress the restore control during an active
* turn (mirrors composer gating). Defaults to enabled.
*/
restoreDisabled?: boolean;
}
function StatsLine({ message }: { message: Message }) {
@@ -155,16 +170,22 @@ function ActionRow({
message,
actions,
hiddenSet,
hasCheckpoint = false,
restoreDisabled = false,
}: {
message: Message;
actions?: MessageActions;
hiddenSet: Set<string>;
hasCheckpoint?: boolean;
restoreDisabled?: boolean;
}) {
const [justCopied, setJustCopied] = useState(false);
const [regenerating, setRegenerating] = useState(false);
const [forking, setForking] = useState(false);
const [deleteOpen, setDeleteOpen] = useState(false);
const [deleting, setDeleting] = useState(false);
const [restoreOpen, setRestoreOpen] = useState(false);
const [restoring, setRestoring] = useState(false);
async function copy() {
try {
@@ -240,12 +261,33 @@ function ActionRow({
}
}
async function confirmRestore() {
if (restoring || !actions?.onRestoreCheckpoint) return;
setRestoring(true);
try {
await actions.onRestoreCheckpoint(message.chat_id, message.id);
setRestoreOpen(false);
} catch (err) {
toast.error(err instanceof Error ? err.message : 'restore failed');
} finally {
setRestoring(false);
}
}
const isAssistant = message.role === 'assistant';
const isUser = message.role === 'user';
const canRegen = isAssistant && message.status !== 'streaming';
const canResend = isUser && message.status === 'complete' && !!message.content?.trim();
const canFork = message.status === 'complete';
const canDelete = message.status !== 'streaming';
// write-edit-robustness #4: show "Restore to here" only for a completed
// assistant message that has a checkpoint AND when the coder wired the
// callback. Disabled (but visible) during an active turn.
const canRestore =
isAssistant &&
hasCheckpoint &&
message.status === 'complete' &&
!!actions?.onRestoreCheckpoint;
return (
<>
@@ -306,6 +348,18 @@ function ActionRow({
<Trash2 className="size-3" />
</button>
)}
{canRestore && (
<button
type="button"
onClick={() => setRestoreOpen(true)}
disabled={restoreDisabled || restoring}
className="inline-flex items-center justify-center size-6 rounded text-muted-foreground hover:bg-muted hover:text-foreground disabled:opacity-40 disabled:cursor-not-allowed max-md:min-h-[44px] max-md:min-w-[44px]"
aria-label="Restore to here"
title="Restore worktree to this point"
>
<History className="size-3" />
</button>
)}
</div>
<Dialog
open={deleteOpen}
@@ -338,6 +392,39 @@ function ActionRow({
</DialogFooter>
</DialogContent>
</Dialog>
<Dialog
open={restoreOpen}
onOpenChange={(open) => {
if (!restoring) setRestoreOpen(open);
}}
>
<DialogContent>
<DialogHeader>
<DialogTitle>Restore to this point?</DialogTitle>
<DialogDescription>
This resets the worktree to before this turn, removes every later
message in this chat, and resets the agent's session. This cannot
be undone.
</DialogDescription>
</DialogHeader>
<DialogFooter>
<Button
variant="outline"
onClick={() => setRestoreOpen(false)}
disabled={restoring}
>
Cancel
</Button>
<Button
variant="destructive"
onClick={() => void confirmRestore()}
disabled={restoring}
>
{restoring ? 'Restoring' : 'Restore'}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
</>
);
}
@@ -550,7 +637,15 @@ function ReasoningBlock({ text, streaming }: { text: string; streaming: boolean
);
}
export function MessageBubble({ message, sessionChats, capHitInfo, actions, hideActions }: Props) {
export function MessageBubble({
message,
sessionChats,
capHitInfo,
actions,
hideActions,
hasCheckpoint,
restoreDisabled,
}: Props) {
const hiddenSet = new Set(hideActions ?? []);
// v1.11: anchored rolling summary row. Checked BEFORE the kind==='compact'
// branch because summary=true never coexists with kind='compact' (new
@@ -652,7 +747,15 @@ export function MessageBubble({ message, sessionChats, capHitInfo, actions, hide
</div>
)}
{!isStreaming && <StatsLine message={message} />}
{!isStreaming && hasContent && <ActionRow message={message} actions={actions} hiddenSet={hiddenSet} />}
{!isStreaming && hasContent && (
<ActionRow
message={message}
actions={actions}
hiddenSet={hiddenSet}
hasCheckpoint={hasCheckpoint}
restoreDisabled={restoreDisabled}
/>
)}
</div>
);
}

View File

@@ -147,11 +147,24 @@ interface Props {
chatId?: string;
footer?: ReactNode;
actions?: MessageActions;
// write-edit-robustness #4: assistant message ids that have a worktree
// checkpoint. The "Restore to here" control renders only on these.
checkpointMessageIds?: Set<string>;
// write-edit-robustness #4: suppress restore during an active turn (mirrors
// composer gating in CoderPane).
restoreDisabled?: boolean;
}
const CODER_HIDDEN_ACTIONS: ('fork' | 'delete')[] = ['fork'];
export function CoderMessageList({ messages, chatId, footer, actions }: Props) {
export function CoderMessageList({
messages,
chatId,
footer,
actions,
checkpointMessageIds,
restoreDisabled,
}: Props) {
const endRef = useRef<HTMLDivElement>(null);
const scrollRef = useRef<HTMLDivElement>(null);
const isNearBottomRef = useRef(true);
@@ -189,6 +202,8 @@ export function CoderMessageList({ messages, chatId, footer, actions }: Props) {
message={item.message as unknown as Message}
actions={actions}
hideActions={CODER_HIDDEN_ACTIONS}
hasCheckpoint={checkpointMessageIds?.has(item.message.id) ?? false}
restoreDisabled={restoreDisabled}
/>
);
}

View File

@@ -381,6 +381,29 @@ function usePendingChanges(sessionId: string) {
return { changes, loading, refresh, approve, reject };
}
// write-edit-robustness #4: which assistant messages in this chat have a
// worktree checkpoint, so CoderMessageList can offer "Restore to here" only on
// those. Refetched on message_complete (same trigger as pending changes) and
// after a successful restore.
function useCheckpoints(sessionId: string, chatId: string | undefined) {
const [messageIds, setMessageIds] = useState<Set<string>>(() => new Set());
const refresh = useCallback(() => {
if (!chatId) {
setMessageIds(new Set());
return Promise.resolve();
}
return api.coder
.getCheckpoints(sessionId, chatId)
.then((res) => setMessageIds(new Set(res.checkpoints.map((c) => c.message_id))))
.catch(() => {/* boocoder may be down / endpoint not ready */});
}, [sessionId, chatId]);
useEffect(() => { void refresh(); }, [refresh]);
return { checkpointMessageIds: messageIds, refreshCheckpoints: refresh };
}
// ---------------------------------------------------------------------------
// Sub-components
// ---------------------------------------------------------------------------
@@ -640,6 +663,7 @@ export function CoderPane({
},
});
const { changes, loading, refresh, approve, reject } = usePendingChanges(sessionId);
const { checkpointMessageIds, refreshCheckpoints } = useCheckpoints(sessionId, chatId);
const [input, setInput] = useState('');
const [sending, setSending] = useState(false);
const [queue, setQueue] = useState<string[]>([]);
@@ -652,15 +676,18 @@ export function CoderPane({
// Refresh pending changes (and agent-session state for the §9b chip) when a
// message_complete arrives — same trigger usePendingChanges already uses.
// write-edit-robustness #4: also refetch checkpoints so a new turn's snapshot
// surfaces its "Restore to here" control.
useEffect(() => {
const lastAssistant = [...messages].reverse().find(
(m): m is CoderMessage => m.role === 'assistant',
);
if (lastAssistant?.status === 'complete') {
refresh();
void refreshCheckpoints();
void refreshAgentSessions(sessionId);
}
}, [messages, refresh, sessionId]);
}, [messages, refresh, refreshCheckpoints, sessionId]);
// The §9b chip only shows once the chat has ≥1 prior turn (a completed
// assistant message). Hidden on a brand-new chat.
@@ -867,6 +894,38 @@ export function CoderPane({
}
}, [activeTaskId]);
// write-edit-robustness #4: reset the worktree to a message's checkpoint and
// trim the transcript past it. The confirm lives in MessageBubble's ActionRow
// (plain Cancel/Restore). The restore route is keyed by checkpoint id, so we
// resolve message→checkpoint via a fresh GET (cheap, and avoids a stale id if
// the set changed). On success, refetch messages so the trimmed transcript
// shows, plus checkpoints (later ones were deleted server-side) and pending
// changes (the worktree was reset).
const handleRestoreCheckpoint = useCallback(async (_chatId: string, messageId: string) => {
if (!chatId || generating) return;
let checkpointId: string | undefined;
try {
const res = await api.coder.getCheckpoints(sessionId, chatId);
checkpointId = res.checkpoints.find((c) => c.message_id === messageId)?.id;
} catch (err) {
toast.error(err instanceof Error ? err.message : 'failed to load checkpoint');
return;
}
if (!checkpointId) {
toast.error('No checkpoint found for this message');
return;
}
try {
await api.coder.restoreCheckpoint(sessionId, checkpointId);
await loadMessages();
await refreshCheckpoints();
refresh();
toast.success('Restored to checkpoint');
} catch (err) {
toast.error(err instanceof Error ? err.message : 'restore failed');
}
}, [chatId, generating, sessionId, loadMessages, refreshCheckpoints, refresh]);
const handleChatInputSlash = useCallback(async (skillName: string, userMessage: string) => {
if (!chatId) return;
// Only BooCoder skills route here; an agent's own commands (not skills) fall
@@ -921,8 +980,11 @@ export function CoderPane({
<CoderMessageList
messages={messages as CoderTimelineWire[]}
chatId={chatId}
checkpointMessageIds={checkpointMessageIds}
restoreDisabled={generating}
actions={{
onResend: async (_chatId, content) => { await sendOneMessage(content); },
onRestoreCheckpoint: handleRestoreCheckpoint,
}}
footer={
activeTaskId && !permissionPrompt && sending === false ? (

View File

@@ -6,6 +6,10 @@ Operating rules for every agent in this registry. Full procedures live in the `c
**Worktrees** — Isolate work in a worktree when it is parallel to in-progress work, risky/experimental, a hotfix interrupting other work, or splits into independent units — just create when clear, propose in one line when ambiguous, skip quick/small single-stream work. Branch from a stable base (default branch); worktrees persist (never auto-remove or auto-merge); they isolate code state, not runtime (ports/DBs/services still collide). Full heuristic: invoke `using-worktrees`.
**Sampling knobs** — Each `## Name` frontmatter block accepts these per-agent sampler fields, threaded into the llama-swap chat-completion request: `temperature`, `top_p`, `top_k`, `min_p`, `presence_penalty`, and (v2.6) `top_n_sigma`, `dry_multiplier`, `dry_base`, `dry_allowed_length`, `dry_penalty_last_n`. The `top_n_sigma` + `dry_*` repetition family curb the doom-loop-prone local model. Omit a field to leave it at the server default. Example: `top_n_sigma: 1.0`, `dry_multiplier: 0.8`, `dry_base: 1.75`, `dry_allowed_length: 2`, `dry_penalty_last_n: -1` (-1 = whole context).
**Reasoning budget** — To cap a reasoning model's thinking tokens, pass `--reasoning-budget` through `llama_extra_args` (already permitted by the deny-list validator; routes the agent to llama-sidecar). Example frontmatter line: `llama_extra_args: ["--reasoning-budget", "2048"]`. This is a sidecar process flag, not a chat-completion body param — distinct from the sampling knobs above.
## Code Reviewer
---
temperature: 0.6

View File

@@ -0,0 +1,45 @@
# Small wins — sampling knobs + PTY stream-json + token UI
**Status:** in progress (started 2026-06-01)
**Source:** `boocode_code_review_v2.md` §1 #11 / #7 / #8 (config-adopt + qwen-code §5g + opencode §3 #4).
Three independent BooCode improvements, disjoint subsystems (apps/server / apps/coder / apps/web).
## #11 — New sampling knobs (apps/server)
Per-agent `top_n_sigma` + the `dry_*` repetition family help the doom-loop-prone local model.
Today the Agent type threads `temperature/top_p/top_k/min_p/presence_penalty` into the inference
request (`stream-phase.ts:396438`). Add `top_n_sigma`, `dry_multiplier`, `dry_base`,
`dry_allowed_length`, `dry_penalty_last_n` as first-class Agent fields (`types/api.ts`), parse them in
`agents.ts:parseFrontmatter` (same bounded per-field numeric pattern + out-of-range warn), and thread
them into the request body **via the same mechanism `top_k`/`min_p` already use** (the agent must
confirm whether that's an AI-SDK `providerOptions`/`extraBody` passthrough — these are llama.cpp
extensions, not standard OpenAI fields — and ride it; surface it if `top_k`/`min_p` turn out to be
silently dropped today). `--reasoning-budget` is a llama-server CLI flag already permitted by the
deny-list validator, so it works via `llama_extra_args: ["--reasoning-budget","N"]` now — document it
in `data/AGENTS.md`. apps/server only.
## #7 — Live PTY stream-json NDJSON parsing (apps/coder)
qwen/claude PTY dispatch slices stdout opaque (`dispatcher.ts` PTY path; qwen already runs
`--output-format stream-json`). Add a parser for the Claude-Code-compatible NDJSON
(`system`/`assistant`/`result`/`stream_event``content_block_delta` text/thinking/tool deltas +
`usage` + `session_id`) that maps to the existing `AgentEvent` union (`agent-backend.ts`). **Live
incremental** (decision 2026-06-01): line-buffer the PTY stdout `data` events, parse each complete
NDJSON line as it arrives, and emit broker frames live (text/reasoning/tool) like the ACP/opencode
paths — plus accumulate for `persistExternalAgentTurn`. claude gets `--output-format stream-json` too.
One parser serves both (same schema). apps/coder only (`pty-dispatch.ts`, `dispatcher.ts`, new
`stream-json-parser.ts` + test).
## #8 — Surface opencode token usage (apps/coder route + apps/web)
`agent_sessions.input_tokens/output_tokens/cost` are accumulated (v2.6.8) but the
`GET /api/sessions/:id/agent-sessions` SELECT + the `AgentSessionInfo` type drop them. Add the 3
columns to both, render condensed beside the existing session chip in `AgentComposerBar`
(ChatThroughput styling: `tabular-nums`, muted, e.g. "12.4K in / 3.2K out / $0.25"). MUST NOT touch
Sam's uncommitted WIP (`ChatTabBar`, `SessionLandingPage`, `Workspace`, `useWorkspacePanes`,
`PaneHeaderActions`).
## Decisions (2026-06-01)
- #7 surfacing: **live incremental** streaming (not parse-at-end).
## Verify
- `pnpm -C apps/server test` (+ new agent-parse tests); `pnpm -C apps/coder test` (+ new parser tests)
- `pnpm -C apps/server build && pnpm -C apps/coder build`; `npx tsc -p apps/web/tsconfig.app.json --noEmit`

View File

@@ -0,0 +1,101 @@
# Write/edit robustness — fuzzy patch applier + worktree checkpoints
**Status:** in progress (started 2026-06-01)
**Source:** `boocode_code_review_v2.md` §1 #3 + #4, §5b/§5d5e (cline, Apache-2.0 — algorithm clean-reimplemented, not vendored).
Two independent BooCoder hardening features for local quantized models.
## #3 — Fuzzy patch applier
**Problem:** `applyOne`'s edit case (`apps/coder/src/services/pending_changes.ts:124`) does exact
`content.includes(oldStr)` → throw, then `content.replace(oldStr, newStr)` (first occurrence).
`rewindOne` (line 206) is the same. Local models (qwen3.6) drift `old_string` by whitespace/
indentation/unicode (curly quotes, en/em-dash, nbsp), so a valid edit fails at apply with
"old_string not found" and is lost.
**Design:** new pure module `apps/coder/src/services/fuzzy-match.ts`:
`locateMatch(content: string, needle: string): { kind: 'exact'|'fuzzy'; start: number; end: number }
| { kind: 'ambiguous'; count: number } | { kind: 'not_found' }`. Match ladder:
1. **Exact** `indexOf`. If exactly one → exact span. If >1 → **ambiguous** (refuse; decision
2026-06-01: safer than silently editing the first).
2. **Per-line whitespace-insensitive** — compare `needle` lines to file line-windows ignoring per-line
`trimEnd`/leading-trailing blank lines.
3. **Unicode canonicalization** — normalize curly→straight quotes, en/em-dash→`-`, nbsp→space on both
sides, then retry the whitespace pass.
4. **Levenshtein** similarity ≥ 0.66 over line-windows sized to `needle`'s line count; best window wins.
Non-exact (fuzzy) matches return the actual file span so the caller replaces the real file text with
`new_string`. `pending_changes.ts` `applyOne`/`rewindOne` use `locateMatch`; `ambiguous`/`not_found`
return `success:false` with a clear message (no throw escaping the existing catch). Unit-tested
(`apps/coder/src/services/__tests__/fuzzy-match.test.ts`), per the `turn-guard.ts` pure-helper pattern.
## #4 — Worktree checkpoint + conversation-trim
**Problem:** `rewind` only reverses BooCode's own `pending_changes` (applied to the project root).
External agents (opencode/goose/qwen/claude) write **directly into the session worktree**
(`/tmp/booworktrees/sess-<id>`); rewind has zero coverage there.
**Schema** (`apps/coder/src/schema.sql`):
```sql
CREATE TABLE IF NOT EXISTS checkpoints (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
session_id UUID,
worktree_id UUID REFERENCES worktrees(id) ON DELETE SET NULL,
message_id UUID, -- anchor: the assistant turn row this checkpoint precedes
commit_sha TEXT NOT NULL, -- shadow-commit capturing the pre-turn worktree tree
label TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp()
);
CREATE INDEX IF NOT EXISTS checkpoints_chat_created_idx ON checkpoints(chat_id, created_at);
```
**Create** (`apps/coder/src/services/checkpoints.ts``createCheckpoint`): hooked into the three
external-agent dispatch paths in `dispatcher.ts` (`runWarmAcpTask` ~821, `runOpenCodeServerTask` ~513,
`runExternalAgent` ~255) — after `ensureSessionWorktree()` and the assistant-message insert (so the
anchor `message_id` exists), before the backend runs. Snapshot captures tracked **+ untracked** via a
temp-index shadow commit, stored in a private GC-safe ref:
```
cd <wt> && TMP=$(mktemp) && GIT_INDEX_FILE="$TMP" git read-tree HEAD \
&& GIT_INDEX_FILE="$TMP" git add -A \
&& TREE=$(GIT_INDEX_FILE="$TMP" git write-tree) \
&& SHA=$(git commit-tree "$TREE" -p HEAD -m "boocode checkpoint") \
&& git update-ref refs/boocode/checkpoints/<id> "$SHA" && rm -f "$TMP" && echo "$SHA"
```
Best-effort: a checkpoint failure logs and never breaks the turn. Native-boocode turns (project-root,
rewind-covered) get no checkpoint.
**Restore** (`POST /api/sessions/:sessionId/checkpoints/:checkpointId/restore`, proxied `/api/coder/*`):
1. Resolve + validate the checkpoint belongs to the session.
2. Reset worktree: `git -C <wt> reset --hard <commit_sha> && git -C <wt> clean -fd` (hostExec+shellEscape).
3. Trim transcript: `DELETE FROM messages WHERE chat_id = <cp.chat_id> AND created_at >=
(SELECT created_at FROM messages WHERE id = <cp.message_id>)` (+ explicit `message_parts` delete if
the FK isn't ON DELETE CASCADE — verify).
4. Reset backend (decision 2026-06-01): `UPDATE agent_sessions SET status='crashed' WHERE
chat_id=<cp.chat_id>` and evict the live pool session for `(chat,agent)` if present, so the next turn
re-establishes a fresh backend — transcript, files, and agent context all consistent at the restore
point. (Warm backends hold context server-side; no partial rewind exists.)
5. Delete now-orphaned later checkpoints: `DELETE FROM checkpoints WHERE chat_id=? AND created_at >
<cp.created_at>`.
6. Return `{ checkpoint_id, messages_deleted, worktree_reset, backend_reset }`.
**Frontend:** per-message "Restore to here" in `CoderMessageList.tsx` (via a new optional
`onRestoreCheckpoint?(chatId, messageId)` on `MessageActions` in `MessageBubble.tsx`), wired in
`CoderPane.tsx`; guarded to `status==='complete'` and to messages that have a checkpoint. After the call
returns, refetch the chat's messages (existing GET) — no new WS frame required.
## Decisions (2026-06-01)
- Multi-exact-match → **refuse as ambiguous** (#3).
- #4 **full** scope incl. conversation-trim.
- Restore **resets** the external-agent backend session (context re-established fresh).
## Parallelization
- **Unit 1 (#3)** — fully independent (`fuzzy-match.ts` + `pending_changes.ts` + test).
- **Unit 2 (#4 backend)** — schema + `checkpoints.ts` (create+restore) + 3 dispatcher hooks + restore route + backend reset. One agent owns all #4 coder backend (shared `checkpoints.ts`).
- **Unit 3 (#4 frontend)** — `CoderMessageList`/`MessageBubble`/`CoderPane`, against the pinned restore contract. Parallel with Unit 2. MUST NOT touch Sam's uncommitted WIP (`ChatTabBar`, `SessionLandingPage`, `Workspace`, `useWorkspacePanes`, `PaneHeaderActions`).
## Verify
- `pnpm -C apps/coder test` (incl. new `fuzzy-match` + any checkpoint pure-helper tests)
- `pnpm -C apps/server build` then `pnpm -C apps/coder build`
- `npx tsc -p apps/web/tsconfig.app.json --noEmit`
- Live smoke (manual, host): external-agent edit → checkpoint row; "Restore to here" → worktree reset + transcript trimmed + next turn fresh.