Files
boocode/apps/server/src/services/codecontext_client.ts
indifferentketchup a450140fc2 v1.13.12: skills audit + token-tracking fix + codecontext + cap50 + UI cleanups
Multi-topic batch. The big-ticket item is the skills audit; the rest are
smaller patches that compounded during the audit work.

## Skills audit (rules→recipes split)

Vendored all 26 skills from /home/samkintop/opt/skills/ into data/skills/
(the boocode-repo-local skill library — see docker-compose change below).
Audited via 5 parallel Claude Code agent-teams running the
mgechev/skills-best-practices 4-step protocol (Discovery → Logic → Edge
Case → self-Architecture-Refinement) per skill, ~2 min wall-clock vs the
~3.7-hour serial estimate.

Result: 14 skills surviving (renamed to gerund form, frontmatter matched),
11 deleted (duplicates, BooCode-irrelevant patterns, Claude-already-does-
natively), 1 migrated to BOOCHAT.md/BOOCODER.md as an always-true rule
(verification-before-completion). Each surviving skill had its description
refined to fix specific trigger gaps surfaced by the protocol — 4
real-bug findings landed (dead refs, stale tags, broken sub-file
references in the original vendored content).

Audit decisions documented in openspec/changes/v1.13.12-skills-audit/
audit-notes.md. Convention codified in BOOCHAT.md/BOOCODER.md "rules vs
recipes" sections — future workflow rules go to those files (100%
present), recipes stay in data/skills/ (~6% invoke rate in multi-turn
per the Codeminer42 measurement).

## Token tracking + stale-stream banner fix (same root cause)

ws-frames.ts IsoTimestamp was z.string().min(1) but postgres returns
timestamp columns as JS Date objects. Every message_complete /
session_updated / chat_updated frame was failing the v1.13.11 Zod gate
and being silently dropped. Symptoms: token tracking blank in the UI
(no usage frames landed); the 60s no-token-activity timer tripped the
stale-stream banner because the frontend's local message state never
saw status='streaming' flip to 'complete'.

Fix: z.preprocess(v => v instanceof Date ? v.toISOString() : v,
z.string().min(1)) applied to the IsoTimestamp primitive. Centralized,
no publisher changes, works identically server + web (the parity test
still passes).

## Codecontext .codecontextignore auto-install

services/codecontext_client.ts now copies the
codecontext/.codecontextignore.template into any project's root on the
first call to that project if no .codecontextignore exists. One file
written per project, idempotent (in-memory Set guard + access-check),
silent fallback on read-only project. Stops the upstream empty-source-
file parser crash on foreign projects' node_modules — previously
required manually copying the template per project.

## Tool-call budget cap 30 → 50

services/inference/budget.ts: BUDGET_READ_ONLY and BUDGET_NO_AGENT
bumped to 50 (from 30). BUDGET_NON_READ_ONLY stays at 10 (no write
tools landed yet). Real recon sessions were hitting 30 with ~3 turns
wasted on codecontext parse failures; legitimate need was ~27, and
Architect-class system overviews want deeper recon. Headroom of 20
absorbs failure-retry turns without changing the safety floor — the
doom-loop guard (3 identical calls → abort) catches the actual
failure mode this cap was guarding against.

v1.14 (Phase C outer agent loop) will supersede this via per-agent
agent.steps. Throwaway-ish patch but unblocks deeper recon today.

## UI cleanups

- ChatPane queued-message dropdown removed. Each queued message now
  has three buttons: edit (pop back into ChatInput via sendToChat
  event), force-send (was the dropdown's only useful action), and
  cancel. Default behavior (send when streaming completes) needs no
  UI — it's the implicit do-nothing path.
- ChatThroughput removed from desktop tab strip (ChatTabBar.tsx).
  Mobile tab switcher still shows it.

## Plumbing

- .gitignore: data/* + !data/AGENTS.md + !data/skills/ negation
  patterns so the vendored skill library + agent registry become
  git-tracked while session DB state stays out.
- docker-compose.yml: removed /opt/skills:/data/skills override
  mount. Skills now live in the boocode repo at data/skills/,
  auditable per-batch. The host-level /opt/skills/ is preserved
  untouched for any other tools that read from it.
- .codecontextignore at repo root: auto-installed when codecontext
  was first called against /opt/boocode itself; matches the template.
- CLAUDE.md: updated to document the v1.13.11 publishFrame wrapper +
  message_parts table + tool_cost_stats view + DB-integration test
  pattern + host-side smoke endpoint quirk. (Pre-existing in working
  tree before this batch; shipped here for completeness.)
2026-05-22 18:58:30 +00:00

168 lines
7.2 KiB
TypeScript

// v1.12 Track B.2: shared HTTP client for the codecontext sidecar. The 8
// per-tool wrappers under tools/codecontext/ all funnel through callCodecontext
// — they're thin adapters that supply toolName + args + projectPath. The
// client owns:
//
// 1. target_dir validation. Codecontext's HTTP shim is naive and forwards
// any target_dir to codecontext, so without this layer a model that
// hallucinated a target_dir could read /opt/anything-on-disk. The
// project root is realpath'd and the requested target_dir is constrained
// to it (same invariant as path_guard.ts but for the codecontext path).
// 2. Inline truncation at 32 kB. Codecontext outputs are markdown reports
// that can balloon on large projects; the model can re-narrow via
// file_path / file_type / limit. Matches the "inline truncation, no
// opaque-id retrieval" decision locked in the 2026-05-21 recon.
// 3. Friendly mapping of codecontext's known failure modes — the empty-
// file parser bug (upstream issue #37) returns a generic error string,
// which we re-surface with a hint to add the file to .codecontextignore.
import { access, copyFile, realpath } from 'node:fs/promises';
import { join } from 'node:path';
import { truncateIfNeeded } from './truncate.js';
// v1.13.12 fix: codecontext crashes on empty source files (upstream issue #37)
// when it can't ignore them. The .codecontextignore.template ships with the
// project at /opt/boocode/codecontext/.codecontextignore.template (path inside
// the container; the host's /opt is bind-mounted). On the first call to any
// project, copy the template in if no per-project ignore exists yet. The user
// can subsequently edit the file to customize. Idempotent — once any file is
// at the project root we never overwrite.
const IGNORE_TEMPLATE_PATH = '/opt/boocode/codecontext/.codecontextignore.template';
const ensuredIgnoreProjects = new Set<string>();
async function ensureIgnoreFile(projectRoot: string): Promise<void> {
if (ensuredIgnoreProjects.has(projectRoot)) return;
const ignorePath = join(projectRoot, '.codecontextignore');
try {
await access(ignorePath);
ensuredIgnoreProjects.add(projectRoot);
return;
} catch {
// missing — install the default
}
try {
await copyFile(IGNORE_TEMPLATE_PATH, ignorePath);
ensuredIgnoreProjects.add(projectRoot);
} catch {
// Template missing or project root read-only — proceed without it. The
// codecontext call may still crash on empty source files; the model gets
// the existing hint-message via the catch below telling it to add to
// .codecontextignore manually.
}
}
export interface CodecontextRequest {
toolName: string;
args: Record<string, unknown>;
projectPath: string;
}
export interface CodecontextResponse {
result: string;
truncated: boolean;
// v1.13.5: optional opaque id pointing at the full pre-slice content on
// tmpfs. Set when truncated=true and storage succeeded.
outputPath?: string;
}
const CODECONTEXT_BASE_URL = process.env['CODECONTEXT_URL'] ?? 'http://codecontext:8080';
const TRUNCATION_LIMIT = 32_000;
const REQUEST_TIMEOUT_MS = 30_000;
export async function callCodecontext(
req: CodecontextRequest,
fetcher: typeof fetch = fetch,
): Promise<CodecontextResponse> {
// Step 1: realpath the project root, then realpath the requested target_dir
// (defaulting to projectPath when the caller didn't pass one — the 8 wrappers
// never pass target_dir; tests can override). A non-existent target_dir
// throws before we hit the network so the model gets a sharp error.
const resolvedProject = await realpath(req.projectPath);
// v1.13.12 fix: install the default .codecontextignore on first call to any
// project so codecontext doesn't crash on empty node_modules files. One file
// written per project, idempotent (set-membership check inside).
await ensureIgnoreFile(resolvedProject);
const requestedTarget = req.args['target_dir'];
const targetDir = typeof requestedTarget === 'string' && requestedTarget.length > 0
? requestedTarget
: req.projectPath;
const resolvedTarget = await realpath(targetDir).catch(() => null);
if (resolvedTarget === null) {
throw new Error(`target_dir does not exist: ${targetDir}`);
}
if (resolvedTarget !== resolvedProject && !resolvedTarget.startsWith(resolvedProject + '/')) {
throw new Error(`target_dir ${targetDir} escapes project root ${resolvedProject}`);
}
// Step 2: re-build args with the resolved target_dir so codecontext sees
// the real absolute path, not a symlink or relative form.
const argsToSend = { ...req.args, target_dir: resolvedTarget };
// Step 3: POST with a hard timeout. AbortController + setTimeout pattern
// matches web_fetch.ts; nothing fancier needed.
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
let response: Response;
try {
response = await fetcher(`${CODECONTEXT_BASE_URL}/v1/${req.toolName}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(argsToSend),
signal: controller.signal,
});
} catch (err) {
clearTimeout(timer);
if (err instanceof Error && (err.name === 'AbortError' || err.name === 'TimeoutError')) {
throw new Error(`codecontext request timed out after ${REQUEST_TIMEOUT_MS}ms`);
}
throw new Error(
`codecontext network error: ${err instanceof Error ? err.message : String(err)}`,
);
}
clearTimeout(timer);
if (!response.ok) {
const text = await response.text().catch(() => '');
throw new Error(`codecontext HTTP ${response.status}: ${text.slice(0, 200)}`);
}
const body = (await response.json()) as { result: string | null; error: string | null };
if (body.error) {
// Upstream issue #37: empty source files crash codecontext's parser. The
// error message reliably contains "content is empty"; surface an
// actionable hint instead of the bare codecontext message.
if (body.error.includes('content is empty')) {
throw new Error(
`codecontext parse failure: ${body.error}. ` +
`Add the offending path to .codecontextignore in the project root and retry.`,
);
}
throw new Error(`codecontext error: ${body.error}`);
}
if (body.result === null) {
return { result: '', truncated: false };
}
// Step 4: inline truncation. The model gets a clear hint about how to
// narrow the next call rather than a silent cut. Mirrors web_fetch.ts.
// v1.13.5: stash the full body on tmpfs when truncating so the model can
// retrieve more via view_truncated_output(id).
if (body.result.length > TRUNCATION_LIMIT) {
const truncated = body.result.slice(0, TRUNCATION_LIMIT);
const omitted = body.result.length - TRUNCATION_LIMIT;
const slicedWithMarker =
`${truncated}\n\n[truncated, ${omitted} chars omitted; narrow with file_path, file_type, or limit]`;
const wrapped = await truncateIfNeeded({
fullContent: body.result,
slicedContent: slicedWithMarker,
wasTruncated: true,
});
return {
result: wrapped.content,
truncated: wrapped.truncated,
...(wrapped.outputPath ? { outputPath: wrapped.outputPath } : {}),
};
}
return { result: body.result, truncated: false };
}