Commit Graph

170 Commits

Author SHA1 Message Date
06116f31b3 v2.0.4-hardening: fuzz suite + integration tests + production readiness
Phase 8 of v2.0. Final hardening pass before production tag.

Path-guard fuzz suite (34 tests): traversal attacks (../ all depths,
encoded %2e%2e, null bytes, absolute escapes, prefix-without-separator,
backslash), secret-file deny list (.env, *.pem, id_rsa*, *.key,
credentials.json, *.kdbx, .netrc), valid-path positives, edge cases
(empty, whitespace, very long, triple-dot, multiple slashes).

write_guard.ts hardened: added null-byte rejection and whitespace-only
rejection (previously only checked empty string).

Pending-changes integration test skeleton: 4 tests covering the full
queue→apply→rewind cycle against a real DB + filesystem. Gated on
DATABASE_URL via describe.runIf (same pattern as apps/server's
tool_cost_stats.test.ts). Skips cleanly when unset.

57 tests passing (23 existing + 34 fuzz), 4 integration skipped.
All builds clean. All services healthy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.4-hardening
2026-05-25 04:31:22 +00:00
47abbb6e3c v2.0.3: CLI client + human inbox + cost tracking + Boomerang new_task
Phase 7 of v2.0. BooCoder gains a terminal-driven UX and subagent
isolation primitive.

CLI (src/cli.ts): standalone entry point for terminal use.
- boocode run "task" [--agent x] [--model y] — create + stream output
- boocode ls [--state x] — formatted task table
- boocode attach <id> — WS stream of running task
- boocode send <id> "msg" — follow-up message to task session
Connects to BOOCODER_URL (default http://100.114.205.53:9502).

Human inbox (routes/inbox.ts): GET /api/inbox (failed/blocked tasks),
POST /api/inbox/:id/retry (reset to pending for re-dispatch).

Cost tracking: dispatcher aggregates tokens_used from all messages in
the task's session after completion, stores in tasks.cost_tokens.
GET /api/stats/costs?group_by=project|agent|day for aggregation.

Boomerang subagent isolation (3 new tools):
- new_task: creates child task with parent_task_id linkage, runs in
  fresh isolated session. Orchestrator sees only output_summary.
- list_tasks: query child tasks of current parent
- check_task_status: read task state + output_summary

The orchestrator pattern: an agent with tools: [new_task, list_tasks,
check_task_status] can ONLY dispatch — can't read files or MCP. This
is the Roo Code Boomerang Tasks capability-restriction principle.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.3
2026-05-25 04:25:18 +00:00
f53c6d6cb9 v2.0.2: BooCoder MCP server — 6 tools over stdio
Phase 6 of v2.0. BooCoder exposes its task primitives as MCP tools
so external agents (Sam's opencode in Termius) can drive the task
queue without going through the web UI.

6 MCP tools registered via McpServer + StdioServerTransport:
- boocoder.create_task — INSERT pending task
- boocoder.list_pending_changes — SELECT pending changes
- boocoder.apply — apply a specific pending change to disk
- boocoder.reject — reject a pending change
- boocoder.dispatch_external_agent — create task with agent for Path B
- boocoder.list_worktrees — list active worktrees from running tasks

Activated by --mcp CLI flag: `node dist/index.js --mcp` starts the
MCP server over stdio instead of the HTTP server. Configure in
opencode: {"mcpServers":{"boocoder":{"type":"stdio","command":"docker",
"args":["exec","-i","boocoder","node","dist/index.js","--mcp"]}}}

Uses McpServer class from @modelcontextprotocol/sdk/server/mcp.js
(high-level .tool() registration API). Zod schemas for input
validation. Process blocks on stdin close, cleanly shuts down DB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.2
2026-05-25 04:17:28 +00:00
3d6055518b v2.0.1: ACP dispatch + PTY fallback + worktree management
Phase 5 of v2.0. External agent dispatch via SSH to host.

ACP dispatch (acp-dispatch.ts): spawns agent via SSH with JSON-RPC
stdio pipe. Wraps opencode/goose in ACP mode. Captures structured
events (file operations, tool calls) mapped to parts taxonomy.
Falls back to PTY if ACP handshake fails.

PTY dispatch (pty-dispatch.ts): raw SSH spawn for agents without ACP
support (claude, pi). Captures stdout/stderr as plain text. Simpler
but less structured than ACP.

SSH helper (ssh.ts): shared spawn wrapper for SSH commands to
samkintop@100.114.205.53 (Tailscale IP, same as booterm). Uses
openssh-client installed in the runtime Dockerfile stage.

Worktree management (worktrees.ts): createWorktree (git worktree add
via SSH), diffWorktree (git diff HEAD...task-branch), cleanupWorktree
(git worktree remove --force). One worktree per task at
/tmp/booworktrees/<taskId>.

Dispatcher updated: checks available_agents.supports_acp to pick
transport. Path B flow: create worktree → dispatch agent → diff
worktree → queue diff into pending_changes → cleanup worktree →
mark task complete.

Agent probe updated: probes via SSH to find host-installed agents
(which opencode && opencode --version over SSH).

Dockerfile: openssh-client added to runtime stage.
Config: SSH_HOST env var (default 100.114.205.53).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.1
2026-05-25 04:10:46 +00:00
752ea74f43 v2.0.0-final: dispatcher + task queue + agent probing
Phase 4 of v2.0. BooCoder can now queue tasks and dispatch them
through the inference loop autonomously.

Dispatcher (services/dispatcher.ts): in-process setInterval(5s) polls
tasks WHERE state='pending', picks one at a time, creates an isolated
session+chat, enqueues inference with the task's input as the user
message, polls for completion, marks state completed/failed with
output_summary. Single-task-at-a-time for v2.0.0; parallel dispatch
is a Phase 5+ concern. Respects onClose hook for graceful shutdown.

Task routes (routes/tasks.ts): POST /api/tasks (create), GET /api/tasks
(list with state/project filters), GET /api/tasks/:id (detail),
POST /api/tasks/:id/cancel (marks cancelled, aborts if running).

Agent probe (services/agent-probe.ts): on startup, probes PATH for
opencode/goose/claude/pi via which + --version. UPSERTs into
available_agents table. Finds nothing inside the container (expected —
Phase 5 addresses host-agent access via ACP/PTY).

Schema: ALTER TABLE tasks ADD COLUMN IF NOT EXISTS session_id (links
task to its auto-created inference session for isolation).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.0-final
2026-05-25 03:55:18 +00:00
73b53089b0 CLAUDE.md: v2.0.0 architecture docs — BooCoder, DB rename, MCP config, workspace deps
Session learnings applied:
- Database renamed boochat (from boocode), new tables documented
- BooCoder architecture section: workspace dep pattern, write tools,
  coder pane integration, proxy routing
- Environment: MCP_CONFIG_PATH, BooCoder health at :9502
- Workflow: Go binary at /snap/go/current/bin, codecontext fork location
- Conventions: workspace exports with types conditions, Docker build order

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 03:51:24 +00:00
457c59fb06 v2.0.0: BooCoder frontend — chat pane + diff pane + session picker
Integrates BooCoder as a 'coder' workspace pane within the existing
BooChat SPA at code.indifferentketchup.com. Renamed the placeholder
'agent' pane kind to 'coder' across all types, menus, hooks, and
mobile switcher (Icon: Code instead of Bot).

CoderPane.tsx: split layout with chat area (messages via WS to
boocoder:9502, input bar posting to /api/coder/sessions/:id/messages)
and diff panel (pending changes with Approve/Reject per change plus
Approve All/Reject All). Reuses MarkdownRenderer for message content.

Proxy: Vite dev config adds /api/coder → boocoder:9502 (ordered above
/api per CLAUDE.md proxy-ordering rule). Production: Fastify route in
apps/server/src/index.ts proxies /api/coder/* to http://boocoder:3000
via fetch() pass-through. WS connects directly to :9502 (same
Tailscale network, no proxy needed for WebSocket upgrade).

WorkspacePaneKind mirror updated in both apps/web and apps/server
types. useWorkspacePanes gains coderPane() factory (replaces the old
agent toast stub). Workspace.tsx switch renders CoderPane for
pane.kind === 'coder'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.0
2026-05-25 03:24:49 +00:00
78455b7efc v2.0.0: BooCoder frontend — chat pane + diff pane + session picker
Phase 3 of v2.0. React + Vite SPA at apps/coder/web/ served by
the coder Fastify server via @fastify/static with SPA fallback.

Chat pane: message list via WS streaming (useSessionStream hook),
input bar, POST /api/sessions/:id/messages on submit, markdown
rendering via react-markdown + remark-gfm, inline tool-call display.

Diff pane: fetches GET /api/sessions/:id/pending, shows pending
changes with file path + operation badge (create/edit/delete),
before/after diff for edits, Approve/Reject per change and
Approve All/Reject All buttons.

Layout: fixed two-pane split (chat 60%, diff 40%). Dark theme
(bg-zinc-900). Desktop-first for v2.0.0.

Session picker (Home page): lists projects and sessions from the
shared DB. No CRUD — use BooChat's UI for that.

Dockerfile updated: builds web app in builder stage, copies dist
to runtime. index.ts registers fastifyStatic + SPA fallback route.

Tailwind v4, React 18, TypeScript strict. ~20 new files, ~370KB
built output. Functional developer tool UI, not polished consumer
product — Phase 7 (v2.0.3) handles polish.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 03:04:52 +00:00
d2108b2f8d verification discipline rules + chat naming from assistant response
BOOCHAT.md + BOOCODER.md: 4 verification rules added to both —
verify against running container not source files, never count dist/,
run commands before claiming success, derive counts from commands.

auto_name.ts: chat titles now derived from the assistant's first
response only (user message dropped from naming input). System prompt
updated to "summarize the topic or outcome — do NOT copy the first
few words verbatim." Produces titles like "Fastify Route Setup"
instead of echoing the assistant's opening sentence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 02:52:49 +00:00
ce31577d1e v2.0.0-beta: write tools, pending-changes queue, inference loop, API routes
Phase 2 of v2.0. BooCoder is now a functional write-capable chatbot.

Write-path guard: resolveWritePath() uses resolve() (no realpath — files may
not exist for creates) + prefix-check + secret-file deny list (.env, *.pem,
id_rsa*, etc.). 23 unit tests cover traversal attacks.

Pending-changes service: queueEdit/Create/Delete → applyOne/All →
rejectOne/All → rewindOne. Edit diffs stored as JSON {old, new}. All writes
queue before touching disk; apply re-validates the path guard.

5 write tools: edit_file, create_file, delete_file, apply_pending, rewind.
Registered alongside 25 read-only tools from BooChat (30 total, alpha-sorted).
Write tools use a module-level inference context for sql+sessionId injection.

Inference loop via workspace dependency: apps/coder imports
createInferenceRunner, createBroker, ALL_TOOLS from @boocode/server (dist/).
apps/server gains declaration: true + exports map with typed subpath entries.
No code duplication — one inference engine shared by both apps.

API routes: POST /api/sessions/:id/messages (user msg → inference), POST stop,
GET/POST pending-changes CRUD (5 endpoints), WebSocket session streaming.

Dockerfile updated to build apps/server first (coder depends on its .d.ts).
Health endpoint reports tool count: {"ok":true,"db":true,"tools":30}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.0-beta
2026-05-25 01:53:38 +00:00
006226cce5 v2.0.0-alpha: BooCoder foundation — container, schema, DB rename
Phase 1 of v2.0. BooCoder is live at port 9502 with a health endpoint.

- Database renamed: ALTER DATABASE boocode RENAME TO boochat (one-time).
  All services updated to connect to /boochat. Docker service name stays
  boocode_db (rename is internal to Postgres, not Docker).

- New apps/coder/ app skeleton: Fastify server with health endpoint,
  postgres connection, schema apply on boot. Mirrors apps/server pattern
  but minimal (no inference loop yet — Phase 2).

- Schema: pending_changes (operation queue before /apply), tasks (dispatch
  DAG with state machine), available_agents (startup-probed agent registry),
  human_inbox view (tasks WHERE state IN blocked/failed). All IF NOT EXISTS,
  idempotent on re-run. Same boochat database, different tables.

- Dockerfile: Node 20 bookworm-slim (glibc for future node-pty in Phase 5).
  Multi-stage build matching the existing boocode image pattern.

- docker-compose.yml: boocoder service on 100.114.205.53:9502, /opt:/opt:rw
  mount (write-capable, policy-gated at tool layer), depends on boocode_db.

- BOOCODER.md: container guidance declaring write-tool capability +
  pending-changes discipline.

All 4 services boot and pass health checks. 9 tables in the shared DB.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0.0-alpha
2026-05-25 01:20:29 +00:00
62d818af23 v2.0 implementation plan: 8 phases from foundation to production
Detailed execution plan for all v2.0 sub-versions:

Phase 1 (v2.0.0-alpha): container skeleton, DB rename, schema migration
Phase 2 (v2.0.0-beta): write tools + pending-changes service + fuzz tests
Phase 3 (v2.0.0): frontend diff pane + chat pane + Caddy routing
Phase 4 (v2.0.0-final): dispatcher worker + task queue + agent probing
Phase 5 (v2.0.1): ACP client + PTY fallback + worktree management
Phase 6 (v2.0.2): MCP server (6 tools, stdio, 10-question eval)
Phase 7 (v2.0.3): CLI + human inbox + cost tracking + observation hooks + Boomerang
Phase 8 (v2.0.x): path-guard fuzz, integration tests, docs, production deploy

~2050 LoC total. Phases 1-4 sequential, 5-7 parallelizable after 4.
Risk register covers path-guard bypass, ACP instability, worktree cleanup,
DB rename, MCP eval, Boomerang context leak.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 01:09:05 +00:00
531d39ace9 v2.0 proposal update: add AGENTS.md extensions, Boomerang pattern, observation hooks, follow-up batches
Additions from second pass of boocode_code_review.md:

- AGENTS.md extensions: output_schema, exit_expression, execution_strategy
  (qodo-ai/agents MIT), expert_model escape hatch (RA.Aid Apache-2.0)
- Subagent isolation via Boomerang Tasks pattern: orchestrator-only-dispatches,
  down-pass/up-pass context discipline, fresh session per subtask
- Observation hooks: 5-event taxonomy from budi (SessionStart, UserPromptSubmit,
  PostToolUse, SubagentStart, Stop) mapped to WS frames
- Follow-up batches table: PR-resolver, HMAC audit log, blind-validation gate,
  majority-vote ensembler, drift detection, anti-slop, globstar gate, Docker
  sandbox, multi-provider LLM
- Additional repo to clone: qodo-ai/agents for agent.toml schema reference

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 23:22:57 +00:00
f2974d6887 v2.0 proposal: BooCoder — write tools, pending changes, ACP dispatch, MCP server
Comprehensive roadmap for the v2.0 major version bump. Covers:
- Schema: pending_changes, tasks, available_agents tables + human_inbox view
- Path A: native write tools (edit_file, create_file, delete_file) queuing
  through pending_changes before /apply flushes to disk
- Path B: external agent dispatch via ACP (opencode, goose) or PTY fallback
  (claude, pi) with per-task git worktrees and automatic diff-on-completion
- BooCoder MCP server: 6 tools exposing task primitives over stdio
- Code lifts: agent-hub (Apache-2.0, task DAG), plandex (MIT, diff UX),
  ACP SDK (Apache-2.0, subprocess protocol), Paseo (AGPL, design-only)
- Sub-versions: v2.0.0 (Path A), v2.0.1 (Path B), v2.0.2 (MCP server),
  v2.0.3 (CLI + polish)
- Estimate: ~2200 LoC total

All v1.x dependencies shipped (v1.13 parts, v1.14 outer loop, v1.15 MCP
client, v1.16 codesight). v2.0 is unblocked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v2.0-proposal
2026-05-24 15:11:16 +00:00
29c7d051b6 v1.16.0-codesight-merge: 4 new codecontext tools — blast radius, hot files, routes, middleware
BooCode wrapper tools for the 4 new MCP tools added to the codecontext
sidecar (Go side committed separately at /opt/forks/codecontext).

- get_blast_radius: reverse-edge BFS — "what breaks if I change this?"
- get_hot_files: most-imported files by incoming edge count
- get_routes: Fastify/Express route extraction via tree-sitter AST
- get_middleware: middleware detection via import + registration patterns

Wrappers follow the existing codecontext pattern: Zod input → callCodecontext
→ ToolDef export. Registered in ALL_TOOLS (alpha-sorted). All 4 are read-only.

codecontext sidecar rebuilt from commit b19e646 with the 4 new Go handlers
(2130 lines, 29 tests). Reviewer fixes applied: defer RUnlock on Tier 2
handlers, extractObjectProperty delegates to extractStringValue for
template-literal route paths.

363/363 server tests passing. No schema changes, no frontend changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.16.0-codesight-merge
2026-05-24 05:19:52 +00:00
d27a977d59 v1.15.0-mcp-multi: multi-server MCP client + stdio transport + config file + tool globs
Generalizes the v1.14.1 single-server Context7 PoC into a multi-server MCP
client registry with per-server graceful degradation. JSON config at
/data/mcp.json (bind-mounted alongside AGENTS.md) matches opencode's
mcpServers schema shape. Config file missing = no MCP (opt-in by presence).

Two transports: Streamable HTTP (remote servers like Context7) and stdio
(local subprocess servers like codecontext). Stdio spawns a persistent child
via the SDK's StdioClientTransport; shutdown hook closes all transports.

Tool prefix generalized from context7_<name> to <serverName>_<toolName> with
a toolToServer reverse map for dispatch routing. AGENTS.md tools: field now
supports glob patterns (context7_*, !web_*) via matchToolGlob — last-match-
wins with ! deny prefix. Replaces exact-match .includes() in stream-phase.ts.

refreshToolNames() in agents.ts rebuilds the DEFAULT_TOOLS snapshot after
appendMcpTools so agents without explicit tools: lists see MCP tools —
reviewer caught that the module-load-time snapshot would permanently exclude
late-registered tools.

Read-only invariant: readOnlyHint === false rejected at discovery. Result
size capped at 5MB. v1.14.1 env vars removed — superseded by config file.
Default data/mcp.json ships with Context7 disabled.

363/363 server tests passing. No schema changes, no frontend changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.15.0-mcp-multi
2026-05-24 04:08:42 +00:00
5692e99a5d v1.14.1-mcp-poc: single-server MCP client against Context7
Validates the MCP-client loop end-to-end against one real MCP server before
the full v1.15 port. New services/mcp-client.ts wraps @modelcontextprotocol/sdk
v1.29.0 with Streamable HTTP transport. On startup (when MCP_CONTEXT7_URL is
set), connects to Context7, discovers tools via tools/list, wraps each as a
ToolDef prefixed context7_<name>, and appends to ALL_TOOLS via appendMcpTools.

Read-only invariant guard rejects any tool with readOnlyHint: false. Tool
dispatch is transparent — executeToolCall routes MCP calls through the ToolDef
execute wrapper, which strips the prefix before calling the MCP server. Result
size capped at 5MB with truncation. Graceful degradation: server down at
startup → zero tools; server down mid-session → error result, model
self-corrects.

Adversarial review caught that a Zod .default() on the URL config made MCP
always-on instead of opt-in — fixed by removing the default. MCP_CONTEXT7_URL
must be explicitly set to enable.

ALL_TOOLS changed from ReadonlyArray to mutable to support late-registration.
appendMcpTools re-sorts and rebuilds TOOLS_BY_NAME after append.

348/348 server tests passing (16 new mcp-client tests). No schema changes,
no frontend changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.14.1-mcp-poc
2026-05-23 21:58:09 +00:00
f4a97808ad v1.14.0-outer-loop: explicit while loop replaces inference recursion
Converts the ad-hoc executeToolPhase → runAssistantTurn recursion into an
explicit while (stepNumber < effectiveCap) loop. A step is one stream-and-
tool-execute iteration; the loop terminates on non-tool finish, step-cap hit,
doom-loop, budget exhaustion, abort, or synthesis success.

MAX_STEPS = 200 hard ceiling (4x old effective limit from budget). Per-agent
steps: field in AGENTS.md frontmatter sets tighter caps (Refactorer: 5,
Architect: 20, others: unset = bounded only by MAX_STEPS). Resolution:
effectiveCap = Math.min(agent.steps ?? Infinity, MAX_STEPS).

executeToolPhase no longer recurses — returns ToolPhaseResult struct
(action: 'continue' | 'paused' | 'synthesis_done') so the caller decides
whether to continue or break. steps: 0 handled as "no tool calls allowed"
via runTextOnlyTurn (one text-only stream phase, tool calls ignored with
warn log).

Step-cap hits produce a sentinel summary (reuses cap_hit kind so
CapHitSentinel.tsx renders without frontend changes; text distinguishes
"Step limit reached" from "Tool budget exhausted"). Doom-loop check migrated
to top of loop body — same predicate, same threshold (3), break instead of
return.

step_start parts are in the schema CHECK but not emitted as message_parts —
writing before the stream phase creates a sequence-0 collision with
partsFromAssistantMessage. Structured log line emitted instead. Adversarial
review caught the collision pre-deploy.

332/332 server tests passing. No frontend changes. No schema changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.14.0-outer-loop
2026-05-23 20:29:21 +00:00
211e903620 v1.13.20-drop-legacy-cols: final phase of v1.13.0 strangler-fig
Removes the dual-write into messages.tool_calls / messages.tool_results JSON
columns and drops the columns. message_parts is now the only source of truth
for tool calls and tool results.

10 dual-write sites stripped (5 in tool-phase.ts, 2 in routes/skills.ts, 2 in
routes/messages.ts, 1 in routes/chats.ts fork-clone). The recon-driven grep
caught 2 sites beyond the original v1.13.2 roadmap inventory and an extra
fixture file (tool_cost_stats.test.ts) with a direct legacy-column INSERT.

messages_with_parts view rewritten to parts-only subselects (COALESCE
fallbacks gone). View runs via CREATE OR REPLACE so it lands before the
column DROPs in startup DDL — Postgres rejects column-drop on view-referenced
cols. v1.12.1 cleanup DO block (DROP CONSTRAINT messages_status_check /
messages_role_check) removed; those one-shots have done their work.

Adversarial review caught a runtime bug the green test suite missed: the
discard_stale endpoint (chats.ts) had a RETURNING ... tool_calls, tool_results
clause that would have crashed on every 60s-no-token-activity recovery in
production. Fixed by switching to two-step UPDATE returning id, then SELECT
from messages_with_parts so parts-synthesized fields keep flowing on the wire.

Message API type retains tool_calls? / tool_results? — the view synthesizes
those keys from parts so the wire shape is unchanged; frontend reads need no
update. Override on the original v1.13.2 plan, captured in the openspec
proposal.

339/339 server tests passing (including 7 DB-integration tests that applied
the schema migration to a live DB and ran the parts-only view end-to-end).
tsc + web build clean.

Pairs with v1.13.0-ai-sdk-v6 (introduced the dual-write) and v1.13.1-B (moved
the read path to messages_with_parts). Umbrella v1.13 tag ships on this same
commit, marking the strangler-fig closed.

CLAUDE.md picks up Sam's pre-existing edits documenting tag-naming and
CHANGELOG conventions — both already in use by v1.13.19 / v1.13.20.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.20-drop-legacy-cols v1.13
2026-05-23 13:03:51 +00:00
ad45b28250 v1.13.19-html-artifact-panes: pane-based artifact viewer with on-request HTML
Every assistant message gets an "Open in pane" affordance that opens the
message in the workspace splitter — Markdown pane (Copy + Download .md) by
default; HTML pane (Download .html only) when the model emits a self-contained
<!DOCTYPE html> or fenced ```html artifact. BOOCHAT.md rule keeps Markdown
default at every length; HTML opt-in on explicit user request.

Backend: services/artifacts.ts (slug derivation + write helpers with
symlink-escape guard via realpath-after-mkdir), routes/artifacts.ts (POST
download + GET stream with nosniff + CSP sandbox defense-in-depth), HTML
detection in finalizeCompletion writing a new message_parts.kind='html_artifact'
row (schema CHECK extended via v1.13.13 pattern), graceful 1MB cap via the
pure decideHtmlArtifactWrite helper. PartKind union extended.

Frontend: MarkdownRenderer.tsx extracted from MessageBubble's inline
MarkdownBody for reuse; MarkdownArtifactPane.tsx + HtmlArtifactPane.tsx with
loading/error states; pane state is reference-only ({chat_id, message_id,
title}) — content fetched on mount to keep workspace_panes jsonb small and
avoid 1MB blobs riding session_workspace_updated frames. iframe sandbox
locked to allow-scripts allow-clipboard-write allow-downloads with no
allow-same-origin, srcDoc not src. openInPane discriminates 404 (expected
fallback) from real errors (toast + bail). PanelRightOpen icon button with
mobile 44px tap-target.

31 new server unit tests including a real-symlink filesystem case; 332/332
server tests passing, tsc clean both sides, pnpm -C apps/web build green.
Smoke deferred to first deploy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.19-html-artifact-panes
2026-05-23 12:43:13 +00:00
1a889dcde3 v1.13.18-codecontext-file-path: resolve file_path against project root in codecontext wrappers
Four codecontext sidecar wrappers — get_file_analysis (required
file_path), get_symbol_info, get_dependencies, and get_semantic_neighborhoods
(optional) — forwarded file_path to the HTTP sidecar unchanged. The
sidecar's internal file index is keyed on absolute paths, so any
relative path from the model returned "File not found in graph".
Three back-to-back failures observed in one chat on 2026-05-22
17:56 UTC, ~48 s of wasted tool budget.

## Resolver

Add resolveProjectPath(projectRoot, rawPath) in codecontext_client.ts:
trim check → absolute/relative branch (both go through resolve() so
dot-segments normalise) → realpath with ENOENT fallthrough → escape
check using the realpathed value. Error shape mirrors the existing
target_dir escape error byte-for-byte; only the field name differs.

Wired into callCodecontext at the args-spread site, guarded on
file_path presence + non-empty. All four wrappers benefit from one
call site; wrappers without file_path (overview, framework, watch,
search) are unaffected.

## Schema trim

.trim() added to all four file_path Zod schemas:

  get_file_analysis:                  z.string().trim().min(1)
  get_symbol_info:                    z.string().trim().optional()
  get_dependencies:                   z.string().trim().optional()
  get_semantic_neighborhoods:         z.string().trim().optional()

Absorbs trailing newlines / whitespace from model output before the
resolver sees the value.

## Adversarial review fixes

Adversarial pass surfaced two P2 findings:

1. Absolute path with `..` resolving outside the project root (e.g.
   `<projectRoot>/../etc/passwd`) that ENOENTs at realpath would slip
   through the literal prefix-check: the raw string starts with
   `<projectRoot>/`. Fix: resolve() the absolute branch's candidate
   too, so dot-segments normalise before the prefix check.

2. No symlink-escape test coverage. Realpath's stated purpose
   (catching in-project symlinks pointing outside the project) was
   never tested. Added: create a tmpdir outside projectRoot,
   symlink projectRoot/evil-link → outside file, assert rejection.

## Tests

codecontext_client.test.ts: 19 tests (10 baseline + 9 new file_path
resolution cases). Cases cover: relative→absolute, absolute-inside,
relative-escape, absolute-outside, ENOENT-fallthrough, empty-string,
wrapper-without-file_path, absolute-with-`..`-ENOENT,
symlink-leaving-root.

codecontext_tools.test.ts: one assertion updated to expect the
resolved-absolute file_path on the wire (previously asserted the raw
relative path passed through, which is exactly the bug being fixed).

Full suite: 301 passed, 7 skipped.

## Affected / unaffected

- get_codebase_overview, get_framework_analysis, watch_changes,
  search_symbols: no file_path arg → resolver guard skips them. No
  behavior change.
- get_semantic_neighborhoods IS in SYNTHESIS_TOOLS — previously-failing
  relative-path calls will now successfully synthesize. Desirable, not
  a regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.18-codecontext-file-path
2026-05-22 21:54:16 +00:00
b52c5df705 v1.13.17-cross-repo-reads: on-demand read access to paths outside the project root
When the agent needed context from another repo, pathGuard rejected every read
with no recovery path. This batch adds a reactive request_read_access flow:
pathGuard's error now hints at the tool, the model emits a structured request,
the inference loop pauses (same mechanism as ask_user_input), the user picks
Allow/Deny via inline chips, and subsequent reads under the granted root succeed
for the rest of the session.

Schema: sessions.allowed_read_paths TEXT[] NOT NULL DEFAULT ARRAY[]::TEXT[]
(idempotent ADD COLUMN IF NOT EXISTS).

Grant unit (design D1): nearest registered projects.path ancestor →
nearest repo-shaped ancestor (.git/ / package.json / go.mod / Cargo.toml)
under PROJECT_ROOT_WHITELIST → else refuse. grant_resolver.ts walks
ancestors with a per-iteration whitelist invariant check so symlinked
input can't escape the whitelist mid-walk (Sam's checkpoint-1 ask).

Path-guard: optional extraRoots arg threaded from session.allowed_read_paths
through executeToolCall to view_file / list_dir / grep / find_files. The
ToolDef.execute signature gets an optional third param; non-FS tools
ignore it. view_file re-anchors the secret-guard check on basename(real)
whenever a relative path starts with "../" so .env / id_rsa* etc. still
deny across grant roots.

Endpoint: POST /api/chats/:id/grant_read_access mirrors /answer_user_input.
On 'allow' it re-resolves the grant root (state may have changed since
prompt — auto-falls to denial reason text on failure, not 500), array_appends
to sessions.allowed_read_paths with in-memory dedup, then publishes
tool_result + session_updated frames and enqueues the next assistant turn.

PATCH /api/sessions/:id allowed_read_paths supports revocation only. Zod
refines absolute + no traversal markers; runtime findUnauthorizedAdditions
guard rejects any entry not already present in the row, so a malicious
curl -X PATCH -d '{"allowed_read_paths":["/etc"]}' returns 400 instead of
bypassing the grant flow (Sam's compliance-review action item).

Frontend: RequestReadAccessCard renders pending (path + reason + Allow/Deny)
and answered (granted/denied summary with the resolved root) variants;
MessageList.flatten/group special-cases the tool name; SettingsPane adds a
per-session grants list with per-row revoke that PATCHes the shortened
array.

Tests: 11 grant_resolver, 8 path_guard, 8 sessions PATCH subset, including
explicit cases for symlink escape mid-walk, walk-bound termination at
whitelist root, /etc bypass attempt via PATCH, and nearest-project
disambiguation. 292 total server tests green.

Pairs with v1.13.16-xml-parser — the model now self-recovers from both
a wrong tool name AND from a refused path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.17-cross-repo-reads
2026-05-22 21:45:52 +00:00
2e1a81de72 v1.13.16-xml-parser: Anthropic <invoke> support + unknown-tool recovery hints
Two-part fix for the model-emitted XML drift the v1.13.15-codecontext-synth
investigation surfaced (1 raw <invoke> leak observed out of 190 qwen3.6
turns — qwen3.6-35b-a3b-mxfp4 drifts to the Anthropic format when prompted
as an Architect-style agent because Claude Code documentation in its
pre-training corpus uses that shape).

## Parser extension

xml-parser.ts now recognizes BOTH XML tool-call flavors:

  - Qwen/Hermes:   <tool_call><function=NAME>...<parameter=K>V</parameter>...</function></tool_call>
  - Anthropic:     <invoke name="NAME"><parameter name="K">V</parameter></invoke>

Both route through the same synthetic-id xml_call_${idx} ToolCall path.
extractToolCallBlocks() and partialXmlOpenerStart() handle both openers
(<tool_call> and <invoke...) so partial buffers don't get prematurely
flushed during streaming.

The existing Qwen parser was tightened to tolerate whitespace around `=`
(<function = name>, <parameter = key>...) so a stray space doesn't get
absorbed into the function name. Name capture is non-whitespace,
non-`>`.

## Unknown-tool recovery hint

New tool-suggestions.ts exports levenshtein() + suggestToolName() +
formatUnknownToolError(). When tool-phase.ts:executeToolCall receives a
toolCall.name that isn't in TOOLS_BY_NAME, the error returned to the
model now includes a "Did you mean: X?" hint based on Levenshtein
distance ≤3 or substring match against Object.keys(TOOLS_BY_NAME).
Targets the qwen3.6 drift to read_file → suggest view_file. Applies to
all unknown tool names, not just <invoke>-derived ones — at the
dispatch layer we no longer know which format produced the call, and
the extra signal is harmless for Qwen-derived calls.

## Test coverage

xml-parser.test.ts: 46 tests, all green. Covers both parsers
(well-formed, malformed, multi-parameter, nested-content), the
partial-opener detector for both flavors, the unified extraction
helper, and the unknown-tool error formatter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.16-xml-parser
2026-05-22 20:59:25 +00:00
61308cf17c v1.13.15-codecontext-synth: remove "tag pending" qualifier in roadmap
Trivial follow-up after the v1.13.15-codecontext-synth tag landed.
Retrospective bullet now describes the shipped state; cleanup-order
tracker marks the batch .

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 20:09:39 +00:00
3992a9fcb7 v1.13.15-codecontext-synth: forced second-inference synthesis for codecontext overview tools
After a codecontext overview-class tool call lands (get_codebase_overview,
get_framework_analysis, get_semantic_neighborhoods), the pipeline runs a
second inference pass that replaces the recursive runAssistantTurn. The
synth pass auto-fetches the top-N source files referenced in the
codecontext output plus project docs (BOOCHAT.md, AGENTS.md,
*roadmap*.md, CONTEXT.md), applies a 32k-token budget with explicit
drop-priority, and streams a structured response that grounds the model
in real load-bearing code rather than relying on the codecontext summary
alone. Smoke #1 (default) and #2 (Architect) both cite the correct
inference/turn.ts + tool-phase.ts + stream-phase.ts files; smoke #6
(fault injection) verifies the fall-through path marks the synth message
status='failed' and yields cleanly to the recursive turn.

## Truncation-aware extraction

codecontext's wrapper inline-truncates results at 32k chars. Without the
expansion step, the top-N file selection only saw the alphabetical head
of the codebase (apps/booterm/dist/*) and auto-fetched the wrong sources.
The pipeline now calls in-process readTruncation(outputPath) before
extracting referenced files, so top-N selection sees the full 80k+ char
output. The 32k truncated head still ships to the synth model — the
expansion is reference-extraction-only, preserving the token-budget
contract. Graceful degradation on readTruncation null/throw: log warn,
fall back to the truncated head.

## Schema deviation from dispatch

The dispatch claimed no schema migration was needed for the new
'synthesis' part kind. Reality: message_parts.kind has an explicit
CHECK constraint (schema.sql:54) that would reject the new value. Added
a DROP CONSTRAINT IF EXISTS + DO $$ pg_constraint idempotency-guarded
re-add matching the CLAUDE.md migration pattern. The inline CREATE TABLE
constraint also updated so fresh installs land with the extended enum.

## User-abort marks synth-message failed

Deviation from review-time spec ("user-abort path does NOT mark the
message failed"). The outer abort handler in error-handler.ts operates
on the parent turn's assistantMessageId, not the new synth row that
runSynthesisPass created. Without explicit marking, the synth row would
sit in status='streaming' until the 5-min stale-streaming sweeper
(v1.13.1-cleanup-bundle), tripping the frontend's 60s no-token-activity
banner in the meantime — exactly the UX bug class the v1.13.1 sweeper
was added to handle. Marking failed on every catch path (including
user-abort) closes the gap. Cost: one extra DB write + one publish on
the rare user-abort-during-synth path.

## Race-safe synth-tool capture

tool-phase.ts uses synthEntries: Array<{tc, output, error?}> with
per-callback push under Promise.all. find() picks the first non-error
entry by call-order (toolCalls array index). Multiple synth-tools in
one batch are uncommon but handled deterministically.

## Roadmap rebase

Updated boocode_roadmap.md retrospective section + cleanup-order tracker
+ schema-changes summary to use the new vMAJOR.MINOR.PATCH-slug tag
names per the 2026-05-22 retag (CHANGELOG.md is the canonical record).
v1.13.15 listed as "this batch, tag pending"; a one-line follow-up
commit will remove that qualifier after the tag lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.15-codecontext-synth
2026-05-22 20:08:47 +00:00
0fa46cd06c v1.13.12: skills audit + token-tracking fix + codecontext + cap50 + UI cleanups
Multi-topic batch. The big-ticket item is the skills audit; the rest are
smaller patches that compounded during the audit work.

## Skills audit (rules→recipes split)

Vendored all 26 skills from /home/samkintop/opt/skills/ into data/skills/
(the boocode-repo-local skill library — see docker-compose change below).
Audited via 5 parallel Claude Code agent-teams running the
mgechev/skills-best-practices 4-step protocol (Discovery → Logic → Edge
Case → self-Architecture-Refinement) per skill, ~2 min wall-clock vs the
~3.7-hour serial estimate.

Result: 14 skills surviving (renamed to gerund form, frontmatter matched),
11 deleted (duplicates, BooCode-irrelevant patterns, Claude-already-does-
natively), 1 migrated to BOOCHAT.md/BOOCODER.md as an always-true rule
(verification-before-completion). Each surviving skill had its description
refined to fix specific trigger gaps surfaced by the protocol — 4
real-bug findings landed (dead refs, stale tags, broken sub-file
references in the original vendored content).

Audit decisions documented in openspec/changes/v1.13.12-skills-audit/
audit-notes.md. Convention codified in BOOCHAT.md/BOOCODER.md "rules vs
recipes" sections — future workflow rules go to those files (100%
present), recipes stay in data/skills/ (~6% invoke rate in multi-turn
per the Codeminer42 measurement).

## Token tracking + stale-stream banner fix (same root cause)

ws-frames.ts IsoTimestamp was z.string().min(1) but postgres returns
timestamp columns as JS Date objects. Every message_complete /
session_updated / chat_updated frame was failing the v1.13.11 Zod gate
and being silently dropped. Symptoms: token tracking blank in the UI
(no usage frames landed); the 60s no-token-activity timer tripped the
stale-stream banner because the frontend's local message state never
saw status='streaming' flip to 'complete'.

Fix: z.preprocess(v => v instanceof Date ? v.toISOString() : v,
z.string().min(1)) applied to the IsoTimestamp primitive. Centralized,
no publisher changes, works identically server + web (the parity test
still passes).

## Codecontext .codecontextignore auto-install

services/codecontext_client.ts now copies the
codecontext/.codecontextignore.template into any project's root on the
first call to that project if no .codecontextignore exists. One file
written per project, idempotent (in-memory Set guard + access-check),
silent fallback on read-only project. Stops the upstream empty-source-
file parser crash on foreign projects' node_modules — previously
required manually copying the template per project.

## Tool-call budget cap 30 → 50

services/inference/budget.ts: BUDGET_READ_ONLY and BUDGET_NO_AGENT
bumped to 50 (from 30). BUDGET_NON_READ_ONLY stays at 10 (no write
tools landed yet). Real recon sessions were hitting 30 with ~3 turns
wasted on codecontext parse failures; legitimate need was ~27, and
Architect-class system overviews want deeper recon. Headroom of 20
absorbs failure-retry turns without changing the safety floor — the
doom-loop guard (3 identical calls → abort) catches the actual
failure mode this cap was guarding against.

v1.14 (Phase C outer agent loop) will supersede this via per-agent
agent.steps. Throwaway-ish patch but unblocks deeper recon today.

## UI cleanups

- ChatPane queued-message dropdown removed. Each queued message now
  has three buttons: edit (pop back into ChatInput via sendToChat
  event), force-send (was the dropdown's only useful action), and
  cancel. Default behavior (send when streaming completes) needs no
  UI — it's the implicit do-nothing path.
- ChatThroughput removed from desktop tab strip (ChatTabBar.tsx).
  Mobile tab switcher still shows it.

## Plumbing

- .gitignore: data/* + !data/AGENTS.md + !data/skills/ negation
  patterns so the vendored skill library + agent registry become
  git-tracked while session DB state stays out.
- docker-compose.yml: removed /opt/skills:/data/skills override
  mount. Skills now live in the boocode repo at data/skills/,
  auditable per-batch. The host-level /opt/skills/ is preserved
  untouched for any other tools that read from it.
- .codecontextignore at repo root: auto-installed when codecontext
  was first called against /opt/boocode itself; matches the template.
- CLAUDE.md: updated to document the v1.13.11 publishFrame wrapper +
  message_parts table + tool_cost_stats view + DB-integration test
  pattern + host-side smoke endpoint quirk. (Pre-existing in working
  tree before this batch; shipped here for completeness.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.14-skills-audit
2026-05-22 18:58:30 +00:00
bc376c878d v1.13.11-b: convert raw broker.publish call sites to typed publishFrame
Second half of the WebSocket-frame-typing batch. Phase A (8b568b3)
landed the schemas + frontend receive validation + publishFrame /
publishUserFrame wrappers. This commit converts the existing publish
call sites so every server-emitted WS frame now goes through Zod
validation at the broker boundary.

Conversion strategy: change once in the inference / skills adapters in
index.ts (so ctx.publish / ctx.publishUser propagate to publishFrame /
publishUserFrame for ALL ~50 inference + auto_name call sites in one
move), then bulk-replace the ~30 direct broker.publish* call sites in
the routes + compaction.

Files touched:
- index.ts: inference + skills route adapters now call publishFrame /
  publishUserFrame internally; raw broker.publishUser('default', ...)
  call in the stale-row sweeper also converted.
- routes/projects.ts (7 sites), routes/chats.ts (9 sites),
  routes/sessions.ts (8 sites): all broker.publishUser(...) → broker.
  publishUserFrame(...).
- services/compaction.ts (3 sites): 2 publishUser, 1 publish.

Real protocol drift surfaced by Zod, fixed in the same commit:

  services/compaction.ts:442 was publishing chat_status with status:
  'working' — the v1.12.1 chat_status widening (CLAUDE.md:55) dropped
  this enum value in favor of streaming|tool_running|waiting_for_input|
  idle|error. The compaction.ts site was missed during v1.12.1; the
  frame had been published with an unknown enum value ever since (the
  frontend useChatStatus quietly ignored it). Corrected to 'streaming'
  — compaction's LLM call has the same dot-state semantic as an
  inference turn. This is exactly the class of bug v1.13.11 exists to
  catch.

Schema relaxation: OpaqueObject (the bag type for nested entities like
Project / Chat / Session / WorkspacePane embedded in WS frames) was
z.object({}).passthrough(), which Zod outputs as {} & {[k:string]:
unknown}. The strict-typed entities don't have index signatures so
TypeScript rejected them at publishFrame call sites. Relaxed to
z.unknown() — runtime validation still accepts the value, dev-time
narrowing happens via the existing hand-maintained types. Trade-off:
frame-level drift detection stays sharp; nested-payload validation
goes to follow-up work as the brief intended.

Schema audit:
  grep -rn "broker\.publish(\|broker\.publishUser(" apps/server/src \
    --include="*.ts" | grep -v "broker.ts\|__tests__\|.bak"
  → 0 results. Every server publish goes through publishFrame /
  publishUserFrame. The remaining ctx.publish / ctx.publishUser sites
  in services/inference/* + services/auto_name.ts route through the
  index.ts adapter, which calls publishFrame internally.

Tests: 219/219 pass (unchanged from v1.13.11-a; the Phase B conversion
is mechanical and doesn't add test cases).

Smoke: clean container boot, no ws-frame-validation-failed entries
under normal traffic. Sidebar list refresh + agent picker open both
pass through useUserEvents without drops.

~70 LoC across 7 files. v1.13.11 closed.
v1.13.13-ws-publish
2026-05-22 15:54:00 +00:00
8b568b36d3 v1.13.11-a: WS frame schemas + frontend receive validation
First half of the WebSocket-frame-typing batch (split per recon — total
scope was ~535 LoC, larger than the roadmap's ~300 estimate, so the
server-side publish-site conversion lands separately in v1.13.11-b).

Phase A scope:

(1) apps/server/src/types/ws-frames.ts (NEW) — Zod schemas for all 27
wire-format WS frame types. Discriminated union (WsFrameSchema) plus
KNOWN_FRAME_TYPES const for diagnostic lookup. UUIDs are z.string().
uuid(); model-emitted tool_call_id stays z.string().min(1) since OpenAI-
compatible APIs emit "call_<random>" not UUID. Per-kind payload narrowing
(tool args, message_parts payloads) intentionally stays z.unknown() —
frame-level drift detection is the goal; deep payload validation is
follow-up work.

(2) apps/web/src/api/ws-frames.ts (NEW) — byte-identical mirror of the
authoritative server file. No path alias from web→server in the existing
tsconfig setup; sync-by-hand was chosen over a new packages/shared/ dir.
A ws-frames.test.ts test asserts the two files match.

(3) apps/server/src/services/broker.ts — adds publishFrame() and
publishUserFrame() methods to the Broker interface. Both validate via
WsFrameSchema and fail-closed: log + drop on invalid. createBroker now
accepts an optional FastifyBaseLogger so validation failures land in
the pino stream (with console.error fallback for unit tests). The
existing publish() / publishUser() raw methods stay legal — they get
converted to the typed variants in v1.13.11-b.

(4) apps/web/src/hooks/useSessionStream.ts + useUserEvents.ts — wrap
ws.onmessage with WsFrameSchema.safeParse. Fail-closed: invalid frames
log + return without dispatching. Hand-maintained WsFrame and
SessionEvent types stay in place; one cast bridges Zod-typed → narrowed
shape (Zod uses OpaqueObject for nested Message[] / WorkspacePane[] etc.,
which are dev-time-narrowed via the existing hand-maintained types).

(5) apps/web/package.json — adds zod ^3.23.8 as a direct dep. Was a
transitive dep via ai-sdk / postgres; promotion makes the import legal.

(6) Tests: 15 new in ws-frames.test.ts covering happy-path per major
frame type, drift-catchers (unknown type, invalid enum, non-UUID, negative
tokens), parts-authoritative read variants, the mirror-file diff check,
and four broker fail-closed scenarios. 219/219 server tests pass (was
204; +15 new).

Two recon corrections to the dispatch brief, both flagged before
implementation:

- No 'parts_appended' frame exists. The brief assumed one; the codebase
  reads parts via the messages_with_parts view after message_complete
  triggers a refetch. MessagePartSchema is therefore unused this batch.
- No 'tool_running' frame exists. The brief listed it as standalone; it
  is in fact a 'chat_status' variant ({ status: 'tool_running' }), already
  covered by ChatStatusFrame.

Smoke: clean container boot, no validation errors in the server log. Real
production frames pass validation (the schemas were derived from the
existing hand-maintained types in api/types.ts and sessionEvents.ts).

v1.13.11-b will follow immediately: convert all ~85 raw broker.publish /
ctx.publish call sites across 11 server files to publishFrame /
publishUserFrame. Mechanical edit; the wiring done here means the diff
in -b is just the call-site swaps.

~310 LoC across 9 files (4 new + 5 modified).
v1.13.12-ws-schemas
2026-05-22 15:48:32 +00:00
34cbecf975 v1.13.15-tools: tiered tool loading via BOOCODE_TOOLS env var
Pattern lift from eyaltoledano/claude-task-master (MIT + Commons Clause
— pattern only, no code lift). Adds BOOCODE_TOOLS env var with three
tiers:

- core (4 tools): view_file, list_dir, grep, find_files. ~2k token
  schema cost.
- standard (15 tools): core + web_search, web_fetch, git_status, all
  8 codecontext_* tools. ~10k token schema cost.
- all (default; current behavior): every tool in ALL_TOOLS (20). ~21k
  token schema cost.

The env var is a CEILING — narrows agent whitelists, never expands.
Default behavior unchanged when var is unset. resolveToolTier is
case-insensitive and falls back to 'all' on unknown values.

CORE_TOOL_NAMES + STANDARD_TOOL_NAMES validated at module load against
TOOLS_BY_NAME via two top-level for-loops that throw on the first
missing name. Module fails to import if a tier references a tool that
doesn't exist in the registry — catches typos and stale tier
definitions at boot rather than silently filtering valid tools out of
agent whitelists.

Wiring: agents.ts parseAgentBlock now reads BOOCODE_TOOLS from
process.env per parse, intersects with the agent's declared frontmatter
tools (or DEFAULT_TOOLS when frontmatter omits the field). Per-parse
read is fine — agents are re-parsed on the existing 60s cache TTL.

Tests: tools.test.ts grows from 1 to 10 tests. Covers resolveToolTier
across tiers/case/unknown values + the CORE-subset-of-STANDARD invariant
+ TOOLS_BY_NAME existence for both tier sets. 204/204 pass (was 195;
+9 new).

Deviation from the brief: the codecontext tools in the actual registry
have NO codecontext_* prefix (the brief's STANDARD list assumed it).
Used the actual names (get_codebase_overview, search_symbols, etc.).
Module-load validation would have failed boot with the prefixed names.

Smoke: with BOOCODE_TOOLS unset, agents return their full 12-tool
whitelists. With BOOCODE_TOOLS=core in .env + container restart, the
same agents narrow to 4 tools (find_files, grep, list_dir, view_file)
— intersection of declared whitelist ∩ core tier. Reverted after
confirmation.

CLAUDE.md updated with BOOCODE_TOOLS in the Environment section's
Optional list. .env.example gained a commented BOOCODE_TOOLS=all line
with the per-tier token-cost table.

~110 LoC across 5 files (4 modified + 1 test expansion). Under the
brief's ~30 LoC estimate for code; the test suite expansion drove
most of the growth.
v1.13.11-tools
2026-05-22 14:59:01 +00:00
5a3f357ce9 v1.13.15-openspec: reformat batch docs to OpenSpec directory structure
Adopt Fission-AI/OpenSpec's openspec/changes/<change-name>/{proposal,
specs,design,tasks}.md shape for BooCode's own batch docs. Zero-dep
documentation reformat; replaces ad-hoc boocode_batchN.md /
handoff_vN.N.N.md convention.

Existing batch docs moved into openspec/changes/archived/ via git mv
(preserves history):
- boocode_batch10.md
- handoff_v1.13.8_prefix_verify.md
- handoff_v1.13.10_per_tool_cost.md

Pre-v1.13.15 docs were NOT split into proposal/tasks/design files. The
work was already shipped; the originals are preserved as archived
snapshots. New v1.13.15+ batches land directly in
openspec/changes/<slug>/proposal.md (+ tasks.md, + design.md when
applicable) per the convention documented in openspec/README.md.

CLAUDE.md gained a one-line pointer to the convention (workflow
section). File grew from 153 → 154 lines, 27,682 → 27,925 chars; both
remain well under the AgentLint hard caps.

specs/ directory is reserved for future OpenSpec CLI adoption (v1.14+).
No CLI dep added in this batch — directory structure only. If/when the
full OpenSpec lifecycle is adopted, that lands as a separate batch.
v1.13.10-openspec
2026-05-22 14:54:17 +00:00
fc11e8dc91 v1.13.15-agentlint: instruction-file audit against AgentLint 31-check standard
Manual audit pass against 0xmariowu/AgentLint's evidence-backed checks
(MIT, drawn from 265 versions of Anthropic's internal Claude Code
system prompt).

Findings and fixes:
- Identity sections ("You are the assistant running inside ...") removed
  from BOOCHAT.md (line 3) and BOOCODER.md (line 5). The model already
  knows where it's running; the openers were emphatic decoration.
- CLAUDE.local.md added to .gitignore (.env was already covered).
  Claude Code's Glob tool ignores .gitignore by default, which means
  any local override file was otherwise readable by any agent walking
  the workspace.
- CLAUDE.md unchanged — already passes all 10 checks. Emphasis density
  0.58/1000 words (under Anthropic's 1.4/1000 endpoint); two IMPORTANT/
  MUST references are load-bearing (tsc-noEmit footgun, v1.13.7
  includeUsage invariant); zero identity sections; zero --no-verify
  references; 27,682 chars (under the 40,000-char silent-drop limit).
  Line count (153) is over the 60-120 target band, but the brief
  explicitly forbids structural rewrites in the audit pass.

Targets not in scope:
- /opt/boocode/AGENTS.md does not exist in this repo (removed in v1.12,
  per CLAUDE.md:152). The global agent registry lives at /data/AGENTS.md
  (bind-mounted from outside the repo); can't be touched by this batch.
- No .github/workflows/ directory — SHA-pin audit (step 8) skipped.

Cumulative effect: model spends fewer tokens parsing instruction-file
ceremony in BOOCHAT/BOOCODER and receives sharper priority signal per
Anthropic's measured-evolution data. Zero code changes.
v1.13.9-agentlint
2026-05-22 14:52:37 +00:00
9ce638c916 v1.13.10: per-tool token cost accounting (rolling 100-call view)
Surfaces per-tool prompt/completion-token rolling averages in
AgentPicker for at-a-glance agent-cost hints. Implementation is a
SQL view on top of messages_with_parts plus a read endpoint and
AgentPicker tooltip extension. No new write site; all source data
already lands via the existing tool-phase.ts:94-95 / error-handler.ts:
109-110 / sentinel-summaries.ts UPDATEs that v1.13.7's includeUsage:
true fix made non-NULL.

(1) schema.sql — new tool_cost_stats view. Window-functions over
messages_with_parts.tool_calls with LATERAL jsonb_array_elements.
Attribution: equal split — multi-tool turn divides tokens N-ways;
the 100-call rolling mean absorbs split noise. Filters: status=
'complete' + metadata.kind NOT IN ('cap_hit','doom_loop') exclude
failed turns and sentinels respectively; tool_calls IS NOT NULL is
defense-in-depth since sentinels are role='system' rows. CREATE OR
REPLACE means schema apply is idempotent.

(2) routes/tools.ts NEW + index.ts wire-in. GET /api/tools/cost_stats
returns { stats: ToolCostStat[] } with mean_prompt_tokens / mean_
completion_tokens computed at read time (sum / n_calls). Sorted by
tool_name ASC. No pagination — ≤30 tools.

(3) __tests__/tool_cost_stats.test.ts NEW — 7 integration tests
keyed off DATABASE_URL env var. Tests skip gracefully when unset
(no-DB default). beforeAll applies the schema via sql.unsafe(read
FileSync(schema.sql)) for self-contained runs. Helper insertAssistant
Turn shared across cases. Covers: empty state, single-tool attribution,
multi-tool equal split, 100-call FIFO window, NULL-tokens exclusion,
parts-authoritative read via messages_with_parts, failed/sentinel
exclusion.

(4) web/api/types.ts + client.ts — ToolCostStat interface + api.tools.
costStats() method binding.

(5) AgentPicker.tsx — fetch costStats on mount, compute per-agent
sum-of-means across whitelisted tools, render muted cost line below
description: "~5.2k prompt / 280 completion · 6/8 tools · last call
3h ago". Skips line entirely when no tool history; preserves existing
native title= for layout backward-compat. formatK/formatAgo colocated.

Tests: 202/202 pass (195 prior + 7 new view-integration). Server +
web tsc clean.

Smoke: schema applied cleanly; GET /api/tools/cost_stats returns
canonical JSON; view + endpoint agree. Single-row result expected
given the v1.13.1-A → v1.13.7 NULL latent regression window; new
traffic populates organically.

Roadmap row at boocode_roadmap.md:114 plus schema row at :474 both
match. View vs table decision documented in handoff_v1.13.10_per_
tool_cost.md (rollback-safe, microsecond-fast at BooCode scale).

~270 LoC across 8 files (5 modified + 3 new).
v1.13.8-tool-cost
2026-05-22 14:42:09 +00:00
8126d78b34 docs: capture v1.13.7-v1.13.9 invariants in CLAUDE.md
Five additions surfacing session-discovered constraints future Claude
sessions need:
- AI SDK v6 includeUsage:true requirement (avoids re-introducing the
  v1.13.1-A→v1.13.7 NULL-tokens regression)
- \n text-delta trim guards in MessageList/MessageBubble + payload.ts
  failed/empty-assistant skip rules (avoid undoing v1.13.7)
- 0.85 × ctx_max overflow formula (v1.13.9) replacing the stale
  ctx_max - 20k line
- New services/system-prompt.ts bullet documenting the v1.13.8
  fingerprint instrumentation surface
- New services/inference/budget.ts bullet with current BUDGET_NO_AGENT=30
  and read-only-tools rationale
2026-05-22 14:07:11 +00:00
b06a4a8e55 v1.13.9: compaction overflow trigger — 0.85 × ctx_max early trigger
Opencode pattern (session/overflow.ts): fire compaction at 85% of
ctx_max, replacing the v1.11.0-era `ctx_max - 20_000` formula.

Old formula: usable = ctx_max - 20_000
  - ctx=262144 → trigger at 242144 (92.4%) — only 7.6% headroom
  - ctx=100000 → trigger at  80000 (80.0%)
  - ctx= 32000 → trigger at  12000 (37.5%) — over-eager
  - ctx<=20000 → trigger at      0 — never fires

New formula: usable = floor(0.85 * ctx_max)
  - ctx=262144 → trigger at 222822 (85.0%) — 15% headroom for summarizer
  - ctx=100000 → trigger at  85000 (85.0%)
  - ctx= 32000 → trigger at  27200 (85.0%)
  - ctx=  8192 → trigger at   6963 (85.0%)

Ratio gives consistent headroom at any context scale. The qwen3.6
daily driver gets ~19k tokens more breathing room before overflow;
small-ctx models no longer degenerate to never-triggering.

usable() is the only consumer of COMPACTION_BUFFER → constant deleted.
New EARLY_TRIGGER_RATIO constant takes its place.

isOverflow() and the maybeFlagForCompaction() call site at
payload.ts:184 are unchanged — formula swap is internal to compaction.ts.
payload.ts comment touched only to drop the stale COMPACTION_BUFFER
reference (PRUNE_TRIGGER_TOKENS stays at 20k as the prune-freed
threshold; independent of the overflow formula).

Tests: 4 new usable() corner cases (262k/100k/8k/zero+negative), plus
5 isOverflow() numbers shifted to match the 85k budget at ctx=100k.
195/195 server tests pass (was 194).

Smoke: ratio math verified by unit tests at all four corners. Live
cap-hit verification deferred — requires accumulating >222k tokens
in a session under qwen3.6-35b-a3b-mxfp4 (was >242k pre-fix); will
surface organically in extended use.
v1.13.7-compaction-trigger
2026-05-22 13:59:14 +00:00
a0c8d212cb v1.13.8: system-prompt prefix stability verify-and-measure
Recon during planning disproved the original v1.13.7 (DB-cache) premise:
buildSystemPrompt already runs over inputs mtime-cached at the file layer
(BOOCHAT.md in system-prompt.ts:25, AGENTS.md global+per-project in
agents.ts:245), and DB scalars are byte-stable until edited. The output
is microsecond pure-string concat with no I/O. Skills aren't in the
prefix; tools live in a separate request body field alpha-sorted by
v1.13.3.

This batch closes the verification gap with instrumentation, not
implementation:

- system-prompt.ts: buildSystemPromptWithFingerprint canonical impl
  computes SHA-256 over the assembled prefix, runs a per-session
  Map<sessionId, lastHash> observer, emits PrefixFingerprint per call
  and PrefixDrift (with field-level changed_inputs) on hash change.
  buildSystemPrompt is now a thin shim returning .prompt.
- agents.ts: getAgentsMtimes accessor — cache-read only, no I/O.
- payload.ts: buildMessagesPayload takes optional log argument; when
  passed, emits prefix-fingerprint (info) + prefix-drift (warn).
- turn.ts + sentinel-summaries.ts: pass ctx.log at 3 production call
  sites; sentinel summaries log too so any drift across cap-hit /
  doom-loop paths surfaces.
- system-prompt.test.ts: 4 new tests (byte-identical, no-drift-on-
  stable, drift-fires-with-changed-inputs, cross-session-no-drift).

194/194 tests pass (was 190).

Smoke: 5 messages in a fresh session produced 7 prefix-fingerprint
logs (extras from buildMessagesPayload being called from sentinel
summary paths), all with identical prefix_hash and prefix_length=2907,
zero prefix-drift. Prefix is byte-stable in steady-state.

Decision: original system_prompt_cache DB table from the roadmap is
permanently dropped. The v1.12.0 mtime caches at the input layer plus
alpha tool ordering at the request body (v1.13.3) already address the
load-bearing cache-stability surfaces. Instrumentation stays so the
claim can be re-verified at any time.
v1.13.6-prefix-stability
2026-05-22 13:42:18 +00:00
0ce6115976 docs: renumber v1.13.8 to verify-and-measure, drop system_prompt_cache table, add v1.13.8 dispatch brief 2026-05-22 13:24:29 +00:00
ff29b48e3a v1.13.7: stability bundle — usage capture + payload/UI sanitization
Five fixes for latent regressions surfaced during the v1.13.x.cosmetic
revert investigation. None alter schema or compaction; all cleanup
against the v1.13.1-A AI SDK migration's hidden surface.

(1) provider.ts — includeUsage: true on createOpenAICompatible.
@ai-sdk/openai-compatible defaults this false, omitting
stream_options.include_usage from the request body; llama-swap never
emitted the usage block, so result.usage.inputTokens/outputTokens
resolved undefined and tokens_used / ctx_used landed NULL in every
assistant row since v1.13.1-A. No historical backfill.

(2) MessageList.tsx — hasText = m.content.trim().length > 0.
AI SDK v6 streaming occasionally emits a leading "\n" text-delta on
tool-call-only turns; the literal newline passed length > 0 and
rendered an empty bubble + ActionRow between every tool call. Trim
catches it without changing semantics for genuine content.

(3) MessageBubble.tsx — same trim on hasContent for the no-tool-calls
path. Defensive symmetry with MessageList.flatten.

(4) payload.ts — buildMessagesPayload skips assistant rows with
status='failed' AND assistant rows with status='complete' + empty
content + no tool_calls. Without this, a trailing empty/failed
assistant + the next attempt's placeholder produced "Cannot have 2
or more assistant messages at the end of the list" rejections from
the OpenAI-compatible upstream after cap-hit + Continue.

(5) budget.ts — BUDGET_NO_AGENT 15 → 30. Every tool in ALL_TOOLS is
read-only today; the 15-cap was forward-looking for write tools that
haven't landed. No-agent mode now matches BUDGET_READ_ONLY.

47 LoC across 5 files. 190/190 server tests pass.

Verified live: new assistant turns populate StatsLine token data;
single-tool-call turns no longer render the stray empty-bubble +
ActionRow between tool calls; Continue after cap-hit no longer hits
the trailing-assistant API rejection.
v1.13.5-stability-bundle
2026-05-22 13:24:19 +00:00
81d837c04e v1.13.6: compaction head-assembly audit + reasoning fix
Audit traced compaction's summary path post-v1.13.1-B read flip:
- Q1: reads from messages_with_parts (view) — clean
- Q2: parts shape correctly threaded through buildHeadPayload — clean
- Q3: reasoning omitted from summary input — FIX NEEDED

v1.13.1-C wired reasoning end-to-end into inference/payload.ts but
missed this read site. Summarizer model couldn't see the reasoning
trail for tool-bearing turns, quietly degrading summary quality for
reasoning-channel models (qwen3.6).

Fix:
- CompactionMessage extended with reasoning_parts field
- SELECT pulls reasoning_parts from messages_with_parts
- buildHeadPayload (now exported for tests) prefixes assistant content
  with <reasoning>...</reasoning>\n\n<content>... when reasoning is
  present; standalone <reasoning>...</reasoning> for tool-call-only
  turns; omits the tag when reasoning is null or empty

4 new render branch tests (190 total).

Smoke deferred: forcing real compaction requires either threshold
pollution or building up a >40k-token chat with reasoning_parts.
Render branches are unit-covered; integration would only re-prove
structural correctness.
v1.13.4-reasoning-fix
2026-05-22 08:18:47 +00:00
f8fc5db929 v1.13.5: opencode truncate.ts port — full tool output retrievable via opaque id
- New services/truncate.ts. Tmpfs storage at /tmp/boocode-truncations/
  (BOOCODE_TRUNCATION_DIR env var overrides for tests). 12-char base32
  opaque ids (~60 bits entropy, "tr_<id>"). Three exports: storeTruncation,
  readTruncation, truncateIfNeeded (wrap-or-passthrough helper).
  cleanupTruncations does TTL-pass (7 days) + orphan-reap (parts query on
  payload->'output'->>'outputPath') in one shot.
- Wired four tools through truncateIfNeeded: view_file (raw full file),
  list_dir (full filtered+secret-filtered entries serialized one-per-line),
  web_fetch (textRaw pre-slice), codecontext_client (body.result pre-slice).
  Each returns the existing sliced view plus an optional outputPath field
  when truncation fires.
- New view_truncated_output ToolDef. Resolves opaque id → on-disk content
  internally; model never sees the truncation dir. Same start_line /
  end_line slicing semantics as view_file. Registered in ALL_TOOLS (alpha
  sort places it after view_file automatically) and READ_ONLY_TOOL_NAMES.
- cleanupTruncations piggybacks on the v1.13.3 stuck-row sweeper's 60s
  setInterval. No-op when truncation dir is empty.

Not wired (TODO follow-up): grep and find_files. file_ops returns post-cap
results to the tool execute path, so the "full content" isn't recoverable
without a refactor of fileOps.grep / fileOps.findFiles to expose the
uncapped result. web_search is silent-slice (no truncated flag); outside
scope. Five sites of seven covered; the remaining two are the only ones
needing a file_ops change.

Tests: 7 new in truncate.test.ts (roundtrip, unknown id, malformed id,
truncateIfNeeded false/true/over-cap/storage-failure paths). 186 total
(was 179). cleanupTruncations file-system half implicitly via TTL pass;
orphan-reap branch covered by the live container smoke.

Smoke verified end-to-end against the live container:
- view_file with start_line=1, end_line=3 on CLAUDE.md → tool_result part
  carried outputPath "tr_cdpn1o04k6ma" + truncated=true.
- /tmp/boocode-truncations/tr_cdpn1o04k6ma exists, 15876 bytes, mode 0o600,
  parent dir mode 0o700.
- Follow-up view_truncated_output(id, start_line=50, end_line=55) returned
  the actual lines 50-55 of CLAUDE.md (the 808notes/BooCode bullets).
- ALL_TOOLS count=20 (was 19); alpha sort places view_truncated_output
  between view_file and watch_changes.

Closes a v1.12 catalog row that was scoped but deferred. The v1.13 parts
table made outputPath ride on the existing tool_result payload with no
schema change beyond the storage helper itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.3-truncate
2026-05-22 07:55:55 +00:00
ec8593cf77 v1.13.4: two-tier compaction prune — opencode pattern half-shipped in v1.11.0
- message_parts.hidden_at timestamptz column (NULL by default) with a
  partial index on (message_id) WHERE hidden_at IS NULL for the common
  visible-parts filter.
- messages_with_parts view changed from COALESCE(parts, legacy) to
  CASE WHEN EXISTS(any parts of kind) THEN visible-parts ELSE legacy.
  COALESCE would have leaked hidden parts back via the legacy fallback
  when every part was pruned (smoke caught it pre-commit). The CASE
  distinguishes "no parts at all → fall back to legacy column for
  pre-v1.13.0 history" from "all parts hidden → return null/empty so
  the row drops out of the model payload" exactly.
- prune.ts: scans tool_result parts newest-first, protects the last 40k
  tokens (PROTECTED_TOKENS), marks older candidates hidden when their
  combined estimate clears 20k (PRUNE_TRIGGER_TOKENS — equal to
  COMPACTION_BUFFER from v1.11.0, so a successful prune is exactly the
  budget the summary path would have freed). Stops at chats.tail_start_id
  so it doesn't double-erase across the last summary boundary. Pure
  decision helper selectPruneTargets exported separately for unit tests.
- Wired into maybeFlagForCompaction: prune runs synchronously when
  overflow is detected; if it freed >= PRUNE_TRIGGER_TOKENS, the
  needs_compaction flag is NOT set and the (expensive) summary inference
  call is skipped this turn. The next turn's overflow check re-evaluates
  from scratch.
- 6 new unit tests in prune.test.ts cover: empty input, protection-only
  (no candidates), candidates below trigger, candidates above trigger,
  candidates straddling a summary boundary, exactly-protection-tokens.
  179 tests total (was 173).

Smoke verified post-rebuild:
- \\d message_parts shows hidden_at + partial index.
- View definition shows AND p.hidden_at IS NULL filters on all three
  subselects.
- Synthetic hide-then-restore confirmed the view drops the tool_result
  jsonb to null when its only part is hidden, and restores when un-hidden.
- EXPLAIN ANALYZE on the 42-message stress chat: 0.325ms (faster than
  v1.13.1-B's 1.018ms — EXISTS short-circuits cleanly for the common
  no-parts case).
- Normal turn (plain text prompt) completes unaffected.

Closes a v1.11.0 design item that was scoped but never implemented. With
v1.13's parts table the prune is dramatically cheaper to write — pre-parts
it would have meant editing JSON blobs in-place; now it's a hidden_at
flag and a view subselect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.2-compaction-prune
2026-05-22 07:02:17 +00:00
a08d809b73 v1.13.3: cleanup bundle — statement timeout + alpha ordering + stuck-row sweeper + repairToolCall
Four independent items, all owed from prior dispatches.

- statement_timeout at the database level via:
    ALTER DATABASE boocode SET statement_timeout = '30s';
  Applied operationally; documented as a comment at the top of schema.sql
  (ALTER DATABASE can't run inside a DO block, so it's not idempotent
  inside applySchema). Re-apply after a volume reset.

- Tool registry alpha-sorted at module load. llama.cpp's prompt cache
  hits on byte-identical prefixes; any reordering of the tool list near
  the top of the system prompt would invalidate every cached turn.
  Single-source sort at the ALL_TOOLS export so toolJsonSchemas() and
  TOOLS_BY_NAME inherit the order automatically. New tools.test.ts
  asserts the invariant; total tests 173 (was 172).

- Periodic in-process stuck-row sweeper. Runs every 60s, marks
  'streaming' rows older than 5 minutes as 'failed', and publishes
  chat_status='idle' on the user channel so the UI dot drops without a
  refresh. Closes the mid-session crash UX gap; the v1.12.1 boot sweep
  only fires once at startup, so sessions used to stay stuck until next
  reboot. setInterval cleaned up via app.addHook('onClose'). Mirrors
  handleAbortOrError's publish pattern.

- experimental_repairToolCall wired through AI SDK v6 streamText. Pass-
  through implementation: log + return the original toolCall so the
  stream keeps going. executeToolPhase's existing error paths (unknown
  tool name → 'unknown tool: X' result; zod-reject → 'tool X rejected
  — field: required') already surface bad calls to the model; the value
  here is preventing the AI SDK from THROWING on parse errors and
  killing the whole stream. Owed since v1.13.1-A.

Smoke verified:
- statement_timeout = '30s' confirmed via SHOW.
- Tool path normal flow intact (list_dir prompt → tool_call → result
  → final assistant). No malformed tool calls in the test run; repair
  log will surface them when qwen3.6 actually emits one.
- Alpha order verified at runtime via the dist bundle: match: true.
- Sweeper logic not traffic-tested (no stuck rows to find), but the
  SQL UPDATE + broker.publishUser pattern is identical to handleAbort
  and the boot sweep — synthesis-only verification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.1-cleanup-bundle
2026-05-22 06:46:03 +00:00
ac1a71f583 v1.13.1-C: port ask_user_input correlation to parts + wire reasoning_parts end-to-end
Pass 1 — ask_user_input correlation port (messages.ts:478, :549):

- The two correlation queries that backed the elicitation flow used to scan
  messages.tool_calls and messages.tool_results JSON columns directly. They
  now JOIN message_parts on payload->>'id' (for the caller assistant) and
  payload->>'tool_call_id' (for the pending tool row). Semantics preserved:
  ORDER BY m.created_at DESC LIMIT 1 still picks the latest issuance, the
  already-answered 409 guard now reads payload.output, and the UPDATE +
  parts replace inside sql.begin is unchanged from v1.13.0.
- Pre-v1.13.0 history has no parts rows and is unreachable to this lookup
  path (404). Acceptable per dispatch decision — no pending elicitation
  from before v1.13.0 will still be open. JSON-column fallback can land as
  a hotfix if it ever surfaces.

Pass 2 — reasoning_parts wired end-to-end:

- types.ts/StreamResult gains `reasoning: string`. stream-phase.ts accumulates
  reasoning-delta text per stream (replacing the v1.13.1-A counter-only
  diagnostic) and returns it on the result.
- parts.ts/partsFromAssistantMessage gains an optional `reasoning` param.
  When present it emits a kind='reasoning' part at sequence 0, ahead of
  the text and tool_call parts.
- error-handler.ts/finalizeCompletion and tool-phase.ts/executeToolPhase
  both thread result.reasoning into the dual-write call so reasoning-channel
  models (qwen3.6) get persistent reasoning rows.
- payload.ts: loadContext SELECT pulls reasoning_parts from the v1.13.1-B
  view; OpenAiMessage gains an optional `reasoning` field; buildMessagesPayload
  collapses reasoning_parts into a single string per assistant message.
- stream-phase.ts/toModelMessages converts assistant messages with reasoning
  into an AI SDK ModelMessage content array starting with a ReasoningPart,
  matching the @ai-sdk/provider-utils AssistantContent union. Reasoning
  models can now replay prior reasoning context across tool-call boundaries.
- types/api.ts and apps/web/src/api/types.ts Message interface gain
  reasoning_parts (optional, nullable). Frontend doesn't render this yet —
  field reserved for a v1.14 UI surface.

Tests: 2 new in parts.test.ts cover reasoning-at-sequence-0 with and
without text content. 172 tests pass (170 prior + 2 new).

Smoke verified against the live container:
- A reasoning-prompt ("walk through 17 × 23 step by step") produced one
  message with kind='reasoning' (361 chars) at sequence 0 and kind='text'
  (429 chars) at sequence 1. Adapter log confirmed reasoning capture.
- The new correlation SQL was validated against existing tool_call /
  tool_result parts: returns the expected message_id + payload shape with
  pending state correctly identified via payload.output IS NULL.
- ask_user_input end-to-end through the UI is Sam's smoke — the Prompt
  Builder agent does not always trigger ask_user_input for these prompts,
  so synthetic verification via SQL substituted for traffic-driven cover.

Annotation: the v1.13.1-A abort-throw site in stream-phase.ts got a
one-liner comment ("AI SDK v6 fullStream returns normally on abort; check
signal explicitly.") to prevent a future refactor removing it.

v1.13.2 drops the dual-write + the JSON columns + collapses the view.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.13.0-ai-sdk-v6
2026-05-22 06:34:10 +00:00
13c3aa5b4e v1.13.1-B: read-path flip from tool_calls/tool_results JSON columns to message_parts
- schema.sql: new messages_with_parts view. tool_calls aggregates parts
  with kind='tool_call' as a jsonb array of {id, name, args}; tool_results
  picks the single sequence=0 part with kind='tool_result' as a jsonb
  {tool_call_id, output, truncated, error?}. COALESCE against the legacy
  jsonb columns means pre-v1.13.0 history (no parts rows) still reads
  correctly via the fallback, and fresh inserts (where parts dual-write
  follows the row INSERT) hit the legacy columns until the parts land.
- reasoning_parts column added to the view but not selected by any caller
  yet — v1.13.1-C extends the Message type and pulls it into the model
  payload alongside the type extension.
- Read sites switched to FROM messages_with_parts:
  - routes/chats.ts:427 (chat history GET)
  - routes/messages.ts:95 (session history GET)
  - routes/ws.ts:27 (WS snapshot on session connect, resume path)
  - services/inference/payload.ts (loadContext for model assembly)
  - services/compaction.ts (compaction's payload assembly)
- chats.ts:394 (discard_stale UPDATE RETURNING) unchanged — UPDATEs target
  messages directly and the returned shape is for a freshly-modified row
  where the legacy column is dual-written and correct.
- messages.ts:478/549 (ask_user_input correlation) intentionally not
  migrated — those query a different shape, ported in v1.13.1-C.
- Writes still target `messages` directly; the view is read-only.

Smoke verified against the live container:
- Equivalence: 5/5 messages with both legacy column and parts row return
  identical tool_calls jsonb between FROM messages and FROM messages_with_parts.
- Perf: EXPLAIN ANALYZE on the 42-message stress chat returns in ~1ms
  (50ms threshold). Bitmap Index Scan on message_parts_msg_seq_idx
  carries the parts lookups.
- API contract: GET /api/chats/:id/messages returns identical
  {id, name, args} tool_calls and {tool_call_id, output, truncated, error}
  tool_results shapes to frontend consumers — no UI changes needed.
- Inference path: sent a view_file prompt; assistant turn 1 emitted the
  tool_call, tool message captured the result, follow-up assistant turn
  read the result back via loadContext (now view-backed) and answered
  correctly. End-to-end loop intact.

v1.13.2 drops the dual-write + the JSON columns + simplifies the view
to just SELECT FROM message_parts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 06:22:47 +00:00
c2c4f78a26 v1.13.1-A: install AI SDK v6 + swap streamText into stream-phase.ts adapter
- Add ai@^6 and @ai-sdk/openai-compatible@^2 to apps/server.
- New services/inference/provider.ts: createOpenAICompatible against
  llama-swap (baseURL threaded from config.LLAMA_SWAP_URL, cached per
  baseURL). No apiKey — Authelia + Tailscale gate llama-swap, not keys.
- streamCompletion rewritten as an adapter over streamText. AI SDK
  fullStream parts (text-delta, tool-call, finish, error) map back to
  the legacy {content?, tool_calls?, finishReason} StreamResult shape
  that executeStreamPhase already consumes. No layer above
  streamCompletion changes.
- toModelMessages converts BooCode's OpenAI-shaped history to AI SDK
  ModelMessage[]; tool messages need toolName which we look up by
  scanning earlier assistant tool_calls for the matching id.
- buildAiTools wraps BooCode's JSON-schema tool defs via
  tool({ inputSchema: jsonSchema(parameters) }) with NO execute —
  BooCode dispatches tools in tool-phase.ts, not the AI SDK loop.
- XML fallback parser preserved as-is — qwen3.6 still emits XML tool
  calls in text content that the structured tool-call layer misses.
- reasoning-delta parts dropped with a debug-level counter — captured
  properly in v1.13.1-C.
- Abort path: streamText({ abortSignal }) wires ctx.signal through, but
  AI SDK v6 swallows the abort (fullStream iterator exits cleanly
  rather than throwing). Post-iteration `if (signal?.aborted) throw` so
  handleAbortOrError owns the row and writes status='cancelled'. Caught
  by smoke D; would have shipped as status='complete' on stop otherwise.
- Usage frame reads result.usage (inputTokens / outputTokens v6 names)
  AFTER stream drain. Single trailing publish through the existing 500ms
  throttle. Known regression: ChatThroughput's live mid-stream tick
  (v1.12.2) is gone — it now shows a single value at stream end.
  TODO(v1.13.1-followup): interpolate outputTokens during streaming
  via a delta-cadence counter (e.g. part.text.length/4 token proxy)
  and publish every 500ms; reconcile against result.usage at finish.
- Write-path dual-write from v1.13.0 unaffected.

Read path stays on JSON columns. v1.13.1-B flips reads to message_parts.

Smoke verified end-to-end against running container:
- A. Plain text: status='complete', 1 text part.
- B. Single tool prompt → multi-tool chain (4 calls): every assistant
     with tool_calls has 2 parts (text+tool_call), every tool row has
     1 part (tool_result).
- C. Multi-step covered by B's chain.
- D. Stop mid-stream: status='cancelled' written via handleAbortOrError
     after the post-iteration abort throw.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 06:17:56 +00:00
1cb6eee24c v1.13.0: message_parts table + dual-write at every tool_calls/tool_results site
Adds a granular message_parts table (one row per text/tool_call/tool_result
chunk) without changing any read path. Old messages.content / tool_calls /
tool_results columns remain authoritative for v1.13.0; this dispatch is
write-only mirroring so the AI SDK migration in v1.13.1 can flip read
authority without a backfill window.

Schema:
  CREATE TABLE message_parts (id, message_id FK ON DELETE CASCADE,
    sequence int, kind text CHECK (text|tool_call|tool_result|reasoning|step_start),
    payload jsonb, created_at, UNIQUE (message_id, sequence))

New module services/inference/parts.ts with two pure derive helpers
(partsFromAssistantMessage, partsFromToolMessage) and insertParts that
fan-outs a multi-row INSERT via postgres-js.

Wired dual-write at every site that writes tool_calls or tool_results:
- tool-phase.ts: assistant finalize UPDATE, executed-tool UPDATE,
  ask_user_input sentinel UPDATE
- messages.ts answer flow: DELETE pending tool_result part + INSERT
  answered one inside the existing sql.begin
- skills.ts: synthetic assistant + tool INSERTs both inside existing tx
- chats.ts fork: CTE clones parts via ROW_NUMBER pairing (source→dest
  message id mapping in one statement, no N+1)
- error-handler.ts finalizeCompletion: text part for plain text-only
  assistant turns

Deviation: tool-phase.ts finalize UPDATEs and finalizeCompletion text-part
write are not wrapped in fresh sql.begin transactions. Safe in v1.13.0
because JSON columns are authoritative for reads. v1.13.1 must wrap these
sites before flipping read authority — TODO comments added at each
unwrapped site referencing v1.13.1.

Tests: 8 new unit tests for the derive helpers in
services/__tests__/parts.test.ts. Existing 162 tests untouched. 170 total.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 05:46:29 +00:00
ca64bf9f0a docs: CLAUDE.md updates from /claude-md-management session
- services/inference.ts → services/inference/ directory map (v1.12.4 split)
- workspace_panes server-side jsonb (was: localStorage-only line)
- chat_status 5-state model + ChatThroughput + discard_stale endpoint
- boot-time stale-streaming sweep documented
- WS frame sync gotcha (server InferenceFrame ↔ web WsFrame)
- session_panes table noted as dropped (not deprecated)
- messages_status_check/role_check drift cleanup noted

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 05:46:14 +00:00
9ef00c0268 v1.12.4: complete inference.ts split into services/inference/
- sentinel-summaries.ts: runCapHitSummary, insertCapHitSentinel,
  runDoomLoopSummary, insertDoomLoopSentinel
- inference.ts → inference/turn.ts: residue is runAssistantTurn,
  runInference, createInferenceRunner orchestration only
- inference/index.ts: re-export shim preserves the public surface
  (createInferenceRunner, runInference, runAssistantTurn,
  detectDoomLoop, DOOM_LOOP_THRESHOLD, buildMessagesPayload, plus
  type-side InferenceContext/InferenceFrame/StreamResult/TurnArgs/
  FramePublisher)
- src/index.ts + auto_name.ts + the two vitest test files updated to
  import from ./services/inference/index.js explicitly (NodeNext ESM
  doesn't honor directory-index resolution)

Final tally: 11 files under services/inference/, the largest being
sentinel-summaries.ts at 523 LoC (two near-clone summary paths kept
side-by-side until a third sentinel justifies factoring out a shared
runWrapUpSummary). turn.ts is now 326 LoC, the next-largest is
stream-phase.ts at 380. Public import surface unchanged.

tool-phase.ts → turn.ts back-edge for runAssistantTurn remains
(cycle is safe; resolved at call time).

Prepares the file structure for v1.13 AI SDK migration — streamText
swap targets stream-phase.ts only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
v1.12.4-inference-split
2026-05-21 22:36:35 +00:00
c87df6981a v1.12.4-rc3: extract stream-phase + tool-phase from inference.ts
- stream-phase.ts: streamCompletion, executeStreamPhase (plus sseLines,
  StreamOptions, ChatCompletionDelta/Chunk as private helpers)
- tool-phase.ts: executeToolPhase + private executeToolCall
- types.ts: shared StreamPhaseState + DB_FLUSH_INTERVAL_MS so the
  summary functions still in inference.ts can reference them without
  pulling from a phase file

Cycle: executeToolPhase recurses into runAssistantTurn, which stays in
inference.ts. Resolved by direct value back-edge — tool-phase.ts does
`import { runAssistantTurn } from '../inference.js'` and runAssistantTurn
is now exported. Safe because the dereference happens inside an async
function body, after both modules have fully evaluated. No
callback-through-args fallback needed.

inference.ts shrinks from ~1401 to ~828 LoC. Final Dispatch D moves the
sentinel summaries out and renames the residue to inference/turn.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 22:28:23 +00:00
8fa7b7fce9 v1.12.4-rc2: extract payload + error-handler from inference.ts
- payload.ts: buildMessagesPayload (re-exported), loadContext,
  maybeFlagForCompaction
- error-handler.ts: handleAbortOrError, finalizeCompletion

Both new files type-import InferenceContext/StreamResult/TurnArgs from
inference.ts; ESM elides type imports so there's no runtime cycle.
handleAbortOrError turned out not to call the summary functions, so
no back-edge needed.

inference.ts shrinks from ~1676 to ~1401 LoC.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 22:09:50 +00:00
ea468ca7fb v1.12.4-rc1: extract budget, sentinels, xml-parser from inference.ts
Pure file moves. No behavior change. inference.ts retains createInferenceRunner
public surface; new files are internal to services/inference/.

- budget.ts: resolveToolBudget
- sentinels.ts: detectDoomLoop (re-exported through inference.ts),
  isCapHitSentinel, isDoomLoopSentinel, isAnySentinel
- xml-parser.ts: parseXmlToolCall, partialXmlOpenerStart

First of four refactor batches preparing inference.ts for the v1.13
AI SDK migration. inference.ts goes from 1780 LoC to ~1620.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:42:41 +00:00