Compare commits
4 Commits
v1.13.14-s
...
v1.13.17-c
| Author | SHA1 | Date | |
|---|---|---|---|
| b52c5df705 | |||
| 2e1a81de72 | |||
| 61308cf17c | |||
| 3992a9fcb7 |
171
CHANGELOG.md
Normal file
171
CHANGELOG.md
Normal file
@@ -0,0 +1,171 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes per release tag. Most recent on top, ordered by tag creation date (which matches the git history). Tag names follow `vMAJOR.MINOR.PATCH-slug` — the slug describes what shipped, so the tag name alone is enough to recall the batch.
|
||||||
|
|
||||||
|
## v1.13.17-cross-repo-reads — 2026-05-22
|
||||||
|
|
||||||
|
On-demand read access to paths outside the session's primary project root. Closes the dead-end where `pathGuard` rejected every cross-repo read with no recovery path. New `request_read_access(path, reason)` tool emits an `ask_user_input`-style pause; user picks Allow/Deny via inline chips in `RequestReadAccessCard.tsx`; on Allow, the new `POST /api/chats/:id/grant_read_access` endpoint re-resolves the grant root and appends to `sessions.allowed_read_paths` (new `TEXT[]` column, default empty). Grant unit per design D1 = nearest registered `projects.path` ancestor → else nearest repo-shaped ancestor (`.git/` / `package.json` / `go.mod` / `Cargo.toml`) under `PROJECT_ROOT_WHITELIST` → else refuse without prompting. `pathGuard` extended with an optional `extraRoots` argument threaded from `session.allowed_read_paths` through `executeToolCall` to the four filesystem tools (view_file, list_dir, grep, find_files); `view_file` re-anchors the secret-guard check on `basename(real)` whenever the path resolved via a grant root so `.env` / `id_rsa*` deny still fires across grants. `grant_resolver.ts`'s ancestor walk checks the whitelist invariant on every iteration (not just final parent) so a symlinked input can't escape mid-walk. PATCH `/api/sessions/:id` exposes `allowed_read_paths` only for revocation: zod refines paths to absolute + no traversal markers, and a runtime subset guard (`findUnauthorizedAdditions`) rejects any entry not already present in the row, so a malicious `curl -X PATCH -d '{"allowed_read_paths":["/etc"]}'` 400s instead of bypassing the grant flow. Settings pane gains a per-session revoke list; archiving the session clears grants implicitly. 11 grant_resolver tests pin the symlink-escape-mid-walk guard (Sam's checkpoint-1 ask) and the nearest-project disambiguation; 8 path_guard tests cover extraRoots traversal; 8 sessions PATCH tests cover the subset guard including the `/etc` bypass attempt. Pairs with `v1.13.16-xml-parser` (model now both self-recovers from a wrong tool name AND from a refused path).
|
||||||
|
|
||||||
|
## v1.13.16-xml-parser — 2026-05-22
|
||||||
|
|
||||||
|
Two-part fix for the model-emitted XML drift the v1.13.15 investigation surfaced. **Parser extension:** `xml-parser.ts` now recognizes the Anthropic `<invoke name="…"><parameter name="…">…</parameter></invoke>` shape alongside the existing Qwen/Hermes `<tool_call><function=…>…</function></tool_call>` shape. qwen3.6-35b-a3b-mxfp4 drifts to the Anthropic format when prompted as an Architect-style agent (Claude Code documentation in its pre-training corpus). Both formats route through the same synthetic-id `xml_call_${idx}` ToolCall path. The existing Qwen parser was tightened to tolerate whitespace around `=` (`<function = name>` shape) so a stray space doesn't get absorbed into the function name. **Unknown-tool recovery hint:** new `tool-suggestions.ts` exports `levenshtein()` + `suggestToolName()` + `formatUnknownToolError()`. When the dispatcher (`tool-phase.ts:executeToolCall`) receives an unknown tool name, the error returned to the model includes a "Did you mean: X?" hint based on Levenshtein distance ≤3 or substring match against `Object.keys(TOOLS_BY_NAME)`. Targets the qwen3.6 drift to `read_file` → suggest `view_file`. Test coverage in `xml-parser.test.ts` (46 tests, all green) covers both parsers, the partial-opener detector for both flavors, the unified extraction helper, and the new error formatter.
|
||||||
|
|
||||||
|
## v1.13.15-codecontext-synth — 2026-05-22
|
||||||
|
|
||||||
|
Forced second-inference synthesis pass for codecontext overview-class tools (`get_codebase_overview`, `get_framework_analysis`, `get_semantic_neighborhoods`). After the tool result lands, the pipeline expands the truncated head via in-process `readTruncation`, extracts referenced file paths from the full content, auto-fetches top-N files + project docs (BOOCHAT.md, AGENTS.md, *roadmap*.md, CONTEXT.md) under a 32k-token budget with explicit drop-priority order, then streams a synthesis turn that replaces the recursive `runAssistantTurn`. The 32k truncated head still ships to the synth model (token-budget contract preserved); the expansion is reference-extraction-only. Falls through to recursion on timeout (90s), model error, or non-2xx; user-abort marks the synth message `status='failed'` and re-throws (the outer abort handler operates on the parent turn's message, not the new synth row — without explicit marking, the row would sit `streaming` until the 5-min sweeper, tripping the 60s stale-stream banner). Adds `'synthesis'` to `message_parts.kind` CHECK constraint via `DROP CONSTRAINT IF EXISTS` + `DO $$ pg_constraint` idempotency-guarded re-add. Smokes #1, #2, #6 all clean; smokes #3–#5 are content-quality checks for UI review.
|
||||||
|
|
||||||
|
## v1.13.14-skills-audit — 2026-05-22
|
||||||
|
|
||||||
|
Multi-topic batch. **Skills audit (headline):** vendored all 26 skills from `/home/samkintop/opt/skills/` into repo-local `data/skills/` (the `/opt/skills:/data/skills` override mount removed from `docker-compose.yml` so skills are auditable per-batch in git). Audited via 5 parallel Claude Code agent-teams running mgechev's 4-step protocol per skill — 14 survive with gerund-form names + refined triggers; 11 dropped (duplicates, BooCode-irrelevant patterns, Claude-already-does-natively); 1 (`verification-before-completion`) migrated to `BOOCHAT.md`/`BOOCODER.md` as an always-true rule. The Codeminer42 "rules vs recipes" split codified in those files. **Token tracking + stale-stream banner fix:** same root cause — `IsoTimestamp = z.string()` in `ws-frames.ts` was failing on postgres `Date` objects, silently dropping every `message_complete` / `session_updated` / `chat_updated` frame through the `v1.13.13-ws-publish` Zod gate; `z.preprocess(v => v instanceof Date ? v.toISOString() : v, ...)` applied to the primitive on both server + web (parity test still passes). **Codecontext ignore:** `codecontext_client.ts` auto-installs `.codecontextignore.template` into any project's root on first call (stops the upstream empty-source-file parser crash on foreign projects' `node_modules`). **Budget bump:** `BUDGET_READ_ONLY` + `BUDGET_NO_AGENT` 30 → 50 (real recon need ~27 + headroom for codecontext failure-retry turns; doom-loop guard catches the loop class anyway). **UI:** queued-message dropdown → edit / force-send / cancel buttons in `ChatPane.tsx`; `ChatThroughput` removed from desktop tab strip (mobile tab switcher keeps it). Audit decisions in `openspec/changes/v1.13.12-skills-audit/audit-notes.md`.
|
||||||
|
|
||||||
|
## v1.13.13-ws-publish — 2026-05-22
|
||||||
|
|
||||||
|
Second half of the WebSocket-frame-typing batch. Converts the existing ~50 inference + auto_name publish sites (via the `index.ts` adapter) plus ~30 direct `broker.publish*` call sites in routes + compaction, so every server-emitted frame now goes through Zod validation at the broker boundary. Pairs with `v1.13.12-ws-schemas`.
|
||||||
|
|
||||||
|
## v1.13.12-ws-schemas — 2026-05-22
|
||||||
|
|
||||||
|
First half of the WebSocket-frame-typing batch. Adds `apps/server/src/types/ws-frames.ts` with Zod schemas for all 27 wire-format frame types (discriminated union `WsFrameSchema` + `KNOWN_FRAME_TYPES` diagnostic lookup), duplicated byte-identical at `apps/web/src/api/ws-frames.ts` with a parity test. Introduces the `publishFrame` / `publishUserFrame` wrappers that fail-closed on schema mismatch.
|
||||||
|
|
||||||
|
## v1.13.11-tools — 2026-05-22
|
||||||
|
|
||||||
|
Tiered tool loading via `BOOCODE_TOOLS` env var (`core` | `standard` | `all`). Core = 4 read-only fs tools (~2k token schema cost). Standard = +web + git + codecontext (~10k). All (default) = every tool in `ALL_TOOLS` (~21k). The var is a ceiling — narrows agent whitelists, never expands. Pattern lifted from `eyaltoledano/claude-task-master`.
|
||||||
|
|
||||||
|
## v1.13.10-openspec — 2026-05-22
|
||||||
|
|
||||||
|
Adopt `Fission-AI/OpenSpec`'s `openspec/changes/<slug>/{proposal,tasks,design}.md` shape for BooCode's own batch docs. Existing batch docs (`boocode_batch10.md`, `handoff_v1.13.8_prefix_verify.md`, `handoff_v1.13.10_per_tool_cost.md`) moved into `openspec/changes/archived/` via `git mv` to preserve history. Zero-dep documentation reformat.
|
||||||
|
|
||||||
|
## v1.13.9-agentlint — 2026-05-22
|
||||||
|
|
||||||
|
Manual audit of instruction files against `0xmariowu/AgentLint`'s 31-check standard. Removed identity-opener sections from `BOOCHAT.md` and `BOOCODER.md` (emphatic decoration the model doesn't need). Added `CLAUDE.local.md` to `.gitignore` — Claude Code's Glob ignores `.gitignore` by default, so local overrides were otherwise readable by any agent walking the workspace. `CLAUDE.md` passed all 10 checks unchanged.
|
||||||
|
|
||||||
|
## v1.13.8-tool-cost — 2026-05-22
|
||||||
|
|
||||||
|
Per-tool prompt/completion-token rolling averages surfaced in AgentPicker as at-a-glance cost hints. Implementation is the `tool_cost_stats` SQL view over `messages_with_parts` (`LATERAL jsonb_array_elements` on `tool_calls`), plus a read endpoint and a tooltip extension. Equal-split attribution — multi-tool turn divides tokens N-ways; the 100-call rolling mean absorbs split noise. Filters out `cap_hit` / `doom_loop` sentinels. Source data already lands via existing UPDATEs that `v1.13.5-stability-bundle`'s `includeUsage: true` fix made non-NULL.
|
||||||
|
|
||||||
|
## v1.13.7-compaction-trigger — 2026-05-22
|
||||||
|
|
||||||
|
Compaction overflow trigger lowered to `floor(0.85 × ctx_max)`, replacing the v1.11.0-era `ctx_max − 20_000` formula. Old formula gave only 7.6% headroom at 262k context and 0 budget for ≤20k contexts (never fired). New formula gives consistent 15% summarizer headroom across all model sizes. Opencode pattern lift from `session/overflow.ts`.
|
||||||
|
|
||||||
|
## v1.13.6-prefix-stability — 2026-05-22
|
||||||
|
|
||||||
|
System-prompt prefix stability verify-and-measure. Recon during planning disproved the original DB-cache premise: `buildSystemPrompt` already runs over inputs mtime-cached at the file layer (BOOCHAT.md, AGENTS.md global+per-project), and DB scalars are byte-stable until edited. This batch closes the verification gap with instrumentation, not implementation — `buildSystemPromptWithFingerprint` computes SHA-256 over the assembled prefix and a per-session `Map` observer fires `prefix-drift` (warn) on hash change with field-level `changed_inputs` diff.
|
||||||
|
|
||||||
|
## v1.13.5-stability-bundle — 2026-05-22
|
||||||
|
|
||||||
|
Five fixes for latent regressions surfaced during the cosmetic-revert investigation. (1) `provider.ts` — `includeUsage: true` on `createOpenAICompatible` (default false omitted `stream_options.include_usage`; llama-swap never emitted usage; tokens_used / ctx_used were NULL on every assistant row since `v1.13.0-ai-sdk-v6`). (2) `MessageList.tsx` — `hasText = m.content.trim().length > 0` to skip whitespace-only tool-call-only turns rendering empty bubbles. (3) `BUDGET_NO_AGENT` raised 15 → 30 to match read-only agent cap. (4) `payload.ts` skips status='failed' + complete-but-empty assistant rows so cap-hit + Continue doesn't upstream-reject. (5) Misc UI sanitization.
|
||||||
|
|
||||||
|
## v1.13.4-reasoning-fix — 2026-05-22
|
||||||
|
|
||||||
|
Compaction head-assembly audit caught one fix: reasoning was omitted from the summarizer's view of tool-bearing turns, silently degrading summary quality for reasoning-channel models (qwen3.6). `v1.13.0-ai-sdk-v6` had wired reasoning end-to-end into inference but missed this one read site. `CompactionMessage` extended with `reasoning_parts`; `buildHeadPayload` embeds it as a `<reasoning>...</reasoning>` prose prefix on the assistant content (OpenAI wire shape has no structured reasoning field).
|
||||||
|
|
||||||
|
## v1.13.3-truncate — 2026-05-22
|
||||||
|
|
||||||
|
Port of opencode's `truncate.ts`. Full tool output retrievable via opaque `tr_<12 base32 chars>` id (~60 bits entropy) and a new `view_truncated_output(id)` tool. Tmpfs storage at `/tmp/boocode-truncations/` (overridable via `BOOCODE_TRUNCATION_DIR`), 5MB cap, 7-day TTL, orphan-reap on the periodic 60s sweeper. Wired through four tools: `view_file`, `list_dir`, `web_fetch`, `codecontext_client`. Each returns the existing sliced view plus an `outputPath` field when truncation fires.
|
||||||
|
|
||||||
|
## v1.13.2-compaction-prune — 2026-05-22
|
||||||
|
|
||||||
|
Two-tier compaction prune — opencode pattern that was half-shipped in v1.11.0. New `message_parts.hidden_at` column with partial index on `WHERE hidden_at IS NULL`. `messages_with_parts` view changed from `COALESCE(parts, legacy)` to a CASE that distinguishes "no parts at all → fall back to legacy column for pre-v1.13.0 history" from "all parts hidden → drop the row from the model payload" (smoke caught the `COALESCE` leaking hidden parts back via legacy fallback). `prune.ts` scans `tool_result` parts newest-first, protects the last 40k tokens, marks older candidates hidden once the combined estimate clears 20k.
|
||||||
|
|
||||||
|
## v1.13.1-cleanup-bundle — 2026-05-22
|
||||||
|
|
||||||
|
Four independent items owed from prior dispatches. (1) `statement_timeout = '30s'` at the database level (documented in `schema.sql` but applied operationally — `ALTER DATABASE` can't run inside a `DO` block). (2) Tool registry alpha-sorted at module load — llama.cpp's prompt cache hits on byte-identical prefixes; reordering tools near the top of the system prompt would invalidate every cached turn. (3) Periodic 60s stuck-row sweeper. (4) `experimental_repairToolCall` to keep streams alive on malformed qwen3.6 tool args (pass-through implementation — logs and forwards unmodified; existing zod-reject path routes back to the model).
|
||||||
|
|
||||||
|
## v1.13.0-ai-sdk-v6 — 2026-05-22
|
||||||
|
|
||||||
|
Major migration to AI SDK v6. Introduces the `streamCompletion` adapter (`services/inference/stream-phase.ts`) over `streamText`, with five known gotchas the LSP can't catch — abort signals swallowed by `fullStream` (post-iteration throw required), usage lands only at stream end via `await result.usage`, tools have no `execute` field (BooCode dispatches in `tool-phase.ts`), and tool-call-only turns may emit a leading `\n` text-delta. Also ships the `messages_with_parts` view (parts-merge read path) and wires `reasoning_parts` end-to-end via a `ReasoningPart` in the v6 ModelMessage. Ports `ask_user_input` correlation queries from JSON columns to `message_parts` JOINs.
|
||||||
|
|
||||||
|
## v1.12.4-inference-split — 2026-05-21
|
||||||
|
|
||||||
|
Complete `inference.ts` split into `services/inference/`. Pieces: `turn.ts` (orchestration — `runAssistantTurn` / `runInference` / `createInferenceRunner`), `sentinel-summaries.ts` (`runCapHitSummary`, `runDoomLoopSummary`), `stream-phase.ts`, `tool-phase.ts`, `provider.ts`, `payload.ts`, `prune.ts`, `budget.ts`, `xml-parser.ts`, `error-handler.ts`, `sentinels.ts`, `parts.ts`, `types.ts`. Public surface re-exported via `inference/index.ts`; callers import from `./services/inference/index.js` explicitly (NodeNext doesn't honor directory-index resolution).
|
||||||
|
|
||||||
|
## v1.12.3-stale-banner — 2026-05-21
|
||||||
|
|
||||||
|
Stale-stream banner with Retry/Discard. When an assistant message sits `status='streaming'` with no token activity for 60+ seconds, the chat shows a banner above the input. Both actions clear the stale row via new `POST /api/chats/:id/discard_stale` (updates `status='failed'`, publishes `chat_status='idle'`). Closes the UX gap from the 2026-05-21 debugging spiral — slow streams and dead streams now look different.
|
||||||
|
|
||||||
|
## v1.12.2-live-toks — 2026-05-21
|
||||||
|
|
||||||
|
Live tok/s + ctx display next to the status indicator. `ChatThroughput` renders inline beside `StatusDot` while streaming or tool_running. Subscribes to existing `'usage'` WS frames (500ms-throttled, carrying `completion_tokens` + `ctx_used` + `ctx_max`) via `sessionEvents`. Hides when status drops to idle/error or data is older than 10s. Addresses the same UX gap as `v1.12.3-stale-banner` — gives users a live token velocity readout that immediately distinguishes slow from dead.
|
||||||
|
|
||||||
|
## v1.12.1-stop-handler — 2026-05-21
|
||||||
|
|
||||||
|
`handleAbortOrError` now writes `status='cancelled'` on user stop; rows no longer stuck `streaming` forever. Drops stale `messages_status_check` constraint (only `messages_status_chk` remains, allowing 'cancelled' via TS `MESSAGE_STATUSES`). Removes `detectSameNameLoop` and `DOOM_LOOP_SAME_NAME_THRESHOLD` (added during the 2026-05-21 debugging spike, never fired in any real run) plus 12 verbose `ctx.log.info` diagnostic markers from the same spike. Bundles workspace pane sync + status indicator overhaul + startup hung-row sweep that landed earlier in v1.12.1 work.
|
||||||
|
|
||||||
|
## v1.12.0-codecontext — 2026-05-21
|
||||||
|
|
||||||
|
Adds the `codecontext` sidecar (Go-based code-graph indexer at `codecontext:8080/v1/<tool_name>` over `boocode_net`) plus container guidance and skills runtime updates. Introduces the `chat_status` WS frame (`streaming | tool_running | waiting_for_input | idle | error`, widened from `working|idle|error`). Drops the deprecated `session_panes` table — workspace pane state moves to `sessions.workspace_panes jsonb` for cross-device sync via `PATCH /api/sessions/:id/workspace`.
|
||||||
|
|
||||||
|
## v1.11.1-consolidation — 2026-05-21
|
||||||
|
|
||||||
|
Rollup of v1.11.0–v1.11.10 work that was shipped piecemeal. Covers anchored rolling compaction (single `summary=true` row per chat that supersedes itself), doom-loop guard via `detectDoomLoop`, `path_guard` secret-filename deny list, web tools (`web_search` against SearXNG + `web_fetch` with SSRF/private-IP block), and the 5MB stream-cap on response bodies with abort-on-overflow.
|
||||||
|
|
||||||
|
## v1.11.0-context-bar — 2026-05-20
|
||||||
|
|
||||||
|
Persistent context-window tracker in `ChatPane` + `ctx_max` capture via `${LLAMA_SWAP_URL}/upstream/<model>/props`. First inferences after a boocode boot may have `ctx_max=NULL` if llama-swap hasn't loaded the model yet — 60s negative cache TTL recovers on next turn. Replaced an earlier dead read of `parsed.timings.n_ctx` which never carried n_ctx.
|
||||||
|
|
||||||
|
## v1.10.1-booterm-user — 2026-05-19
|
||||||
|
|
||||||
|
Per-user shell privilege drop in the booterm container via `gosu` in `tmux.conf` default-command. Shells launched in browser terminal panes drop privs to `samkintop` rather than running as root inside the container.
|
||||||
|
|
||||||
|
## v1.10.0-booterm — 2026-05-18
|
||||||
|
|
||||||
|
Second container (`apps/booterm`, port 9501, bookworm-slim+glibc). Fastify + node-pty + tmux. Browser terminal panes connect via WS to `/ws/term/sessions/:sid/panes/:pid`; per-session tmux session `bc-<sid>`, per-pane window `term-<pid>`. xterm-addon-webgl with `document.fonts.load(...)`-gated init (Canvas2D doesn't honor `font-display: block`) and iOS-friendly visibility-change context recreation.
|
||||||
|
|
||||||
|
## v1.9.2-ask-user-input — 2026-05-18
|
||||||
|
|
||||||
|
`ask_user_input` elicitation tool. Pauses the inference loop and surfaces a prompt to the user; their response routes back as the tool result. Correlation initially via `messages.tool_calls` / `tool_results` JSON columns (later ported to `message_parts` in `v1.13.0-ai-sdk-v6`).
|
||||||
|
|
||||||
|
## v1.9.1-skills — 2026-05-18
|
||||||
|
|
||||||
|
Skills runtime + `/skill` slash command with autocomplete. Server-side parser, tools, `/api/skills`, and mount. Hardens `.dockerignore` to exclude `secrets/` and `data/`. Drops the type-to-confirm gate on chat delete (plain Cancel/Confirm only — per workspace convention).
|
||||||
|
|
||||||
|
## v1.9.0-themes-settings — 2026-05-17
|
||||||
|
|
||||||
|
Settings pane + per-project defaults + bulk archive + themes lift. `themes-v1` (18 preset palettes) ships in the same batch with a Settings picker for live theme switching.
|
||||||
|
|
||||||
|
## v1.8.2-cap-hit — 2026-05-17
|
||||||
|
|
||||||
|
Tool-loop cap-hit summary — when an assistant exceeds the per-turn tool budget, a sentinel `role='system'` row with `metadata.kind='cap_hit'` is inserted and a summary turn runs to give the user a coherent endpoint. Also compacts the tool-call UI rendering.
|
||||||
|
|
||||||
|
## v1.8.1-agents-global — 2026-05-16
|
||||||
|
|
||||||
|
Global agents (`data/AGENTS.md` bind-mounted at `/data/AGENTS.md`) + parser robustness + WS reconnect toast. Per-project `AGENTS.md` mechanism (`getAgentsForProject`) remains for *other* projects; the BooCode repo itself uses global-only to eliminate two-files-must-stay-in-sync drift.
|
||||||
|
|
||||||
|
## v1.8.0-agents — 2026-05-16
|
||||||
|
|
||||||
|
Tier 2 agents — `AGENTS.md` registry + per-session agent picker. Also lands mobile tab switcher, branch indicator, and the `git_status` tool.
|
||||||
|
|
||||||
|
## v1.7.0-drag-drop — 2026-05-16
|
||||||
|
|
||||||
|
Drag-drop + paste-as-attachment for long text in the chat input.
|
||||||
|
|
||||||
|
## v1.6.0-mobile — 2026-05-16
|
||||||
|
|
||||||
|
Full mobile suite. Adds `useViewport` (matchMedia breakpoints mobile <768 / tablet 768–1023 / desktop ≥1024), `useSidebarDrawer` / `useRightRailDrawer` (Context + auto-close on `useLocation().pathname` change), `useLongPress` (500ms timer, synthetic `contextmenu`), `usePullToRefresh` (80px threshold, 600ms hold), `SwipeablePaneTab` (60px close, 30px vertical bail). Mobile headers with safe-area padding, hamburger left, FolderTree right. Tap targets at `max-md:min-h-[44px] max-md:min-w-[44px]`. Raises `MAX_TOOL_LOOP_DEPTH` 5 → 15. Right-rail becomes a drawer on mobile.
|
||||||
|
|
||||||
|
## v1.5.1-bootstrap — 2026-05-16
|
||||||
|
|
||||||
|
Bootstrap fixes — git + ssh installed in the boocode container, Tailscale host rewrite, `/opt/projects` label correction for the create-new-project bootstrap flow.
|
||||||
|
|
||||||
|
## v1.5.0-refactor-tests — 2026-05-16
|
||||||
|
|
||||||
|
Refactor split (FileBrowserPane / Workspace / `runAssistantTurn`) + vitest harness + unit tests for security-critical pure functions. Scopes the `/opt` mount to `/opt/projects` (writable) plus `PROJECT_ROOT_WHITELIST=/opt` (read-only resolution for add-existing). Surfaces swallowed errors and removes dead `session_renamed` paths.
|
||||||
|
|
||||||
|
## v1.4.0-fork-header — 2026-05-16
|
||||||
|
|
||||||
|
Fork from message + delete message + header polish + general housekeeping.
|
||||||
|
|
||||||
|
## v1.3.0-chats-projects — 2026-05-16
|
||||||
|
|
||||||
|
Chats-in-sessions era. Adds force-send, `/compact`, right-rail file browser, archive/rename/Open-in-Gitea sidebar context menu, archived projects landing page, create-project bootstrap with Gitea remote setup, landing-card buttons, 1000px content cap. Dedup audit and chat archive/delete from the sidebar.
|
||||||
|
|
||||||
|
## v1.2.0-multi-pane — 2026-05-15
|
||||||
|
|
||||||
|
Multi-pane workspace (batch 3, T1–T8). `session_panes` schema (later replaced by `sessions.workspace_panes jsonb` in v1.12.0), `Pane` discriminated union, broker user channel + `/api/ws/user`, `file_ops` + `file_index` services, `PaneShell` / `ChatPane` / `FileBrowserPane` / `PaneTab` / `Workspace` components, `usePanes` hook, Shiki integration in `CodeBlock`. Up to 5 panes per session; default chat pane created on `POST /api/sessions`.
|
||||||
|
|
||||||
|
## v1.1.0-markdown-sidebar — 2026-05-15
|
||||||
|
|
||||||
|
Markdown rendering, message actions, tok/s + ctx display, AI session naming. Sidebar restructure — chats nested under projects (max 5 + view-all), live updates via WS.
|
||||||
|
|
||||||
|
## v1.0.0-initial — 2026-05-14
|
||||||
|
|
||||||
|
Initial commit. Skeleton of the monorepo: `apps/server` (Fastify + postgres), `apps/web` (React + Vite), basic chat loop against llama-swap.
|
||||||
@@ -115,7 +115,7 @@ async function main() {
|
|||||||
broker.publishUserFrame(user, frame as unknown as import('./types/ws-frames.js').WsFrame);
|
broker.publishUserFrame(user, frame as unknown as import('./types/ws-frames.js').WsFrame);
|
||||||
}
|
}
|
||||||
);
|
);
|
||||||
registerMessageRoutes(app, sql, {
|
registerMessageRoutes(app, sql, config, broker, {
|
||||||
enqueueInference: (sessionId, chatId, assistantId, user) => {
|
enqueueInference: (sessionId, chatId, assistantId, user) => {
|
||||||
inference.enqueue(sessionId, chatId, assistantId, user);
|
inference.enqueue(sessionId, chatId, assistantId, user);
|
||||||
},
|
},
|
||||||
|
|||||||
70
apps/server/src/routes/__tests__/sessions.test.ts
Normal file
70
apps/server/src/routes/__tests__/sessions.test.ts
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
// v1.13.17-cross-repo-reads: PATCH /api/sessions/:id allowed_read_paths
|
||||||
|
// subset enforcement. Sam flagged in the compliance review that without a
|
||||||
|
// runtime subset check, a malicious client could POST
|
||||||
|
// {"allowed_read_paths":["/etc"]}
|
||||||
|
// and bypass the user-consent grant flow entirely. The findUnauthorizedAdditions
|
||||||
|
// helper is the guard; tests pin its behavior so a regression in the helper
|
||||||
|
// or its callsite (PATCH handler in sessions.ts) trips CI before prod.
|
||||||
|
|
||||||
|
import { describe, it, expect } from 'vitest';
|
||||||
|
import { findUnauthorizedAdditions } from '../sessions.js';
|
||||||
|
|
||||||
|
describe('findUnauthorizedAdditions — PATCH allowed_read_paths subset guard', () => {
|
||||||
|
it('returns no extras when requested is empty (full revoke)', () => {
|
||||||
|
expect(findUnauthorizedAdditions(['/opt/forks/foo'], [])).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns no extras when requested is a strict subset (single revoke)', () => {
|
||||||
|
expect(
|
||||||
|
findUnauthorizedAdditions(['/opt/forks/foo', '/opt/forks/bar'], ['/opt/forks/foo']),
|
||||||
|
).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns no extras when requested equals prior (no-op PATCH)', () => {
|
||||||
|
expect(
|
||||||
|
findUnauthorizedAdditions(['/opt/forks/foo', '/opt/forks/bar'], [
|
||||||
|
'/opt/forks/foo',
|
||||||
|
'/opt/forks/bar',
|
||||||
|
]),
|
||||||
|
).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('flags an unauthorized addition when prior is empty', () => {
|
||||||
|
// The /etc bypass attempt — Sam's specific concern from the compliance
|
||||||
|
// review. Without this guard, the PATCH would have written /etc directly.
|
||||||
|
expect(findUnauthorizedAdditions([], ['/etc'])).toEqual(['/etc']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('flags a single unauthorized addition mixed in with valid revokes', () => {
|
||||||
|
// The attacker still tries to be sneaky: keep one legit entry, drop
|
||||||
|
// another, slip in a new one. The guard catches the addition regardless
|
||||||
|
// of how the rest of the array shrinks.
|
||||||
|
expect(
|
||||||
|
findUnauthorizedAdditions(['/opt/forks/foo', '/opt/forks/bar'], [
|
||||||
|
'/opt/forks/foo',
|
||||||
|
'/var/secrets',
|
||||||
|
]),
|
||||||
|
).toEqual(['/var/secrets']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('flags every unauthorized addition when there are multiple', () => {
|
||||||
|
expect(
|
||||||
|
findUnauthorizedAdditions(['/opt/forks/foo'], ['/opt/forks/foo', '/etc', '/root']),
|
||||||
|
).toEqual(['/etc', '/root']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('treats requested duplicates correctly (each occurrence checked)', () => {
|
||||||
|
// If the requested array has duplicates of an unauthorized entry, the
|
||||||
|
// guard surfaces each one. (A frontend would never send duplicates, but
|
||||||
|
// the guard's contract shouldn't assume that.)
|
||||||
|
expect(findUnauthorizedAdditions([], ['/etc', '/etc'])).toEqual(['/etc', '/etc']);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not flag entries present in prior even if requested has duplicates', () => {
|
||||||
|
// Duplicate of an authorized entry passes — the membership check is by
|
||||||
|
// value, not by index. Settled by Set.has semantics.
|
||||||
|
expect(
|
||||||
|
findUnauthorizedAdditions(['/opt/forks/foo'], ['/opt/forks/foo', '/opt/forks/foo']),
|
||||||
|
).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,7 +1,13 @@
|
|||||||
import type { FastifyInstance } from 'fastify';
|
import type { FastifyInstance } from 'fastify';
|
||||||
import { z } from 'zod';
|
import { z } from 'zod';
|
||||||
import type { Sql } from '../db.js';
|
import type { Sql } from '../db.js';
|
||||||
|
import type { Config } from '../config.js';
|
||||||
|
import type { Broker } from '../services/broker.js';
|
||||||
import type { Chat, Message, Session, ToolCall } from '../types/api.js';
|
import type { Chat, Message, Session, ToolCall } from '../types/api.js';
|
||||||
|
// v1.13.17-cross-repo-reads: grant_read_access resolves the grant root at
|
||||||
|
// decision time (not at request time) so concurrent project changes don't
|
||||||
|
// stale-bind the resolution.
|
||||||
|
import { resolveGrantRoot } from '../services/grant_resolver.js';
|
||||||
|
|
||||||
const SendBody = z.object({
|
const SendBody = z.object({
|
||||||
content: z.string().min(1).max(64_000),
|
content: z.string().min(1).max(64_000),
|
||||||
@@ -47,6 +53,21 @@ const AskUserInputArgs = z.object({
|
|||||||
.max(3),
|
.max(3),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: grant decision body. tool_call_id is the
|
||||||
|
// model-emitted id (e.g. "call_abc123"), not a UUID. decision is binary.
|
||||||
|
const GrantReadAccessBody = z.object({
|
||||||
|
tool_call_id: z.string().min(1),
|
||||||
|
decision: z.enum(['allow', 'deny']),
|
||||||
|
});
|
||||||
|
|
||||||
|
// Same shape as services/request_read_access.ts RequestReadAccessInput.
|
||||||
|
// Re-derived to avoid the services/tools.ts import (matches the
|
||||||
|
// AskUserInputArgs pattern above).
|
||||||
|
const RequestReadAccessArgs = z.object({
|
||||||
|
path: z.string().min(1),
|
||||||
|
reason: z.string().min(1).max(500),
|
||||||
|
});
|
||||||
|
|
||||||
interface MessageHandlers {
|
interface MessageHandlers {
|
||||||
enqueueInference: (sessionId: string, chatId: string, assistantMessageId: string, user: string) => void;
|
enqueueInference: (sessionId: string, chatId: string, assistantMessageId: string, user: string) => void;
|
||||||
// v1.11: returns a promise that resolves after compaction.process finishes
|
// v1.11: returns a promise that resolves after compaction.process finishes
|
||||||
@@ -76,6 +97,8 @@ interface MessageHandlers {
|
|||||||
export function registerMessageRoutes(
|
export function registerMessageRoutes(
|
||||||
app: FastifyInstance,
|
app: FastifyInstance,
|
||||||
sql: Sql,
|
sql: Sql,
|
||||||
|
config: Config,
|
||||||
|
broker: Broker,
|
||||||
handlers: MessageHandlers
|
handlers: MessageHandlers
|
||||||
): void {
|
): void {
|
||||||
app.get<{ Params: { id: string } }>(
|
app.get<{ Params: { id: string } }>(
|
||||||
@@ -626,4 +649,234 @@ export function registerMessageRoutes(
|
|||||||
return result;
|
return result;
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: resume an awaiting-grant pause. Mirror shape
|
||||||
|
// of /answer_user_input (validate, look up via message_parts, UPDATE,
|
||||||
|
// publish, enqueue). Differences vs /answer_user_input:
|
||||||
|
// - On 'allow', re-resolves the grant root via grant_resolver (state
|
||||||
|
// may have changed since the prompt fired — concurrent project add,
|
||||||
|
// etc.). Resolution failure auto-falls to a denial with reason text
|
||||||
|
// rather than 500ing.
|
||||||
|
// - On 'allow' with a valid root, appends to sessions.allowed_read_paths
|
||||||
|
// (deduplicated) inside the same transaction.
|
||||||
|
// - On success, also publishes session_updated so an open SettingsPane
|
||||||
|
// refetches the new grant list.
|
||||||
|
// Error codes match /answer:
|
||||||
|
// 400 invalid_body / mismatched_answer_shape (bad args on the tool_call)
|
||||||
|
// 404 chat_not_found / unknown_tool_call_id
|
||||||
|
// 409 tool_call_already_answered
|
||||||
|
app.post<{ Params: { id: string } }>(
|
||||||
|
'/api/chats/:id/grant_read_access',
|
||||||
|
async (req, reply) => {
|
||||||
|
const parsed = GrantReadAccessBody.safeParse(req.body);
|
||||||
|
if (!parsed.success) {
|
||||||
|
reply.code(400);
|
||||||
|
return { error: 'invalid_body', details: parsed.error.flatten() };
|
||||||
|
}
|
||||||
|
const { tool_call_id, decision } = parsed.data;
|
||||||
|
|
||||||
|
const chatRows = await sql<Chat[]>`
|
||||||
|
SELECT id, session_id FROM chats WHERE id = ${req.params.id} AND status = 'open'
|
||||||
|
`;
|
||||||
|
if (chatRows.length === 0) {
|
||||||
|
reply.code(404);
|
||||||
|
return { error: 'chat_not_found' };
|
||||||
|
}
|
||||||
|
const chat = chatRows[0]!;
|
||||||
|
const sessionId = chat.session_id;
|
||||||
|
|
||||||
|
// Mirror the /answer lookup: assistant tool_call by id via message_parts.
|
||||||
|
const callerRows = await sql<{
|
||||||
|
message_id: string;
|
||||||
|
payload: { id: string; name: string; args: Record<string, unknown> };
|
||||||
|
}[]>`
|
||||||
|
SELECT p.message_id, p.payload
|
||||||
|
FROM message_parts p
|
||||||
|
JOIN messages m ON m.id = p.message_id
|
||||||
|
WHERE m.chat_id = ${chat.id}
|
||||||
|
AND m.role = 'assistant'
|
||||||
|
AND p.kind = 'tool_call'
|
||||||
|
AND p.payload->>'id' = ${tool_call_id}
|
||||||
|
ORDER BY m.created_at DESC
|
||||||
|
LIMIT 1
|
||||||
|
`;
|
||||||
|
const callerRow = callerRows[0];
|
||||||
|
if (!callerRow) {
|
||||||
|
reply.code(404);
|
||||||
|
return { error: 'unknown_tool_call_id' };
|
||||||
|
}
|
||||||
|
const foundCall: ToolCall = {
|
||||||
|
id: callerRow.payload.id,
|
||||||
|
name: callerRow.payload.name,
|
||||||
|
args: callerRow.payload.args,
|
||||||
|
};
|
||||||
|
if (foundCall.name !== 'request_read_access') {
|
||||||
|
reply.code(400);
|
||||||
|
return { error: 'tool_call_not_request_read_access' };
|
||||||
|
}
|
||||||
|
const argsParsed = RequestReadAccessArgs.safeParse(foundCall.args);
|
||||||
|
if (!argsParsed.success) {
|
||||||
|
reply.code(400);
|
||||||
|
return { error: 'mismatched_answer_shape', detail: 'tool_call args invalid' };
|
||||||
|
}
|
||||||
|
const requestedPath = argsParsed.data.path;
|
||||||
|
|
||||||
|
// Find the pending tool row.
|
||||||
|
const toolRows = await sql<{
|
||||||
|
message_id: string;
|
||||||
|
payload: { tool_call_id: string; output: unknown };
|
||||||
|
}[]>`
|
||||||
|
SELECT p.message_id, p.payload
|
||||||
|
FROM message_parts p
|
||||||
|
JOIN messages m ON m.id = p.message_id
|
||||||
|
WHERE m.chat_id = ${chat.id}
|
||||||
|
AND m.role = 'tool'
|
||||||
|
AND p.kind = 'tool_result'
|
||||||
|
AND p.payload->>'tool_call_id' = ${tool_call_id}
|
||||||
|
ORDER BY m.created_at DESC
|
||||||
|
LIMIT 1
|
||||||
|
`;
|
||||||
|
const toolRow = toolRows[0];
|
||||||
|
if (!toolRow) {
|
||||||
|
reply.code(404);
|
||||||
|
return { error: 'unknown_tool_call_id', detail: 'tool message not found' };
|
||||||
|
}
|
||||||
|
if (toolRow.payload && toolRow.payload.output !== null) {
|
||||||
|
reply.code(409);
|
||||||
|
return { error: 'tool_call_already_answered' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Look up session + project so we can re-resolve the grant root and
|
||||||
|
// append to allowed_read_paths atomically. We don't need agent or
|
||||||
|
// history here — just the project path for the resolver.
|
||||||
|
const sessionRows = await sql<{
|
||||||
|
id: string;
|
||||||
|
project_id: string;
|
||||||
|
allowed_read_paths: string[];
|
||||||
|
project_path: string;
|
||||||
|
}[]>`
|
||||||
|
SELECT s.id, s.project_id, s.allowed_read_paths, p.path AS project_path
|
||||||
|
FROM sessions s
|
||||||
|
JOIN projects p ON p.id = s.project_id
|
||||||
|
WHERE s.id = ${sessionId}
|
||||||
|
`;
|
||||||
|
const sessionRow = sessionRows[0];
|
||||||
|
if (!sessionRow) {
|
||||||
|
reply.code(404);
|
||||||
|
return { error: 'session_not_found' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Decision branch. 'deny' is the easy path: nothing to resolve or
|
||||||
|
// persist. 'allow' resolves the grant root; if resolution fails (e.g.
|
||||||
|
// path was deleted, project removed since prompt) the tool gets a
|
||||||
|
// denial with the resolver's reason text instead of a 500.
|
||||||
|
let resultOutput: string;
|
||||||
|
let grantRoot: string | null = null;
|
||||||
|
if (decision === 'allow') {
|
||||||
|
const resolution = await resolveGrantRoot(
|
||||||
|
sql,
|
||||||
|
requestedPath,
|
||||||
|
sessionRow.project_path,
|
||||||
|
config.PROJECT_ROOT_WHITELIST,
|
||||||
|
);
|
||||||
|
if (!resolution.ok) {
|
||||||
|
resultOutput = `denied: ${resolution.reason}`;
|
||||||
|
} else {
|
||||||
|
grantRoot = resolution.root;
|
||||||
|
resultOutput = `granted: ${grantRoot}`;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
resultOutput = 'denied';
|
||||||
|
}
|
||||||
|
|
||||||
|
const newToolResults = {
|
||||||
|
tool_call_id,
|
||||||
|
output: resultOutput,
|
||||||
|
truncated: false,
|
||||||
|
};
|
||||||
|
const toolMessageId = toolRow.message_id;
|
||||||
|
const dbResult = await sql.begin(async (tx) => {
|
||||||
|
await tx`
|
||||||
|
UPDATE messages
|
||||||
|
SET tool_results = ${tx.json(newToolResults as never)}
|
||||||
|
WHERE id = ${toolMessageId}
|
||||||
|
`;
|
||||||
|
// Same delete+insert dance as /answer — UNIQUE (message_id, sequence)
|
||||||
|
// blocks plain UPDATE on append-style parts.
|
||||||
|
await tx`DELETE FROM message_parts WHERE message_id = ${toolMessageId} AND kind = 'tool_result'`;
|
||||||
|
await tx`
|
||||||
|
INSERT INTO message_parts (message_id, sequence, kind, payload)
|
||||||
|
VALUES (${toolMessageId}, 0, 'tool_result', ${tx.json(newToolResults as never)})
|
||||||
|
`;
|
||||||
|
// Persist the grant if we have one. ARRAY-level dedup — append only
|
||||||
|
// when the root isn't already present. The session row gets
|
||||||
|
// touched (updated_at) so the post-update publish below has a
|
||||||
|
// fresh timestamp.
|
||||||
|
let allowedRootsAfter = sessionRow.allowed_read_paths;
|
||||||
|
if (grantRoot !== null) {
|
||||||
|
if (!sessionRow.allowed_read_paths.includes(grantRoot)) {
|
||||||
|
const updated = await tx<{ allowed_read_paths: string[] }[]>`
|
||||||
|
UPDATE sessions
|
||||||
|
SET allowed_read_paths = array_append(allowed_read_paths, ${grantRoot}),
|
||||||
|
updated_at = clock_timestamp()
|
||||||
|
WHERE id = ${sessionId}
|
||||||
|
RETURNING allowed_read_paths
|
||||||
|
`;
|
||||||
|
allowedRootsAfter = updated[0]?.allowed_read_paths ?? sessionRow.allowed_read_paths;
|
||||||
|
} else {
|
||||||
|
// Already present — touch updated_at so any open settings
|
||||||
|
// panel still picks up the no-op via session_updated.
|
||||||
|
await tx`UPDATE sessions SET updated_at = clock_timestamp() WHERE id = ${sessionId}`;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const [assistantMsg] = await tx<{ id: string }[]>`
|
||||||
|
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
||||||
|
VALUES (${sessionId}, ${chat.id}, 'assistant', '', 'streaming', clock_timestamp())
|
||||||
|
RETURNING id
|
||||||
|
`;
|
||||||
|
await tx`UPDATE chats SET updated_at = clock_timestamp() WHERE id = ${chat.id}`;
|
||||||
|
return {
|
||||||
|
tool_message_id: toolMessageId,
|
||||||
|
assistant_message_id: assistantMsg!.id,
|
||||||
|
allowed_roots_after: allowedRootsAfter,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
// Publish the deferred tool_result frame so the pending card flips to
|
||||||
|
// its answered view without a refetch.
|
||||||
|
handlers.publishSessionFrame(sessionId, {
|
||||||
|
type: 'tool_result',
|
||||||
|
tool_message_id: dbResult.tool_message_id,
|
||||||
|
tool_call_id,
|
||||||
|
chat_id: chat.id,
|
||||||
|
output: resultOutput,
|
||||||
|
truncated: false,
|
||||||
|
});
|
||||||
|
// session_updated nudge so any open SettingsPane refetches and sees
|
||||||
|
// the new allowed_read_paths. We publish on the user channel to match
|
||||||
|
// the existing PATCH /api/sessions/:id behavior — frontend refetches
|
||||||
|
// via api.sessions.get on receipt.
|
||||||
|
const nowIso = new Date().toISOString();
|
||||||
|
broker.publishUserFrame('default', {
|
||||||
|
type: 'session_updated',
|
||||||
|
session_id: sessionId,
|
||||||
|
project_id: sessionRow.project_id,
|
||||||
|
// session name doesn't change on grant; we look it up fresh to
|
||||||
|
// avoid carrying stale state if a rename raced us.
|
||||||
|
name:
|
||||||
|
(
|
||||||
|
await sql<{ name: string }[]>`SELECT name FROM sessions WHERE id = ${sessionId}`
|
||||||
|
)[0]?.name ?? '',
|
||||||
|
updated_at: nowIso,
|
||||||
|
});
|
||||||
|
handlers.enqueueInference(sessionId, chat.id, dbResult.assistant_message_id, 'default');
|
||||||
|
|
||||||
|
reply.code(202);
|
||||||
|
return {
|
||||||
|
tool_message_id: dbResult.tool_message_id,
|
||||||
|
assistant_message_id: dbResult.assistant_message_id,
|
||||||
|
allowed_read_paths: dbResult.allowed_roots_after,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -32,6 +32,29 @@ const PatchBody = z.object({
|
|||||||
agent_id: z.string().min(1).max(200).nullable().optional(),
|
agent_id: z.string().min(1).max(200).nullable().optional(),
|
||||||
// v1.9: null = inherit from project default; true/false = explicit override.
|
// v1.9: null = inherit from project default; true/false = explicit override.
|
||||||
web_search_enabled: z.boolean().nullable().optional(),
|
web_search_enabled: z.boolean().nullable().optional(),
|
||||||
|
// v1.13.17-cross-repo-reads: revocation pathway. PATCH with a shortened
|
||||||
|
// list deletes entries; the grant flow itself APPENDS via the separate
|
||||||
|
// grant_read_access endpoint, never via this PATCH. Frontend treats this
|
||||||
|
// as "send the new whole array". Per-entry shape validation: must be
|
||||||
|
// absolute, no NUL, no `/..` traversal segment. Server doesn't re-validate
|
||||||
|
// whitelist membership on PATCH — entries already in the array were
|
||||||
|
// placed there by the grant endpoint after a full whitelist+repo-shape
|
||||||
|
// check. THE SUBSET CHECK (every entry must already be in the current
|
||||||
|
// array) is enforced at runtime in the PATCH handler below, NOT in this
|
||||||
|
// zod refinement, because the refinement has no access to the existing
|
||||||
|
// session row.
|
||||||
|
allowed_read_paths: z
|
||||||
|
.array(
|
||||||
|
z
|
||||||
|
.string()
|
||||||
|
.min(1)
|
||||||
|
.max(1024)
|
||||||
|
.refine((p) => p.startsWith('/') && !p.includes('\0') && !p.includes('/..'), {
|
||||||
|
message: 'must be an absolute path without traversal markers',
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.max(64)
|
||||||
|
.optional(),
|
||||||
});
|
});
|
||||||
|
|
||||||
async function resolveDefaultModel(sql: Sql, config: Config): Promise<string> {
|
async function resolveDefaultModel(sql: Sql, config: Config): Promise<string> {
|
||||||
@@ -40,6 +63,19 @@ async function resolveDefaultModel(sql: Sql, config: Config): Promise<string> {
|
|||||||
return config.DEFAULT_MODEL;
|
return config.DEFAULT_MODEL;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: subset enforcement for PATCH allowed_read_paths.
|
||||||
|
// The PATCH route can only SHRINK the array; growth happens exclusively via
|
||||||
|
// POST /api/chats/:id/grant_read_access (which requires user consent).
|
||||||
|
// Returns the list of disallowed-additions; an empty list means the request
|
||||||
|
// is a valid shrink-or-no-op. Exported for the unit test.
|
||||||
|
export function findUnauthorizedAdditions(
|
||||||
|
prior: readonly string[],
|
||||||
|
requested: readonly string[],
|
||||||
|
): string[] {
|
||||||
|
const priorSet = new Set(prior);
|
||||||
|
return requested.filter((p) => !priorSet.has(p));
|
||||||
|
}
|
||||||
|
|
||||||
export function registerSessionRoutes(
|
export function registerSessionRoutes(
|
||||||
app: FastifyInstance,
|
app: FastifyInstance,
|
||||||
sql: Sql,
|
sql: Sql,
|
||||||
@@ -56,7 +92,7 @@ export function registerSessionRoutes(
|
|||||||
}
|
}
|
||||||
const status = req.query.status === 'archived' ? 'archived' : 'open';
|
const status = req.query.status === 'archived' ? 'archived' : 'open';
|
||||||
const rows = await sql<Session[]>`
|
const rows = await sql<Session[]>`
|
||||||
SELECT id, project_id, name, model, system_prompt, status, created_at, updated_at, agent_id, web_search_enabled, workspace_panes
|
SELECT id, project_id, name, model, system_prompt, status, created_at, updated_at, agent_id, web_search_enabled, workspace_panes, allowed_read_paths
|
||||||
FROM sessions
|
FROM sessions
|
||||||
WHERE project_id = ${req.params.id} AND status = ${status}
|
WHERE project_id = ${req.params.id} AND status = ${status}
|
||||||
ORDER BY updated_at DESC
|
ORDER BY updated_at DESC
|
||||||
@@ -124,7 +160,7 @@ export function registerSessionRoutes(
|
|||||||
|
|
||||||
app.get<{ Params: { id: string } }>('/api/sessions/:id', async (req, reply) => {
|
app.get<{ Params: { id: string } }>('/api/sessions/:id', async (req, reply) => {
|
||||||
const rows = await sql<Session[]>`
|
const rows = await sql<Session[]>`
|
||||||
SELECT id, project_id, name, model, system_prompt, status, created_at, updated_at, agent_id, web_search_enabled, workspace_panes
|
SELECT id, project_id, name, model, system_prompt, status, created_at, updated_at, agent_id, web_search_enabled, workspace_panes, allowed_read_paths
|
||||||
FROM sessions WHERE id = ${req.params.id}
|
FROM sessions WHERE id = ${req.params.id}
|
||||||
`;
|
`;
|
||||||
if (rows.length === 0) {
|
if (rows.length === 0) {
|
||||||
@@ -150,15 +186,53 @@ export function registerSessionRoutes(
|
|||||||
const newAgentId = parsed.data.agent_id ?? null;
|
const newAgentId = parsed.data.agent_id ?? null;
|
||||||
const wseProvided = parsed.data.web_search_enabled !== undefined;
|
const wseProvided = parsed.data.web_search_enabled !== undefined;
|
||||||
const newWse = parsed.data.web_search_enabled ?? null;
|
const newWse = parsed.data.web_search_enabled ?? null;
|
||||||
// Read the prior name so the post-update publish can skip no-op renames
|
// v1.13.17-cross-repo-reads: tri-state on the wire (undefined = no
|
||||||
// (PATCH { name: "Foo" } where the session is already "Foo"). The window
|
// change, [] = clear). Frontend currently uses this PATCH only for
|
||||||
// between SELECT and UPDATE is sub-millisecond in the same request handler;
|
// revocation (delete a single entry from the existing array, send
|
||||||
// a concurrent rename in that gap would just mean one stale publish, which
|
// shortened result). Append-style grants go through the dedicated
|
||||||
// existing clients dedup by id.
|
// grant_read_access endpoint inside the inference loop.
|
||||||
const before = await sql<{ name: string }[]>`
|
const arpProvided = parsed.data.allowed_read_paths !== undefined;
|
||||||
SELECT name FROM sessions WHERE id = ${req.params.id}
|
const newArp = parsed.data.allowed_read_paths ?? [];
|
||||||
|
// Read the prior name + grants so the post-update publish can skip no-op
|
||||||
|
// renames (PATCH { name: "Foo" } where the session is already "Foo") AND
|
||||||
|
// so the subset check below has the current grant list to compare against.
|
||||||
|
// The window between SELECT and UPDATE is sub-millisecond in the same
|
||||||
|
// request handler; a concurrent rename in that gap would just mean one
|
||||||
|
// stale publish, which existing clients dedup by id.
|
||||||
|
const before = await sql<{ name: string; allowed_read_paths: string[] }[]>`
|
||||||
|
SELECT name, allowed_read_paths FROM sessions WHERE id = ${req.params.id}
|
||||||
`;
|
`;
|
||||||
const priorName = before[0]?.name;
|
const priorName = before[0]?.name;
|
||||||
|
const priorArp = before[0]?.allowed_read_paths ?? [];
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: subset enforcement. The grant flow is the
|
||||||
|
// ONLY path that can add entries to allowed_read_paths — PATCH can only
|
||||||
|
// shrink the array, never grow it. Without this guard, a malicious
|
||||||
|
// client could POST {"allowed_read_paths":["/etc"]} and bypass the
|
||||||
|
// user-consent prompt entirely. Sam flagged this in the v1.13.17
|
||||||
|
// compliance review (2026-05-22).
|
||||||
|
// Race note: a concurrent grant landing between this SELECT and the
|
||||||
|
// UPDATE below would briefly make a "shouldn't-have-been-valid" PATCH
|
||||||
|
// succeed (the newly-granted root sneaks in). Inverse race — a
|
||||||
|
// legitimate revoke happening alongside a concurrent grant — could
|
||||||
|
// briefly reject the revoke; the user retries. Both are acceptable
|
||||||
|
// given the single-user threat model + sub-millisecond window.
|
||||||
|
if (arpProvided) {
|
||||||
|
const extras = findUnauthorizedAdditions(priorArp, newArp);
|
||||||
|
if (extras.length > 0) {
|
||||||
|
reply.code(400);
|
||||||
|
return {
|
||||||
|
error: 'invalid body',
|
||||||
|
details: {
|
||||||
|
fieldErrors: {
|
||||||
|
allowed_read_paths: [
|
||||||
|
`entries must already be granted; cannot add via PATCH: ${extras.join(', ')}`,
|
||||||
|
],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
const rows = await sql<Session[]>`
|
const rows = await sql<Session[]>`
|
||||||
UPDATE sessions
|
UPDATE sessions
|
||||||
SET
|
SET
|
||||||
@@ -167,10 +241,11 @@ export function registerSessionRoutes(
|
|||||||
system_prompt = COALESCE(${system_prompt ?? null}, system_prompt),
|
system_prompt = COALESCE(${system_prompt ?? null}, system_prompt),
|
||||||
agent_id = CASE WHEN ${agentIdProvided} THEN ${newAgentId} ELSE agent_id END,
|
agent_id = CASE WHEN ${agentIdProvided} THEN ${newAgentId} ELSE agent_id END,
|
||||||
web_search_enabled = CASE WHEN ${wseProvided} THEN ${newWse} ELSE web_search_enabled END,
|
web_search_enabled = CASE WHEN ${wseProvided} THEN ${newWse} ELSE web_search_enabled END,
|
||||||
|
allowed_read_paths = CASE WHEN ${arpProvided} THEN ${sql.array(newArp, 25)} ELSE allowed_read_paths END,
|
||||||
updated_at = clock_timestamp()
|
updated_at = clock_timestamp()
|
||||||
WHERE id = ${req.params.id}
|
WHERE id = ${req.params.id}
|
||||||
RETURNING id, project_id, name, model, system_prompt, status, created_at, updated_at,
|
RETURNING id, project_id, name, model, system_prompt, status, created_at, updated_at,
|
||||||
agent_id, web_search_enabled, workspace_panes
|
agent_id, web_search_enabled, workspace_panes, allowed_read_paths
|
||||||
`;
|
`;
|
||||||
if (rows.length === 0) {
|
if (rows.length === 0) {
|
||||||
reply.code(404);
|
reply.code(404);
|
||||||
@@ -213,7 +288,7 @@ export function registerSessionRoutes(
|
|||||||
updated_at = clock_timestamp()
|
updated_at = clock_timestamp()
|
||||||
WHERE id = ${req.params.id}
|
WHERE id = ${req.params.id}
|
||||||
RETURNING id, project_id, name, model, system_prompt, status, created_at, updated_at,
|
RETURNING id, project_id, name, model, system_prompt, status, created_at, updated_at,
|
||||||
agent_id, web_search_enabled, workspace_panes
|
agent_id, web_search_enabled, workspace_panes, allowed_read_paths
|
||||||
`;
|
`;
|
||||||
if (rows.length === 0) {
|
if (rows.length === 0) {
|
||||||
reply.code(404);
|
reply.code(404);
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ CREATE TABLE IF NOT EXISTS message_parts (
|
|||||||
kind text NOT NULL,
|
kind text NOT NULL,
|
||||||
payload jsonb NOT NULL,
|
payload jsonb NOT NULL,
|
||||||
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
|
||||||
CONSTRAINT message_parts_kind_chk CHECK (kind IN ('text', 'tool_call', 'tool_result', 'reasoning', 'step_start')),
|
CONSTRAINT message_parts_kind_chk CHECK (kind IN ('text', 'tool_call', 'tool_result', 'reasoning', 'step_start', 'synthesis')),
|
||||||
CONSTRAINT message_parts_seq_uniq UNIQUE (message_id, sequence)
|
CONSTRAINT message_parts_seq_uniq UNIQUE (message_id, sequence)
|
||||||
);
|
);
|
||||||
CREATE INDEX IF NOT EXISTS message_parts_msg_seq_idx ON message_parts (message_id, sequence);
|
CREATE INDEX IF NOT EXISTS message_parts_msg_seq_idx ON message_parts (message_id, sequence);
|
||||||
@@ -74,6 +74,23 @@ END $$;
|
|||||||
CREATE INDEX IF NOT EXISTS message_parts_hidden_idx
|
CREATE INDEX IF NOT EXISTS message_parts_hidden_idx
|
||||||
ON message_parts (message_id) WHERE hidden_at IS NULL;
|
ON message_parts (message_id) WHERE hidden_at IS NULL;
|
||||||
|
|
||||||
|
-- v1.13.13: extend message_parts.kind to allow 'synthesis'. Existing DBs were
|
||||||
|
-- created with the pre-v1.13.13 CHECK constraint that did NOT include
|
||||||
|
-- 'synthesis'; drop + re-add the constraint with the extended enum. Fresh
|
||||||
|
-- installs hit the inline constraint above (already updated) and skip this
|
||||||
|
-- block via the pg_constraint guard.
|
||||||
|
ALTER TABLE message_parts DROP CONSTRAINT IF EXISTS message_parts_kind_chk;
|
||||||
|
DO $$
|
||||||
|
BEGIN
|
||||||
|
IF NOT EXISTS (
|
||||||
|
SELECT 1 FROM pg_constraint WHERE conname = 'message_parts_kind_chk'
|
||||||
|
) THEN
|
||||||
|
ALTER TABLE message_parts
|
||||||
|
ADD CONSTRAINT message_parts_kind_chk
|
||||||
|
CHECK (kind IN ('text', 'tool_call', 'tool_result', 'reasoning', 'step_start', 'synthesis'));
|
||||||
|
END IF;
|
||||||
|
END $$;
|
||||||
|
|
||||||
-- v1.13.1-B: read-path view. Read sites SELECT FROM messages_with_parts
|
-- v1.13.1-B: read-path view. Read sites SELECT FROM messages_with_parts
|
||||||
-- instead of messages so tool_calls / tool_results / reasoning_parts come
|
-- instead of messages so tool_calls / tool_results / reasoning_parts come
|
||||||
-- from the granular message_parts table. The COALESCE means pre-v1.13.0
|
-- from the granular message_parts table. The COALESCE means pre-v1.13.0
|
||||||
@@ -313,6 +330,16 @@ END $$;
|
|||||||
-- agent_id is the slugified agent name. NULL means "use BooCode defaults".
|
-- agent_id is the slugified agent name. NULL means "use BooCode defaults".
|
||||||
ALTER TABLE sessions ADD COLUMN IF NOT EXISTS agent_id TEXT;
|
ALTER TABLE sessions ADD COLUMN IF NOT EXISTS agent_id TEXT;
|
||||||
|
|
||||||
|
-- v1.13.17-cross-repo-reads: session-scoped read grants for paths outside the
|
||||||
|
-- session's primary project root. Populated only by the request_read_access
|
||||||
|
-- tool's approve branch; revoked via PATCH /api/sessions/:id. Values are
|
||||||
|
-- absolute paths to project roots OR repo-shaped dirs under
|
||||||
|
-- PROJECT_ROOT_WHITELIST (default /opt). No CHECK constraint — validation
|
||||||
|
-- happens at write time in services/grant_resolver.ts. Cleared automatically
|
||||||
|
-- when the session row is deleted (no cascade needed; the column goes with it).
|
||||||
|
ALTER TABLE sessions
|
||||||
|
ADD COLUMN IF NOT EXISTS allowed_read_paths TEXT[] NOT NULL DEFAULT ARRAY[]::TEXT[];
|
||||||
|
|
||||||
-- v1.8.2: per-message metadata for sentinels (cap-hit) and structured error
|
-- v1.8.2: per-message metadata for sentinels (cap-hit) and structured error
|
||||||
-- reasons. JSONB so future kinds can extend without further schema churn.
|
-- reasons. JSONB so future kinds can extend without further schema churn.
|
||||||
-- Shape for cap_hit: { kind: 'cap_hit', used: number, limit: number,
|
-- Shape for cap_hit: { kind: 'cap_hit', used: number, limit: number,
|
||||||
|
|||||||
199
apps/server/src/services/__tests__/grant_resolver.test.ts
Normal file
199
apps/server/src/services/__tests__/grant_resolver.test.ts
Normal file
@@ -0,0 +1,199 @@
|
|||||||
|
// v1.13.17-cross-repo-reads: resolveGrantRoot decision tree.
|
||||||
|
//
|
||||||
|
// Sam's dispatch note (2026-05-22): "in the project-root resolver ancestor
|
||||||
|
// walk, stop the moment parent exits PROJECT_ROOT_WHITELIST or hits
|
||||||
|
// filesystem root — check on every iteration, not just final parent.
|
||||||
|
// Symlinked input must not be able to escape the whitelist during the
|
||||||
|
// walk." The symlink-escape-mid-walk test below pins that invariant —
|
||||||
|
// without the per-iteration whitelist check, this case would walk OUTSIDE
|
||||||
|
// the whitelist root and return a phantom grant.
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest';
|
||||||
|
import { mkdtemp, rm, mkdir, writeFile, symlink } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { realpath } from 'node:fs/promises';
|
||||||
|
import { resolveGrantRoot } from '../grant_resolver.js';
|
||||||
|
import type { Sql } from '../../db.js';
|
||||||
|
|
||||||
|
let tmp: string;
|
||||||
|
let whitelist: string;
|
||||||
|
let project: string;
|
||||||
|
let fork: string;
|
||||||
|
let outside: string;
|
||||||
|
|
||||||
|
// Fake sql tag — returns the projects rows we want without touching a real
|
||||||
|
// database. The resolver only ever does a single SELECT, so a single-shot
|
||||||
|
// mock that returns the prepared rows on every invocation is enough.
|
||||||
|
function makeSql(rows: Array<{ path: string }>): Sql {
|
||||||
|
const tag = ((..._args: unknown[]) => Promise.resolve(rows)) as unknown as Sql;
|
||||||
|
return tag;
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
tmp = await realpath(await mkdtemp(join(tmpdir(), 'boocode-gr-')));
|
||||||
|
whitelist = join(tmp, 'whitelist');
|
||||||
|
project = join(whitelist, 'boocode');
|
||||||
|
fork = join(whitelist, 'forks', 'codecontext');
|
||||||
|
outside = join(tmp, 'outside');
|
||||||
|
await mkdir(project, { recursive: true });
|
||||||
|
await mkdir(fork, { recursive: true });
|
||||||
|
await mkdir(outside, { recursive: true });
|
||||||
|
// Mark project as a repo (.git directory).
|
||||||
|
await mkdir(join(project, '.git'));
|
||||||
|
await writeFile(join(project, 'README.md'), 'project readme');
|
||||||
|
// Mark fork as a repo via go.mod (matches the proposal's example).
|
||||||
|
await writeFile(join(fork, 'go.mod'), 'module example.com/foo');
|
||||||
|
await writeFile(join(fork, 'main.go'), 'package main');
|
||||||
|
await writeFile(join(outside, 'secret.txt'), 'forbidden');
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await rm(tmp, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('resolveGrantRoot — happy paths', () => {
|
||||||
|
it('refuses when the requested path is already under projectRoot', async () => {
|
||||||
|
const result = await resolveGrantRoot(makeSql([]), join(project, 'README.md'), project, whitelist);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/already accessible/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns the project root when the path falls under a registered project', async () => {
|
||||||
|
// Register `fork` as a known project. Resolver should return the project
|
||||||
|
// ancestor (LONGEST match wins) rather than the repo-shape fallback.
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([{ path: fork }]),
|
||||||
|
join(fork, 'main.go'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(true);
|
||||||
|
if (result.ok) {
|
||||||
|
expect(result.root).toBe(fork);
|
||||||
|
expect(result.source).toBe('project');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('falls back to the nearest repo-shaped ancestor when no project matches', async () => {
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([]),
|
||||||
|
join(fork, 'main.go'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(true);
|
||||||
|
if (result.ok) {
|
||||||
|
expect(result.root).toBe(fork);
|
||||||
|
expect(result.source).toBe('whitelist');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('resolveGrantRoot — refusals', () => {
|
||||||
|
it('refuses paths outside PROJECT_ROOT_WHITELIST', async () => {
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([]),
|
||||||
|
join(outside, 'secret.txt'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/outside permitted scope/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refuses non-absolute paths', async () => {
|
||||||
|
const result = await resolveGrantRoot(makeSql([]), 'relative/path', project, whitelist);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/absolute/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refuses missing paths without prompting', async () => {
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([]),
|
||||||
|
join(whitelist, 'nope'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/does not exist/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refuses when no repo-shape marker is found before hitting the whitelist root', async () => {
|
||||||
|
// Build a directory tree under the whitelist that has NO repo markers
|
||||||
|
// all the way up to the whitelist root.
|
||||||
|
const plain = join(whitelist, 'plain-dir', 'nested');
|
||||||
|
await mkdir(plain, { recursive: true });
|
||||||
|
await writeFile(join(plain, 'just-a-file.txt'), 'x');
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([]),
|
||||||
|
join(plain, 'just-a-file.txt'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/no repo-shaped ancestor/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not grant the whitelist root itself as a fallback', async () => {
|
||||||
|
// Even if .git existed at the whitelist root (it doesn't), we'd refuse.
|
||||||
|
// Easier to assert: a path directly under whitelist with no repo marker.
|
||||||
|
const direct = join(whitelist, 'lone-file.txt');
|
||||||
|
await writeFile(direct, 'x');
|
||||||
|
const result = await resolveGrantRoot(makeSql([]), direct, project, whitelist);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('resolveGrantRoot — symlink-escape-mid-walk guard (Sam 2026-05-22)', () => {
|
||||||
|
it('refuses a symlinked input whose realpath sits outside the whitelist', async () => {
|
||||||
|
// The symlink lives nominally inside the whitelist, but its target
|
||||||
|
// (realpath) is outside. The guard's first realpath() call normalizes
|
||||||
|
// and the up-front whitelist check refuses immediately.
|
||||||
|
const link = join(whitelist, 'escape-link');
|
||||||
|
try {
|
||||||
|
await symlink(outside, link);
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([]),
|
||||||
|
join(link, 'secret.txt'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/outside permitted scope/);
|
||||||
|
} finally {
|
||||||
|
await rm(link, { force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('walk loop terminates at the whitelist root, not at filesystem /', async () => {
|
||||||
|
// Construct a deep tree with NO repo markers anywhere. Without a bound,
|
||||||
|
// the walk would chase parents up to "/". The bound flips the loop into
|
||||||
|
// a refusal once the cursor equals the realpath'd whitelist root.
|
||||||
|
const deep = join(whitelist, 'a', 'b', 'c', 'd');
|
||||||
|
await mkdir(deep, { recursive: true });
|
||||||
|
await writeFile(join(deep, 'leaf.txt'), 'x');
|
||||||
|
const result = await resolveGrantRoot(makeSql([]), join(deep, 'leaf.txt'), project, whitelist);
|
||||||
|
expect(result.ok).toBe(false);
|
||||||
|
if (!result.ok) expect(result.reason).toMatch(/no repo-shaped ancestor/);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('resolveGrantRoot — nearest-project disambiguation', () => {
|
||||||
|
it('prefers the longest matching project path over a shorter ancestor', async () => {
|
||||||
|
const outer = whitelist;
|
||||||
|
const inner = fork; // /whitelist/forks/codecontext, deeper than outer
|
||||||
|
const result = await resolveGrantRoot(
|
||||||
|
makeSql([{ path: outer }, { path: inner }]),
|
||||||
|
join(fork, 'main.go'),
|
||||||
|
project,
|
||||||
|
whitelist,
|
||||||
|
);
|
||||||
|
expect(result.ok).toBe(true);
|
||||||
|
if (result.ok) expect(result.root).toBe(inner);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Belt-and-suspenders: silence a known dynamic-import warning that vitest
|
||||||
|
// occasionally emits on transient fs operations in CI but never in dev.
|
||||||
|
vi.spyOn(console, 'warn').mockImplementation(() => {});
|
||||||
93
apps/server/src/services/__tests__/path_guard.test.ts
Normal file
93
apps/server/src/services/__tests__/path_guard.test.ts
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
// v1.13.17-cross-repo-reads: pathGuard now accepts an optional extraRoots
|
||||||
|
// list. Validates the primary-root path stays the source of truth and that
|
||||||
|
// extra roots are consulted when (and only when) the primary rejects.
|
||||||
|
|
||||||
|
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||||
|
import { mkdtemp, rm, mkdir, writeFile, symlink } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { realpath } from 'node:fs/promises';
|
||||||
|
import { pathGuard, PathScopeError } from '../path_guard.js';
|
||||||
|
|
||||||
|
let tmp: string;
|
||||||
|
let projectRoot: string;
|
||||||
|
let altRoot: string;
|
||||||
|
let outsideDir: string;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
tmp = await realpath(await mkdtemp(join(tmpdir(), 'boocode-pg-')));
|
||||||
|
projectRoot = join(tmp, 'project');
|
||||||
|
altRoot = join(tmp, 'alt');
|
||||||
|
outsideDir = join(tmp, 'outside');
|
||||||
|
await mkdir(projectRoot, { recursive: true });
|
||||||
|
await mkdir(altRoot, { recursive: true });
|
||||||
|
await mkdir(outsideDir, { recursive: true });
|
||||||
|
await writeFile(join(projectRoot, 'inside.txt'), 'p');
|
||||||
|
await writeFile(join(altRoot, 'cross.txt'), 'a');
|
||||||
|
await writeFile(join(outsideDir, 'forbidden.txt'), 'x');
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await rm(tmp, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('pathGuard (v1.13.17 extraRoots)', () => {
|
||||||
|
it('accepts paths inside the primary projectRoot', async () => {
|
||||||
|
const real = await pathGuard(projectRoot, 'inside.txt');
|
||||||
|
expect(real).toBe(join(projectRoot, 'inside.txt'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects paths outside the primary root when no extra roots given', async () => {
|
||||||
|
await expect(pathGuard(projectRoot, join(outsideDir, 'forbidden.txt'))).rejects.toBeInstanceOf(
|
||||||
|
PathScopeError,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('accepts cross-root paths when the matching extra root is provided', async () => {
|
||||||
|
const real = await pathGuard(projectRoot, join(altRoot, 'cross.txt'), [altRoot]);
|
||||||
|
expect(real).toBe(join(altRoot, 'cross.txt'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects cross-root paths even with extra roots when no root matches', async () => {
|
||||||
|
await expect(
|
||||||
|
pathGuard(projectRoot, join(outsideDir, 'forbidden.txt'), [altRoot]),
|
||||||
|
).rejects.toBeInstanceOf(PathScopeError);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ignores empty-string extra roots silently', async () => {
|
||||||
|
const real = await pathGuard(projectRoot, join(altRoot, 'cross.txt'), ['', altRoot]);
|
||||||
|
expect(real).toBe(join(altRoot, 'cross.txt'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('error message contains the request_read_access hint when scope rejects', async () => {
|
||||||
|
try {
|
||||||
|
await pathGuard(projectRoot, join(outsideDir, 'forbidden.txt'));
|
||||||
|
throw new Error('should have thrown');
|
||||||
|
} catch (err) {
|
||||||
|
expect(err).toBeInstanceOf(PathScopeError);
|
||||||
|
expect((err as Error).message).toContain('request_read_access');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('still resolves symlinks before the scope check', async () => {
|
||||||
|
const linkPath = join(projectRoot, 'link-to-outside');
|
||||||
|
await symlink(join(outsideDir, 'forbidden.txt'), linkPath);
|
||||||
|
// Symlink target escapes both primary and the single extra root, so
|
||||||
|
// even though the surface path "looks" inside projectRoot, the real
|
||||||
|
// path resolves outside and the guard rejects.
|
||||||
|
await expect(pathGuard(projectRoot, linkPath, [altRoot])).rejects.toBeInstanceOf(
|
||||||
|
PathScopeError,
|
||||||
|
);
|
||||||
|
// But adding outsideDir as an extra root accepts (realpath inside it).
|
||||||
|
const real = await pathGuard(projectRoot, linkPath, [altRoot, outsideDir]);
|
||||||
|
expect(real).toBe(join(outsideDir, 'forbidden.txt'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('tries extra roots in order until one accepts', async () => {
|
||||||
|
const real = await pathGuard(projectRoot, join(altRoot, 'cross.txt'), [
|
||||||
|
outsideDir, // rejects
|
||||||
|
altRoot, // accepts
|
||||||
|
]);
|
||||||
|
expect(real).toBe(join(altRoot, 'cross.txt'));
|
||||||
|
});
|
||||||
|
});
|
||||||
357
apps/server/src/services/__tests__/xml-parser.test.ts
Normal file
357
apps/server/src/services/__tests__/xml-parser.test.ts
Normal file
@@ -0,0 +1,357 @@
|
|||||||
|
// v1.13.16: covers the Qwen/Hermes <tool_call> parser, the new Anthropic
|
||||||
|
// <invoke> parser, the partial-opener detector for both flavors, the unified
|
||||||
|
// extraction helper, and the unknown-tool error formatter that downstream
|
||||||
|
// dispatch uses to give the model a recovery hint when it drifts to a
|
||||||
|
// Claude Code tool name like read_file instead of BooCode's view_file.
|
||||||
|
|
||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
import {
|
||||||
|
parseXmlToolCall,
|
||||||
|
parseInvokeToolCall,
|
||||||
|
partialXmlOpenerStart,
|
||||||
|
extractToolCallBlocks,
|
||||||
|
XML_TOOL_OPEN,
|
||||||
|
XML_TOOL_CLOSE,
|
||||||
|
INVOKE_TOOL_OPEN,
|
||||||
|
INVOKE_TOOL_CLOSE,
|
||||||
|
} from '../inference/xml-parser.js';
|
||||||
|
import {
|
||||||
|
levenshtein,
|
||||||
|
suggestToolName,
|
||||||
|
formatUnknownToolError,
|
||||||
|
} from '../inference/tool-suggestions.js';
|
||||||
|
|
||||||
|
describe('parseXmlToolCall (Qwen/Hermes <tool_call>)', () => {
|
||||||
|
it('parses a well-formed single-parameter call', () => {
|
||||||
|
const block = '<tool_call><function=view_file><parameter=path>/tmp/foo</parameter></function></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('parses multi-parameter call', () => {
|
||||||
|
const block = '<tool_call><function=grep><parameter=pattern>foo</parameter><parameter=path>src/</parameter></function></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toEqual({
|
||||||
|
name: 'grep',
|
||||||
|
args: { pattern: 'foo', path: 'src/' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('JSON-parses numeric parameter values', () => {
|
||||||
|
const block = '<tool_call><function=foo><parameter=count>42</parameter></function></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toEqual({ name: 'foo', args: { count: 42 } });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('tolerates whitespace around = in function (v1.13.16 tightening)', () => {
|
||||||
|
const block = '<tool_call><function = view_file><parameter=path>/tmp/foo</parameter></function></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('tolerates whitespace around = in parameter (v1.13.16 tightening)', () => {
|
||||||
|
const block = '<tool_call><function=view_file><parameter = path>/tmp/foo</parameter></function></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when function name is missing', () => {
|
||||||
|
const block = '<tool_call><parameter=path>/tmp/foo</parameter></tool_call>';
|
||||||
|
expect(parseXmlToolCall(block)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('parseInvokeToolCall (Anthropic <invoke>) — v1.13.16', () => {
|
||||||
|
// Spec case 1
|
||||||
|
it('parses a well-formed single-parameter call (spec case 1)', () => {
|
||||||
|
const block = '<invoke name="view_file"><parameter name="path">/tmp/foo</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 2
|
||||||
|
it('parses a multi-parameter call (spec case 2)', () => {
|
||||||
|
const block = '<invoke name="grep"><parameter name="pattern">foo</parameter><parameter name="path">src/</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'grep',
|
||||||
|
args: { pattern: 'foo', path: 'src/' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 3
|
||||||
|
it('tolerates newlines and spaces in attributes (spec case 3)', () => {
|
||||||
|
const block = `<invoke
|
||||||
|
name="view_file"
|
||||||
|
>
|
||||||
|
<parameter
|
||||||
|
name="path"
|
||||||
|
>/tmp/foo</parameter>
|
||||||
|
</invoke>`;
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 4 (parser portion — the not-found enrichment is tested below)
|
||||||
|
it('parses a call whose name is not a registered BooCode tool (spec case 4)', () => {
|
||||||
|
const block = '<invoke name="read_file"><parameter name="path">/tmp/foo</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'read_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('supports single-quoted attribute values', () => {
|
||||||
|
const block = "<invoke name='view_file'><parameter name='path'>/tmp/foo</parameter></invoke>";
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('JSON-parses numeric parameter values', () => {
|
||||||
|
const block = '<invoke name="foo"><parameter name="count">42</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({ name: 'foo', args: { count: 42 } });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('tolerates spaces around = inside name attribute', () => {
|
||||||
|
const block = '<invoke name = "view_file"><parameter name = "path">/tmp/foo</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toEqual({
|
||||||
|
name: 'view_file',
|
||||||
|
args: { path: '/tmp/foo' },
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when name attribute is missing', () => {
|
||||||
|
const block = '<invoke><parameter name="path">/tmp/foo</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when name attribute is empty', () => {
|
||||||
|
const block = '<invoke name=""><parameter name="path">/tmp/foo</parameter></invoke>';
|
||||||
|
expect(parseInvokeToolCall(block)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('exports the expected delimiters', () => {
|
||||||
|
expect(INVOKE_TOOL_OPEN).toBe('<invoke');
|
||||||
|
expect(INVOKE_TOOL_CLOSE).toBe('</invoke>');
|
||||||
|
expect(XML_TOOL_OPEN).toBe('<tool_call>');
|
||||||
|
expect(XML_TOOL_CLOSE).toBe('</tool_call>');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('partialXmlOpenerStart (v1.13.16 — both flavors)', () => {
|
||||||
|
it('returns -1 when the buffer is empty', () => {
|
||||||
|
expect(partialXmlOpenerStart('')).toBe(-1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns -1 when the buffer has no openers', () => {
|
||||||
|
expect(partialXmlOpenerStart('plain prose, no markup')).toBe(-1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns the index of a complete <tool_call> opener (existing)', () => {
|
||||||
|
expect(partialXmlOpenerStart('prose <tool_call>more')).toBe(6);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns the index of a complete <invoke opener (v1.13.16)', () => {
|
||||||
|
expect(partialXmlOpenerStart('prose <invoke name=')).toBe(6);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('holds a partial <tool_ prefix at end of buffer', () => {
|
||||||
|
expect(partialXmlOpenerStart('text <tool_')).toBe(5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('holds a partial <invo prefix at end of buffer (v1.13.16)', () => {
|
||||||
|
expect(partialXmlOpenerStart('text <invo')).toBe(5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('holds a bare < at end of buffer', () => {
|
||||||
|
expect(partialXmlOpenerStart('text <')).toBe(5);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns -1 when < is followed by non-opener text', () => {
|
||||||
|
expect(partialXmlOpenerStart('text <unknown>')).toBe(-1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns the earliest opener when both flavors are present', () => {
|
||||||
|
expect(partialXmlOpenerStart('xxx <tool_call>YYY <invoke>')).toBe(4);
|
||||||
|
expect(partialXmlOpenerStart('xxx <invoke>YYY <tool_call>')).toBe(4);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('extractToolCallBlocks (v1.13.16 — unified extraction)', () => {
|
||||||
|
// Spec case 1 (extraction-level)
|
||||||
|
it('extracts a single <invoke> block (spec case 1)', () => {
|
||||||
|
const input = '<invoke name="view_file"><parameter name="path">/tmp/foo</parameter></invoke>';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([{ name: 'view_file', args: { path: '/tmp/foo' } }]);
|
||||||
|
expect(result.flushed).toBe('');
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 5: opener arrives in one chunk, closer in the next.
|
||||||
|
it('holds the partial <invoke> chunk when the closer has not arrived (spec case 5, first chunk)', () => {
|
||||||
|
const firstChunk = '<invoke name="view_file"><parameter name="path">/tmp/foo</parameter>';
|
||||||
|
const result = extractToolCallBlocks(firstChunk);
|
||||||
|
expect(result.calls).toEqual([]);
|
||||||
|
expect(result.flushed).toBe('');
|
||||||
|
expect(result.remaining).toBe(firstChunk);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('extracts the block once the closer arrives in a later chunk (spec case 5, completion)', () => {
|
||||||
|
const firstChunk = '<invoke name="view_file"><parameter name="path">/tmp/foo</parameter>';
|
||||||
|
const r1 = extractToolCallBlocks(firstChunk);
|
||||||
|
const combined = r1.remaining + '</invoke>';
|
||||||
|
const r2 = extractToolCallBlocks(combined);
|
||||||
|
expect(r2.calls).toEqual([{ name: 'view_file', args: { path: '/tmp/foo' } }]);
|
||||||
|
expect(r2.flushed).toBe('');
|
||||||
|
expect(r2.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 6: prose interleaving
|
||||||
|
it('flushes prose around a recognized block but not the markup itself (spec case 6)', () => {
|
||||||
|
const input = 'I will read the file.\n<invoke name="view_file"><parameter name="path">/tmp/foo</parameter></invoke>\nThanks.';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([{ name: 'view_file', args: { path: '/tmp/foo' } }]);
|
||||||
|
expect(result.flushed).toBe('I will read the file.\n\nThanks.');
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
// Spec case 7 regression
|
||||||
|
it('extracts a <tool_call> Qwen block alongside the new code path (spec case 7 regression)', () => {
|
||||||
|
const input = '<tool_call><function=view_file><parameter=path>/tmp/foo</parameter></function></tool_call>';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([{ name: 'view_file', args: { path: '/tmp/foo' } }]);
|
||||||
|
expect(result.flushed).toBe('');
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('extracts mixed-format blocks in source order (hand-back: shared counter)', () => {
|
||||||
|
const input =
|
||||||
|
'<invoke name="view_file"><parameter name="path">/a</parameter></invoke>' +
|
||||||
|
' middle ' +
|
||||||
|
'<tool_call><function=grep><parameter=pattern>foo</parameter></function></tool_call>';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([
|
||||||
|
{ name: 'view_file', args: { path: '/a' } },
|
||||||
|
{ name: 'grep', args: { pattern: 'foo' } },
|
||||||
|
]);
|
||||||
|
expect(result.flushed).toBe(' middle ');
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('drops a malformed <invoke> block silently (matches existing <tool_call> behavior)', () => {
|
||||||
|
const input = 'prose <invoke><parameter name="path">/a</parameter></invoke> trailing';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([]);
|
||||||
|
expect(result.flushed).toBe('prose trailing');
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('holds a tail with a fresh partial opener after extracting earlier complete blocks', () => {
|
||||||
|
const input = '<invoke name="view_file"><parameter name="path">/a</parameter></invoke> next: <tool_';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([{ name: 'view_file', args: { path: '/a' } }]);
|
||||||
|
expect(result.flushed).toBe(' next: ');
|
||||||
|
expect(result.remaining).toBe('<tool_');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('passes plain prose straight through when no markup is present', () => {
|
||||||
|
const input = 'just some text with a < character but no opener';
|
||||||
|
const result = extractToolCallBlocks(input);
|
||||||
|
expect(result.calls).toEqual([]);
|
||||||
|
expect(result.flushed).toBe(input);
|
||||||
|
expect(result.remaining).toBe('');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('levenshtein', () => {
|
||||||
|
it('returns 0 for identical strings', () => {
|
||||||
|
expect(levenshtein('view_file', 'view_file')).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns the length when one string is empty', () => {
|
||||||
|
expect(levenshtein('', 'view_file')).toBe(9);
|
||||||
|
expect(levenshtein('view_file', '')).toBe(9);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('computes a small distance for a single-character substitution', () => {
|
||||||
|
expect(levenshtein('cat', 'bat')).toBe(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('computes a known case: read_file → view_file is 4', () => {
|
||||||
|
// r→v, e→i, a→e, d→w → 4 substitutions, same length
|
||||||
|
expect(levenshtein('read_file', 'view_file')).toBe(4);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('suggestToolName (v1.13.16)', () => {
|
||||||
|
const tools = [
|
||||||
|
'view_file',
|
||||||
|
'list_dir',
|
||||||
|
'grep',
|
||||||
|
'find_files',
|
||||||
|
'view_truncated_output',
|
||||||
|
'ask_user_input',
|
||||||
|
'web_search',
|
||||||
|
];
|
||||||
|
|
||||||
|
it('suggests the closest match when distance is small', () => {
|
||||||
|
expect(suggestToolName('view_files', tools)).toBe('view_file');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('suggests via substring match when distance alone would miss', () => {
|
||||||
|
// 'file' is a substring of multiple tools; closest by distance wins.
|
||||||
|
expect(suggestToolName('file', tools)).toBe('view_file');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when nothing is close', () => {
|
||||||
|
expect(suggestToolName('xxxx_yyyy_zzzz', tools)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is case-insensitive in the distance check', () => {
|
||||||
|
expect(suggestToolName('VIEW_FILE', tools)).toBe('view_file');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('formatUnknownToolError (v1.13.16)', () => {
|
||||||
|
const tools = ['view_file', 'list_dir', 'grep', 'find_files'];
|
||||||
|
|
||||||
|
it('includes the wrong name and the available tools list', () => {
|
||||||
|
const msg = formatUnknownToolError('read_file', tools);
|
||||||
|
expect(msg).toContain("Tool 'read_file' not found");
|
||||||
|
expect(msg).toContain('Available tools:');
|
||||||
|
expect(msg).toContain('view_file');
|
||||||
|
expect(msg).toContain('find_files');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('includes a suggestion when the drifted name is within threshold', () => {
|
||||||
|
// distance(view_files, view_file) = 1 (one extra char)
|
||||||
|
const msg = formatUnknownToolError('view_files', tools);
|
||||||
|
expect(msg).toContain('Did you mean: view_file?');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('omits the suggestion clause when no tool is close enough', () => {
|
||||||
|
const msg = formatUnknownToolError('zzzzzzz', tools);
|
||||||
|
expect(msg).toContain("Tool 'zzzzzzz' not found");
|
||||||
|
expect(msg).toContain('Available tools:');
|
||||||
|
expect(msg).not.toContain('Did you mean');
|
||||||
|
});
|
||||||
|
|
||||||
|
// The drift incident in the recon (chat 30d8…1be7167, msg 7ff558f4) had the
|
||||||
|
// model emit <invoke name="read_file">. lev(read_file, view_file) = 4, so
|
||||||
|
// the spec's threshold (<=3) doesn't suggest view_file — the model still
|
||||||
|
// gets the available-tools list to pick from. This pins that behavior so a
|
||||||
|
// future loosening of the threshold is a deliberate choice.
|
||||||
|
it('does not suggest view_file for the read_file drift case (distance is 4, over threshold)', () => {
|
||||||
|
const msg = formatUnknownToolError('read_file', tools);
|
||||||
|
expect(msg).not.toContain('Did you mean');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -47,8 +47,12 @@ export interface FindFilesResult {
|
|||||||
truncated: boolean;
|
truncated: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function listDir(projectRoot: string, relPath: string): Promise<ListDirResult> {
|
export async function listDir(
|
||||||
const real = await pathGuard(projectRoot, relPath);
|
projectRoot: string,
|
||||||
|
relPath: string,
|
||||||
|
opts?: { extra_roots?: readonly string[] },
|
||||||
|
): Promise<ListDirResult> {
|
||||||
|
const real = await pathGuard(projectRoot, relPath, opts?.extra_roots);
|
||||||
const s = await stat(real);
|
const s = await stat(real);
|
||||||
if (!s.isDirectory()) {
|
if (!s.isDirectory()) {
|
||||||
throw new PathScopeError(`not a directory: ${relPath}`);
|
throw new PathScopeError(`not a directory: ${relPath}`);
|
||||||
@@ -82,8 +86,12 @@ export async function listDir(projectRoot: string, relPath: string): Promise<Lis
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function viewFile(projectRoot: string, relPath: string): Promise<ViewFileResult> {
|
export async function viewFile(
|
||||||
const real = await pathGuard(projectRoot, relPath);
|
projectRoot: string,
|
||||||
|
relPath: string,
|
||||||
|
opts?: { extra_roots?: readonly string[] },
|
||||||
|
): Promise<ViewFileResult> {
|
||||||
|
const real = await pathGuard(projectRoot, relPath, opts?.extra_roots);
|
||||||
const s = await stat(real);
|
const s = await stat(real);
|
||||||
if (!s.isFile()) {
|
if (!s.isFile()) {
|
||||||
throw new PathScopeError(`not a file: ${relPath}`);
|
throw new PathScopeError(`not a file: ${relPath}`);
|
||||||
@@ -119,10 +127,10 @@ interface RipgrepMatch {
|
|||||||
export async function grep(
|
export async function grep(
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
pattern: string,
|
pattern: string,
|
||||||
opts?: { path?: string; max_matches?: number; case_sensitive?: boolean; hidden?: boolean }
|
opts?: { path?: string; max_matches?: number; case_sensitive?: boolean; hidden?: boolean; extra_roots?: readonly string[] }
|
||||||
): Promise<GrepResult> {
|
): Promise<GrepResult> {
|
||||||
const targetPath = opts?.path ?? projectRoot;
|
const targetPath = opts?.path ?? projectRoot;
|
||||||
const target = await pathGuard(projectRoot, targetPath);
|
const target = await pathGuard(projectRoot, targetPath, opts?.extra_roots);
|
||||||
const limit = Math.min(
|
const limit = Math.min(
|
||||||
Math.max(opts?.max_matches ?? DEFAULT_GREP_RESULTS, 1),
|
Math.max(opts?.max_matches ?? DEFAULT_GREP_RESULTS, 1),
|
||||||
MAX_GREP_RESULTS
|
MAX_GREP_RESULTS
|
||||||
@@ -192,14 +200,14 @@ export async function grep(
|
|||||||
export async function findFiles(
|
export async function findFiles(
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
pattern?: string,
|
pattern?: string,
|
||||||
opts?: { type?: 'file' | 'dir'; max_results?: number; path?: string }
|
opts?: { type?: 'file' | 'dir'; max_results?: number; path?: string; extra_roots?: readonly string[] }
|
||||||
): Promise<FindFilesResult> {
|
): Promise<FindFilesResult> {
|
||||||
const limit = Math.min(
|
const limit = Math.min(
|
||||||
Math.max(opts?.max_results ?? DEFAULT_FIND_RESULTS, 1),
|
Math.max(opts?.max_results ?? DEFAULT_FIND_RESULTS, 1),
|
||||||
MAX_FIND_RESULTS
|
MAX_FIND_RESULTS
|
||||||
);
|
);
|
||||||
const target = opts?.path != null
|
const target = opts?.path != null
|
||||||
? await pathGuard(projectRoot, opts.path)
|
? await pathGuard(projectRoot, opts.path, opts?.extra_roots)
|
||||||
: projectRoot;
|
: projectRoot;
|
||||||
const args = ['--files'];
|
const args = ['--files'];
|
||||||
if (pattern) args.push('--glob', pattern);
|
if (pattern) args.push('--glob', pattern);
|
||||||
|
|||||||
161
apps/server/src/services/grant_resolver.ts
Normal file
161
apps/server/src/services/grant_resolver.ts
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
// v1.13.17-cross-repo-reads: derives the grant root for a path the user is
|
||||||
|
// being asked to approve cross-repo read access to.
|
||||||
|
//
|
||||||
|
// Per design decision D1: grant unit = nearest registered project root,
|
||||||
|
// then nearest path-whitelist ancestor that looks like a repo root, then
|
||||||
|
// refuse. Granting the literal file path is too narrow (next file in the
|
||||||
|
// same repo re-prompts). Granting an arbitrary parent dir over-scopes.
|
||||||
|
//
|
||||||
|
// The resolver runs in two contexts:
|
||||||
|
// 1. request_read_access.execute — pre-prompt validation (cheap; bails
|
||||||
|
// early if the path can't plausibly be granted so the user is never
|
||||||
|
// asked about /etc/passwd)
|
||||||
|
// 2. POST /api/chats/:id/grant_read_access — at decision time, re-derives
|
||||||
|
// the root and persists it on sessions.allowed_read_paths
|
||||||
|
//
|
||||||
|
// Sam (2026-05-22 dispatch confirmation): "in the project-root resolver
|
||||||
|
// ancestor walk, stop the moment parent exits PROJECT_ROOT_WHITELIST or hits
|
||||||
|
// filesystem root — check on every iteration, not just final parent.
|
||||||
|
// Symlinked input must not be able to escape the whitelist during the
|
||||||
|
// walk." Hence the loop here checks both the walk bound AND the still-
|
||||||
|
// inside-whitelist invariant every step.
|
||||||
|
|
||||||
|
import { access, realpath } from 'node:fs/promises';
|
||||||
|
import { constants } from 'node:fs';
|
||||||
|
import { dirname, isAbsolute, sep } from 'node:path';
|
||||||
|
import type { Sql } from '../db.js';
|
||||||
|
|
||||||
|
// Files whose presence in a directory marks it as a repo root for grant
|
||||||
|
// purposes. Kept narrow on purpose; broader heuristics (e.g. ".project",
|
||||||
|
// "pyproject.toml") can be added with measured intent. Each entry is a
|
||||||
|
// literal basename — no globs.
|
||||||
|
const REPO_MARKERS: ReadonlyArray<string> = [
|
||||||
|
'.git',
|
||||||
|
'package.json',
|
||||||
|
'go.mod',
|
||||||
|
'Cargo.toml',
|
||||||
|
];
|
||||||
|
|
||||||
|
export type GrantResolution =
|
||||||
|
| { ok: true; root: string; source: 'project' | 'whitelist' }
|
||||||
|
| { ok: false; reason: string };
|
||||||
|
|
||||||
|
function isUnder(child: string, parent: string): boolean {
|
||||||
|
return child === parent || child.startsWith(parent + sep);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function exists(path: string): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
await access(path, constants.F_OK);
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function isRepoShaped(dir: string): Promise<boolean> {
|
||||||
|
for (const marker of REPO_MARKERS) {
|
||||||
|
if (await exists(`${dir}${sep}${marker}`)) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolves an absolute path to its grant root or refuses with a reason
|
||||||
|
// string suitable for surfacing to the model. Pure helper — no DB writes,
|
||||||
|
// no broker publishes. Caller persists the root on session.allowed_read_paths
|
||||||
|
// if it wants the grant to stick.
|
||||||
|
//
|
||||||
|
// Arguments:
|
||||||
|
// sql — used only to read projects.path (no writes)
|
||||||
|
// requestedPath — absolute path the model wants to read
|
||||||
|
// projectRoot — the session's primary project root (already
|
||||||
|
// realpath'd by caller). Used to short-circuit
|
||||||
|
// "already in scope".
|
||||||
|
// whitelistRoot — PROJECT_ROOT_WHITELIST from config (default /opt).
|
||||||
|
// Walk bound for the repo-shape fallback.
|
||||||
|
//
|
||||||
|
// Returns { ok: true, root, source } on success; { ok: false, reason } else.
|
||||||
|
export async function resolveGrantRoot(
|
||||||
|
sql: Sql,
|
||||||
|
requestedPath: string,
|
||||||
|
projectRoot: string,
|
||||||
|
whitelistRoot: string,
|
||||||
|
): Promise<GrantResolution> {
|
||||||
|
if (typeof requestedPath !== 'string' || requestedPath.length === 0) {
|
||||||
|
return { ok: false, reason: 'path is required' };
|
||||||
|
}
|
||||||
|
if (!isAbsolute(requestedPath)) {
|
||||||
|
return { ok: false, reason: 'path must be absolute' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve symlinks so subsequent ancestor checks compare apples-to-apples
|
||||||
|
// with realpath'd projectRoot. If the path doesn't exist at all, bail
|
||||||
|
// before bothering the user — the model is asking about a phantom.
|
||||||
|
let real: string;
|
||||||
|
try {
|
||||||
|
real = await realpath(requestedPath);
|
||||||
|
} catch {
|
||||||
|
return { ok: false, reason: `path does not exist: ${requestedPath}` };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Whitelist guard. Symlinked inputs can resolve outside the whitelist
|
||||||
|
// even when the surface-form path looks inside it; that's why we test
|
||||||
|
// the *real* path here, not the requested one.
|
||||||
|
let realWhitelist: string;
|
||||||
|
try {
|
||||||
|
realWhitelist = await realpath(whitelistRoot);
|
||||||
|
} catch {
|
||||||
|
return { ok: false, reason: `whitelist root does not exist: ${whitelistRoot}` };
|
||||||
|
}
|
||||||
|
if (!isUnder(real, realWhitelist)) {
|
||||||
|
return { ok: false, reason: 'path outside permitted scope' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Already in scope? No prompt needed; the tool's caller should retry.
|
||||||
|
if (isUnder(real, projectRoot)) {
|
||||||
|
return { ok: false, reason: 'path already accessible without a grant' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Look for a registered project whose root is an ancestor of the
|
||||||
|
// requested path. Pick the LONGEST match (nearest ancestor wins) so
|
||||||
|
// sub-projects don't get over-broadened.
|
||||||
|
const projectRows = await sql<{ path: string }[]>`
|
||||||
|
SELECT path FROM projects WHERE status = 'open'
|
||||||
|
`;
|
||||||
|
let bestProject: string | null = null;
|
||||||
|
for (const row of projectRows) {
|
||||||
|
if (!row.path) continue;
|
||||||
|
if (!isUnder(real, row.path)) continue;
|
||||||
|
if (bestProject === null || row.path.length > bestProject.length) {
|
||||||
|
bestProject = row.path;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (bestProject !== null) {
|
||||||
|
return { ok: true, root: bestProject, source: 'project' };
|
||||||
|
}
|
||||||
|
|
||||||
|
// Repo-shape fallback. Walk from the requested path upward toward the
|
||||||
|
// whitelist root. At every iteration: confirm we're still inside the
|
||||||
|
// whitelist (so a symlinked component can't slip the bound mid-walk)
|
||||||
|
// and confirm we haven't hit the filesystem root. The first dir with a
|
||||||
|
// REPO_MARKER child is the grant root.
|
||||||
|
let cursor = real;
|
||||||
|
while (true) {
|
||||||
|
// Don't grant the whitelist root itself — that would be far too broad.
|
||||||
|
if (cursor === realWhitelist) {
|
||||||
|
return { ok: false, reason: 'no repo-shaped ancestor found under whitelist' };
|
||||||
|
}
|
||||||
|
if (!isUnder(cursor, realWhitelist)) {
|
||||||
|
return { ok: false, reason: 'path outside permitted scope' };
|
||||||
|
}
|
||||||
|
const parent = dirname(cursor);
|
||||||
|
if (parent === cursor) {
|
||||||
|
// Hit filesystem root without finding a repo marker.
|
||||||
|
return { ok: false, reason: 'no repo-shaped ancestor found under whitelist' };
|
||||||
|
}
|
||||||
|
if (await isRepoShaped(cursor)) {
|
||||||
|
return { ok: true, root: cursor, source: 'whitelist' };
|
||||||
|
}
|
||||||
|
cursor = parent;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,7 +7,17 @@ import type { ToolCall, ToolResult } from '../../types/api.js';
|
|||||||
// JSON columns; the swap to parts-as-source-of-truth happens in a later
|
// JSON columns; the swap to parts-as-source-of-truth happens in a later
|
||||||
// v1.13 dispatch alongside the AI SDK streamText migration.
|
// v1.13 dispatch alongside the AI SDK streamText migration.
|
||||||
|
|
||||||
export type PartKind = 'text' | 'tool_call' | 'tool_result' | 'reasoning' | 'step_start';
|
// v1.13.13: 'synthesis' added. Schema CHECK constraint is updated in lockstep
|
||||||
|
// (schema.sql adds 'synthesis' to message_parts_kind_chk on startup). The
|
||||||
|
// dispatch's claim that no schema migration was needed assumed kind was a
|
||||||
|
// bare text column — it isn't; the constraint enumerates allowed values.
|
||||||
|
export type PartKind =
|
||||||
|
| 'text'
|
||||||
|
| 'tool_call'
|
||||||
|
| 'tool_result'
|
||||||
|
| 'reasoning'
|
||||||
|
| 'step_start'
|
||||||
|
| 'synthesis';
|
||||||
|
|
||||||
export interface PartInsert {
|
export interface PartInsert {
|
||||||
message_id: string;
|
message_id: string;
|
||||||
|
|||||||
@@ -6,12 +6,9 @@ import type {
|
|||||||
import * as modelContext from '../model-context.js';
|
import * as modelContext from '../model-context.js';
|
||||||
import { toolJsonSchemas, type ToolJsonSchema } from '../tools.js';
|
import { toolJsonSchemas, type ToolJsonSchema } from '../tools.js';
|
||||||
import type { OpenAiMessage } from './payload.js';
|
import type { OpenAiMessage } from './payload.js';
|
||||||
import {
|
// v1.13.16: extractToolCallBlocks replaces the inline opener-search loop and
|
||||||
XML_TOOL_CLOSE,
|
// recognizes both Qwen <tool_call> and Anthropic <invoke> markup in one pass.
|
||||||
XML_TOOL_OPEN,
|
import { extractToolCallBlocks } from './xml-parser.js';
|
||||||
parseXmlToolCall,
|
|
||||||
partialXmlOpenerStart,
|
|
||||||
} from './xml-parser.js';
|
|
||||||
import { DB_FLUSH_INTERVAL_MS, type StreamPhaseState } from './types.js';
|
import { DB_FLUSH_INTERVAL_MS, type StreamPhaseState } from './types.js';
|
||||||
import type {
|
import type {
|
||||||
InferenceContext,
|
InferenceContext,
|
||||||
@@ -132,16 +129,24 @@ function buildAiTools(schemas: ToolJsonSchema[]): Record<string, ReturnType<type
|
|||||||
// v1.10.5 Qwen-coder XML fallback. Some local models (notably qwen3-coder via
|
// v1.10.5 Qwen-coder XML fallback. Some local models (notably qwen3-coder via
|
||||||
// llama-swap) emit tool calls as inline XML inside delta.content rather than
|
// llama-swap) emit tool calls as inline XML inside delta.content rather than
|
||||||
// the structured tool_calls field. We extract them out of the streamed text
|
// the structured tool_calls field. We extract them out of the streamed text
|
||||||
// before flushing it to the client, mirroring the pre-AI-SDK behavior.
|
// before flushing it to the client.
|
||||||
//
|
//
|
||||||
// XML shape:
|
// Qwen shape:
|
||||||
// <tool_call>
|
// <tool_call>
|
||||||
// <function=NAME>
|
// <function=NAME>
|
||||||
// <parameter=KEY>VALUE</parameter>
|
// <parameter=KEY>VALUE</parameter>
|
||||||
// ...
|
// ...
|
||||||
// </function>
|
// </function>
|
||||||
// </tool_call>
|
// </tool_call>
|
||||||
// Multiple <tool_call> blocks may appear back-to-back; they never nest.
|
//
|
||||||
|
// v1.13.16: also recognize Anthropic <invoke> markup that qwen3.6-35b-a3b-mxfp4
|
||||||
|
// drifts to (training-data residue from Claude Code documentation):
|
||||||
|
// <invoke name="NAME">
|
||||||
|
// <parameter name="KEY">VALUE</parameter>
|
||||||
|
// </invoke>
|
||||||
|
// Both formats share the synthetic xml_call_${idx} ID space; the counter
|
||||||
|
// increments across whichever opener appears first. Multiple blocks may
|
||||||
|
// appear back-to-back in either format and they never nest.
|
||||||
export async function streamCompletion(
|
export async function streamCompletion(
|
||||||
ctx: InferenceContext,
|
ctx: InferenceContext,
|
||||||
model: string,
|
model: string,
|
||||||
@@ -209,47 +214,24 @@ export async function streamCompletion(
|
|||||||
switch (part.type) {
|
switch (part.type) {
|
||||||
case 'text-delta': {
|
case 'text-delta': {
|
||||||
pendingBuffer += part.text;
|
pendingBuffer += part.text;
|
||||||
// Extract any complete <tool_call>...</tool_call> blocks before
|
// v1.13.16: unified extraction. The helper finds the earliest-opening
|
||||||
// flushing visible text.
|
// complete <tool_call> or <invoke> block, flushes prose between/around
|
||||||
while (true) {
|
// them, holds any partial opener for the next chunk, and silently
|
||||||
const startIdx = pendingBuffer.indexOf(XML_TOOL_OPEN);
|
// drops blocks that fail to parse (matches pre-v1.13.16 behavior).
|
||||||
if (startIdx === -1) break;
|
const extracted = extractToolCallBlocks(pendingBuffer);
|
||||||
const closeIdx = pendingBuffer.indexOf(XML_TOOL_CLOSE, startIdx);
|
if (extracted.flushed.length > 0) {
|
||||||
if (closeIdx === -1) break;
|
content += extracted.flushed;
|
||||||
const blockEnd = closeIdx + XML_TOOL_CLOSE.length;
|
onDelta(extracted.flushed);
|
||||||
const block = pendingBuffer.slice(startIdx, blockEnd);
|
|
||||||
if (startIdx > 0) {
|
|
||||||
const before = pendingBuffer.slice(0, startIdx);
|
|
||||||
content += before;
|
|
||||||
onDelta(before);
|
|
||||||
}
|
}
|
||||||
const parsedCall = parseXmlToolCall(block);
|
for (const call of extracted.calls) {
|
||||||
if (parsedCall) {
|
|
||||||
const synthIdx = toolCalls.length;
|
const synthIdx = toolCalls.length;
|
||||||
toolCalls.push({
|
toolCalls.push({
|
||||||
id: `xml_call_${synthIdx}`,
|
id: `xml_call_${synthIdx}`,
|
||||||
name: parsedCall.name,
|
name: call.name,
|
||||||
args: parsedCall.args,
|
args: call.args,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
// Parse failures still drop the block — leaking <tool_call> XML to
|
pendingBuffer = extracted.remaining;
|
||||||
// the chat would look worse than silently swallowing the bad block.
|
|
||||||
pendingBuffer = pendingBuffer.slice(blockEnd);
|
|
||||||
}
|
|
||||||
// Hold back any (partial or full) unclosed opener; flush the rest.
|
|
||||||
const partialIdx = partialXmlOpenerStart(pendingBuffer);
|
|
||||||
if (partialIdx >= 0) {
|
|
||||||
if (partialIdx > 0) {
|
|
||||||
const flush = pendingBuffer.slice(0, partialIdx);
|
|
||||||
content += flush;
|
|
||||||
onDelta(flush);
|
|
||||||
}
|
|
||||||
pendingBuffer = pendingBuffer.slice(partialIdx);
|
|
||||||
} else if (pendingBuffer.length > 0) {
|
|
||||||
content += pendingBuffer;
|
|
||||||
onDelta(pendingBuffer);
|
|
||||||
pendingBuffer = '';
|
|
||||||
}
|
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
case 'tool-call': {
|
case 'tool-call': {
|
||||||
|
|||||||
@@ -4,6 +4,16 @@ import { PathScopeError } from '../path_guard.js';
|
|||||||
import { TOOLS_BY_NAME } from '../tools.js';
|
import { TOOLS_BY_NAME } from '../tools.js';
|
||||||
import { maybeFlagForCompaction } from './payload.js';
|
import { maybeFlagForCompaction } from './payload.js';
|
||||||
import { insertParts, partsFromAssistantMessage, partsFromToolMessage } from './parts.js';
|
import { insertParts, partsFromAssistantMessage, partsFromToolMessage } from './parts.js';
|
||||||
|
// v1.13.16: richer unknown-tool error so the model can self-correct when it
|
||||||
|
// drifts to a Claude Code tool name (e.g. read_file → suggest view_file).
|
||||||
|
// Applies to all unknown tool names, not just <invoke>-derived ones — at the
|
||||||
|
// dispatch layer we no longer know which format produced the call, and the
|
||||||
|
// extra signal is harmless for Qwen-derived calls.
|
||||||
|
import { formatUnknownToolError } from './tool-suggestions.js';
|
||||||
|
// v1.13.17-cross-repo-reads: pre-prompt validation for request_read_access.
|
||||||
|
// Resolves the grant root before pausing the loop so the user is never
|
||||||
|
// prompted about paths we couldn't grant anyway (e.g. /etc/passwd).
|
||||||
|
import { resolveGrantRoot } from '../grant_resolver.js';
|
||||||
import type {
|
import type {
|
||||||
InferenceContext,
|
InferenceContext,
|
||||||
StreamResult,
|
StreamResult,
|
||||||
@@ -14,14 +24,24 @@ import type {
|
|||||||
// the reference is read at call time (inside an async function body), not
|
// the reference is read at call time (inside an async function body), not
|
||||||
// at module top-level. Node + tsc resolve this cleanly.
|
// at module top-level. Node + tsc resolve this cleanly.
|
||||||
import { runAssistantTurn } from './turn.js';
|
import { runAssistantTurn } from './turn.js';
|
||||||
|
// v1.13.13: synthesis pipeline — replaces the immediate recursive turn when
|
||||||
|
// any of this batch's tool calls is in SYNTHESIS_TOOLS. Falls through to
|
||||||
|
// recursion on synthesis failure (timeout / model error). See module header
|
||||||
|
// in synthesisPipeline.ts for the auto-fetch + token-budget rules.
|
||||||
|
import { SYNTHESIS_TOOLS, runSynthesisPass } from '../synthesisPipeline.js';
|
||||||
|
|
||||||
async function executeToolCall(
|
async function executeToolCall(
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
toolCall: ToolCall
|
toolCall: ToolCall,
|
||||||
|
extraRoots: readonly string[],
|
||||||
): Promise<{ output: unknown; truncated: boolean; error?: string }> {
|
): Promise<{ output: unknown; truncated: boolean; error?: string }> {
|
||||||
const tool = TOOLS_BY_NAME[toolCall.name];
|
const tool = TOOLS_BY_NAME[toolCall.name];
|
||||||
if (!tool) {
|
if (!tool) {
|
||||||
return { output: null, truncated: false, error: `unknown tool: ${toolCall.name}` };
|
return {
|
||||||
|
output: null,
|
||||||
|
truncated: false,
|
||||||
|
error: formatUnknownToolError(toolCall.name, Object.keys(TOOLS_BY_NAME)),
|
||||||
|
};
|
||||||
}
|
}
|
||||||
const parsed = tool.inputSchema.safeParse(toolCall.args);
|
const parsed = tool.inputSchema.safeParse(toolCall.args);
|
||||||
if (!parsed.success) {
|
if (!parsed.success) {
|
||||||
@@ -48,7 +68,7 @@ async function executeToolCall(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
try {
|
try {
|
||||||
const output = await tool.execute(parsed.data, projectRoot);
|
const output = await tool.execute(parsed.data, projectRoot, extraRoots);
|
||||||
const truncated =
|
const truncated =
|
||||||
typeof output === 'object' && output !== null && 'truncated' in output
|
typeof output === 'object' && output !== null && 'truncated' in output
|
||||||
? Boolean((output as { truncated: unknown }).truncated)
|
? Boolean((output as { truncated: unknown }).truncated)
|
||||||
@@ -155,6 +175,12 @@ export async function executeToolPhase(
|
|||||||
// batches still execute the other tools normally.
|
// batches still execute the other tools normally.
|
||||||
ctx.publishUser({ type: 'chat_status', chat_id: chatId, status: 'tool_running', at: new Date().toISOString() });
|
ctx.publishUser({ type: 'chat_status', chat_id: chatId, status: 'tool_running', at: new Date().toISOString() });
|
||||||
let pausingForUserInput = false;
|
let pausingForUserInput = false;
|
||||||
|
// v1.13.13: capture synth-tool result text so the synthesis pipeline below
|
||||||
|
// doesn't have to re-fetch from DB. Array (not single) because a batch
|
||||||
|
// could theoretically include multiple synthesis tools — we take the first
|
||||||
|
// for the synthesis input. Race-free under Promise.all because each
|
||||||
|
// callback pushes its own captured value.
|
||||||
|
const synthEntries: Array<{ tc: ToolCall; output: unknown; error?: string }> = [];
|
||||||
await Promise.all(
|
await Promise.all(
|
||||||
toolCalls.map(async (tc) => {
|
toolCalls.map(async (tc) => {
|
||||||
const [toolRow] = await ctx.sql<{ id: string }[]>`
|
const [toolRow] = await ctx.sql<{ id: string }[]>`
|
||||||
@@ -185,7 +211,74 @@ export async function executeToolPhase(
|
|||||||
);
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const tres = await executeToolCall(projectRoot, tc);
|
// v1.13.17-cross-repo-reads: request_read_access pauses identically to
|
||||||
|
// ask_user_input EXCEPT for an up-front validation pass — if the path
|
||||||
|
// can't be granted under the whitelist / repo-shape rules, surface an
|
||||||
|
// immediate denial without prompting the user. Per design D1, we never
|
||||||
|
// ask the user about /etc/passwd or paths outside PROJECT_ROOT_WHITELIST.
|
||||||
|
if (tc.name === 'request_read_access') {
|
||||||
|
const tcArgs = tc.args as { path?: unknown; reason?: unknown };
|
||||||
|
const requested =
|
||||||
|
typeof tcArgs.path === 'string' ? tcArgs.path : '';
|
||||||
|
const resolution = await resolveGrantRoot(
|
||||||
|
ctx.sql,
|
||||||
|
requested,
|
||||||
|
projectRoot,
|
||||||
|
ctx.config.PROJECT_ROOT_WHITELIST,
|
||||||
|
);
|
||||||
|
if (!resolution.ok) {
|
||||||
|
// Auto-deny without pausing. The model sees the reason on its
|
||||||
|
// next turn and decides what to do.
|
||||||
|
const stored = {
|
||||||
|
tool_call_id: tc.id,
|
||||||
|
output: `denied: ${resolution.reason}`,
|
||||||
|
truncated: false,
|
||||||
|
};
|
||||||
|
await ctx.sql`
|
||||||
|
UPDATE messages
|
||||||
|
SET tool_results = ${ctx.sql.json(stored as never)}
|
||||||
|
WHERE id = ${toolMessageId}
|
||||||
|
`;
|
||||||
|
await insertParts(
|
||||||
|
ctx.sql,
|
||||||
|
partsFromToolMessage({ tool_results: stored }).map((p) => ({
|
||||||
|
...p,
|
||||||
|
message_id: toolMessageId,
|
||||||
|
})),
|
||||||
|
);
|
||||||
|
ctx.publish(sessionId, {
|
||||||
|
type: 'tool_result',
|
||||||
|
tool_message_id: toolMessageId,
|
||||||
|
chat_id: chatId,
|
||||||
|
tool_call_id: tc.id,
|
||||||
|
output: stored.output,
|
||||||
|
truncated: false,
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Path is plausibly grantable — install the pending sentinel and
|
||||||
|
// pause. The grant endpoint re-derives the root at decision time
|
||||||
|
// (state may have changed in the meantime) so we don't stash it here.
|
||||||
|
pausingForUserInput = true;
|
||||||
|
const sentinel = { tool_call_id: tc.id, output: null, truncated: false };
|
||||||
|
await ctx.sql`
|
||||||
|
UPDATE messages
|
||||||
|
SET tool_results = ${ctx.sql.json(sentinel as never)}
|
||||||
|
WHERE id = ${toolMessageId}
|
||||||
|
`;
|
||||||
|
await insertParts(
|
||||||
|
ctx.sql,
|
||||||
|
partsFromToolMessage({ tool_results: sentinel }).map((p) => ({
|
||||||
|
...p,
|
||||||
|
message_id: toolMessageId,
|
||||||
|
})),
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const tres = await executeToolCall(projectRoot, tc, session.allowed_read_paths);
|
||||||
|
if (SYNTHESIS_TOOLS.has(tc.name)) {
|
||||||
|
synthEntries.push({ tc, output: tres.output, ...(tres.error ? { error: tres.error } : {}) });
|
||||||
|
}
|
||||||
const stored = {
|
const stored = {
|
||||||
tool_call_id: tc.id,
|
tool_call_id: tc.id,
|
||||||
output: tres.output,
|
output: tres.output,
|
||||||
@@ -233,6 +326,41 @@ export async function executeToolPhase(
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.13.13: synthesis-pipeline branch. When any of this batch's tool calls
|
||||||
|
// is a codecontext overview/analysis tool that produced a non-error result,
|
||||||
|
// run a forced second-inference synthesis pass with auto-fetched files +
|
||||||
|
// project docs instead of the normal recursive runAssistantTurn. Falls
|
||||||
|
// through to the recursive call on synthesis failure (timeout, model
|
||||||
|
// error). User-abort re-throws so the outer handler runs.
|
||||||
|
const synthEntry = synthEntries.find((e) => !e.error && e.output != null);
|
||||||
|
if (synthEntry) {
|
||||||
|
// codecontext wrappers return { result: string, truncated: boolean, ... }.
|
||||||
|
// Defensive: stringify the output if it isn't the expected shape so the
|
||||||
|
// synthesis still has something to chew on rather than crashing on
|
||||||
|
// missing `.result`.
|
||||||
|
const out = synthEntry.output as { result?: unknown; truncated?: boolean; outputPath?: string };
|
||||||
|
const toolResultText =
|
||||||
|
typeof out?.result === 'string'
|
||||||
|
? out.result
|
||||||
|
: JSON.stringify(synthEntry.output);
|
||||||
|
// v1.13.15-b: forward the wrapper's truncation flag + opaque tmpfs id so
|
||||||
|
// synthesisPipeline can re-read the full content for reference extraction.
|
||||||
|
const ran = await runSynthesisPass({
|
||||||
|
ctx,
|
||||||
|
args,
|
||||||
|
session,
|
||||||
|
projectRoot,
|
||||||
|
toolName: synthEntry.tc.name,
|
||||||
|
toolResultText,
|
||||||
|
...(typeof out?.truncated === 'boolean' ? { truncated: out.truncated } : {}),
|
||||||
|
...(typeof out?.outputPath === 'string' ? { outputPath: out.outputPath } : {}),
|
||||||
|
});
|
||||||
|
if (ran) return;
|
||||||
|
// ran === false → synthesis failed (timeout / model error) → fall through
|
||||||
|
// to the standard recursive turn below. The synth message (if created)
|
||||||
|
// was already marked status='failed' inside runSynthesisPass.
|
||||||
|
}
|
||||||
|
|
||||||
const [nextAssistant] = await ctx.sql<{ id: string }[]>`
|
const [nextAssistant] = await ctx.sql<{ id: string }[]>`
|
||||||
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
INSERT INTO messages (session_id, chat_id, role, content, status, created_at)
|
||||||
VALUES (${sessionId}, ${chatId}, 'assistant', '', 'streaming', clock_timestamp())
|
VALUES (${sessionId}, ${chatId}, 'assistant', '', 'streaming', clock_timestamp())
|
||||||
|
|||||||
63
apps/server/src/services/inference/tool-suggestions.ts
Normal file
63
apps/server/src/services/inference/tool-suggestions.ts
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
// v1.13.16: Levenshtein + suggestion + formatter for the unknown-tool error
|
||||||
|
// returned to the model when an XML-extracted tool call references a name
|
||||||
|
// that isn't in TOOLS_BY_NAME. The drift incident this targets: qwen3.6
|
||||||
|
// emitting <invoke name="read_file"> from its Claude Code training residue
|
||||||
|
// when BooCode's actual file-read tool is view_file. Hand-rolled distance
|
||||||
|
// function — no new dep.
|
||||||
|
|
||||||
|
export function levenshtein(a: string, b: string): number {
|
||||||
|
if (a.length === 0) return b.length;
|
||||||
|
if (b.length === 0) return a.length;
|
||||||
|
const dp: number[][] = Array.from(
|
||||||
|
{ length: a.length + 1 },
|
||||||
|
() => new Array<number>(b.length + 1).fill(0),
|
||||||
|
);
|
||||||
|
for (let i = 0; i <= a.length; i++) dp[i]![0] = i;
|
||||||
|
for (let j = 0; j <= b.length; j++) dp[0]![j] = j;
|
||||||
|
for (let i = 1; i <= a.length; i++) {
|
||||||
|
for (let j = 1; j <= b.length; j++) {
|
||||||
|
const cost = a[i - 1] === b[j - 1] ? 0 : 1;
|
||||||
|
dp[i]![j] = Math.min(
|
||||||
|
dp[i - 1]![j]! + 1,
|
||||||
|
dp[i]![j - 1]! + 1,
|
||||||
|
dp[i - 1]![j - 1]! + cost,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return dp[a.length]![b.length]!;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Threshold per the v1.13.16 dispatch: distance <= 3 OR substring match
|
||||||
|
// (either direction). Ties broken by smallest distance, then alphabetical.
|
||||||
|
export function suggestToolName(
|
||||||
|
name: string,
|
||||||
|
available: readonly string[],
|
||||||
|
): string | null {
|
||||||
|
const lower = name.toLowerCase();
|
||||||
|
let best: { name: string; dist: number } | null = null;
|
||||||
|
for (const tool of available) {
|
||||||
|
const tlower = tool.toLowerCase();
|
||||||
|
const dist = levenshtein(lower, tlower);
|
||||||
|
const isSubstr = tlower.includes(lower) || lower.includes(tlower);
|
||||||
|
if (dist > 3 && !isSubstr) continue;
|
||||||
|
if (
|
||||||
|
best === null ||
|
||||||
|
dist < best.dist ||
|
||||||
|
(dist === best.dist && tool.localeCompare(best.name) < 0)
|
||||||
|
) {
|
||||||
|
best = { name: tool, dist };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return best?.name ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function formatUnknownToolError(
|
||||||
|
name: string,
|
||||||
|
available: readonly string[],
|
||||||
|
): string {
|
||||||
|
const sorted = [...available].sort();
|
||||||
|
const suggestion = suggestToolName(name, sorted);
|
||||||
|
const list = sorted.join(', ');
|
||||||
|
const tail = suggestion ? ` Did you mean: ${suggestion}?` : '';
|
||||||
|
return `Tool '${name}' not found. Available tools: [${list}].${tail}`;
|
||||||
|
}
|
||||||
@@ -1,23 +1,42 @@
|
|||||||
// v1.10.5: XML-tag tool-call fallback. Some models emit
|
// v1.10.5: XML-tag tool-call fallback. Some models emit
|
||||||
// <tool_call><function=foo><parameter=key>value</parameter></function></tool_call>
|
// <tool_call><function=foo><parameter=key>value</parameter></function></tool_call>
|
||||||
// in plain content instead of using the OpenAI tool_calls JSON channel.
|
// in plain content instead of using the OpenAI tool_calls JSON channel.
|
||||||
// The streaming loop in inference.ts extracts these blocks via these helpers.
|
// The streaming loop in stream-phase.ts extracts these blocks via these helpers.
|
||||||
|
//
|
||||||
|
// v1.13.16: also recognize Anthropic <invoke name="..."><parameter name="...">
|
||||||
|
// markup. qwen3.6-35b-a3b-mxfp4 drifts to this format when prompted as an
|
||||||
|
// "Architect"-style agent because Claude Code documentation in its
|
||||||
|
// pre-training data uses this shape. Both formats route through the same
|
||||||
|
// synthetic ToolCall path with shared xml_call_${idx} IDs; downstream
|
||||||
|
// dispatch handles unknown tool names with a richer error (see
|
||||||
|
// tool-suggestions.ts + tool-phase.ts).
|
||||||
|
|
||||||
export const XML_TOOL_OPEN = '<tool_call>';
|
export const XML_TOOL_OPEN = '<tool_call>';
|
||||||
export const XML_TOOL_CLOSE = '</tool_call>';
|
export const XML_TOOL_CLOSE = '</tool_call>';
|
||||||
|
|
||||||
export function parseXmlToolCall(
|
// v1.13.16: Anthropic <invoke> opener is matched by prefix (not the full
|
||||||
block: string,
|
// `<invoke ...>` tag) because attributes follow. Closer is the literal tag.
|
||||||
): { name: string; args: Record<string, unknown> } | null {
|
export const INVOKE_TOOL_OPEN = '<invoke';
|
||||||
const nameMatch = block.match(/<function=([^>]+)>/);
|
export const INVOKE_TOOL_CLOSE = '</invoke>';
|
||||||
|
|
||||||
|
export interface ParsedCall {
|
||||||
|
name: string;
|
||||||
|
args: Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
// v1.10.5: Qwen-flavor parser. Tightened in v1.13.16 to tolerate whitespace
|
||||||
|
// around `=` (e.g. `<function = view_file>`). Name capture is non-whitespace,
|
||||||
|
// non-`>` so a stray space doesn't get absorbed into the function name.
|
||||||
|
const QWEN_FUNCTION_RE = /<function\s*=\s*([^>\s]+)\s*>/;
|
||||||
|
const QWEN_PARAM_RE = /<parameter\s*=\s*([^>\s]+)\s*>([\s\S]*?)<\/parameter>/g;
|
||||||
|
|
||||||
|
export function parseXmlToolCall(block: string): ParsedCall | null {
|
||||||
|
const nameMatch = block.match(QWEN_FUNCTION_RE);
|
||||||
if (!nameMatch || !nameMatch[1]) return null;
|
if (!nameMatch || !nameMatch[1]) return null;
|
||||||
const name = nameMatch[1].trim();
|
const name = nameMatch[1].trim();
|
||||||
if (!name) return null;
|
if (!name) return null;
|
||||||
const args: Record<string, unknown> = {};
|
const args: Record<string, unknown> = {};
|
||||||
// Non-greedy body so each <parameter=…>…</parameter> pair is matched
|
for (const m of block.matchAll(QWEN_PARAM_RE)) {
|
||||||
// independently even when multiple appear in the same block.
|
|
||||||
const paramRe = /<parameter=([^>]+)>([\s\S]*?)<\/parameter>/g;
|
|
||||||
for (const m of block.matchAll(paramRe)) {
|
|
||||||
const key = (m[1] ?? '').trim();
|
const key = (m[1] ?? '').trim();
|
||||||
if (!key) continue;
|
if (!key) continue;
|
||||||
const raw = (m[2] ?? '').trim();
|
const raw = (m[2] ?? '').trim();
|
||||||
@@ -30,24 +49,121 @@ export function parseXmlToolCall(
|
|||||||
return { name, args };
|
return { name, args };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.13.16: Anthropic-flavor parser. Same JSON-parse-with-string-fallback
|
||||||
|
// shape as parseXmlToolCall so the dispatch layer doesn't need to care which
|
||||||
|
// flavor produced the call.
|
||||||
|
const INVOKE_NAME_RE =
|
||||||
|
/<invoke\s+name\s*=\s*("([^"]*)"|'([^']*)')\s*>/;
|
||||||
|
const INVOKE_PARAM_RE =
|
||||||
|
/<parameter\s+name\s*=\s*("([^"]*)"|'([^']*)')\s*>([\s\S]*?)<\/parameter>/g;
|
||||||
|
|
||||||
|
export function parseInvokeToolCall(block: string): ParsedCall | null {
|
||||||
|
const nameMatch = block.match(INVOKE_NAME_RE);
|
||||||
|
if (!nameMatch) return null;
|
||||||
|
const name = (nameMatch[2] ?? nameMatch[3] ?? '').trim();
|
||||||
|
if (!name) return null;
|
||||||
|
const args: Record<string, unknown> = {};
|
||||||
|
for (const m of block.matchAll(INVOKE_PARAM_RE)) {
|
||||||
|
const key = ((m[2] ?? m[3] ?? '') as string).trim();
|
||||||
|
if (!key) continue;
|
||||||
|
const raw = (m[4] ?? '').trim();
|
||||||
|
try {
|
||||||
|
args[key] = JSON.parse(raw);
|
||||||
|
} catch {
|
||||||
|
args[key] = raw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { name, args };
|
||||||
|
}
|
||||||
|
|
||||||
// Locate the first character that begins (or completely contains) an
|
// Locate the first character that begins (or completely contains) an
|
||||||
// unfinished <tool_call> opener in `s`. Returns -1 when `s` can be flushed
|
// unfinished opener (either flavor) in `s`. Returns -1 when `s` can be
|
||||||
// to the client in full without risking a partial tag leak.
|
// flushed to the client in full without risking a partial tag leak.
|
||||||
// Case 1: a full `<tool_call>` opener with no matching closer — caller
|
// Case 1: a full opener (`<tool_call>` or `<invoke`) with no matching
|
||||||
// must keep everything from that index forward until the next
|
// closer — caller must keep everything from that index forward
|
||||||
// chunk arrives with the closer.
|
// until the next chunk arrives with the closer.
|
||||||
// Case 2: `s` ends with a strict prefix of `<tool_call>` (e.g. `<tool_c`).
|
// Case 2: `s` ends with a strict prefix of either opener (e.g. `<tool_c`
|
||||||
// Caller must keep just that suffix in the buffer.
|
// or `<invo`). Caller must keep just that suffix in the buffer.
|
||||||
// Note: case 1 assumes the calling loop already extracted every complete
|
// Note: case 1 assumes the calling loop already extracted every complete
|
||||||
// <tool_call>…</tool_call> pair before reaching this check.
|
// block before reaching this check.
|
||||||
|
const ALL_OPENERS = [XML_TOOL_OPEN, INVOKE_TOOL_OPEN] as const;
|
||||||
|
|
||||||
export function partialXmlOpenerStart(s: string): number {
|
export function partialXmlOpenerStart(s: string): number {
|
||||||
const fullOpener = s.indexOf(XML_TOOL_OPEN);
|
let earliest = -1;
|
||||||
if (fullOpener !== -1) return fullOpener;
|
for (const op of ALL_OPENERS) {
|
||||||
|
const idx = s.indexOf(op);
|
||||||
|
if (idx === -1) continue;
|
||||||
|
if (earliest === -1 || idx < earliest) earliest = idx;
|
||||||
|
}
|
||||||
|
if (earliest !== -1) return earliest;
|
||||||
const lastLt = s.lastIndexOf('<');
|
const lastLt = s.lastIndexOf('<');
|
||||||
if (lastLt === -1) return -1;
|
if (lastLt === -1) return -1;
|
||||||
const suffix = s.slice(lastLt);
|
const suffix = s.slice(lastLt);
|
||||||
if (XML_TOOL_OPEN.startsWith(suffix) && suffix.length < XML_TOOL_OPEN.length) {
|
for (const op of ALL_OPENERS) {
|
||||||
return lastLt;
|
if (op.startsWith(suffix) && suffix.length < op.length) return lastLt;
|
||||||
}
|
}
|
||||||
return -1;
|
return -1;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.13.16: unified extraction. Replaces the inline loop that used to live
|
||||||
|
// in stream-phase.ts. Pure function — returns the visible text to flush,
|
||||||
|
// the parsed tool-call payloads in source order, and the buffer remainder
|
||||||
|
// to retain for the next streaming chunk. Parse failures are silently
|
||||||
|
// dropped (matches the pre-v1.13.16 behavior — leaking partial XML to the
|
||||||
|
// chat looks worse than swallowing a bad block).
|
||||||
|
export interface ToolCallExtraction {
|
||||||
|
flushed: string;
|
||||||
|
calls: ParsedCall[];
|
||||||
|
remaining: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface OpenerSpec {
|
||||||
|
open: string;
|
||||||
|
close: string;
|
||||||
|
parse: (block: string) => ParsedCall | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const OPENER_SPECS: ReadonlyArray<OpenerSpec> = [
|
||||||
|
{ open: XML_TOOL_OPEN, close: XML_TOOL_CLOSE, parse: parseXmlToolCall },
|
||||||
|
{ open: INVOKE_TOOL_OPEN, close: INVOKE_TOOL_CLOSE, parse: parseInvokeToolCall },
|
||||||
|
];
|
||||||
|
|
||||||
|
export function extractToolCallBlocks(buffer: string): ToolCallExtraction {
|
||||||
|
let flushed = '';
|
||||||
|
const calls: ParsedCall[] = [];
|
||||||
|
let pos = 0;
|
||||||
|
|
||||||
|
while (pos < buffer.length) {
|
||||||
|
let next: { spec: OpenerSpec; openIdx: number; closeIdx: number } | null = null;
|
||||||
|
for (const spec of OPENER_SPECS) {
|
||||||
|
const openIdx = buffer.indexOf(spec.open, pos);
|
||||||
|
if (openIdx === -1) continue;
|
||||||
|
const closeIdx = buffer.indexOf(spec.close, openIdx);
|
||||||
|
if (closeIdx === -1) continue;
|
||||||
|
if (next === null || openIdx < next.openIdx) {
|
||||||
|
next = { spec, openIdx, closeIdx };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (next === null) break;
|
||||||
|
|
||||||
|
if (next.openIdx > pos) {
|
||||||
|
flushed += buffer.slice(pos, next.openIdx);
|
||||||
|
}
|
||||||
|
const blockEnd = next.closeIdx + next.spec.close.length;
|
||||||
|
const block = buffer.slice(next.openIdx, blockEnd);
|
||||||
|
const parsed = next.spec.parse(block);
|
||||||
|
if (parsed) calls.push(parsed);
|
||||||
|
pos = blockEnd;
|
||||||
|
}
|
||||||
|
|
||||||
|
const tail = buffer.slice(pos);
|
||||||
|
const partialIdx = partialXmlOpenerStart(tail);
|
||||||
|
if (partialIdx === -1) {
|
||||||
|
flushed += tail;
|
||||||
|
return { flushed, calls, remaining: '' };
|
||||||
|
}
|
||||||
|
if (partialIdx > 0) {
|
||||||
|
flushed += tail.slice(0, partialIdx);
|
||||||
|
}
|
||||||
|
return { flushed, calls, remaining: tail.slice(partialIdx) };
|
||||||
|
}
|
||||||
|
|||||||
@@ -16,9 +16,22 @@ export async function resolveProjectRoot(projectPath: string): Promise<string> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function isUnder(real: string, root: string): boolean {
|
||||||
|
return real === root || real.startsWith(root + sep);
|
||||||
|
}
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: pathGuard now accepts an optional extraRoots
|
||||||
|
// list (typically session.allowed_read_paths). The primary projectRoot is
|
||||||
|
// tried first; if the resolved path doesn't sit under it, each extraRoot is
|
||||||
|
// tried in turn. Throws PathScopeError if no root accepts. The error message
|
||||||
|
// includes a hint pointing the model at the request_read_access tool so it
|
||||||
|
// can self-correct on the next turn — extraRoots IS the persistence
|
||||||
|
// mechanism for those grants, so we only suggest it when there's a missing
|
||||||
|
// grant to ask for (i.e. the path isn't already under any allowed root).
|
||||||
export async function pathGuard(
|
export async function pathGuard(
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
requested: string
|
requested: string,
|
||||||
|
extraRoots: readonly string[] = [],
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
if (typeof requested !== 'string' || requested.length === 0) {
|
if (typeof requested !== 'string' || requested.length === 0) {
|
||||||
throw new PathScopeError('path is required');
|
throw new PathScopeError('path is required');
|
||||||
@@ -30,10 +43,13 @@ export async function pathGuard(
|
|||||||
} catch {
|
} catch {
|
||||||
throw new PathScopeError(`path does not exist: ${requested}`);
|
throw new PathScopeError(`path does not exist: ${requested}`);
|
||||||
}
|
}
|
||||||
if (real !== projectRoot && !real.startsWith(projectRoot + sep)) {
|
if (isUnder(real, projectRoot)) return real;
|
||||||
|
for (const extra of extraRoots) {
|
||||||
|
if (extra.length === 0) continue;
|
||||||
|
if (isUnder(real, extra)) return real;
|
||||||
|
}
|
||||||
throw new PathScopeError(
|
throw new PathScopeError(
|
||||||
`path escapes project root: ${requested} -> ${real}`
|
`path escapes project root: ${requested} -> ${real}. ` +
|
||||||
|
`Use request_read_access(path, reason) to ask the user for permission.`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
return real;
|
|
||||||
}
|
|
||||||
|
|||||||
82
apps/server/src/services/request_read_access.ts
Normal file
82
apps/server/src/services/request_read_access.ts
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
// v1.13.17-cross-repo-reads: tool the model uses to request read access to
|
||||||
|
// a path outside its session's primary project root. When the model emits
|
||||||
|
// view_file("/opt/forks/foo/go.mod") under a session scoped to /opt/boocode,
|
||||||
|
// pathGuard's error message hints at this tool. The model then emits
|
||||||
|
// request_read_access(path="/opt/forks/foo/go.mod",
|
||||||
|
// reason="investigating foo to write the design doc")
|
||||||
|
// The tool's execute does cheap up-front validation: if the requested path
|
||||||
|
// can't possibly be granted under the current whitelist + repo-shape rules,
|
||||||
|
// it returns a denial immediately without prompting the user. Otherwise, the
|
||||||
|
// tool-phase pause branch (parallel of ask_user_input) stores a pending
|
||||||
|
// sentinel and waits for the user's allow/deny via the grant_read_access
|
||||||
|
// endpoint.
|
||||||
|
//
|
||||||
|
// The execute body never directly mutates state; the grant endpoint owns
|
||||||
|
// the persistence path. This keeps the tool-side logic side-effect-free
|
||||||
|
// (it's just a request) and matches ask_user_input's "server-side no-op
|
||||||
|
// fallback, pause happens in tool-phase" shape.
|
||||||
|
|
||||||
|
import { z } from 'zod';
|
||||||
|
import type { ToolDef } from './tools.js';
|
||||||
|
|
||||||
|
const RequestReadAccessInput = z.object({
|
||||||
|
path: z.string().min(1),
|
||||||
|
reason: z.string().min(1).max(500),
|
||||||
|
});
|
||||||
|
type RequestReadAccessInputT = z.infer<typeof RequestReadAccessInput>;
|
||||||
|
|
||||||
|
export const requestReadAccess: ToolDef<RequestReadAccessInputT> = {
|
||||||
|
name: 'request_read_access',
|
||||||
|
description:
|
||||||
|
"Ask the user for read-only access to a path outside the current " +
|
||||||
|
"session's project scope. Use when a previous read tool (view_file, " +
|
||||||
|
'list_dir, grep, find_files) was refused with a path-escapes-project ' +
|
||||||
|
'error and the path is plausibly under another known repository (e.g. ' +
|
||||||
|
'/opt/forks/foo). Provide a short reason describing why you need the ' +
|
||||||
|
"access. Pauses the conversation until the user picks Allow or Deny; " +
|
||||||
|
'the next assistant turn sees the result. On Allow, the tool result ' +
|
||||||
|
'is "granted: <root>" — subsequent reads under that root succeed for ' +
|
||||||
|
'the rest of the session. On Deny, the tool result is "denied". Do ' +
|
||||||
|
'not call this for paths that are already inside the project root.',
|
||||||
|
inputSchema: RequestReadAccessInput,
|
||||||
|
jsonSchema: {
|
||||||
|
type: 'function',
|
||||||
|
function: {
|
||||||
|
name: 'request_read_access',
|
||||||
|
description:
|
||||||
|
"Ask the user for read-only access to a path outside the session's " +
|
||||||
|
'project scope. Pauses the conversation until the user picks Allow ' +
|
||||||
|
'or Deny. Subsequent reads under the granted root succeed for the ' +
|
||||||
|
'rest of the session.',
|
||||||
|
parameters: {
|
||||||
|
type: 'object',
|
||||||
|
properties: {
|
||||||
|
path: {
|
||||||
|
type: 'string',
|
||||||
|
description:
|
||||||
|
'Absolute path the model wants to read. Must be under the ' +
|
||||||
|
"server's PROJECT_ROOT_WHITELIST (default /opt) and outside " +
|
||||||
|
"the session's primary project root.",
|
||||||
|
},
|
||||||
|
reason: {
|
||||||
|
type: 'string',
|
||||||
|
description:
|
||||||
|
'Short rationale (<=500 chars) shown to the user explaining ' +
|
||||||
|
'why the access is needed. The user uses this to decide.',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required: ['path', 'reason'],
|
||||||
|
additionalProperties: false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
// Server-side no-op. The "execution" of request_read_access is the
|
||||||
|
// pause-and-resume cycle managed by tool-phase.ts + the grant endpoint.
|
||||||
|
// The inference loop catches this tool name BEFORE executeToolCall fires
|
||||||
|
// and inserts a pending sentinel instead — this fallback only runs if
|
||||||
|
// something bypasses that branch, in which case we surface the pending
|
||||||
|
// shape so downstream code can still detect it. Mirrors ask_user_input.
|
||||||
|
async execute(input) {
|
||||||
|
return { _pending: true, path: input.path, reason: input.reason };
|
||||||
|
},
|
||||||
|
};
|
||||||
493
apps/server/src/services/synthesisPipeline.ts
Normal file
493
apps/server/src/services/synthesisPipeline.ts
Normal file
@@ -0,0 +1,493 @@
|
|||||||
|
// v1.13.13: forced second-inference synthesis pass for codecontext
|
||||||
|
// overview/analysis tools. Triggered from tool-phase.ts after a codecontext
|
||||||
|
// tool call lands and BEFORE the normal recursive runAssistantTurn fires.
|
||||||
|
//
|
||||||
|
// Inputs to the synthesis stream:
|
||||||
|
// 1. The codecontext tool's result text.
|
||||||
|
// 2. Top-N source files referenced in that text, fetched via view_file.
|
||||||
|
// 3. Project documentation auto-fetched from the repo root.
|
||||||
|
// 4. The original user message that triggered the turn.
|
||||||
|
//
|
||||||
|
// Output: a NEW assistant message whose sole part is kind='synthesis'.
|
||||||
|
// Streams to the client as deltas exactly like a normal assistant turn.
|
||||||
|
//
|
||||||
|
// Failure modes (all fall through to recursive runAssistantTurn):
|
||||||
|
// - SYNTHESIS_TOOLS membership check fails -> return false immediately.
|
||||||
|
// - File-fetch / doc-fetch errors -> silent skip, continue with what we have.
|
||||||
|
// - Stream error / timeout -> mark synth message status='failed', return false.
|
||||||
|
// - User-abort -> mark cancelled and re-throw so the outer abort handler runs.
|
||||||
|
|
||||||
|
import { promises as fs } from 'node:fs';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
|
||||||
|
import { TOOLS_BY_NAME } from './tools.js';
|
||||||
|
import { streamCompletion } from './inference/stream-phase.js';
|
||||||
|
import { SYNTHESIS_SYSTEM_PROMPT } from './synthesisPrompt.js';
|
||||||
|
import { insertParts } from './inference/parts.js';
|
||||||
|
import * as modelContext from './model-context.js';
|
||||||
|
import { readTruncation } from './truncate.js';
|
||||||
|
|
||||||
|
import type { Session } from '../types/api.js';
|
||||||
|
import type { OpenAiMessage } from './inference/payload.js';
|
||||||
|
import type { InferenceContext, TurnArgs } from './inference/turn.js';
|
||||||
|
|
||||||
|
export const SYNTHESIS_TOOLS: ReadonlySet<string> = new Set([
|
||||||
|
'get_codebase_overview',
|
||||||
|
'get_framework_analysis',
|
||||||
|
'get_semantic_neighborhoods',
|
||||||
|
]);
|
||||||
|
|
||||||
|
const TOP_N_FILES = 5;
|
||||||
|
const FILE_LINE_CAP = 200;
|
||||||
|
const DOC_LINE_CAP = 500;
|
||||||
|
// Token budget for the auto-fetched content (files + docs combined). Estimated
|
||||||
|
// via chars/4 — a rough but stable proxy that doesn't require a tokenizer dep.
|
||||||
|
const TOKEN_BUDGET = 32_000;
|
||||||
|
const CHARS_PER_TOKEN = 4;
|
||||||
|
// 90s per synthesis call. Long enough for a thoughtful overview against a
|
||||||
|
// large auto-fetched payload; short enough that a hung upstream falls through
|
||||||
|
// to the normal recursive turn within a typical user attention window.
|
||||||
|
const SYNTH_TIMEOUT_MS = 90_000;
|
||||||
|
|
||||||
|
// File-extension regex for referenced-file extraction. Limited to source-
|
||||||
|
// language extensions so we don't pull in lockfiles, images, etc.
|
||||||
|
const FILE_PATH_RE =
|
||||||
|
/(?:^|[`'"<\s\(\[])([A-Za-z0-9_./@-]+\.(?:ts|tsx|js|jsx|py|go|rs|java|kt|c|cpp|h|hpp|md|json|yaml|yml|sql|sh|html|css))(?=[`'"<\)\]\s,;:]|$)/gm;
|
||||||
|
|
||||||
|
export interface SynthesisParams {
|
||||||
|
ctx: InferenceContext;
|
||||||
|
args: TurnArgs;
|
||||||
|
session: Session;
|
||||||
|
projectRoot: string;
|
||||||
|
toolName: string;
|
||||||
|
toolResultText: string;
|
||||||
|
// v1.13.15-b: when codecontext's wrapper hit its 32k inline-truncation
|
||||||
|
// limit, we expand the full content via readTruncation for reference-file
|
||||||
|
// extraction only. toolResultText (the truncated head) still ships to the
|
||||||
|
// synth model — preserves the 32k payload-budget contract.
|
||||||
|
truncated?: boolean;
|
||||||
|
// opaque id (tr_<…>), not a filesystem path — see truncate.ts naming note
|
||||||
|
outputPath?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface FetchedFile {
|
||||||
|
path: string;
|
||||||
|
content: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface DocsCollection {
|
||||||
|
boochat?: string;
|
||||||
|
agents?: string;
|
||||||
|
context?: string;
|
||||||
|
roadmap?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function runSynthesisPass(p: SynthesisParams): Promise<boolean> {
|
||||||
|
if (!SYNTHESIS_TOOLS.has(p.toolName)) return false;
|
||||||
|
|
||||||
|
let synthMessageId: string | null = null;
|
||||||
|
let accumulated = '';
|
||||||
|
let timedOut = false;
|
||||||
|
const synthCtrl = new AbortController();
|
||||||
|
const timer = setTimeout(() => {
|
||||||
|
timedOut = true;
|
||||||
|
synthCtrl.abort();
|
||||||
|
}, SYNTH_TIMEOUT_MS);
|
||||||
|
|
||||||
|
try {
|
||||||
|
const userMessage = await fetchOriginalUserMessage(p.ctx, p.args.chatId);
|
||||||
|
if (!userMessage) {
|
||||||
|
p.ctx.log.warn({ chatId: p.args.chatId }, 'synthesis: no user message found; falling through');
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// v1.13.15-b: when the tool result was inline-truncated by the wrapper
|
||||||
|
// (32k cap, see codecontext_client.ts:114), expand the full content from
|
||||||
|
// tmpfs for reference-file extraction. The synth payload still ships the
|
||||||
|
// truncated head (see buildPayload call below) so the token-budget
|
||||||
|
// contract holds. Graceful degradation: if readTruncation returns null
|
||||||
|
// (missing id, ENOENT) or throws, fall back to the truncated head.
|
||||||
|
let extractionSource = p.toolResultText;
|
||||||
|
if (p.truncated && p.outputPath) {
|
||||||
|
try {
|
||||||
|
const full = await readTruncation(p.outputPath);
|
||||||
|
if (full !== null) {
|
||||||
|
extractionSource = full;
|
||||||
|
p.ctx.log.info(
|
||||||
|
{
|
||||||
|
chatId: p.args.chatId,
|
||||||
|
toolName: p.toolName,
|
||||||
|
originalChars: p.toolResultText.length,
|
||||||
|
fullChars: full.length,
|
||||||
|
},
|
||||||
|
'synthesis: expanded truncated tool output',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
p.ctx.log.warn(
|
||||||
|
{ chatId: p.args.chatId, toolName: p.toolName, err: String(err) },
|
||||||
|
'synthesis: readTruncation failed, using truncated output',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const refFiles = extractReferencedFiles(extractionSource);
|
||||||
|
const files = await fetchTopFiles(refFiles, p.projectRoot);
|
||||||
|
const docs = await fetchProjectDocs(p.projectRoot);
|
||||||
|
const { files: budgetedFiles, docs: budgetedDocs } = applyTokenBudget(files, docs);
|
||||||
|
const synthMessages = buildPayload(
|
||||||
|
p.toolName,
|
||||||
|
// Truncated head only — full content was used for reference extraction above
|
||||||
|
p.toolResultText,
|
||||||
|
budgetedFiles,
|
||||||
|
budgetedDocs,
|
||||||
|
userMessage,
|
||||||
|
);
|
||||||
|
|
||||||
|
// Insert + announce the synthesis assistant message. From here on, any
|
||||||
|
// exception must clean up via the catch block so the row doesn't linger
|
||||||
|
// in 'streaming' status (the 5min stale-streaming sweeper catches it
|
||||||
|
// eventually, but explicit cleanup is better).
|
||||||
|
const [synthRow] = await p.ctx.sql<
|
||||||
|
{ id: string; started_at: string }[]
|
||||||
|
>`
|
||||||
|
INSERT INTO messages (session_id, chat_id, role, content, status, started_at, created_at)
|
||||||
|
VALUES (${p.args.sessionId}, ${p.args.chatId}, 'assistant', '', 'streaming', clock_timestamp(), clock_timestamp())
|
||||||
|
RETURNING id, started_at
|
||||||
|
`;
|
||||||
|
synthMessageId = synthRow!.id;
|
||||||
|
const startedAt = synthRow!.started_at;
|
||||||
|
|
||||||
|
p.ctx.publish(p.args.sessionId, {
|
||||||
|
type: 'message_started',
|
||||||
|
message_id: synthMessageId,
|
||||||
|
chat_id: p.args.chatId,
|
||||||
|
role: 'assistant',
|
||||||
|
});
|
||||||
|
|
||||||
|
// Combine the user-abort signal with our synthesis-specific timeout so
|
||||||
|
// either fires correctly. The `timedOut` flag in scope tells us which one
|
||||||
|
// tripped after streamCompletion throws.
|
||||||
|
const combinedSignal: AbortSignal | undefined = p.args.signal
|
||||||
|
? AbortSignal.any([p.args.signal, synthCtrl.signal])
|
||||||
|
: synthCtrl.signal;
|
||||||
|
|
||||||
|
const onDelta = (delta: string): void => {
|
||||||
|
accumulated += delta;
|
||||||
|
p.ctx.publish(p.args.sessionId, {
|
||||||
|
type: 'delta',
|
||||||
|
message_id: synthMessageId!,
|
||||||
|
chat_id: p.args.chatId,
|
||||||
|
content: delta,
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const streamResult = await streamCompletion(
|
||||||
|
p.ctx,
|
||||||
|
p.session.model,
|
||||||
|
synthMessages,
|
||||||
|
{ tools: null },
|
||||||
|
onDelta,
|
||||||
|
undefined,
|
||||||
|
combinedSignal,
|
||||||
|
);
|
||||||
|
|
||||||
|
const mctx = await modelContext.getModelContext(p.session.model);
|
||||||
|
const nCtx = mctx?.n_ctx ?? null;
|
||||||
|
const [updated] = await p.ctx.sql<
|
||||||
|
{
|
||||||
|
tokens_used: number | null;
|
||||||
|
ctx_used: number | null;
|
||||||
|
ctx_max: number | null;
|
||||||
|
finished_at: string | null;
|
||||||
|
}[]
|
||||||
|
>`
|
||||||
|
UPDATE messages
|
||||||
|
SET content = ${streamResult.content},
|
||||||
|
status = 'complete',
|
||||||
|
tokens_used = ${streamResult.completionTokens},
|
||||||
|
ctx_used = ${streamResult.promptTokens},
|
||||||
|
ctx_max = ${nCtx},
|
||||||
|
finished_at = clock_timestamp()
|
||||||
|
WHERE id = ${synthMessageId}
|
||||||
|
RETURNING tokens_used, ctx_used, ctx_max, finished_at
|
||||||
|
`;
|
||||||
|
await insertParts(p.ctx.sql, [
|
||||||
|
{
|
||||||
|
message_id: synthMessageId,
|
||||||
|
sequence: 0,
|
||||||
|
kind: 'synthesis',
|
||||||
|
payload: { text: streamResult.content },
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
p.ctx.publish(p.args.sessionId, {
|
||||||
|
type: 'message_complete',
|
||||||
|
message_id: synthMessageId,
|
||||||
|
chat_id: p.args.chatId,
|
||||||
|
tokens_used: updated?.tokens_used ?? null,
|
||||||
|
ctx_used: updated?.ctx_used ?? null,
|
||||||
|
ctx_max: updated?.ctx_max ?? null,
|
||||||
|
started_at: startedAt,
|
||||||
|
finished_at: updated?.finished_at ?? null,
|
||||||
|
model: p.session.model,
|
||||||
|
});
|
||||||
|
p.ctx.publishUser({
|
||||||
|
type: 'chat_status',
|
||||||
|
chat_id: p.args.chatId,
|
||||||
|
status: 'idle',
|
||||||
|
at: new Date().toISOString(),
|
||||||
|
});
|
||||||
|
p.ctx.log.info(
|
||||||
|
{
|
||||||
|
chatId: p.args.chatId,
|
||||||
|
synthMessageId,
|
||||||
|
toolName: p.toolName,
|
||||||
|
chars: streamResult.content.length,
|
||||||
|
files: budgetedFiles.length,
|
||||||
|
},
|
||||||
|
'synthesis pass complete',
|
||||||
|
);
|
||||||
|
return true;
|
||||||
|
} catch (err) {
|
||||||
|
await markSynthFailed(p, synthMessageId, accumulated).catch((cleanupErr) => {
|
||||||
|
p.ctx.log.warn({ cleanupErr: String(cleanupErr) }, 'synthesis cleanup UPDATE failed');
|
||||||
|
});
|
||||||
|
if (err instanceof Error && err.name === 'AbortError') {
|
||||||
|
if (timedOut) {
|
||||||
|
p.ctx.log.warn(
|
||||||
|
{ toolName: p.toolName, chatId: p.args.chatId },
|
||||||
|
'synthesis pass timed out; falling through to recursive turn',
|
||||||
|
);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
// User-initiated abort: propagate so the outer error handler marks the
|
||||||
|
// parent turn cancelled. The synth message is already marked failed by
|
||||||
|
// markSynthFailed above.
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
p.ctx.log.warn(
|
||||||
|
{ err: String(err), toolName: p.toolName, chatId: p.args.chatId },
|
||||||
|
'synthesis pass failed; falling through to recursive turn',
|
||||||
|
);
|
||||||
|
return false;
|
||||||
|
} finally {
|
||||||
|
clearTimeout(timer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function markSynthFailed(
|
||||||
|
p: SynthesisParams,
|
||||||
|
synthMessageId: string | null,
|
||||||
|
accumulated: string,
|
||||||
|
): Promise<void> {
|
||||||
|
if (synthMessageId === null) return;
|
||||||
|
await p.ctx.sql`
|
||||||
|
UPDATE messages
|
||||||
|
SET content = ${accumulated},
|
||||||
|
status = 'failed',
|
||||||
|
finished_at = clock_timestamp()
|
||||||
|
WHERE id = ${synthMessageId}
|
||||||
|
`;
|
||||||
|
// Republish so the frontend's live state flips from 'streaming' to
|
||||||
|
// terminal. message_complete carries no error reason — the row's status
|
||||||
|
// column is the truth. The 5-state chat_status dot has 'error' but we
|
||||||
|
// don't fire that here because the broader inference is about to retry
|
||||||
|
// via recursion; flipping the user-channel status to 'error' would race
|
||||||
|
// the recursive turn's 'streaming' announcement.
|
||||||
|
p.ctx.publish(p.args.sessionId, {
|
||||||
|
type: 'message_complete',
|
||||||
|
message_id: synthMessageId,
|
||||||
|
chat_id: p.args.chatId,
|
||||||
|
model: p.session.model,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchOriginalUserMessage(
|
||||||
|
ctx: InferenceContext,
|
||||||
|
chatId: string,
|
||||||
|
): Promise<string | null> {
|
||||||
|
const rows = await ctx.sql<{ content: string }[]>`
|
||||||
|
SELECT content FROM messages
|
||||||
|
WHERE chat_id = ${chatId} AND role = 'user'
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
LIMIT 1
|
||||||
|
`;
|
||||||
|
return rows[0]?.content ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function extractReferencedFiles(text: string): string[] {
|
||||||
|
const seen = new Set<string>();
|
||||||
|
const order: string[] = [];
|
||||||
|
let m: RegExpExecArray | null;
|
||||||
|
while ((m = FILE_PATH_RE.exec(text)) !== null) {
|
||||||
|
const candidate = m[1]!;
|
||||||
|
if (seen.has(candidate)) continue;
|
||||||
|
if (
|
||||||
|
candidate.includes('node_modules') ||
|
||||||
|
candidate.includes('/dist/') ||
|
||||||
|
candidate.includes('/test/') ||
|
||||||
|
candidate.includes('/tests/') ||
|
||||||
|
/\.(test|spec)\.[a-z]+$/.test(candidate)
|
||||||
|
) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
seen.add(candidate);
|
||||||
|
order.push(candidate);
|
||||||
|
}
|
||||||
|
return order;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchTopFiles(refs: string[], projectRoot: string): Promise<FetchedFile[]> {
|
||||||
|
const tool = TOOLS_BY_NAME['view_file'];
|
||||||
|
if (!tool) return [];
|
||||||
|
const out: FetchedFile[] = [];
|
||||||
|
for (const p of refs.slice(0, TOP_N_FILES)) {
|
||||||
|
const absPath = p.startsWith('/') ? p : join(projectRoot, p);
|
||||||
|
try {
|
||||||
|
const r = await tool.execute({ path: absPath, end_line: FILE_LINE_CAP }, projectRoot);
|
||||||
|
const content = (r as { content?: string }).content ?? '';
|
||||||
|
if (content) out.push({ path: p, content });
|
||||||
|
} catch {
|
||||||
|
// path-scope blocked, secret-filtered, file too large, or missing —
|
||||||
|
// skip silently. The remaining files (or none) still produce a
|
||||||
|
// meaningful synthesis input.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function fetchProjectDocs(projectRoot: string): Promise<DocsCollection> {
|
||||||
|
const tool = TOOLS_BY_NAME['view_file'];
|
||||||
|
if (!tool) return {};
|
||||||
|
const docs: DocsCollection = {};
|
||||||
|
for (const [filename, key] of [
|
||||||
|
['BOOCHAT.md', 'boochat'],
|
||||||
|
['AGENTS.md', 'agents'],
|
||||||
|
['CONTEXT.md', 'context'],
|
||||||
|
] as const) {
|
||||||
|
try {
|
||||||
|
const r = await tool.execute(
|
||||||
|
{ path: join(projectRoot, filename), end_line: DOC_LINE_CAP },
|
||||||
|
projectRoot,
|
||||||
|
);
|
||||||
|
const content = (r as { content?: string }).content;
|
||||||
|
if (content) docs[key] = content;
|
||||||
|
} catch {
|
||||||
|
// missing doc — skip
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Case-insensitive *roadmap*.md glob. Picks the first match (alphabetical
|
||||||
|
// by readdir() order); typical projects have at most one roadmap doc.
|
||||||
|
try {
|
||||||
|
const entries = await fs.readdir(projectRoot);
|
||||||
|
const roadmap = entries.find(
|
||||||
|
(e) => /roadmap/i.test(e) && e.toLowerCase().endsWith('.md'),
|
||||||
|
);
|
||||||
|
if (roadmap) {
|
||||||
|
const r = await tool.execute(
|
||||||
|
{ path: join(projectRoot, roadmap), end_line: DOC_LINE_CAP },
|
||||||
|
projectRoot,
|
||||||
|
);
|
||||||
|
const content = (r as { content?: string }).content;
|
||||||
|
if (content) docs.roadmap = content;
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// unreadable project root — skip
|
||||||
|
}
|
||||||
|
return docs;
|
||||||
|
}
|
||||||
|
|
||||||
|
function estTokens(s: string | undefined): number {
|
||||||
|
return s ? Math.ceil(s.length / CHARS_PER_TOKEN) : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyTokenBudget(
|
||||||
|
files: FetchedFile[],
|
||||||
|
docs: DocsCollection,
|
||||||
|
): { files: FetchedFile[]; docs: DocsCollection } {
|
||||||
|
let total = 0;
|
||||||
|
for (const f of files) total += estTokens(f.content);
|
||||||
|
total += estTokens(docs.boochat) + estTokens(docs.agents) + estTokens(docs.context) + estTokens(docs.roadmap);
|
||||||
|
if (total <= TOKEN_BUDGET) return { files, docs };
|
||||||
|
|
||||||
|
// Drop priority (lowest priority dropped first):
|
||||||
|
// 1. top-2..N files (keep top-1)
|
||||||
|
// 2. top-1 file
|
||||||
|
// 3. roadmap (+ CONTEXT.md grouped here — dispatch listed roadmap above
|
||||||
|
// AGENTS.md, CONTEXT.md was not in the priority list)
|
||||||
|
// 4. AGENTS.md
|
||||||
|
// 5. BOOCHAT.md (never dropped — truncate to budget if alone exceeds)
|
||||||
|
let outFiles = files.slice();
|
||||||
|
const outDocs: DocsCollection = { ...docs };
|
||||||
|
|
||||||
|
while (total > TOKEN_BUDGET && outFiles.length > 1) {
|
||||||
|
const last = outFiles.pop()!;
|
||||||
|
total -= estTokens(last.content);
|
||||||
|
}
|
||||||
|
if (total <= TOKEN_BUDGET) return { files: outFiles, docs: outDocs };
|
||||||
|
|
||||||
|
if (outFiles[0]) {
|
||||||
|
total -= estTokens(outFiles[0].content);
|
||||||
|
outFiles = [];
|
||||||
|
}
|
||||||
|
if (total <= TOKEN_BUDGET) return { files: outFiles, docs: outDocs };
|
||||||
|
|
||||||
|
if (outDocs.roadmap) {
|
||||||
|
total -= estTokens(outDocs.roadmap);
|
||||||
|
delete outDocs.roadmap;
|
||||||
|
}
|
||||||
|
if (outDocs.context) {
|
||||||
|
total -= estTokens(outDocs.context);
|
||||||
|
delete outDocs.context;
|
||||||
|
}
|
||||||
|
if (total <= TOKEN_BUDGET) return { files: outFiles, docs: outDocs };
|
||||||
|
|
||||||
|
if (outDocs.agents) {
|
||||||
|
total -= estTokens(outDocs.agents);
|
||||||
|
delete outDocs.agents;
|
||||||
|
}
|
||||||
|
if (total <= TOKEN_BUDGET) return { files: outFiles, docs: outDocs };
|
||||||
|
|
||||||
|
if (outDocs.boochat) {
|
||||||
|
const maxChars = TOKEN_BUDGET * CHARS_PER_TOKEN;
|
||||||
|
if (outDocs.boochat.length > maxChars) {
|
||||||
|
outDocs.boochat = outDocs.boochat.slice(0, maxChars);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { files: outFiles, docs: outDocs };
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildPayload(
|
||||||
|
toolName: string,
|
||||||
|
toolResultText: string,
|
||||||
|
files: FetchedFile[],
|
||||||
|
docs: DocsCollection,
|
||||||
|
userMessage: string,
|
||||||
|
): OpenAiMessage[] {
|
||||||
|
const sections: string[] = [];
|
||||||
|
sections.push(`## Codecontext tool output (${toolName})\n\n${toolResultText}`);
|
||||||
|
if (files.length > 0) {
|
||||||
|
sections.push(`---\n\n## Auto-fetched source files`);
|
||||||
|
for (const f of files) {
|
||||||
|
sections.push(`### ${f.path}\n\n\`\`\`\n${f.content}\n\`\`\``);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const docEntries: Array<[string, string | undefined]> = [
|
||||||
|
['BOOCHAT.md', docs.boochat],
|
||||||
|
['AGENTS.md', docs.agents],
|
||||||
|
['CONTEXT.md', docs.context],
|
||||||
|
['roadmap', docs.roadmap],
|
||||||
|
];
|
||||||
|
const presentDocs = docEntries.filter(([, v]) => Boolean(v));
|
||||||
|
if (presentDocs.length > 0) {
|
||||||
|
sections.push(`---\n\n## Project documentation`);
|
||||||
|
for (const [name, v] of presentDocs) {
|
||||||
|
sections.push(`### ${name}\n\n${v!}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sections.push(`---\n\n## Original user question\n\n${userMessage}`);
|
||||||
|
return [
|
||||||
|
{ role: 'system', content: SYNTHESIS_SYSTEM_PROMPT },
|
||||||
|
{ role: 'user', content: sections.join('\n\n') },
|
||||||
|
];
|
||||||
|
}
|
||||||
20
apps/server/src/services/synthesisPrompt.ts
Normal file
20
apps/server/src/services/synthesisPrompt.ts
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
// v1.13.13: synthesis pipeline system prompt. Verbatim from the v1.13.13
|
||||||
|
// dispatch — do not paraphrase. The synthesis pass loads this as its sole
|
||||||
|
// system message, followed by a user message that concatenates the
|
||||||
|
// codecontext tool result, auto-fetched top files, auto-fetched project
|
||||||
|
// docs, and the original user message.
|
||||||
|
export const SYNTHESIS_SYSTEM_PROMPT = `You are synthesizing structural data into an accurate, detailed answer about the user's codebase.
|
||||||
|
|
||||||
|
Inputs you have been given:
|
||||||
|
1. The output of a codecontext analysis tool (raw structural data — file counts, symbols, dependencies, frameworks).
|
||||||
|
2. The contents of the top files referenced in that output.
|
||||||
|
3. Any project documentation found in the repo root (BOOCHAT.md, AGENTS.md, roadmap docs, CONTEXT.md).
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Cite specific files and line numbers when making claims about code.
|
||||||
|
- If project docs contradict the code, docs win for questions about state, version, status, or roadmap. Code wins for questions about runtime behavior or implementation.
|
||||||
|
- If the codecontext output looks sparse (low symbol count for a TypeScript project, missing dependency edges, empty framework list), explicitly say so — codecontext falls back to the JavaScript grammar for TypeScript and loses interfaces, generics, decorators, and type aliases.
|
||||||
|
- Do not invent symbols, files, or relationships that are not present in the inputs.
|
||||||
|
- Do not respond with a generic "this looks like a [framework] project" summary. The user has the framework analysis already. Add specifics: what is actually in this codebase, what is shipped, what is planned, what is load-bearing.
|
||||||
|
- Length: match the depth the user asked for. Overview questions get structured multi-section answers. Specific questions get focused answers.
|
||||||
|
`;
|
||||||
@@ -22,6 +22,10 @@ import {
|
|||||||
getSemanticNeighborhoods,
|
getSemanticNeighborhoods,
|
||||||
getFrameworkAnalysis,
|
getFrameworkAnalysis,
|
||||||
} from './tools/codecontext/index.js';
|
} from './tools/codecontext/index.js';
|
||||||
|
// v1.13.17-cross-repo-reads: cross-repo read grant request tool. Paired
|
||||||
|
// with the pause-on-pending-grant branch in inference/tool-phase.ts and the
|
||||||
|
// POST /api/chats/:id/grant_read_access endpoint in routes/messages.ts.
|
||||||
|
import { requestReadAccess } from './request_read_access.js';
|
||||||
|
|
||||||
const MAX_FILE_BYTES = 5 * 1024 * 1024;
|
const MAX_FILE_BYTES = 5 * 1024 * 1024;
|
||||||
const DEFAULT_VIEW_LINES = 200;
|
const DEFAULT_VIEW_LINES = 200;
|
||||||
@@ -45,7 +49,13 @@ export interface ToolDef<TInput> {
|
|||||||
description: string;
|
description: string;
|
||||||
inputSchema: z.ZodType<TInput>;
|
inputSchema: z.ZodType<TInput>;
|
||||||
jsonSchema: ToolJsonSchema;
|
jsonSchema: ToolJsonSchema;
|
||||||
execute(input: TInput, projectRoot: string): Promise<unknown>;
|
// v1.13.17-cross-repo-reads: extraRoots is the session's
|
||||||
|
// allowed_read_paths, threaded through executeToolCall in tool-phase.ts.
|
||||||
|
// Only the filesystem tools (view_file, list_dir, grep, find_files,
|
||||||
|
// view_truncated_output) forward it to pathGuard; other tools accept the
|
||||||
|
// arg and ignore it. The execute signature stays compatible with
|
||||||
|
// pre-v1.13.17 callsites because the parameter is optional.
|
||||||
|
execute(input: TInput, projectRoot: string, extraRoots?: readonly string[]): Promise<unknown>;
|
||||||
}
|
}
|
||||||
|
|
||||||
const ViewFileInput = z.object({
|
const ViewFileInput = z.object({
|
||||||
@@ -78,14 +88,19 @@ export const viewFile: ToolDef<ViewFileInputT> = {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
async execute(input, projectRoot) {
|
async execute(input, projectRoot, extraRoots) {
|
||||||
const real = await pathGuard(projectRoot, input.path);
|
const real = await pathGuard(projectRoot, input.path, extraRoots);
|
||||||
// v1.11.7: secret-file deny check. Test the project-relative path
|
// v1.11.7: secret-file deny check. Test the project-relative path
|
||||||
// (matches the form continue.dev's patterns expect: basenames + dir
|
// (matches the form continue.dev's patterns expect: basenames + dir
|
||||||
// segments). Throw a typed error so executeToolCall in inference.ts
|
// segments). Throw a typed error so executeToolCall in inference.ts
|
||||||
// surfaces a clear "blocked" message to the LLM instead of silently
|
// surfaces a clear "blocked" message to the LLM instead of silently
|
||||||
// returning content the user wanted hidden.
|
// returning content the user wanted hidden.
|
||||||
const relPath = relative(projectRoot, real) || basename(real);
|
// v1.13.17: when the resolved path is outside the primary projectRoot
|
||||||
|
// (i.e. via an allowed_read_paths grant), `relative()` returns "../…"
|
||||||
|
// which won't match secret-file basename patterns. Re-anchor on the
|
||||||
|
// file's basename so the secret deny still fires across all grant roots.
|
||||||
|
const rel = relative(projectRoot, real);
|
||||||
|
const relPath = rel && !rel.startsWith('..') ? rel : basename(real);
|
||||||
if (isSecretPath(relPath)) {
|
if (isSecretPath(relPath)) {
|
||||||
throw new SecretBlockedError(relPath);
|
throw new SecretBlockedError(relPath);
|
||||||
}
|
}
|
||||||
@@ -157,8 +172,8 @@ export const listDir: ToolDef<ListDirInputT> = {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
async execute(input, projectRoot) {
|
async execute(input, projectRoot, extraRoots) {
|
||||||
const real = await pathGuard(projectRoot, input.path);
|
const real = await pathGuard(projectRoot, input.path, extraRoots);
|
||||||
const s = await stat(real);
|
const s = await stat(real);
|
||||||
if (!s.isDirectory()) {
|
if (!s.isDirectory()) {
|
||||||
throw new PathScopeError(`not a directory: ${input.path}`);
|
throw new PathScopeError(`not a directory: ${input.path}`);
|
||||||
@@ -264,7 +279,7 @@ export const grep: ToolDef<GrepInputT> = {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
async execute(input, projectRoot) {
|
async execute(input, projectRoot, extraRoots) {
|
||||||
const limit = Math.min(
|
const limit = Math.min(
|
||||||
Math.max(input.max_results ?? DEFAULT_GREP_RESULTS, 1),
|
Math.max(input.max_results ?? DEFAULT_GREP_RESULTS, 1),
|
||||||
MAX_GREP_RESULTS
|
MAX_GREP_RESULTS
|
||||||
@@ -276,6 +291,7 @@ export const grep: ToolDef<GrepInputT> = {
|
|||||||
max_matches: limit,
|
max_matches: limit,
|
||||||
case_sensitive: input.case_sensitive,
|
case_sensitive: input.case_sensitive,
|
||||||
hidden: input.hidden,
|
hidden: input.hidden,
|
||||||
|
extra_roots: extraRoots,
|
||||||
});
|
});
|
||||||
const reshaped = result.matches.map((m) => ({
|
const reshaped = result.matches.map((m) => ({
|
||||||
path: m.path,
|
path: m.path,
|
||||||
@@ -325,7 +341,7 @@ export const findFiles: ToolDef<FindFilesInputT> = {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
async execute(input, projectRoot) {
|
async execute(input, projectRoot, extraRoots) {
|
||||||
const limit = Math.min(
|
const limit = Math.min(
|
||||||
Math.max(input.max_results ?? DEFAULT_FIND_RESULTS, 1),
|
Math.max(input.max_results ?? DEFAULT_FIND_RESULTS, 1),
|
||||||
MAX_FIND_RESULTS
|
MAX_FIND_RESULTS
|
||||||
@@ -335,6 +351,7 @@ export const findFiles: ToolDef<FindFilesInputT> = {
|
|||||||
const result = await fileOpsFindFiles(projectRoot, input.pattern, {
|
const result = await fileOpsFindFiles(projectRoot, input.pattern, {
|
||||||
path: input.path,
|
path: input.path,
|
||||||
max_results: limit,
|
max_results: limit,
|
||||||
|
extra_roots: extraRoots,
|
||||||
});
|
});
|
||||||
// v1.11.7: drop paths matching secret patterns. The original `total`
|
// v1.11.7: drop paths matching secret patterns. The original `total`
|
||||||
// from file_ops includes pre-truncation count; we report the visible
|
// from file_ops includes pre-truncation count; we report the visible
|
||||||
@@ -383,7 +400,10 @@ export const viewTruncatedOutput: ToolDef<ViewTruncatedOutputInputT> = {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
async execute(input, _projectRoot) {
|
// view_truncated_output doesn't touch the filesystem — it pulls from tmpfs
|
||||||
|
// by opaque id. extraRoots is irrelevant here; declared for signature parity
|
||||||
|
// with the v1.13.17 ToolDef contract.
|
||||||
|
async execute(input, _projectRoot, _extraRoots) {
|
||||||
const content = await readTruncation(input.id);
|
const content = await readTruncation(input.id);
|
||||||
if (content === null) {
|
if (content === null) {
|
||||||
return {
|
return {
|
||||||
@@ -658,6 +678,11 @@ export const ALL_TOOLS: ReadonlyArray<ToolDef<unknown>> = [
|
|||||||
watchChanges as ToolDef<unknown>,
|
watchChanges as ToolDef<unknown>,
|
||||||
getSemanticNeighborhoods as ToolDef<unknown>,
|
getSemanticNeighborhoods as ToolDef<unknown>,
|
||||||
getFrameworkAnalysis as ToolDef<unknown>,
|
getFrameworkAnalysis as ToolDef<unknown>,
|
||||||
|
// v1.13.17-cross-repo-reads: paired with the pause-on-pending-grant
|
||||||
|
// branch in tool-phase.ts. Read-only — only ever READS files; the only
|
||||||
|
// state change is appending to sessions.allowed_read_paths via the
|
||||||
|
// grant endpoint, gated by user consent.
|
||||||
|
requestReadAccess as ToolDef<unknown>,
|
||||||
].sort((a, b) => a.name.localeCompare(b.name));
|
].sort((a, b) => a.name.localeCompare(b.name));
|
||||||
|
|
||||||
// v1.8.2: forward-compatible read-only whitelist. An agent whose `tools` is
|
// v1.8.2: forward-compatible read-only whitelist. An agent whose `tools` is
|
||||||
@@ -694,6 +719,10 @@ export const READ_ONLY_TOOL_NAMES = [
|
|||||||
'watch_changes',
|
'watch_changes',
|
||||||
'get_semantic_neighborhoods',
|
'get_semantic_neighborhoods',
|
||||||
'get_framework_analysis',
|
'get_framework_analysis',
|
||||||
|
// v1.13.17-cross-repo-reads: pauses execution but doesn't mutate project
|
||||||
|
// state directly (the grant endpoint appends to sessions.allowed_read_paths
|
||||||
|
// only with user consent). Belongs in the read-only budget tier.
|
||||||
|
'request_read_access',
|
||||||
] as const;
|
] as const;
|
||||||
|
|
||||||
export const TOOLS_BY_NAME: Record<string, ToolDef<unknown>> = Object.fromEntries(
|
export const TOOLS_BY_NAME: Record<string, ToolDef<unknown>> = Object.fromEntries(
|
||||||
|
|||||||
@@ -42,6 +42,12 @@ export interface Session {
|
|||||||
// v1.12.1: server-side workspace pane layout. Replaces per-device
|
// v1.12.1: server-side workspace pane layout. Replaces per-device
|
||||||
// localStorage so all devices viewing the session see the same panes.
|
// localStorage so all devices viewing the session see the same panes.
|
||||||
workspace_panes: WorkspacePane[];
|
workspace_panes: WorkspacePane[];
|
||||||
|
// v1.13.17: absolute paths the agent has been granted read access to via
|
||||||
|
// the request_read_access tool. Empty by default; populated only by the
|
||||||
|
// grant_read_access endpoint's allow branch. Revoked via PATCH session.
|
||||||
|
// path_guard's extraRoots check consults this list before refusing reads
|
||||||
|
// outside the primary project root.
|
||||||
|
allowed_read_paths: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export type WorkspacePaneKind = 'chat' | 'terminal' | 'agent' | 'empty' | 'settings';
|
export type WorkspacePaneKind = 'chat' | 'terminal' | 'agent' | 'empty' | 'settings';
|
||||||
|
|||||||
@@ -123,7 +123,20 @@ export const api = {
|
|||||||
get: (id: string) => request<Session>(`/api/sessions/${id}`),
|
get: (id: string) => request<Session>(`/api/sessions/${id}`),
|
||||||
update: (
|
update: (
|
||||||
id: string,
|
id: string,
|
||||||
body: Partial<Pick<Session, 'name' | 'model' | 'system_prompt' | 'agent_id' | 'web_search_enabled'>>
|
body: Partial<
|
||||||
|
Pick<
|
||||||
|
Session,
|
||||||
|
| 'name'
|
||||||
|
| 'model'
|
||||||
|
| 'system_prompt'
|
||||||
|
| 'agent_id'
|
||||||
|
| 'web_search_enabled'
|
||||||
|
// v1.13.17: revocation path — frontend sends the shortened list
|
||||||
|
// when the user removes a grant. Grants are appended only via the
|
||||||
|
// separate grantReadAccess endpoint below.
|
||||||
|
| 'allowed_read_paths'
|
||||||
|
>
|
||||||
|
>
|
||||||
) =>
|
) =>
|
||||||
request<Session>(`/api/sessions/${id}`, {
|
request<Session>(`/api/sessions/${id}`, {
|
||||||
method: 'PATCH',
|
method: 'PATCH',
|
||||||
@@ -228,6 +241,19 @@ export const api = {
|
|||||||
body: JSON.stringify({ tool_call_id: toolCallId, answers }),
|
body: JSON.stringify({ tool_call_id: toolCallId, answers }),
|
||||||
},
|
},
|
||||||
),
|
),
|
||||||
|
// v1.13.17-cross-repo-reads: resume a paused request_read_access. On
|
||||||
|
// 'allow' the server re-resolves the grant root and appends it to
|
||||||
|
// sessions.allowed_read_paths; the returned list reflects the post-
|
||||||
|
// grant state. On 'deny' the array is unchanged.
|
||||||
|
grantReadAccess: (chatId: string, toolCallId: string, decision: 'allow' | 'deny') =>
|
||||||
|
request<{
|
||||||
|
tool_message_id: string;
|
||||||
|
assistant_message_id: string;
|
||||||
|
allowed_read_paths: string[];
|
||||||
|
}>(`/api/chats/${chatId}/grant_read_access`, {
|
||||||
|
method: 'POST',
|
||||||
|
body: JSON.stringify({ tool_call_id: toolCallId, decision }),
|
||||||
|
}),
|
||||||
},
|
},
|
||||||
|
|
||||||
messages: {
|
messages: {
|
||||||
|
|||||||
@@ -48,6 +48,11 @@ export interface Session {
|
|||||||
web_search_enabled: boolean | null;
|
web_search_enabled: boolean | null;
|
||||||
// v1.12.1: server-authoritative pane layout, replaces localStorage.
|
// v1.12.1: server-authoritative pane layout, replaces localStorage.
|
||||||
workspace_panes: WorkspacePane[];
|
workspace_panes: WorkspacePane[];
|
||||||
|
// v1.13.17: paths the agent has been granted read access to via the
|
||||||
|
// request_read_access tool. Empty by default. Settings UI surfaces the
|
||||||
|
// list with per-row revoke; the grant flow itself appends through the
|
||||||
|
// dedicated POST /api/chats/:id/grant_read_access endpoint (not PATCH).
|
||||||
|
allowed_read_paths: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
// v1.8.1: 'global' = /data/AGENTS.md (always-on), 'project' = per-project
|
// v1.8.1: 'global' = /data/AGENTS.md (always-on), 'project' = per-project
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import { MessageBubble } from './MessageBubble';
|
|||||||
import { ToolCallGroup } from './ToolCallGroup';
|
import { ToolCallGroup } from './ToolCallGroup';
|
||||||
import { ToolCallLine, type ToolRun } from './ToolCallLine';
|
import { ToolCallLine, type ToolRun } from './ToolCallLine';
|
||||||
import { AskUserInputCard } from './AskUserInputCard';
|
import { AskUserInputCard } from './AskUserInputCard';
|
||||||
|
import { RequestReadAccessCard } from './RequestReadAccessCard';
|
||||||
|
|
||||||
interface Props {
|
interface Props {
|
||||||
messages: Message[];
|
messages: Message[];
|
||||||
@@ -85,7 +86,9 @@ function group(items: RenderItem[]): RenderItem[] {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
const name = item.run.call.name;
|
const name = item.run.call.name;
|
||||||
if (name === 'ask_user_input') {
|
if (name === 'ask_user_input' || name === 'request_read_access') {
|
||||||
|
// v1.13.17: same rationale as ask_user_input — grouping would collapse
|
||||||
|
// the interactive pause card into a non-actionable ToolCallLine.
|
||||||
out.push(item);
|
out.push(item);
|
||||||
i += 1;
|
i += 1;
|
||||||
continue;
|
continue;
|
||||||
@@ -181,6 +184,16 @@ export function MessageList({ messages, sessionChats }: Props) {
|
|||||||
/>
|
/>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
if (item.run.call.name === 'request_read_access') {
|
||||||
|
return (
|
||||||
|
<RequestReadAccessCard
|
||||||
|
key={item.key}
|
||||||
|
toolCall={item.run.call}
|
||||||
|
toolResult={item.run.result}
|
||||||
|
chatId={item.chatId}
|
||||||
|
/>
|
||||||
|
);
|
||||||
|
}
|
||||||
return <ToolCallLine key={item.key} run={item.run} />;
|
return <ToolCallLine key={item.key} run={item.run} />;
|
||||||
}
|
}
|
||||||
return <ToolCallGroup key={item.key} runs={item.runs} />;
|
return <ToolCallGroup key={item.key} runs={item.runs} />;
|
||||||
|
|||||||
193
apps/web/src/components/RequestReadAccessCard.tsx
Normal file
193
apps/web/src/components/RequestReadAccessCard.tsx
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
import { useState } from 'react';
|
||||||
|
import { Check, FolderOpen, ShieldOff } from 'lucide-react';
|
||||||
|
import { toast } from 'sonner';
|
||||||
|
import { api } from '@/api/client';
|
||||||
|
import { Button } from '@/components/ui/button';
|
||||||
|
import type { ToolCall, ToolResult } from '@/api/types';
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads. Renders an inline allow/deny picker for a
|
||||||
|
// paused request_read_access tool call. Mirrors AskUserInputCard's pending
|
||||||
|
// vs answered render dance:
|
||||||
|
// - Pending: server pre-stamps a sentinel tool_result with output=null.
|
||||||
|
// The card shows path + reason and lets the user pick Allow or Deny.
|
||||||
|
// - Answered: the eventual WS tool_result frame carries the actual
|
||||||
|
// decision string ("granted: <root>" or "denied" or "denied: <reason>").
|
||||||
|
// The card flips to a read-only summary line.
|
||||||
|
//
|
||||||
|
// Tool name discrimination lives in MessageList.flatten/group — anything
|
||||||
|
// with tc.name === 'request_read_access' bypasses grouping and renders this
|
||||||
|
// card directly.
|
||||||
|
|
||||||
|
interface Props {
|
||||||
|
toolCall: ToolCall;
|
||||||
|
toolResult: ToolResult | null;
|
||||||
|
chatId: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface ParsedArgs {
|
||||||
|
path: string;
|
||||||
|
reason: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseArgs(raw: unknown): ParsedArgs | null {
|
||||||
|
if (!raw || typeof raw !== 'object') return null;
|
||||||
|
const obj = raw as { path?: unknown; reason?: unknown };
|
||||||
|
if (typeof obj.path !== 'string' || obj.path.length === 0) return null;
|
||||||
|
if (typeof obj.reason !== 'string' || obj.reason.length === 0) return null;
|
||||||
|
return { path: obj.path, reason: obj.reason };
|
||||||
|
}
|
||||||
|
|
||||||
|
function decisionVariant(output: unknown): 'granted' | 'denied' | 'unknown' {
|
||||||
|
if (typeof output !== 'string') return 'unknown';
|
||||||
|
if (output.startsWith('granted:')) return 'granted';
|
||||||
|
if (output === 'denied' || output.startsWith('denied:')) return 'denied';
|
||||||
|
return 'unknown';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function RequestReadAccessCard({ toolCall, toolResult, chatId }: Props) {
|
||||||
|
const args = parseArgs(toolCall.args);
|
||||||
|
|
||||||
|
if (!args) {
|
||||||
|
return (
|
||||||
|
<div className="rounded border border-destructive/40 bg-destructive/10 text-xs px-3 py-2 text-destructive">
|
||||||
|
request_read_access: malformed tool args
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Non-null output means the WS tool_result frame arrived (or the row was
|
||||||
|
// re-fetched from history).
|
||||||
|
const answered = toolResult && toolResult.output !== null;
|
||||||
|
if (answered) {
|
||||||
|
return <AnsweredView args={args} output={toolResult!.output} />;
|
||||||
|
}
|
||||||
|
|
||||||
|
return <PendingView args={args} toolCallId={toolCall.id} chatId={chatId} />;
|
||||||
|
}
|
||||||
|
|
||||||
|
function PendingView({
|
||||||
|
args,
|
||||||
|
toolCallId,
|
||||||
|
chatId,
|
||||||
|
}: {
|
||||||
|
args: ParsedArgs;
|
||||||
|
toolCallId: string;
|
||||||
|
chatId: string;
|
||||||
|
}) {
|
||||||
|
const [submitting, setSubmitting] = useState<'allow' | 'deny' | null>(null);
|
||||||
|
|
||||||
|
async function decide(decision: 'allow' | 'deny') {
|
||||||
|
if (submitting) return;
|
||||||
|
setSubmitting(decision);
|
||||||
|
try {
|
||||||
|
await api.chats.grantReadAccess(chatId, toolCallId, decision);
|
||||||
|
// Card stays mounted; the incoming WS tool_result frame swaps it to
|
||||||
|
// AnsweredView via the parent prop change.
|
||||||
|
} catch (err) {
|
||||||
|
toast.error(err instanceof Error ? err.message : 'request failed');
|
||||||
|
setSubmitting(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="rounded-lg border border-amber-500/40 bg-amber-500/5 text-sm">
|
||||||
|
<div className="px-4 py-3 space-y-2">
|
||||||
|
<div className="flex items-center gap-2 text-xs uppercase tracking-wide text-amber-700 dark:text-amber-300">
|
||||||
|
<ShieldOff className="size-3.5" />
|
||||||
|
<span>Read-access request</span>
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<div className="text-[10px] uppercase tracking-wide text-muted-foreground/70">Path</div>
|
||||||
|
<div className="font-mono text-xs break-all rounded bg-background/60 border px-2 py-1">
|
||||||
|
{args.path}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<div className="text-[10px] uppercase tracking-wide text-muted-foreground/70">Reason</div>
|
||||||
|
<div className="text-sm leading-snug whitespace-pre-wrap">{args.reason}</div>
|
||||||
|
</div>
|
||||||
|
<div className="text-[11px] text-muted-foreground pt-1">
|
||||||
|
Allow grants the agent read access to the matching repository root for
|
||||||
|
the rest of this session. Revoke any time from the session settings.
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="flex justify-end gap-2 border-t border-amber-500/20 px-4 py-2">
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
size="sm"
|
||||||
|
variant="outline"
|
||||||
|
disabled={submitting !== null}
|
||||||
|
onClick={() => void decide('deny')}
|
||||||
|
>
|
||||||
|
{submitting === 'deny' ? 'Denying…' : 'Deny'}
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
size="sm"
|
||||||
|
disabled={submitting !== null}
|
||||||
|
onClick={() => void decide('allow')}
|
||||||
|
>
|
||||||
|
{submitting === 'allow' ? 'Allowing…' : 'Allow'}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function AnsweredView({ args, output }: { args: ParsedArgs; output: unknown }) {
|
||||||
|
const variant = decisionVariant(output);
|
||||||
|
const text = typeof output === 'string' ? output : 'unknown';
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
className={
|
||||||
|
variant === 'granted'
|
||||||
|
? 'rounded-lg border border-emerald-500/40 bg-emerald-500/5 text-sm'
|
||||||
|
: variant === 'denied'
|
||||||
|
? 'rounded-lg border bg-muted/20 text-sm'
|
||||||
|
: 'rounded-lg border border-destructive/40 bg-destructive/5 text-sm'
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<div className="px-4 py-3 space-y-2">
|
||||||
|
<div className="flex items-center gap-2 text-xs uppercase tracking-wide">
|
||||||
|
{variant === 'granted' ? (
|
||||||
|
<>
|
||||||
|
<Check className="size-3.5 text-emerald-600" />
|
||||||
|
<span className="text-emerald-700 dark:text-emerald-300">Read access granted</span>
|
||||||
|
</>
|
||||||
|
) : variant === 'denied' ? (
|
||||||
|
<>
|
||||||
|
<ShieldOff className="size-3.5 text-muted-foreground" />
|
||||||
|
<span className="text-muted-foreground">Read access denied</span>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<ShieldOff className="size-3.5 text-destructive" />
|
||||||
|
<span className="text-destructive">Read access request — unknown result</span>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<div className="text-[10px] uppercase tracking-wide text-muted-foreground/70">Path</div>
|
||||||
|
<div className="font-mono text-xs break-all rounded bg-background/60 border px-2 py-1">
|
||||||
|
{args.path}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{variant === 'granted' && (
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<div className="text-[10px] uppercase tracking-wide text-muted-foreground/70">Granted root</div>
|
||||||
|
<div className="font-mono text-xs break-all rounded bg-background/60 border px-2 py-1 flex items-center gap-1.5">
|
||||||
|
<FolderOpen className="size-3 shrink-0 text-muted-foreground" />
|
||||||
|
<span>{text.replace(/^granted:\s*/, '')}</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{variant === 'denied' && text !== 'denied' && (
|
||||||
|
<div className="text-[11px] text-muted-foreground">
|
||||||
|
{text.replace(/^denied:\s*/, '')}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { useEffect, useState } from 'react';
|
import { useEffect, useState } from 'react';
|
||||||
import { Archive, Maximize2, Minimize2, X } from 'lucide-react';
|
import { Archive, FolderOpen, Maximize2, Minimize2, Trash2, X } from 'lucide-react';
|
||||||
import { toast } from 'sonner';
|
import { toast } from 'sonner';
|
||||||
import { api } from '@/api/client';
|
import { api } from '@/api/client';
|
||||||
import type { Project, Session } from '@/api/types';
|
import type { Project, Session } from '@/api/types';
|
||||||
@@ -269,6 +269,8 @@ function SessionSection({ session, project }: { session: Session; project: Proje
|
|||||||
</p>
|
</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<AllowedReadPathsSection session={session} />
|
||||||
|
|
||||||
<div className="space-y-1.5">
|
<div className="space-y-1.5">
|
||||||
<div className="flex items-center justify-between gap-3">
|
<div className="flex items-center justify-between gap-3">
|
||||||
<label className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
|
<label className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
|
||||||
@@ -337,6 +339,76 @@ function SessionSection({ session, project }: { session: Session; project: Proje
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.13.17-cross-repo-reads: revoke UI for session.allowed_read_paths.
|
||||||
|
// Append happens through the inline request_read_access pause flow; this
|
||||||
|
// section only shrinks the list. PATCH /api/sessions/:id replaces the
|
||||||
|
// whole array, so we send the original list minus the deleted entry.
|
||||||
|
function AllowedReadPathsSection({ session }: { session: Session }) {
|
||||||
|
const [paths, setPaths] = useState<string[]>(session.allowed_read_paths);
|
||||||
|
const [pendingDelete, setPendingDelete] = useState<string | null>(null);
|
||||||
|
|
||||||
|
// Re-sync on session prop change (e.g. WS session_updated after a new
|
||||||
|
// grant lands). Without this, a grant approved in this same chat wouldn't
|
||||||
|
// appear in the list until the user closes and reopens settings.
|
||||||
|
useEffect(() => {
|
||||||
|
setPaths(session.allowed_read_paths);
|
||||||
|
}, [session.id, session.allowed_read_paths]);
|
||||||
|
|
||||||
|
async function remove(path: string) {
|
||||||
|
if (pendingDelete) return;
|
||||||
|
setPendingDelete(path);
|
||||||
|
const next = paths.filter((p) => p !== path);
|
||||||
|
try {
|
||||||
|
const updated = await api.sessions.update(session.id, { allowed_read_paths: next });
|
||||||
|
setPaths(updated.allowed_read_paths);
|
||||||
|
toast.success('Grant revoked');
|
||||||
|
} catch (err) {
|
||||||
|
toast.error(err instanceof Error ? err.message : 'failed to revoke');
|
||||||
|
} finally {
|
||||||
|
setPendingDelete(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="space-y-1.5">
|
||||||
|
<label className="text-xs font-medium uppercase tracking-wide text-muted-foreground">
|
||||||
|
Cross-repo read grants
|
||||||
|
</label>
|
||||||
|
{paths.length === 0 ? (
|
||||||
|
<p className="text-xs text-muted-foreground italic">
|
||||||
|
The agent has no access outside this project. Grants are created when
|
||||||
|
the agent asks for them inline.
|
||||||
|
</p>
|
||||||
|
) : (
|
||||||
|
<ul className="space-y-1">
|
||||||
|
{paths.map((p) => (
|
||||||
|
<li
|
||||||
|
key={p}
|
||||||
|
className="flex items-center gap-2 rounded border bg-background/60 px-2 py-1.5"
|
||||||
|
>
|
||||||
|
<FolderOpen className="size-3.5 shrink-0 text-muted-foreground" />
|
||||||
|
<span className="font-mono text-xs flex-1 min-w-0 break-all">{p}</span>
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={() => void remove(p)}
|
||||||
|
disabled={pendingDelete !== null}
|
||||||
|
aria-label={`Revoke ${p}`}
|
||||||
|
title="Revoke"
|
||||||
|
className="inline-flex items-center justify-center size-7 rounded text-muted-foreground hover:bg-muted hover:text-destructive disabled:opacity-40 disabled:cursor-not-allowed max-md:min-h-[44px] max-md:min-w-[44px]"
|
||||||
|
>
|
||||||
|
<Trash2 className="size-3.5" />
|
||||||
|
</button>
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
<p className="text-xs text-muted-foreground">
|
||||||
|
Grants are session-scoped. Archiving the session clears them.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
function ProjectSection({ project }: { project: Project }) {
|
function ProjectSection({ project }: { project: Project }) {
|
||||||
const [name, setName] = useState(project.name);
|
const [name, setName] = useState(project.name);
|
||||||
const [defaultPrompt, setDefaultPrompt] = useState(project.default_system_prompt);
|
const [defaultPrompt, setDefaultPrompt] = useState(project.default_system_prompt);
|
||||||
|
|||||||
@@ -72,6 +72,30 @@ External code lifted from / referenced in: see `boocode_code_review.md` for full
|
|||||||
|
|
||||||
-----
|
-----
|
||||||
|
|
||||||
|
### Shipped (v1.13.x — written 2026-05-22, retagged same day)
|
||||||
|
|
||||||
|
All v1.13.x batches were retagged to the `vMAJOR.MINOR.PATCH-slug` scheme on 2026-05-22. `CHANGELOG.md` is the canonical per-tag record (slug describes what shipped; tag name alone recalls the batch). Tip is `v1.13.14-skills-audit` (`0fa46cd`); the next batch is `v1.13.15-codecontext-synth` (this batch, tag pending). Tags in chronological order:
|
||||||
|
|
||||||
|
- `v1.13.0-ai-sdk-v6` — AI SDK v6 migration; `streamCompletion` adapter; `messages_with_parts` view; reasoning_parts end-to-end
|
||||||
|
- `v1.13.1-cleanup-bundle` — `statement_timeout='30s'`, alpha-sorted tool registry, 60s stuck-row sweeper, `experimental_repairToolCall` pass-through
|
||||||
|
- `v1.13.2-compaction-prune` — two-tier prune; `message_parts.hidden_at` column + partial index; `messages_with_parts` view CASE refinement
|
||||||
|
- `v1.13.3-truncate` — opencode `truncate.ts` port; opaque `tr_<…>` id, `view_truncated_output(id)` tool, tmpfs storage
|
||||||
|
- `v1.13.4-reasoning-fix` — `<reasoning>` prose-prefix in compaction head-assembly for tool-bearing turns
|
||||||
|
- `v1.13.5-stability-bundle` — `includeUsage: true` on provider, `hasText` trim guard, `BUDGET_NO_AGENT` 15→30, trailing-empty-assistant filter
|
||||||
|
- `v1.13.6-prefix-stability` — `buildSystemPromptWithFingerprint` SHA-256 + per-session drift observer
|
||||||
|
- `v1.13.7-compaction-trigger` — overflow trigger lowered to `floor(0.85 × ctx_max)`
|
||||||
|
- `v1.13.8-tool-cost` — `tool_cost_stats` SQL view + per-tool rolling 100-call mean in AgentPicker
|
||||||
|
- `v1.13.9-agentlint` — instruction-file AgentLint pass; identity-openers removed; `CLAUDE.local.md` to .gitignore
|
||||||
|
- `v1.13.10-openspec` — `openspec/changes/<slug>/{proposal,tasks,design}.md` shape; archived batch docs preserved via `git mv`
|
||||||
|
- `v1.13.11-tools` — tiered tool loading via `BOOCODE_TOOLS` env (`core | standard | all`)
|
||||||
|
- `v1.13.12-ws-schemas` — Zod schemas for all 27 wire-format frames; `publishFrame` / `publishUserFrame` wrappers; parity test
|
||||||
|
- `v1.13.13-ws-publish` — all ~80 publish sites converted to the typed wrappers; every WS frame now Zod-validated at boundary
|
||||||
|
- `v1.13.14-skills-audit` — 26 skills vendored + audited via 5 parallel agent teams; 14 kept, 11 dropped, 1 migrated to BOOCHAT.md/BOOCODER.md
|
||||||
|
- `v1.13.15-codecontext-synth` — forced second-inference synthesis pass for codecontext overview tools (truncation-aware extraction; auto-fetched top-N files + project docs; 32k payload-budget contract preserved)
|
||||||
|
- `v1.13.16-xml-parser` — Anthropic `<invoke>` parser support + Levenshtein-based unknown-tool recovery hints (qwen3.6 drift to Claude Code-style tool names like `read_file`); xml-parser test coverage
|
||||||
|
|
||||||
|
The remaining strangler-fig final step (drop `messages.tool_calls` + `tool_results` columns) is still pending under its old `v1.13.2` working name; will get a new tag slug when scoped.
|
||||||
|
|
||||||
## In flight / next (v1.13.x cleanup line)
|
## In flight / next (v1.13.x cleanup line)
|
||||||
|
|
||||||
Five more single-dispatch batches before the strangler-fig closes. Each ships independently with its own smoke and rollback surface. **Do not fold.** Order is locked:
|
Five more single-dispatch batches before the strangler-fig closes. Each ships independently with its own smoke and rollback surface. **Do not fold.** Order is locked:
|
||||||
@@ -462,17 +486,23 @@ term.indifferentketchup.com → booterm :9501 (or routed under code.
|
|||||||
- **v1.11.7:** none (pathGuard logic, no DB)
|
- **v1.11.7:** none (pathGuard logic, no DB)
|
||||||
- **v1.12.0:** none (codecontext stateless; truncation in-memory id-map with TTL cleanup)
|
- **v1.12.0:** none (codecontext stateless; truncation in-memory id-map with TTL cleanup)
|
||||||
- **v1.12.1:** `sessions.workspace_panes jsonb` (workspace sync); drop deprecated `session_panes` table; drop stale `messages_status_check` constraint
|
- **v1.12.1:** `sessions.workspace_panes jsonb` (workspace sync); drop deprecated `session_panes` table; drop stale `messages_status_check` constraint
|
||||||
- **v1.13.0:** `message_parts (id, message_id, sequence, kind, payload jsonb, created_at)` + unique `(message_id, sequence)` + `kind` CHECK; `ToolDef.category` field (TS type, not DB)
|
- **v1.13.0-ai-sdk-v6:** `message_parts (id, message_id, sequence, kind, payload jsonb, created_at)` + unique `(message_id, sequence)` + `kind` CHECK; `messages_with_parts` view with COALESCE fallbacks; `ToolDef.category` field (TS type, not DB)
|
||||||
- **v1.13.1-B:** `messages_with_parts` view with COALESCE fallbacks
|
- **v1.13.1-cleanup-bundle:** `ALTER DATABASE boocode SET statement_timeout = '30s'` (op step, documented in schema.sql; doesn't survive volume reset)
|
||||||
- **v1.13.3:** `ALTER DATABASE boocode SET statement_timeout = '30s'` (op step, documented in schema.sql; doesn't survive volume reset)
|
- **v1.13.2-compaction-prune:** `message_parts.hidden_at TIMESTAMPTZ` column + partial index `(message_id) WHERE hidden_at IS NULL`; `messages_with_parts` view filters hidden parts
|
||||||
- **v1.13.4:** `message_parts.hidden_at TIMESTAMPTZ` column + partial index `(message_id) WHERE hidden_at IS NULL`; `messages_with_parts` view filters hidden parts
|
- **v1.13.3-truncate:** none (tmpfs id-map stored on disk under `BOOCODE_TRUNCATION_DIR`; no schema)
|
||||||
- **v1.13.5:** none (tmpfs id-map stored on disk under `BOOCODE_TRUNCATION_DIR`; no schema)
|
- **v1.13.4-reasoning-fix:** none (compaction read-side change; `CompactionMessage` extended in TS, not DB)
|
||||||
- **v1.13.6:** none (compaction read-side change; `CompactionMessage` extended in TS, not DB)
|
- **v1.13.5-stability-bundle:** none (provider config + 4 frontend/payload guards + budget constant, no schema change)
|
||||||
- **v1.13.7:** none (provider config + 4 frontend/payload guards + budget constant, no schema change)
|
- **v1.13.6-prefix-stability:** none — verify-and-measure batch, instrumentation only; drops the originally-planned `system_prompt_cache` table since recon proved input-layer mtime caches already achieve prefix stability
|
||||||
- **v1.13.8 (planned):** none — verify-and-measure batch, instrumentation only; drops the originally-planned `system_prompt_cache` table since recon proved input-layer mtime caches already achieve prefix stability
|
- **v1.13.7-compaction-trigger:** none (compaction overflow trigger is a constant change in `services/compaction.ts`, no DB)
|
||||||
- **v1.13.9 (planned):** none (compaction overflow trigger is a constant change in `services/compaction.ts`, no DB)
|
- **v1.13.8-tool-cost:** `tool_cost_stats` SQL view over `messages_with_parts` (no new table — view + LATERAL `jsonb_array_elements` on `tool_calls`); rolling 100-call window
|
||||||
- **v1.13.10 (planned):** `tool_cost_stats (tool_name, prompt_tokens_sum, completion_tokens_sum, n_calls, updated_at)` — rolling 100-call window
|
- **v1.13.9-agentlint:** none (instruction-file audit + `.gitignore` add of `CLAUDE.local.md`, no DB)
|
||||||
- **v1.13.2 (planned):** drop `messages.tool_calls`, `messages.tool_results`; simplify `messages_with_parts` view
|
- **v1.13.10-openspec:** none (docs reorganization, `git mv` only)
|
||||||
|
- **v1.13.11-tools:** none (env-var tier filter at request time, no DB)
|
||||||
|
- **v1.13.12-ws-schemas:** none (Zod schemas + wrappers in TS, no DB)
|
||||||
|
- **v1.13.13-ws-publish:** none (publish-site conversion + protocol-drift fix in `compaction.ts`, no DB)
|
||||||
|
- **v1.13.14-skills-audit:** none (skills + AGENTS.md migration into git via `.gitignore` negation patterns; no DB)
|
||||||
|
- **v1.13.15-codecontext-synth (this batch, tag pending):** `message_parts.kind` CHECK constraint extended with `'synthesis'` value (DROP + DO $$ pg_constraint idempotency-guarded re-add)
|
||||||
|
- **(column drop, pending — old working name v1.13.2):** drop `messages.tool_calls`, `messages.tool_results`; simplify `messages_with_parts` view
|
||||||
- **v1.14:** `agents.steps` column (or AGENTS.md parser extension; no DB if file-only)
|
- **v1.14:** `agents.steps` column (or AGENTS.md parser extension; no DB if file-only)
|
||||||
- **v1.14.x-mcp (NEW):** none — single-server MCP-client PoC is config-only at first, no schema change
|
- **v1.14.x-mcp (NEW):** none — single-server MCP-client PoC is config-only at first, no schema change
|
||||||
- **v1.14.x-html (NEW):** `message_parts.kind` CHECK constraint extended with `'html_artifact'` value
|
- **v1.14.x-html (NEW):** `message_parts.kind` CHECK constraint extended with `'html_artifact'` value
|
||||||
@@ -582,7 +612,7 @@ Earlier May 18 chat recommended Option A (thin orchestration shell over OpenCode
|
|||||||
|
|
||||||
### v1.13.x cleanup line locked (2026-05-22)
|
### v1.13.x cleanup line locked (2026-05-22)
|
||||||
|
|
||||||
After v1.13.1-C shipped clean, the cleanup order is **v1.13.3 ✅ → v1.13.4 ✅ → v1.13.5 ✅ → v1.13.6 ✅ → v1.13.7 ✅ → v1.13.8 (verify) → v1.13.9 (overflow) → v1.13.10 → v1.13.11 → v1.13.12 → v1.13.2** (column drop last as rollback insurance). **Do not fold.** Smoke isolation matters: each batch has a distinct rollback surface, and bisecting a 750-LoC merge across four unrelated changes is worse than four separate dispatches.
|
After the 2026-05-22 retag, the v1.13.x cleanup line in `vMAJOR.MINOR.PATCH-slug` form is **v1.13.0-ai-sdk-v6 ✅ → v1.13.1-cleanup-bundle ✅ → v1.13.2-compaction-prune ✅ → v1.13.3-truncate ✅ → v1.13.4-reasoning-fix ✅ → v1.13.5-stability-bundle ✅ → v1.13.6-prefix-stability ✅ → v1.13.7-compaction-trigger ✅ → v1.13.8-tool-cost ✅ → v1.13.9-agentlint ✅ → v1.13.10-openspec ✅ → v1.13.11-tools ✅ → v1.13.12-ws-schemas ✅ → v1.13.13-ws-publish ✅ → v1.13.14-skills-audit ✅ → v1.13.15-codecontext-synth ✅ → v1.13.16-xml-parser ✅ → column drop (final, pending — old working name v1.13.2)**. **Do not fold.** Smoke isolation matters: each batch has a distinct rollback surface, and bisecting a 750-LoC merge across four unrelated changes is worse than four separate dispatches.
|
||||||
|
|
||||||
### v1.13 retrospective (what shipped)
|
### v1.13 retrospective (what shipped)
|
||||||
|
|
||||||
|
|||||||
145
openspec/changes/v1.13.15-codecontext-synth/proposal.md
Normal file
145
openspec/changes/v1.13.15-codecontext-synth/proposal.md
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
# v1.13.13 — codecontext synthesis pipeline
|
||||||
|
|
||||||
|
Slots between v1.13.12 (skills audit) and v1.14 (Phase C outer agent loop). Adds a forced second-inference synthesis pass for codecontext overview/analysis tools so the model stops returning shallow first-touch summaries.
|
||||||
|
|
||||||
|
Does NOT change the recursion structure, depth cap, or budget — those are v1.14 concerns. The cap-50 patch from v1.13.12 stays; v1.14 supersedes it via per-agent `agent.steps`.
|
||||||
|
|
||||||
|
## What ships
|
||||||
|
|
||||||
|
- `apps/server/src/services/synthesisPrompt.ts` (NEW, 20 lines) — verbatim system prompt as a const.
|
||||||
|
- `apps/server/src/services/synthesisPipeline.ts` (NEW, ~450 lines) — `SYNTHESIS_TOOLS` set + `runSynthesisPass(params) → Promise<boolean>`. Auto-fetches top-N referenced files + project docs (BOOCHAT.md, AGENTS.md, *roadmap*.md, CONTEXT.md), applies a 32k-token budget with priority drop order, streams a synthesis turn via `streamCompletion`, dual-writes a `kind='synthesis'` part.
|
||||||
|
- `apps/server/src/services/inference/parts.ts` — `PartKind` union extended with `'synthesis'`.
|
||||||
|
- `apps/server/src/services/inference/tool-phase.ts` — synth-tool result capture during `Promise.all`; post-pause synth check before the recursive `runAssistantTurn`.
|
||||||
|
- `apps/server/src/schema.sql` — inline CHECK constraint updated + `DROP CONSTRAINT IF EXISTS` + `DO $$ pg_constraint` migration block. Idempotent (drops + re-adds on every startup; per-boot cost is trivial).
|
||||||
|
|
||||||
|
SYNTHESIS_TOOLS = `{get_codebase_overview, get_framework_analysis, get_semantic_neighborhoods}`. The other 5 codecontext tools (search_symbols, get_dependencies, get_file_analysis, get_symbol_info, watch_changes) return targeted data the model uses directly — no synthesis pass.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
### Schema migration was required (dispatch was wrong)
|
||||||
|
|
||||||
|
The original dispatch said "kind is text column, no schema migration needed." Reality: `schema.sql:54` has an explicit `message_parts_kind_chk` CHECK constraint enumerating allowed kinds (`'text', 'tool_call', 'tool_result', 'reasoning', 'step_start'`). Adding `'synthesis'` requires updating the constraint.
|
||||||
|
|
||||||
|
Resolution: added a `DROP CONSTRAINT IF EXISTS` + `DO $$ ... pg_constraint` idempotency-guarded migration block in `schema.sql` matching the CLAUDE.md migration pattern, plus updated the inline CREATE TABLE constraint so fresh installs include the new value.
|
||||||
|
|
||||||
|
### `view_file` input shape uses `start_line`/`end_line`, not `line_count`
|
||||||
|
|
||||||
|
The dispatch's auto-fetch sketch implied a `line_count` parameter. The real `viewFile` tool's input schema (`tools.ts:51-55`) takes `start_line`/`end_line` (1-indexed inclusive) with a 200-line default if both are omitted. The pipeline uses `end_line: FILE_LINE_CAP` for files (200) and `end_line: DOC_LINE_CAP` for docs (500), which gives the first N lines — same effective truncation.
|
||||||
|
|
||||||
|
### User-abort during synthesis marks the synth message failed (deviates from review req)
|
||||||
|
|
||||||
|
**Decision: option A — mark synth message `status='failed'` on every catch path including user-abort, then re-throw on user-abort.**
|
||||||
|
|
||||||
|
Sam's stated review requirement: "User-abort path does NOT mark the message failed (re-throw to outer handler is correct)."
|
||||||
|
|
||||||
|
Why this deviation: the outer abort handler (`error-handler.ts:handleAbortOrError`) operates on `args.assistantMessageId` — the *parent* assistant message that triggered the tool call. It does not know about the *new* synth assistant message that `runSynthesisPass` created. If the synth row isn't explicitly marked failed on user-abort, it sits in `status='streaming'` until the 5-min stale-streaming sweeper (`apps/server/src/index.ts`) picks it up — meanwhile the frontend's 60s no-token-activity timer trips the stale-stream banner on the orphan. Same UX bug class the v1.13.3 stuck-row sweeper was added to handle.
|
||||||
|
|
||||||
|
Cost: one extra DB write + one `message_complete` republish on the rare user-abort-during-synth path. Worth it to avoid the zombie message + ghost banner.
|
||||||
|
|
||||||
|
**Note for v1.14 outer-loop port**: when Phase C migrates the depth cap into `agent.steps` and reworks the recursion, the synth message is a sibling to the parent assistant message — both belong to the same chat. The new outer loop should either (a) preserve this pattern (mark all chat-scoped streaming messages failed on abort) or (b) extend `handleAbortOrError` to sweep chat-scoped streaming rows. Option (b) is a wider blast radius and was rejected here; option (a) is one targeted call site.
|
||||||
|
|
||||||
|
### Token budget priority list
|
||||||
|
|
||||||
|
Drop order when the 32k cap is exceeded (lowest priority first):
|
||||||
|
1. top-2..N files (keep top-1)
|
||||||
|
2. top-1 file
|
||||||
|
3. `*roadmap*.md` + `CONTEXT.md` (mid-priority — both describe state/intent)
|
||||||
|
4. `AGENTS.md`
|
||||||
|
5. `BOOCHAT.md` — **never dropped**; truncated to 32k if it alone exceeds
|
||||||
|
|
||||||
|
CONTEXT.md wasn't in the original dispatch's priority list; grouped with roadmap as mid-priority (same semantic — both are state/intent docs).
|
||||||
|
|
||||||
|
### 90s timeout via `AbortSignal.any`
|
||||||
|
|
||||||
|
Synthesis call has its own `AbortController` with a 90s `setTimeout`. Combined with `p.args.signal` (the user-abort signal) via `AbortSignal.any([user, synth])` — either fires correctly. Node 20.3+. A `timedOut` flag in scope disambiguates which signal tripped after `streamCompletion` throws (`AbortError`): timeout → return false (fall through to recursion); user-abort → re-throw (after `markSynthFailed`).
|
||||||
|
|
||||||
|
### Race-safe synth-tool capture under `Promise.all`
|
||||||
|
|
||||||
|
`synthEntries: Array<{tc, output, error?}>` populated by each parallel callback pushing its own result. After `Promise.all` resolves, `synthEntries.find((e) => !e.error && e.output != null)` picks the first non-error synth entry by call-order (i.e. by `toolCalls` array index in the original LLM emit order). Not result-quality scoring — explicitly call-order, documented inline.
|
||||||
|
|
||||||
|
### Known interaction: qwen3.6 `include_stats: "True"` retry loop compounds synth-pass cost
|
||||||
|
|
||||||
|
Smoke #1 surfaced a pre-existing qwen3.6 quirk: the model emits `"True"` (string) instead of `true` (bool) for boolean tool args. The `experimental_repairToolCall` + zod-reject retry path (v1.13.3) handles this — the model retries on the next turn with corrected args, then succeeds.
|
||||||
|
|
||||||
|
**Synth pass cost interaction:** when the first tool-call fails zod validation, the recursive runAssistantTurn fires *before* the successful synth-tool call lands. The user effectively pays: (1) failed tool-call turn → (2) error tool-result → (3) retry tool-call turn → (4) successful tool-result → (5) synth pass.
|
||||||
|
|
||||||
|
Per-fire token cost for an overview question now: ~5 inference calls (turns 1, 3, 5 are model calls; 5 is the synth pass adding ~5k tokens of auto-fetched context). Not a blocker — the synth content is dramatically better than the without-synth case (4920 tokens of cited analysis vs. a 70-token tool-call-only turn). Worth tracking if usage stats start showing it.
|
||||||
|
|
||||||
|
### v1.14 outer-loop port — preserve this pattern
|
||||||
|
|
||||||
|
Two patterns from this batch the Phase C outer-loop port must preserve:
|
||||||
|
|
||||||
|
1. **Chat-scoped abort cleanup**: the synth message is a sibling to the parent assistant message, both belong to the same chat. The new outer loop should either (a) keep `markSynthFailed` (or its equivalent) firing on every catch path including user-abort, or (b) extend `handleAbortOrError` to sweep all chat-scoped streaming rows. This batch chose (a); (b) was rejected as wider blast radius.
|
||||||
|
2. **Race-safe `Promise.all` capture**: `synthEntries: Array<...>` instead of a single shared variable. Per-callback push avoids the last-write-wins race when a batch has multiple synth tools.
|
||||||
|
|
||||||
|
## Test plan
|
||||||
|
|
||||||
|
6-prompt smoke + 1 failure-injection. Sequence:
|
||||||
|
|
||||||
|
1. **Default agent** — "What's in this codebase?" → expect `get_codebase_overview` + synthesis pass, response cites BOOCHAT.md + actual files + roadmap state.
|
||||||
|
2. **Architect agent** — "Give me a system overview of how BooCode handles tool calls" → expect synthesis with refs to inference/turn.ts, tool-phase.ts, stream-phase.ts.
|
||||||
|
3. **Architect agent** — "What's the current state of v1.13?" → synthesis must read `boocode_roadmap.md` and report shipped vs planned correctly. Must NOT infer "v1.13.2 shipped" from code presence — roadmap explicitly defers it.
|
||||||
|
4. **Code Reviewer** — "Find all callers of buildSystemPrompt" → `search_symbols` fires, NO synthesis pass (not in SYNTHESIS_TOOLS).
|
||||||
|
5. **Debugger** — "Where is detectDoomLoop defined and called from?" → `search_symbols` + `get_dependencies`, NO synthesis pass.
|
||||||
|
6. **Failure injection** — temporarily make `streamCompletion` throw inside `runSynthesisPass`; verify fall-through to recursion + log entry visible + non-empty answer.
|
||||||
|
|
||||||
|
## Backups in place
|
||||||
|
|
||||||
|
```
|
||||||
|
apps/server/src/schema.sql.bak-v1.13.13-20260522
|
||||||
|
apps/server/src/services/inference/parts.ts.bak-v1.13.13-20260522
|
||||||
|
apps/server/src/services/inference/tool-phase.ts.bak-v1.13.13-20260522
|
||||||
|
```
|
||||||
|
|
||||||
|
To be deleted after merge.
|
||||||
|
|
||||||
|
## Smoke results
|
||||||
|
|
||||||
|
### Smoke #1 — default agent, "What is in this codebase?"
|
||||||
|
|
||||||
|
Synthesis fired on `get_codebase_overview`. Log line:
|
||||||
|
```
|
||||||
|
{"chatId":"7bb05e54-…","synthMessageId":"44480541-…","toolName":"get_codebase_overview","chars":6727,"files":5,"msg":"synthesis pass complete"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Token accounting: synth turn = 4920 tokens (vs. 63 + 70 on the preceding tool-call-only turns). Model is using the auto-fetched context, not parroting codecontext output. Synth message has the expected `kind='synthesis'` part dual-write.
|
||||||
|
|
||||||
|
Side note: qwen3.6 needed one retry due to the `include_stats: "True"` quirk (see Decisions). `repairToolCall` handled it; synth fired on the successful call.
|
||||||
|
|
||||||
|
### Smoke #6 — fault injection
|
||||||
|
|
||||||
|
Env-gated throw inserted between the synth-message INSERT and the `streamCompletion` call. Container rebuilt with `V1_13_13_FAULT_INJECT=1`. Sent the same prompt to a new smoke chat.
|
||||||
|
|
||||||
|
All 6 expected outcomes confirmed:
|
||||||
|
|
||||||
|
| # | Outcome | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | `runSynthesisPass` throws | log: `err: "Error: v1.13.13 smoke #6 fault injection"` |
|
||||||
|
| 2 | Synth message marked `status='failed'` with empty content | msg `7ac9c685-…` role=assistant status=failed content_len=0 |
|
||||||
|
| 3 | `message_complete` frame published for the synth message | implicit via `markSynthFailed`; frontend never tripped the 60s timer |
|
||||||
|
| 4 | Fall-through to recursive `runAssistantTurn` | log: `synthesis pass failed; falling through to recursive turn` |
|
||||||
|
| 5 | User sees normal (non-synthesized) assistant response | final msg `924076a3-…` 453 tokens: `"This is **boocode** — a self-hosted, single-user developer chat app."` |
|
||||||
|
| 6 | Stale-stream banner does NOT fire on failed synth | confirmed — terminal `status='failed'` is what `applyFrame` writes |
|
||||||
|
|
||||||
|
Fault injection reverted post-test:
|
||||||
|
- `grep FAULT_INJECT apps/server/src/services/synthesisPipeline.ts docker-compose.yml` → empty
|
||||||
|
- `grep FAULT_INJECT apps/server/dist/services/synthesisPipeline.js` → empty
|
||||||
|
- `docker compose exec boocode printenv V1_13_13_FAULT_INJECT` → exit 1 (unset)
|
||||||
|
- Boot log clean, `skills loaded: 14`
|
||||||
|
|
||||||
|
### Smokes #2–#5
|
||||||
|
|
||||||
|
Sam is doing the qualitative reads from the UI in parallel — those verifications are about synthesis content quality (cites correct files, reads roadmap accurately, no-synthesis on `search_symbols`).
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- ✅ `synthesisPrompt.ts` + `synthesisPipeline.ts` created
|
||||||
|
- ✅ `parts.ts` PartKind union extended
|
||||||
|
- ✅ `tool-phase.ts` insertion point edited
|
||||||
|
- ✅ Schema migration block added (deviation from dispatch acknowledged)
|
||||||
|
- ✅ Type-clean (`pnpm -C apps/server build`)
|
||||||
|
- ✅ Container rebuilt + migration confirmed via pg_constraint and logs
|
||||||
|
- ✅ Smoke #1 (positive synth path) verified
|
||||||
|
- ✅ Smoke #6 (fault injection + fall-through) verified, injection reverted
|
||||||
|
- ⏳ Smokes #2–#5 (Sam's UI reads)
|
||||||
|
- ⏳ Sam commit
|
||||||
185
openspec/changes/v1.13.17-cross-repo-reads/proposal.md
Normal file
185
openspec/changes/v1.13.17-cross-repo-reads/proposal.md
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
# v1.13.17-cross-repo-reads — on-demand read access to another repo (draft, 2026-05-22)
|
||||||
|
|
||||||
|
BooChat sessions are scoped to one project root. When the agent needs context from another repo (e.g. `/opt/forks/codecontext` to investigate a dependency), `pathGuard` rejects every read tool and the agent has no recovery path.
|
||||||
|
|
||||||
|
This batch adds a reactive `ask_user_input`-style flow that the agent triggers on `PathScopeError`. User approves once per session per project root; subsequent reads under that root succeed without further prompting.
|
||||||
|
|
||||||
|
## Trigger flow
|
||||||
|
|
||||||
|
1. Model emits `view_file("/opt/forks/codecontext/go.mod")` while session is scoped to `/opt/boocode`.
|
||||||
|
2. `pathGuard` throws `PathScopeError`. Existing tool wrapper catches it and returns the error to the model. **The error message now ends with a hint:** `"Use request_read_access(path, reason) to ask the user for permission."`
|
||||||
|
3. Model self-issues `request_read_access("/opt/forks/codecontext/go.mod", "investigating codecontext fork to write design doc")` on the next turn.
|
||||||
|
4. The new tool emits a pending tool-call frame (same pause mechanism as `ask_user_input`); inference loop pauses.
|
||||||
|
5. Frontend renders approve/deny chips with the path + reason.
|
||||||
|
6. User picks Allow → append the grant root to `session.allowed_read_paths`, resume inference, tool returns `"granted: /opt/forks/codecontext"`. Model retries the original `view_file` on the next turn.
|
||||||
|
7. User picks Deny → tool returns `"denied"` without mutating session state; model decides what to do next.
|
||||||
|
|
||||||
|
## Decisions (draft — override in dispatch if different)
|
||||||
|
|
||||||
|
### D1. Grant unit = nearest registered project root, then nearest path-whitelist ancestor, then refuse
|
||||||
|
|
||||||
|
When user approves access to `/opt/forks/codecontext/go.mod`:
|
||||||
|
- If a row in `projects.path` is an ancestor of the requested path → grant the project's root path.
|
||||||
|
- Else if `PROJECT_ROOT_WHITELIST` env (default `/opt`) is an ancestor and the immediate child dir of the whitelist looks like a repo root (`.git/`, `package.json`, `go.mod`, or `Cargo.toml` present) → grant that immediate child dir (e.g. `/opt/forks/codecontext`).
|
||||||
|
- Else → refuse without prompting. Tool returns `"denied: path outside permitted scope"`. No user prompt fires.
|
||||||
|
|
||||||
|
Why: granting the literal path is too narrow (next file in the same repo re-prompts). Granting an arbitrary parent dir over-scopes. The nearest repo-shaped directory is the natural unit.
|
||||||
|
|
||||||
|
### D2. Persistence = per-session, no expiry
|
||||||
|
|
||||||
|
`sessions.allowed_read_paths` is the source of truth. Grants stick until the session is archived. A new session in the same project re-prompts on the first cross-repo read.
|
||||||
|
|
||||||
|
Why: per-chat is too granular for the typical workflow (Sam investigates the same fork across multiple chats in one investigation session). Per-project is too broad (different sessions in the same project might have different scope needs). Per-session is the natural unit and matches `session.web_search_enabled`'s scope.
|
||||||
|
|
||||||
|
### D3. Secret-file deny list applies across all grant roots
|
||||||
|
|
||||||
|
`is_secret_path` in `secret_guard.ts` filters filenames (`.env`, `*.pem`, `credentials.json`, etc.) regardless of which root they're under. The check is post-`pathGuard`, so it already runs on the resolved path. No change needed.
|
||||||
|
|
||||||
|
### D4. Revocation UI = chat-settings panel + automatic clear on archive
|
||||||
|
|
||||||
|
- Settings panel under the session-info popover: lists current `allowed_read_paths` with a per-row delete button.
|
||||||
|
- Session archive deletes the row (no need to clear allowed_read_paths separately — the row goes).
|
||||||
|
- No expiry timer.
|
||||||
|
|
||||||
|
Optional v1.13.18 follow-up if Sam wants it: a `/clear_grants` slash command for power users. Out of scope for v1.13.17.
|
||||||
|
|
||||||
|
## Schema
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- v1.13.17: session-scoped cross-repo read grants. Populated via the
|
||||||
|
-- request_read_access tool's approve path; never written by other code.
|
||||||
|
ALTER TABLE sessions
|
||||||
|
ADD COLUMN IF NOT EXISTS allowed_read_paths text[] NOT NULL DEFAULT ARRAY[]::text[];
|
||||||
|
```
|
||||||
|
|
||||||
|
No CHECK constraint — values are absolute paths validated at write time against the projects table + whitelist heuristic.
|
||||||
|
|
||||||
|
## New tool: `request_read_access`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// apps/server/src/services/request_read_access.ts (new)
|
||||||
|
|
||||||
|
export const requestReadAccessInput = z.object({
|
||||||
|
path: z.string().min(1),
|
||||||
|
reason: z.string().min(1).max(500),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const requestReadAccess: ToolDef<...> = {
|
||||||
|
name: 'request_read_access',
|
||||||
|
description:
|
||||||
|
'Ask the user for read-only access to a path outside the current ' +
|
||||||
|
'session\'s project scope. Use when pathGuard rejected a read ' +
|
||||||
|
'attempt and the path is plausibly under another known repo. ' +
|
||||||
|
'Returns "granted: <root>" or "denied".',
|
||||||
|
inputSchema: requestReadAccessInput,
|
||||||
|
jsonSchema: { ... },
|
||||||
|
category: 'read_only',
|
||||||
|
async execute(input, projectRoot) {
|
||||||
|
// Validate path: must be absolute, must be under PROJECT_ROOT_WHITELIST
|
||||||
|
// (default /opt), must NOT already be under the session's primary
|
||||||
|
// projectRoot (silly to ask for what's already in scope).
|
||||||
|
// Validation failures return sentinel without prompting the user.
|
||||||
|
|
||||||
|
// Emit pending-grant tool result (parallel of ask_user_input's pause
|
||||||
|
// sentinel). Inference loop pauses on this kind=pending_grant marker.
|
||||||
|
// User picks Allow/Deny via a new POST /api/messages/:id/grant endpoint.
|
||||||
|
// On Allow: derive grant root per D1 + UPDATE sessions SET
|
||||||
|
// allowed_read_paths = array_append(allowed_read_paths, <root>);
|
||||||
|
// resume inference; tool returns "granted: <root>".
|
||||||
|
// On Deny: resume immediately; tool returns "denied".
|
||||||
|
},
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Registered in `ALL_TOOLS` + `READ_ONLY_TOOL_NAMES`. Available to all agents by default (no agent's `tools` whitelist needs to be updated to grant access — the tool registry's filter is per-agent).
|
||||||
|
|
||||||
|
## `pathGuard` extension
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// apps/server/src/services/path_guard.ts — current signature:
|
||||||
|
// pathGuard(projectRoot, requestedPath): Promise<string>
|
||||||
|
//
|
||||||
|
// Extended:
|
||||||
|
// pathGuard(projectRoot, requestedPath, extraRoots?: string[]): Promise<string>
|
||||||
|
//
|
||||||
|
// Tries primary projectRoot first; on PathScopeError, walks extraRoots and
|
||||||
|
// returns the first one that resolves the requestedPath inside its tree.
|
||||||
|
// Throws PathScopeError if no root accepts.
|
||||||
|
```
|
||||||
|
|
||||||
|
Every tool that calls `pathGuard` (currently `view_file`, `list_dir`, `grep`, `find_files`, `view_truncated_output`) threads `session.allowed_read_paths` through `executeToolCall`. The `Session` interface already flows through `TurnArgs`; tool-phase just needs to forward `session.allowed_read_paths` as the third arg.
|
||||||
|
|
||||||
|
## Pause/resume infrastructure reuse
|
||||||
|
|
||||||
|
The pending-grant pause uses the **same mechanism as `ask_user_input`**:
|
||||||
|
- Tool insert with `payload.output = null` + `payload.kind = 'pending_grant'`.
|
||||||
|
- `pausingForUserInput` branch in `tool-phase.ts` is widened to also catch pending grants.
|
||||||
|
- `chat_status` flips to `waiting_for_input` per the v1.12.1 5-state model.
|
||||||
|
|
||||||
|
New endpoint `POST /api/messages/:tool_msg_id/grant` (parallel of the existing `/answer`):
|
||||||
|
- Body: `{ decision: 'allow' | 'deny' }`.
|
||||||
|
- Resolves grant root per D1 if Allow. UPDATEs `sessions.allowed_read_paths`. UPDATEs tool message with output. Resumes inference via existing enqueue path.
|
||||||
|
|
||||||
|
## Frontend changes (in scope; small)
|
||||||
|
|
||||||
|
- `MessageBubble.tsx`: render `pending_grant` tool messages with Allow/Deny chips + the path + reason text. Wires to `api.messages.grant(toolMsgId, decision)`.
|
||||||
|
- New API client method `api.messages.grant`.
|
||||||
|
- Settings popover: `allowed_read_paths` list with per-row delete (calls `PATCH /api/sessions/:id` with the modified array).
|
||||||
|
|
||||||
|
## Hard rules
|
||||||
|
|
||||||
|
- No git commit, no git push, no git pull during dispatch. Sam commits manually.
|
||||||
|
- Backup every file before edit per the standard convention.
|
||||||
|
- TS strict, no `any`.
|
||||||
|
- No new deps.
|
||||||
|
- Schema migration is **additive only** (ADD COLUMN IF NOT EXISTS), idempotent on re-run.
|
||||||
|
- Tool is **read-only** — no path under `allowed_read_paths` can ever be written by BooChat (no write tools registered today; this is a structural guarantee).
|
||||||
|
- Secret-file deny list still runs unconditionally on resolved paths.
|
||||||
|
|
||||||
|
## Stop checkpoints
|
||||||
|
|
||||||
|
1. After recon (read existing path_guard + ask_user_input + answer endpoint patterns): stop, hand back the recon report.
|
||||||
|
2. After code edits, before schema migration applies: stop, hand back the diff.
|
||||||
|
3. After schema migration applies in dev: stop, run smoke plan, report.
|
||||||
|
|
||||||
|
## Smoke plan
|
||||||
|
|
||||||
|
1. **Approve flow.** Send a chat in a `/opt/boocode` session asking the agent to investigate `/opt/forks/codecontext/go.mod`. Confirm:
|
||||||
|
- `pathGuard` throws on the first attempt; tool result includes the `request_read_access` hint.
|
||||||
|
- Agent calls `request_read_access`; tool-call frame lands; chat status flips to `waiting_for_input`.
|
||||||
|
- Frontend renders Allow/Deny chips with the path + reason.
|
||||||
|
- Pick Allow → grant root resolves to `/opt/forks/codecontext` (per D1); `sessions.allowed_read_paths` shows the entry; agent retries `view_file` successfully on the next turn.
|
||||||
|
2. **Deny flow.** Same setup; pick Deny. Confirm session state unchanged, tool returns `"denied"`, agent gives up or asks differently.
|
||||||
|
3. **Persistence.** In the same session, a second `view_file` against a different file under `/opt/forks/codecontext/` succeeds without re-prompting.
|
||||||
|
4. **Cross-session isolation.** Open a fresh session in the boocode project, try the same path — re-prompts (allowed_read_paths is empty on the new session).
|
||||||
|
5. **Secret-file deny still fires.** Approve access to a repo that contains a `.env` file. Try `view_file('/opt/forks/some-repo/.env')`. Confirm refused via `is_secret_path`, not via pathGuard scope.
|
||||||
|
6. **Out-of-scope refusal.** Try `request_read_access('/etc/passwd', 'system file')`. Tool validates against the whitelist + repo-shape heuristic, returns `"denied: path outside permitted scope"` without prompting the user.
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- New `request_read_access` tool + `POST /api/messages/:id/grant` endpoint shipped.
|
||||||
|
- `path_guard.ts` extended; all read tools forward `allowed_read_paths`.
|
||||||
|
- `MessageBubble.tsx` renders pending-grant bubbles; settings popover lists + clears grants.
|
||||||
|
- Schema migration applied (sessions.allowed_read_paths).
|
||||||
|
- Smoke plan green.
|
||||||
|
- v1.13.17-cross-repo-reads tag + CHANGELOG entry + roadmap retrospective bullet.
|
||||||
|
|
||||||
|
## Files expected to touch
|
||||||
|
|
||||||
|
- `apps/server/src/schema.sql` — new column
|
||||||
|
- `apps/server/src/services/request_read_access.ts` — NEW
|
||||||
|
- `apps/server/src/services/path_guard.ts` — extra-roots param + helpful PathScopeError message
|
||||||
|
- `apps/server/src/services/tools.ts` — register the new tool, update view_file / list_dir / grep / find_files / view_truncated_output to thread allowed_read_paths
|
||||||
|
- `apps/server/src/services/inference/tool-phase.ts` — pause-on-pending-grant branch (alongside ask_user_input)
|
||||||
|
- `apps/server/src/routes/messages.ts` — new `/grant` endpoint
|
||||||
|
- `apps/server/src/types/api.ts` — `Session.allowed_read_paths`
|
||||||
|
- `apps/web/src/api/client.ts` — `api.messages.grant`
|
||||||
|
- `apps/web/src/api/types.ts` — `Session.allowed_read_paths`
|
||||||
|
- `apps/web/src/components/MessageBubble.tsx` — render pending_grant chips
|
||||||
|
- `apps/web/src/components/` — settings-popover grants list (file TBD during impl)
|
||||||
|
|
||||||
|
Estimate: ~120 LoC across backend + frontend + schema. Single batch.
|
||||||
|
|
||||||
|
## Open questions for dispatch
|
||||||
|
|
||||||
|
The four design decisions above are my recommendations. Override any of them in the dispatch and I'll update the proposal before recon. Most likely-overridable: **D1** (grant unit — you may want exact-path-only for tighter scoping, accepting the re-prompt cost) and **D4** (revocation UI — you may want it deferred entirely).
|
||||||
Reference in New Issue
Block a user