boocode/boocode_roadmap.md

# BooCode v1.x — Roadmap

Last updated: 2026-05-20

## Overview

BooCode is a standalone code-chat tool at `/opt/boocode/`. Read-only by design — pick a project, chat with a local LLM that has file-inspection tools, get streaming responses over WebSocket.

Live at `https://code.indifferentketchup.com` (Caddy → Authelia → Tailscale → `100.114.205.53:9500`).

**Architectural commitments:**

- No embeddings. The model uses file-view tools (`view_file`, `list_dir`, `grep`, `find_files`) + sidecar analyzers (codecontext, codesight). Walked away from the RAG pipeline May 2026.
- Read-only in v1.x. Write tools land in BooCoder (separate container, post-v1.x).
- One Postgres (`boocode_db`), one frontend SPA, container-per-service for new capabilities.

External code lifted from / referenced in: see `boocode_code_review.md` for full inventory.

-----

## Shipped (status as of 2026-05-20)

| Version | Theme | Notes |
|---|---|---|
| v1.0 | Initial scaffold | live |
| Batches 1–4.4 | Markdown, sidebar, panes, chats-inside-sessions, archive, fork/delete, header polish, settings drawer | merged |
| v1.5 | resolveProjectPath, BOOTSTRAP_ROOT, vitest pin | merged |
| v1.6, v1.6.1, v1.6.2 | Mobile pass + RightRail mobile drawer | merged |
| v1.7 | Drag-drop file + paste-as-attachment | merged |
| v1.8, v1.8.1, v1.8.2 | Settings drawer, git_status tool, WS reconnect, **per-turn budget reset + Continue affordance + CapHitSentinel** | merged |
| v1.9.1 | Skills system (`/opt/skills/` + `skill_find`/`skill_use`/`skill_resource` tools + `/skill` slash command) | merged |
| v1.9.7 | `ask_user_input` elicitation tool | merged |
| **Batch 9 (Agents Tier 2)** | `AGENTS.md` + 6 builtin agents + AgentPicker in ChatInput toolbar + `sessions.agent_id` | **merged in `92bd3b1`**, included in v1.9.1/v1.9.7/v1.10.x tags |
| v1.10.0 | BooTerm: separate container, xterm.js + node-pty + tmux | merged |
| v1.10.1 | BooTerm-user (spawn as samkintop, login bash, Claude Code/opencode PATH) | merged |
| v1.10.4, v1.10.5 | Mobile terminal + XML tool-call fallback parser | merged |
| **v1.11.0** | **opencode-style compaction port** (auto-overflow, anchored summary, tail preservation) | merged |
| v1.11.1 | Compaction follow-up (working indicator during compaction, unit tests, .bak cleanup) | merged |
| v1.11.2 | ContextBar (persistent context-usage indicator) | merged |
| v1.11.3 | `ctx_max` capture via `/upstream/<model>/props` (replaces dead `timings.n_ctx` read) | merged |

-----

## In flight / queued

| Version | Theme | Status |
|---|---|---|
| ~~v1.11.4~~ | ~~Per-turn budget + Continue affordance~~ | **CANCELLED** — already shipped in v1.8.2 |
| **v1.11.5** | ContextBar relocate (above agent-picker row), thicker, always-visible, remove ChatContextPopover | **dispatched** |
| v1.11.6 | Doom-loop guard from opencode (3 identical tool calls → sentinel, abort recursion) | drafted |
| v1.11.7 | pathGuard secrets filter (continue.dev's `DEFAULT_SECURITY_IGNORE_FILETYPES`) | drafted |
| v1.11.x | Tag consolidation point (everything since v1.11.0) | queued |

-----

## Major work after v1.11.x

| Version | Theme | LoC est. |
|---|---|---|
| **v1.12** | codecontext sidecar + tool output truncation + repair tool call (Integration 1 + 3 from May review, fused) | ~600 |
| v1.13 | Phase B groundwork — parts table + AI SDK adoption + per-tool `read_only`/`write` tagging | ~1500 |
| v1.14 | Phase C — outer agent loop (multi-step until non-tool finish, AGENTS.md `steps` field, reasoning as part type) | ~800 |
| v1.15 | Phase D — permission ruleset + MCP client (lays foundation for BooCoder) | ~600 |
| v1.16 | Batch 11b — codesight repo_health (call graph, circular deps, dead code) | ~400 |
| **v2.0** | Batch 14 — BooCoder pending changes (new container, write tools, plandex pattern) | ~1200 |
| v2.1 | Batch 15 — BooCoder runtime isolation (per-session Docker sandbox, OpenHands pattern) | ~600 |
| v2.x | Batch 16/17 — Multi-provider LLM (optional, pi-ai) and Workflow graphs (far future, agent-framework concepts) | tbd |

-----

## Roadmap doc deviations and corrections

This roadmap was significantly out of sync with reality until 2026-05-20. Key corrections folded in:

1. **Batch 9 (Agents Tier 2) is done**, not "next up." Shipped as commit `92bd3b1`, included in v1.9.1 forward. The original "Track A: Batch 9 next" recommendation was correct but the doc never got updated.
2. **v1.6.2 merged.** No longer "in flight."
3. **Batch 5 (fork/delete), Batch 6 (drag-drop), Batch 7 (settings drawer), Batch 8 (web search), Batch 10 (BooTerm) all shipped**, scattered across the v1.6–v1.10 version line. Original "Track A polish then agents" plan was abandoned; work happened opportunistically.
4. **v1.11.0 was a major unplanned addition** — opencode-style compaction (auto-overflow detection + anchored rolling summary + tail preservation). This is NOT a batch from the old roadmap. It opened a new patch line (v1.11.x) of small follow-ups in front of the original Batches 11–17.
5. **Batch 11 (codecontext sidecar) moves to v1.12.** Bundles with truncation and repair-tool-call lift (both from opencode) since they share concerns and the `tool_choice='required'` confirmation makes repair-tool-call viable.
6. **Phase B (parts table + AI SDK + tool-call lifecycle) becomes v1.13.** This absorbs the old Batch 13 (append-only event log) — same outcome (typed message parts), different mental framing.
7. **Phase C and Phase D are new** (numbered v1.14/v1.15). They originate from the opencode integration analysis, not from the original 17-batch plan. Phase C delivers the outer agent loop with explicit step boundaries. Phase D delivers the permission ruleset + MCP client needed for codecontext to be useful and for BooCoder to gate writes.
8. **BooCoder (v2.0/v2.1)** is the second-major-version line. New container, new safety story (pending changes + per-session Docker sandbox). Maps to original Batches 14/15.

-----

## v1.11.x patches in detail

### v1.11.0 — opencode-style compaction port ✅

**What shipped:** Auto-detection of context overflow (`isOverflow(usage, model)`) triggers compaction on the *next* user turn. Compaction preserves the last 2 turns verbatim and produces an anchored Markdown summary (8-section template lifted verbatim from opencode `compaction.ts`) that replaces older head messages. Summary is rolling — each new compaction updates the prior summary, not stacks. Schema additions: `messages.compacted_at`, `messages.summary`, `messages.tail_start_id`, `chats.needs_compaction`. WS `compacted` frame fires sonner toast on completion.

**Key divergences from opencode:** Per-chat (not per-session) compaction state because BooCode history is per-chat. UUID `tail_start_id` not BIGINT. No `parent_id` on messages. Context limit comes from `messages.ctx_max` (last-known `n_ctx`), not a `model.context_limit` field.

### v1.11.1 — Compaction follow-up ✅

Working-state `chat_status: working/idle` frames around the LLM call inside `compaction.process()`. 24 new vitest cases for the six pure functions (`usable`, `isOverflow`, `estimate`, `turns`, `select`, `buildPrompt`). 7 `.bak-v1.11` files deleted.

### v1.11.2 — ContextBar ✅

New `ContextBar.tsx` rendering above MessageList. Shows `{used} / {max} ({pct}%)` with color tiers computed against `max - 20k` reserve (matches `compaction.usable()`): muted <60%, amber 60-80%, orange 80-95%, red ≥95%. Tooltip shows "Auto-compaction at ~N%". Mobile breakpoints: `< 380px` shows "Ctx" + numbers; `380-639px` adds parenthetical %; `≥ 640px` shows full "Context" label.

### v1.11.3 — ctx_max capture fix ✅

Discovered the dead code at `inference.ts:479-481` and `compaction.ts:300` reading `parsed.timings.n_ctx` never fired — llama-server emits `prompt_n / predicted_n / *_ms / *_per_second` in timings but NOT `n_ctx`. New `model-context.ts` module fetches `GET /upstream/<model>/props` with 3s timeout, positive cache (no TTL), 60s negative cache. Wired into all 4 ctx_max write sites (3 in inference.ts, 1 in compaction.ts). 12 new vitest cases. 7 historical rows backfilled to `ctx_max = 262144` (single-day backfill, only qwen3.6-35b-a3b-mxfp4 in use).

### v1.11.4 — CANCELLED

Original scope: per-turn budget reset + Continue affordance + CapHitSentinel card. Recon revealed all three are already shipped (v1.8.2 timestamps in inference.ts comments). Dead version slot.

### v1.11.5 — ContextBar relocate (DISPATCHED)

Relocate ContextBar from above MessageList to above the agent-picker row. Bump height from ~4px bar to ~10-12px. Always-visible (zero-state when no assistant messages + use `model_context_limit` from v1.11.3 cache). Remove `ChatContextPopover` entirely (redundant signal; mobile-hostile).

### v1.11.6 — Doom-loop guard (QUEUED)

Detect 3 identical tool calls in a row within one turn (same name + same args via JSON.stringify). On detection: abort tool-call recursion, insert `metadata.kind='doom_loop'` sentinel, trigger summary turn via existing `runCapHitSummary` path. New `DoomLoopSentinel.tsx` component (no Continue button — looping shouldn't be retried with same tools). Per-turn sliding window, scoped to current turn's tool-call accumulator.

**Lift source:** opencode `processor.ts`, `DOOM_LOOP_THRESHOLD = 3` constant.

### v1.11.7 — pathGuard secrets filter (QUEUED)

Extend pathGuard with `DEFAULT_SECURITY_IGNORE_FILETYPES` from continue.dev `core/indexing/ignore.ts`. Three-tier matcher: exact basenames (`credentials`, `secrets.yml`), extensions (`.env`, `.pem`, `.key`, `.crt`, etc.), prefix patterns (`id_rsa`, `id_dsa`, `id_ecdsa`, `id_ed25519`). Blocked files appear in `list_dir` and `find_files` results with `(blocked)` annotation. `view_file` returns `{ error: 'blocked_secret_file', ... }`. `grep` cannot read blocked file contents. No override mechanism in v1.x (use host shell).

**Why it matters:** `/opt:/opt:ro` mount currently exposes `boolab/.env`, `dubdrive/users.json`, `authelia/state`, every other service's secrets to any tool past path validation. Cheap close on that surface area.

-----

## v1.12 — codecontext sidecar + truncation + repair tool call

Three lifts fused because they share concerns:

1. **codecontext sidecar** — new container, single-instance, path-addressed multi-project. Mount `/opt/projects:/workspace:ro`. 8 tools wired as static `ToolDef` wrappers in `apps/server/src/services/tools/codecontext/` (one file per tool). HTTP client to `http://codecontext:8765`. New module `apps/server/src/services/codecontext_bridge.ts` translates `project_id` → `/workspace/<relative>/` paths.

2. **Tool output truncation** — opencode `truncate.ts` pattern. Cap at 2000 lines / 50KB. Larger outputs: write full content server-side, return preview + opaque `id`. New tool `view_truncated_output(id)` retrieves full content by server-mapped id. **No pathGuard exception** for `/tmp` directory — the opaque-id approach avoids exposing a writable filesystem location to the model. Only codecontext outputs need truncation; native tools (view_file 200 lines, grep 200 results, list_dir 500 entries, find_files 200 results) already cap reasonably.

3. **`experimental_repairToolCall` equivalent** — when model emits malformed tool call (JSON parse fails or Zod validation fails), return a synthetic tool result instead of an error: `{ error, raw_args, tool_name, hint: 'Retry with valid JSON arguments.' }`. Model self-corrects on next step. Add one line to system prompt instructing self-correction on malformed-args results. Confirmed working precondition: `tool_choice: "required"` accepted by llama-swap (verified 2026-05-20 against qwen3.6-35b-a3b-mxfp4).

**Hand-roll, not AI SDK adoption.** AI SDK migration deferred to v1.13.

**AGENTS.md updates:** Each of the 6 builtin agents gets a curated codecontext tool whitelist:
- Architect: all 8
- Debugger: `search_symbols`, `get_dependencies`
- Code Reviewer: `get_file_analysis`
- Refactorer: `get_semantic_neighborhoods`, `get_dependencies`
- Security Auditor: `get_file_analysis`, `search_symbols`, `get_dependencies`
- Prompt Builder: none (no structural reasoning relevance)

**Dependencies:** v1.11.x merged. No others.

**Estimated:** 600 LoC across 3-4 dispatches under the v1.12 umbrella.

-----

## v1.13 — Phase B: parts table + AI SDK + per-tool tagging

**Goal:** typed message parts replace JSON blobs on `messages.tool_calls` / `tool_results`. Adopt Vercel AI SDK `streamText`. Tag tools as `read_only` or `write` at definition time.

**Scope:**

1. Schema: new `message_parts` table (`id, message_id, kind, payload JSONB, sequence`). Kinds: `text`, `tool_call`, `tool_result`, `reasoning`, `step_start`. The `messages` table becomes header-only.
2. Inference loop rewritten on AI SDK `streamText`. `streamCompletion` becomes a thin wrapper. Native AI SDK `experimental_repairToolCall` replaces v1.12's hand-rolled version.
3. Tool registry: `ToolDef<T>` gains `category: 'read_only' | 'write'` field. BooCode v1.x rejects any `write` tool at registry time (defense in depth for the BooCoder split). Alpha-sort tool list before sending to model (prompt-cache stability).
4. Reasoning content (`reasoning_content` from Qwen3.6) captured as its own part type instead of dropped or inlined.

**Migration risk:** non-trivial. inference.ts is ~1400 lines with custom XML fallback, SSE parsing, compaction integration. Plan dedicated cutover window. Compaction.ts must update to assemble head from parts.

**Replaces:** Original Batch 13 (append-only event log) — same outcome, different vocabulary.

**Dependencies:** v1.12 merged.

-----

## v1.14 — Phase C: outer agent loop

**Goal:** explicit multi-step loop per opencode `prompt.ts` `runLoop()`. Replace the current ad-hoc tool-call recursion.

**Scope:**

1. Outer loop continues until model returns non-tool finish OR step cap hit. Step ≠ tool call: one step can contain multiple tool calls in parallel.
2. `agent.steps ?? Infinity` per-agent step cap. AGENTS.md gains `steps:` field. Refactorer `steps: 5`, Architect `steps: 20`, etc.
3. Step-boundary events (`step_start`, `step_finish`) explicit in the parts stream. Per-step snapshot for revert (planned for BooCoder; backend-only in v1.14).
4. Doom-loop guard (v1.11.6) migrates from "abort recursion" to "raise within loop iteration." Same predicate, different control flow.

**Dependencies:** v1.13 merged.

-----

## v1.15 — Phase D: permission ruleset + MCP client

**Goal:** wildcard permission ruleset (opencode `evaluate.ts` pattern) and a proper MCP client implementation. Foundation for BooCoder to gate writes; immediate value for codecontext to be re-wired as a real MCP server.

**Scope:**

1. Wildcard rule matcher: `{ permission, pattern, action: 'allow' | 'deny' | 'ask' }`. Last-match-wins. Per-agent rulesets layer under per-session rulesets.
2. MCP client implementation: SSE transport, `tools/list` discovery, `tools/call` invocation. codecontext sidecar gets re-pointed from static wrappers (v1.12) to real MCP. New connectors become a config-only addition.
3. UI: permission-ask flow when a tool requires `ask` action. Modal or inline card with Allow once / Allow always / Deny.
4. v1.x stays read-only by default (no `write` tools in the registry yet).

**Absorbs:** Original Batch 12 (tool approval + plan/act mode) — same outcome via permission rules instead of mode enum.

**Dependencies:** v1.13 merged (parts table for permission events). Independent of v1.14.

-----

## v1.16 — Batch 11b: codesight repo_health

Call graph, circular dependency detection, dead code flagging. Port `analyze.mjs` from spirituslab/codesight. New tool `repo_health(project_id)`. In-process Node (not sidecar). Cache results keyed by `(project_id, file_hashes_sig)`.

**Dependencies:** v1.12 merged (can reuse codecontext parse output where overlapping).

-----

## v2.0 — BooCoder pending changes

New container `boocoder` at `100.114.205.53:9502`. Owns write tools (`edit_file`, `create_file`, `delete_file`, `apply_pending`, `rewind`). Edits queue in `pending_changes` table; nothing touches disk until `/apply`. Per-pane diff UI with Approve/Reject. BooCode chat stays read-only (`/opt:/opt:ro`).

**Lift source:** plandex pending-changes data model.

**Dependencies:** v1.13 (parts) + v1.15 (permissions).

-----

## v2.1 — BooCoder runtime isolation

Per-session Docker sandbox spawned by BooCoder on first write. Only project path mounted, not `/opt`. Idle-timeout 30 min. Standard OpenHands runtime contract: HTTP API inside container, BooCoder calls in.

**Lift source:** OpenHands V1 runtime pattern.

**Dependencies:** v2.0.

-----

## v2.x — Optional / far future

- **Multi-provider LLM** (pi-ai pattern): Only if a concrete need for Anthropic / OpenAI / Mistral direct surfaces. llama-swap covers everything today.
- **Workflow graphs** (microsoft/agent-framework concepts): Multi-agent coordination. Conceptual reference only. Realistically a v3.x topic.

-----

## Architecture target state

### Containers

| Container | Port | Mount | Purpose | Status |
|---|---|---|---|---|
| `boocode` | `100.114.205.53:9500` | `/opt:/opt:ro` | Chat + read-only tools + SPA | Live |
| `boocode_db` | `127.0.0.1:5500` | `boocode_pgdata` volume | Postgres 16-alpine | Live |
| `booterm` | `100.114.205.53:9501` | `/opt/repos:/opt/repos:rw` | Terminals (tmux + node-pty) | Live (v1.10.0) |
| `codecontext` | `:8765` (internal) | `/opt/projects:/workspace:ro` | MCP server for architect tools | v1.12 |
| `boocoder` | `100.114.205.53:9502` | per-session sandbox | Write tools | v2.0 |

### Schema additions by version

- **v1.11.0:** `messages.compacted_at`, `messages.summary`, `messages.tail_start_id`, `chats.needs_compaction`
- **v1.11.7:** none (pathGuard logic, no DB)
- **v1.12:** none (codecontext is stateless on disk; truncation uses in-memory id→path map with TTL cleanup)
- **v1.13:** `message_parts` table; `messages` becomes header-only
- **v1.14:** `agents.steps` column (or AGENTS.md parser extension; no DB if file-only)
- **v1.15:** `permissions` table, `agent_permissions` join, `session_permissions` join
- **v1.16:** `repo_health_cache (project_id, file_hashes_sig, payload JSONB, created_at)`
- **v2.0:** `pending_changes (id, session_id, file_path, diff TEXT, status, created_at)`

-----

## Lift sources (summary)

Full inventory in `boocode_code_review.md`. Headline items:

| Source | Used for | Where |
|---|---|---|
| **`sst/opencode`** (MIT, TS) | **Compaction algorithms** | **v1.11.0 (shipped)** |
| `sst/opencode` (MIT, TS) | Doom-loop guard | v1.11.6 |
| `sst/opencode` (MIT, TS) | `repairToolCall`, truncate.ts, MCP client, permission evaluate, runLoop | v1.12/v1.13/v1.14/v1.15 |
| `continuedev/continue` (Apache-2.0) | `DEFAULT_SECURITY_IGNORE_FILETYPES` | v1.11.7 |
| `nmakod/codecontext` (MIT, Go) | Architect: codebase map sidecar | v1.12 |
| `spirituslab/codesight` (MIT-ish, TS) | Architect: repo health analyzer | v1.16 |
| `Aider-AI/aider` (Apache-2.0) | Fallback `.scm` grammars | v1.12 (fallback) |
| `cline/cline` (Apache-2.0) | Plan/Act pattern (absorbed into v1.15 permissions) | v1.15 |
| `plandex-ai/plandex` (MIT) | Pending-changes data model | v2.0 |
| `OpenHands/OpenHands` (MIT) | Sandbox runtime contract | v2.1 |
| `aimasteracc/tree-sitter-analyzer` (MIT) | Outline-first patterns | v1.12 (alt) |
| `earendil-works/pi` (MIT) | Multi-provider LLM | v2.x (optional) |

**Original Batch 13 (event log from OpenHands) replaced** by v1.13 (parts table). Same outcome, different framing.

-----

## Decisions log

- **Embeddings dropped from BooCode** (May 2026). Replaced RAG with file-view tools + sidecar analyzers.
- **Original Batch 11 (aider PageRank port) replaced** by codecontext sidecar approach.
- **Original Batch 12 (codebase indexer w/ Harrier) removed.** No embedding infrastructure in BooCode v1.x.
- **Globstar parked** — not an architect tool. Future verify-before-commit candidate only.
- **codeprysm rejected** — embedding-based. Node/edge taxonomy noted as reference if we ever build our own graph.
- **Batch 9 decoupled from Batch 7 (2026-05-16); shipped in `92bd3b1`.** Builtin defaults: six agents (Code Reviewer, Debugger, Refactorer, Architect, Security Auditor, Prompt Builder) with no `model` field. Session model wins by default.
- **opencode lift opened** (2026-05-20). Started with compaction (v1.11.0). Continuing through v1.15. Five distinct algorithms: compaction, doom-loop guard, repairToolCall, runLoop, permission evaluate. Plus `truncate.ts` and `MCP client`. Each lifts the algorithm, not the Effect-TS plumbing.
- **AI SDK adoption deferred to v1.13.** Hand-roll repairToolCall in v1.12 first. Migrate everything together when parts table lands.
- **`tool_choice='required'` confirmed supported** by llama-swap (qwen3.6-35b-a3b-mxfp4, 2026-05-20). Unblocks repair tool call viability.
- **v1.11.4 cancelled** (2026-05-20). Per-turn budget reset + Continue affordance + CapHitSentinel were already shipped in v1.8.2. Roadmap was 14 versions stale at time of recon.

-----

## Workflow

Each batch:

1. Verify previous batch merged. `git log --oneline main -5`.
2. Cut branch from main. Single-branch-per-dispatch convention.
3. Dispatch via Paseo to Claude Code at `/opt/boocode`.
4. Claude Code recon → blocking questions → implement → hand back.
5. Compliance review in separate Claude chat (paste handback).
6. Build: `docker compose build --no-cache boocode` (no-cache avoids the v1.11.2 stale-bundle trap).
7. Restart: `docker compose up -d boocode`.
8. Smoke test in browser (hard refresh).
9. Sam commits and pushes. **Never** `git pull` / `git push` / `git commit` on his behalf.

Sam reviews all diffs.