indifferentketchup/boocode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v2.1.0-provider-picker: BooCoder systemd migration + provider picker

Browse Source 
- BooCoder moves from Docker to host systemd service (boocoder.service)
- Agent dispatch (ACP + PTY) switches from SSH to direct spawn/exec
- SSH helpers marked @deprecated (kept for one release cycle)
- Provider registry (5 providers: boocode, opencode, goose, claude, qwen)
- Agent probe with direct which/exec + model discovery (qwen settings, static claude models)
- GET /api/providers route with installed status, models, transport fallback
- ProviderPicker frontend component in CoderPane header
- External provider messages route through tasks row instead of inference enqueue
- Smart scroll: MessageList only auto-scrolls when near bottom (150px threshold)
- DB: available_agents gets models, label, transport columns
- Bug fix: loadContext SELECT includes allowed_read_paths
- Bug fix: cap hit sentinel inserted before buildMessagesPayload
- docker-compose.yml: boocoder service commented out, BOOCODER_URL env var added
- CLAUDE.md: updated docs for systemd, provider registry, JSONB gotcha, loadContext
This commit is contained in:
indifferentketchup
2026-05-25 19:20:53 +00:00
parent e423579e99
commit d8ffee1950
21 changed files with 687 additions and 222 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
CLAUDE.md
View File

@@ -66,16 +66,24 @@ Key services:
- **`messages_with_parts` view** (v1.13.1-B; `schema.sql`). Read sites that need `tool_calls` / `tool_results` / `reasoning_parts` SELECT from this view, NOT `messages` directly. v1.13.20 dropped the legacy `messages.tool_calls` / `messages.tool_results` JSON columns; the view now reads parts-only subselects. Writes target `message_parts` exclusively via `insertParts` (or via the helpers `partsFromAssistantMessage` / `partsFromToolMessage`). The `Message` wire type still carries `tool_calls?` / `tool_results?` because the view synthesizes them from parts — frontend reads are unchanged. Shapes: `tool_calls jsonb[]`, `tool_results jsonb` single object, `reasoning_parts jsonb[]` of `{text}`. If you ever need to UPDATE a message and return its full Message shape, do a two-step UPDATE returning `id` followed by SELECT from the view — RETURNING off the bare `messages` table no longer carries the tool fields.
- **`services/file_ops.ts`** — Shared file operation implementations used by both inference tools and HTTP routes.
- **`services/auto_name.ts`** — Non-streaming LLM call to generate 4-word session titles after first assistant reply.
- **`services/provider-registry.ts`** — Static registry of provider metadata (label, transport, model source). `PROVIDERS` array, `PROVIDERS_BY_NAME` map. 5 providers: boocode (native), opencode (acp), goose (pty), claude (pty), qwen (pty).
- **`services/agent-probe.ts`** — Startup probe using direct `exec()` (not SSH). Discovers installed agents on host, their versions, ACP support, and models. Qwen models read from `~/.qwen/settings.json`. Claude models are static from the registry. Results persisted to `available_agents` table.
- **`routes/providers.ts`** — `GET /api/providers` returns installed providers with models. Transport field reflects actual capability (checks `supports_acp` from DB, not just registry preference).
- **Provider picker dispatch**: when `provider !== 'boocode'`, the message route creates a `tasks` row (with `session_id` set) instead of calling `inference.enqueue`. The dispatcher picks it up and dispatches via ACP or PTY using the agent's `install_path`.
Route registration: all routes registered in `index.ts` via `register*Routes(app, sql, ...)` functions. Routes are in `routes/*.ts`.
### BooCoder (`apps/coder/src/`)
- Write-capable coding agent. Separate Fastify server at port 9502, same docker network (`boocode_net`).
- **Workspace dependency on `@boocode/server`**: imports `createInferenceRunner`, `createBroker`, `ALL_TOOLS`, `appendMcpTools` from the server's compiled `dist/`. apps/server's `package.json` has an `exports` map with `types` conditions for NodeNext resolution. apps/server must build FIRST (Dockerfile builds server → coder).
- Write-capable coding agent. Runs as a **systemd service on the host** (`boocoder.service`), NOT in Docker. Fastify server at port 9502, connects to postgres at `127.0.0.1:5500`.
- **Workspace dependency on `@boocode/server`**: imports `createInferenceRunner`, `createBroker`, `ALL_TOOLS`, `appendMcpTools` from the server's compiled `dist/`. apps/server's `package.json` has an `exports` map with `types` conditions for NodeNext resolution. apps/server must build FIRST.
- Build + deploy: `pnpm -C apps/server build && pnpm -C apps/coder build && sudo systemctl restart boocoder`. Env file at `apps/coder/.env.host`. Service file at `/etc/systemd/system/boocoder.service`.
- Agent dispatch spawns binaries directly using `install_path` from `available_agents` — no `spawn('sh', ['-c', ...])` (fails under systemd). Follows Paseo's pattern: `spawn(fullBinaryPath, argsArray, { cwd })`.
- systemd hardening: only `NoNewPrivileges=true` is safe. `ProtectSystem`, `ProtectHome`, `PrivateTmp` all break agent dispatch (agents need full filesystem access to read configs, write to worktrees).
- `apps/server/tsconfig.json` has `declaration: true` so `.d.ts` files exist for workspace consumers.
- Write tools (`edit_file`, `create_file`, `delete_file`, `apply_pending`, `rewind`) queue in `pending_changes` table. Nothing hits disk until `apply_pending` is called. `write_guard.ts` validates paths (resolve + prefix-check, no realpath since files may not exist for creates).
- Frontend: NOT a separate SPA. BooCoder is a `'coder'` pane type within BooChat's SPA (`apps/web/`). `CoderPane.tsx` in `apps/web/src/components/panes/`. API requests go through `/api/coder/*` proxy (Vite dev + Fastify production) which rewrites to `http://boocoder:3000/api/*`. WS connects directly to `:9502`.
- Frontend: NOT a separate SPA. BooCoder is a `'coder'` pane type within BooChat's SPA (`apps/web/`). `CoderPane.tsx` in `apps/web/src/components/panes/`. API requests go through `/api/coder/*` proxy (Vite dev + Fastify production) which rewrites to the boocoder host service (`BOOCODER_URL` env var, default `http://100.114.205.53:9502`). WS connects directly to `:9502`.
- `apps/coder/web/` is a STANDALONE fallback SPA served at `:9502` directly. The PRIMARY BooCoder frontend is the `CoderPane` in BooChat's SPA (`apps/web/src/components/panes/CoderPane.tsx`), accessible via the "Coder" pane in the workspace at `code.indifferentketchup.com`. Both exist; the pane is what Sam uses.
### Frontend (`apps/web/src/`)
@@ -122,7 +130,11 @@ Schema CHECK migration order when renaming allowed values: (1) `ALTER TABLE ...
Required: `DATABASE_URL`, `LLAMA_SWAP_URL`. Optional: `PORT` (3000), `HOST` (0.0.0.0), `PROJECT_ROOT_WHITELIST` (/opt, read-only scope for add-existing path resolution), `BOOTSTRAP_ROOT` (/opt/projects, writable scope for create-new-project bootstrap mkdir target — host must `mkdir -p /opt/projects` before container start), `DEFAULT_MODEL`, `LOG_LEVEL`, `SEARXNG_URL` (default `http://100.114.205.53:8888` — internal Tailscale Fathom; the public `search.indifferentketchup.com` is behind Authelia and unusable from server context), `BOOCODE_TOOLS` (`core` | `standard` | `all`, default `all`; v1.13.15-tools tier filter — ceiling, never expands an agent's whitelist), `MCP_CONFIG_PATH` (optional; default `/data/mcp.json` — JSON config for MCP servers matching opencode's `mcpServers` shape; file missing = no MCP).
BooCoder at port 9502: `curl http://100.114.205.53:9502/api/health`. Same Tailscale IP binding as BooChat. Health reports tool count: `{"ok":true,"db":true,"tools":30}`.
BooCoder at port 9502: `curl http://100.114.205.53:9502/api/health`. Runs as `boocoder.service` on the host (not Docker). Deploy: `pnpm -C apps/server build && pnpm -C apps/coder build && sudo systemctl restart boocoder`. Health reports tool count: `{"ok":true,"db":true,"tools":33}`.
- `FAST_MODEL` (optional) — cheaper model for titles, summaries, labeling (auto_name.ts, tool-summaries.ts). Falls back to session model or DEFAULT_MODEL when unset. Set to a small model on llama-swap (e.g. `nemotron-nano-4b`) to avoid loading the 35B for 20-token calls.
- Qwen Code dispatch: `OPENAI_BASE_URL=http://100.101.41.16:8401/v1 OPENAI_API_KEY=dummy qwen -p "<task>" --output-format stream-json`. Install: `npm install -g @qwen-code/qwen-code@latest`. Node ≥22 required on host (container stays Node 20; dispatch via SSH). No `--yolo` flag — non-interactive mode (`-p`) runs autonomously without approval prompts. ACP bridge is HTTP daemon (not stdio); use PTY dispatch.
- Arena (v2.0.5): `POST /api/arena {project_id, input, contestants: [{agent?, model?}]}` dispatches the same task to N models/agents in parallel. Each contestant gets its own task + worktree. `GET /api/arena/:id` for results. `POST /api/arena/:id/select/:task_id` picks winner.
## Workflow
@@ -133,7 +145,7 @@ BooCoder at port 9502: `curl http://100.114.205.53:9502/api/health`. Same Tailsc
- Deploy: `cd /opt/boocode && docker compose up --build -d` (or `docker compose build --no-cache boocode && docker compose up -d` if you suspect a layer-cache issue).
- Git push to Gitea: `GIT_SSH_COMMAND="ssh -i /opt/boocode/secrets/boocode_gitea -o IdentitiesOnly=yes" git push origin <branch>`. The default agent identity is rejected; the in-repo deploy key (`secrets/`, gitignored) is the working one. Transient `Connection reset by peer` retries cleanly after `sleep 5`.
- Don't accumulate `.bak-*` files. Clean them up in the same batch or immediately after merge.
- DB-integration tests opt-in via env var: `DATABASE_URL='postgres://boocode:devpass@localhost:5500/boocode' pnpm -C apps/server test`. Host port is 5500 (mapped from `boocode_db:5432`); password is `${POSTGRES_PASSWORD}` from `.env` (`devpass`), NOT the literal in `.env`'s `DATABASE_URL=postgres://boocode:Ketchup1479@boocode_db:5432/...` line. Pattern: `describe.runIf(!!process.env.DATABASE_URL)(...)` with a `beforeAll` that applies the schema via `sql.unsafe(readFileSync(schemaPath))`. Tests skip cleanly when var is unset. `tool_cost_stats.test.ts` is the reference.
- DB-integration tests opt-in via env var: `DATABASE_URL='postgres://boocode:devpass@localhost:5500/boochat' pnpm -C apps/server test`. Host port is 5500 (mapped from `boocode_db:5432`); password is `${POSTGRES_PASSWORD}` from `.env` (`devpass`), NOT the literal in `.env`'s `DATABASE_URL=postgres://boocode:Ketchup1479@boocode_db:5432/...` line. Pattern: `describe.runIf(!!process.env.DATABASE_URL)(...)` with a `beforeAll` that applies the schema via `sql.unsafe(readFileSync(schemaPath))`. Tests skip cleanly when var is unset. `tool_cost_stats.test.ts` is the reference.
- Host-side smoke endpoint: `curl http://100.114.205.53:9500/api/...`. The boocode container's port mapping binds to the Tailscale IP, not `0.0.0.0`, so `localhost:9500` doesn't work from the host shell. Same for booterm at `:9501`.
- Fastify global JSON parser tolerates empty bodies (overridden in `index.ts`); bodyless POSTs (archive, unarchive, stop) work without setting `Content-Type` tricks on the client.
- Event dedup discipline: for any mutation the server publishes via `broker.publishUser`, do NOT add a local `sessionEvents.emit(...)` after the API call — `useUserEvents` forwards the WS frame onto the bus. Frontend mutation handlers must be idempotent (dedup by id, no-op on already-present).
@@ -169,4 +181,5 @@ BooCoder at port 9502: `curl http://100.114.205.53:9502/api/health`. Same Tailsc
- Agent registry lives at `data/AGENTS.md` (global, bind-mounted at `/data/AGENTS.md`). No per-project `AGENTS.md` in this repo — removed in v1.12 to eliminate the two-files-must-stay-in-sync drift. The `getAgentsForProject` per-project override mechanism remains for *other* projects.
- MCP stdio transport uses newline-delimited JSON (NDJSON), NOT LSP-style `Content-Length` headers. The `codecontext/shim.go` framing implementation is the reference; per the MCP spec (modelcontextprotocol.io/specification/server/transports).
- **Workspace dependency pattern** (`apps/coder` → `@boocode/server`): the consuming package adds `"@boocode/server": "workspace:*"` in `package.json`. The provider's `package.json` needs `exports` with `types` + `default` conditions per subpath: `"./inference": { "types": "./dist/.../index.d.ts", "default": "./dist/.../index.js" }`. Without the `types` condition, NodeNext resolution can't find `.d.ts` files and tsc fails with "Cannot find module" in the consumer.
- **Docker build order for workspace deps**: the Dockerfile must `COPY` + `RUN pnpm build` the provider app BEFORE the consumer app. `apps/coder/Dockerfile` builds `apps/server` first, then `apps/coder`.
- **JSONB columns**: use `sql.json(value as never)` — NOT `${JSON.stringify(value)}::jsonb` which double-serializes (stores a JSON string instead of a JSON object/array). Pattern established in `parts.ts`, `settings.ts`.
- **`payload.ts:loadContext` SELECT**: must include every `Session` field that downstream code reads. The tool phase reads `session.allowed_read_paths`; if the SELECT omits it, cross-repo read grants silently fail. The `Session` TypeScript type doesn't catch this because `sql<Session[]>` doesn't enforce column coverage.