Removes the dual-write into messages.tool_calls / messages.tool_results JSON columns and drops the columns. message_parts is now the only source of truth for tool calls and tool results. 10 dual-write sites stripped (5 in tool-phase.ts, 2 in routes/skills.ts, 2 in routes/messages.ts, 1 in routes/chats.ts fork-clone). The recon-driven grep caught 2 sites beyond the original v1.13.2 roadmap inventory and an extra fixture file (tool_cost_stats.test.ts) with a direct legacy-column INSERT. messages_with_parts view rewritten to parts-only subselects (COALESCE fallbacks gone). View runs via CREATE OR REPLACE so it lands before the column DROPs in startup DDL — Postgres rejects column-drop on view-referenced cols. v1.12.1 cleanup DO block (DROP CONSTRAINT messages_status_check / messages_role_check) removed; those one-shots have done their work. Adversarial review caught a runtime bug the green test suite missed: the discard_stale endpoint (chats.ts) had a RETURNING ... tool_calls, tool_results clause that would have crashed on every 60s-no-token-activity recovery in production. Fixed by switching to two-step UPDATE returning id, then SELECT from messages_with_parts so parts-synthesized fields keep flowing on the wire. Message API type retains tool_calls? / tool_results? — the view synthesizes those keys from parts so the wire shape is unchanged; frontend reads need no update. Override on the original v1.13.2 plan, captured in the openspec proposal. 339/339 server tests passing (including 7 DB-integration tests that applied the schema migration to a live DB and ran the parts-only view end-to-end). tsc + web build clean. Pairs with v1.13.0-ai-sdk-v6 (introduced the dual-write) and v1.13.1-B (moved the read path to messages_with_parts). Umbrella v1.13 tag ships on this same commit, marking the strangler-fig closed. CLAUDE.md picks up Sam's pre-existing edits documenting tag-naming and CHANGELOG conventions — both already in use by v1.13.19 / v1.13.20. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
127 lines
7.8 KiB
Markdown
127 lines
7.8 KiB
Markdown
# v1.13.20-drop-legacy-cols — drop messages.tool_calls + messages.tool_results
|
|
|
|
Final phase of the v1.13.0 strangler-fig migration. Removes the dual-write into `messages.tool_calls` / `messages.tool_results` JSON columns and drops the columns themselves. After this batch, `message_parts` is the only source of truth for tool-call and tool-result data.
|
|
|
|
Tag `v1.13` (umbrella) ships on the same commit per the original roadmap entry.
|
|
|
|
## Why
|
|
|
|
v1.13.0 (AI SDK v6 migration) introduced `message_parts` as the new canonical store for tool calls, tool results, reasoning, text, synthesis, and now html_artifact. To stay safe during the migration, every write site also dual-wrote to the legacy `messages.tool_calls` / `messages.tool_results` JSON columns, and `messages_with_parts` view COALESCEs over both. Reads have been migrated; dual-writes are pure overhead at this point.
|
|
|
|
Verification query (per the original v1.13.2 plan) returns `0 / 0` orphan rows. Today's DB is also empty (0 messages on the live instance), so the COUNT query alone is weakly informative — the safety check shifts to a code-level audit: every dual-write site listed in the v1.13.2 roadmap entry must be located and its parts-write half kept, JSON-column half removed.
|
|
|
|
## Scope
|
|
|
|
### S1. Remove dual-write from every site
|
|
|
|
Per the v1.13.2 roadmap entry, dual-writes live at:
|
|
|
|
- `services/inference/tool-phase.ts` — 3 sites
|
|
- `services/inference/error-handler.ts` — `finalizeCompletion`
|
|
- `routes/skills.ts` — 2 sites
|
|
- `routes/messages.ts` — answer flow
|
|
- `routes/chats.ts` — fork flow
|
|
|
|
Implementer must grep for every UPDATE / INSERT that touches `tool_calls` or `tool_results` columns and verify it has a paired `insertParts(...)` call. Keep the parts write, remove the column write. If a site only writes to the JSON column with no parts pair — STOP and escalate (would indicate a bug in the v1.13.0 dual-write rollout we haven't caught).
|
|
|
|
### S2. Simplify `messages_with_parts` view
|
|
|
|
Current view COALESCEs parts-table rows over legacy JSON columns to support pre-v1.13.0 history. After this batch, the JSON columns no longer exist — drop the COALESCE fallbacks. The view should read only from `message_parts` joined to `messages`.
|
|
|
|
### S3. Drop the columns
|
|
|
|
```sql
|
|
ALTER TABLE messages DROP COLUMN tool_calls;
|
|
ALTER TABLE messages DROP COLUMN tool_results;
|
|
```
|
|
|
|
Idempotent via `IF EXISTS`. Apply unconditionally on startup (matches the rest of `schema.sql`'s shape).
|
|
|
|
### S4. Remove from API types
|
|
|
|
`Message` interface in `apps/server/src/types/api.ts` AND `apps/web/src/api/types.ts` — drop `tool_calls?` and `tool_results?` fields. The API boundary is unchanged because every consumer already reads parts-derived values through `messages_with_parts`. Mirror byte-for-byte.
|
|
|
|
### S5. Drop the stale `messages_status_check` cleanup DO block from v1.12.1 if still present
|
|
|
|
Per the v1.13.2 roadmap entry, there's a v1.12.1 `DO $$ DROP CONSTRAINT messages_status_check` block that was meant to clean up the old anonymous constraint. If still present in `schema.sql`, remove — it's been one-shot effective.
|
|
|
|
### S6. Update test fixtures
|
|
|
|
`inference.test.ts` and `compaction.test.ts` (and any other test file the grep finds) construct Message-shaped fixtures with `tool_calls: null, tool_results: null` literals. Rewrite ~30 fixtures to construct via `message_parts` rows where the test actually exercises tool calls. For tests that don't exercise tool calls at all, just drop the now-absent fields.
|
|
|
|
`partsFromAssistantMessage` and `partsFromToolMessage` helpers in `parts.ts` currently take `tool_calls` and `tool_results` as args (because that's what the legacy Message shape carried). Keep their input shapes — they're useful constructors. The change is at the call sites, not the helpers.
|
|
|
|
## Non-goals
|
|
|
|
- **No changes to `message_parts` schema.** It's correct as-is.
|
|
- **No changes to the `messages_with_parts` view name or interface.** Just the implementation simplifies.
|
|
- **No removal of `partsFromAssistantMessage` / `partsFromToolMessage`.** They're useful as constructors; their job becomes producing parts from raw ToolCall/ToolResult objects, not from a legacy Message row.
|
|
- **No frontend changes beyond the type mirror.** Web reads parts via `messages_with_parts` already.
|
|
- **No reads from the legacy columns in any code path.** Verify with grep.
|
|
|
|
## Hard rules
|
|
|
|
- No git commits during dispatch. Sam commits manually (handled by controller after all dispatches done).
|
|
- Backups: every modified file → `.bak-v1.13.20-20260523`.
|
|
- TS strict, no `any`.
|
|
- No new deps.
|
|
- Schema migration: additive-or-destructive but idempotent (`IF EXISTS` on the column drops).
|
|
- Run the full server test suite after — must be green.
|
|
- Frontend: `tsc -p apps/web/tsconfig.app.json --noEmit` + `pnpm -C apps/web build` clean.
|
|
|
|
## Stop checkpoints
|
|
|
|
1. **After recon** (grep-driven inventory of dual-write call sites + read sites still touching the legacy columns): stop, hand back inventory. The roadmap listed 7+ sites; verify nothing's been missed.
|
|
2. **After code edits, before schema migration**: stop, hand back diff + test results. Confirm the parts write at every former dual-write site still happens.
|
|
3. **After schema migration applies in dev**: stop, run tests, run a fresh `applySchema()` cycle (boot twice), confirm idempotent.
|
|
|
|
## Smoke plan
|
|
|
|
1. **Fresh boot.** Restart the boocode container, confirm `applySchema()` completes without error.
|
|
2. **Idempotent boot.** Restart again, confirm no error on the second pass (column DROP IF EXISTS is a no-op).
|
|
3. **Send a chat that triggers a tool call.** Confirm:
|
|
- Assistant message lands with content + reasoning + tool_call parts (all in `message_parts`).
|
|
- Tool result lands as a `tool_result` part.
|
|
- `messages_with_parts` returns the same shape the frontend expects (verify by reading the live chat in the UI).
|
|
4. **DB inspection.** `\d messages` — confirm `tool_calls` and `tool_results` columns are gone.
|
|
5. **Compaction roundtrip.** Trigger a compaction-eligible turn (long context); confirm the rolling summary still anchors correctly and uses parts as input.
|
|
|
|
## Done when
|
|
|
|
- All dual-write sites converted to parts-only writes.
|
|
- View simplified, columns dropped, types updated.
|
|
- Test suite green.
|
|
- Frontend typecheck + build clean.
|
|
- Smoke green.
|
|
- Tagged `v1.13.20-drop-legacy-cols` AND the umbrella `v1.13` on the same commit.
|
|
- CHANGELOG.md entry + roadmap retrospective bullet.
|
|
|
|
## Files expected to touch
|
|
|
|
**Backend:**
|
|
- `apps/server/src/schema.sql` — DROP columns + simplify view + remove v1.12.1 cleanup block
|
|
- `apps/server/src/services/inference/tool-phase.ts` — remove 3 dual-write sites
|
|
- `apps/server/src/services/inference/error-handler.ts` — remove dual-write in `finalizeCompletion`
|
|
- `apps/server/src/routes/skills.ts` — remove 2 dual-write sites
|
|
- `apps/server/src/routes/messages.ts` — remove dual-write in answer flow
|
|
- `apps/server/src/routes/chats.ts` — remove dual-write in fork
|
|
- `apps/server/src/types/api.ts` — drop `tool_calls?` / `tool_results?` from Message
|
|
- `apps/server/src/services/__tests__/inference.test.ts` — fixture rewrites
|
|
- `apps/server/src/services/__tests__/compaction.test.ts` — fixture rewrites
|
|
- `apps/server/src/services/__tests__/parts.test.ts` — likely some fixture updates
|
|
- `apps/server/src/services/__tests__/tool_cost_stats.test.ts` — likely some fixture updates
|
|
- `apps/server/src/services/__tests__/system-prompt.test.ts` — likely some fixture updates
|
|
|
|
**Frontend:**
|
|
- `apps/web/src/api/types.ts` — mirror Message change
|
|
|
|
**Docs:**
|
|
- `BOOCHAT.md` — no change expected (rules don't mention the legacy columns)
|
|
- `boocode_roadmap.md` — retrospective bullet
|
|
- `CHANGELOG.md` — new section
|
|
- `CLAUDE.md` — drop the v1.13.0 dual-write notes that no longer apply (audit the surrounding paragraphs)
|
|
|
|
## Estimate
|
|
|
|
~150 LoC net (mostly deletions). Mechanical work — same per-batch shape as v1.13.18.
|