indifferentketchup
9ce638c916
Surfaces per-tool prompt/completion-token rolling averages in
AgentPicker for at-a-glance agent-cost hints. Implementation is a
SQL view on top of messages_with_parts plus a read endpoint and
AgentPicker tooltip extension. No new write site; all source data
already lands via the existing tool-phase.ts:94-95 / error-handler.ts:
109-110 / sentinel-summaries.ts UPDATEs that v1.13.7's includeUsage:
true fix made non-NULL.
(1) schema.sql — new tool_cost_stats view. Window-functions over
messages_with_parts.tool_calls with LATERAL jsonb_array_elements.
Attribution: equal split — multi-tool turn divides tokens N-ways;
the 100-call rolling mean absorbs split noise. Filters: status=
'complete' + metadata.kind NOT IN ('cap_hit','doom_loop') exclude
failed turns and sentinels respectively; tool_calls IS NOT NULL is
defense-in-depth since sentinels are role='system' rows. CREATE OR
REPLACE means schema apply is idempotent.
(2) routes/tools.ts NEW + index.ts wire-in. GET /api/tools/cost_stats
returns { stats: ToolCostStat[] } with mean_prompt_tokens / mean_
completion_tokens computed at read time (sum / n_calls). Sorted by
tool_name ASC. No pagination — ≤30 tools.
(3) __tests__/tool_cost_stats.test.ts NEW — 7 integration tests
keyed off DATABASE_URL env var. Tests skip gracefully when unset
(no-DB default). beforeAll applies the schema via sql.unsafe(read
FileSync(schema.sql)) for self-contained runs. Helper insertAssistant
Turn shared across cases. Covers: empty state, single-tool attribution,
multi-tool equal split, 100-call FIFO window, NULL-tokens exclusion,
parts-authoritative read via messages_with_parts, failed/sentinel
exclusion.
(4) web/api/types.ts + client.ts — ToolCostStat interface + api.tools.
costStats() method binding.
(5) AgentPicker.tsx — fetch costStats on mount, compute per-agent
sum-of-means across whitelisted tools, render muted cost line below
description: "~5.2k prompt / 280 completion · 6/8 tools · last call
3h ago". Skips line entirely when no tool history; preserves existing
native title= for layout backward-compat. formatK/formatAgo colocated.
Tests: 202/202 pass (195 prior + 7 new view-integration). Server +
web tsc clean.
Smoke: schema applied cleanly; GET /api/tools/cost_stats returns
canonical JSON; view + endpoint agree. Single-row result expected
given the v1.13.1-A → v1.13.7 NULL latent regression window; new
traffic populates organically.
Roadmap row at boocode_roadmap.md:114 plus schema row at :474 both
match. View vs table decision documented in handoff_v1.13.10_per_
tool_cost.md (rollback-safe, microsecond-fast at BooCode scale).
~270 LoC across 8 files (5 modified + 3 new).
41 lines
1.3 KiB
TypeScript
41 lines
1.3 KiB
TypeScript
import type { FastifyInstance } from 'fastify';
|
|
import type { Sql } from '../db.js';
|
|
|
|
export interface ToolCostStat {
|
|
tool_name: string;
|
|
mean_prompt_tokens: number;
|
|
mean_completion_tokens: number;
|
|
n_calls: number;
|
|
updated_at: string;
|
|
}
|
|
|
|
// v1.13.10: per-tool token cost rolling window read endpoint. Backed by the
|
|
// tool_cost_stats view in schema.sql (last 100 calls per tool, equal-split
|
|
// attribution across multi-tool turns, sentinel/failed-turn excluded).
|
|
// Consumed by AgentPicker for at-a-glance per-agent cost hints.
|
|
export function registerToolsRoutes(app: FastifyInstance, sql: Sql): void {
|
|
app.get('/api/tools/cost_stats', async () => {
|
|
const rows = await sql<
|
|
{
|
|
tool_name: string;
|
|
prompt_tokens_sum: number;
|
|
completion_tokens_sum: number;
|
|
n_calls: number;
|
|
updated_at: string;
|
|
}[]
|
|
>`
|
|
SELECT tool_name, prompt_tokens_sum, completion_tokens_sum, n_calls, updated_at
|
|
FROM tool_cost_stats
|
|
ORDER BY tool_name ASC
|
|
`;
|
|
const stats: ToolCostStat[] = rows.map((r) => ({
|
|
tool_name: r.tool_name,
|
|
mean_prompt_tokens: Math.round(r.prompt_tokens_sum / r.n_calls),
|
|
mean_completion_tokens: Math.round(r.completion_tokens_sum / r.n_calls),
|
|
n_calls: r.n_calls,
|
|
updated_at: r.updated_at,
|
|
}));
|
|
return { stats };
|
|
});
|
|
}
|