v1.1 batch 1: markdown, message actions, tok/s+ctx, AI naming

Four features land together on this branch: 1. Markdown rendering — assistant messages go through react-markdown + remark-gfm. Fenced code blocks render via existing CodeBlock (with copy button); inline `code` is styled inline. User messages stay plain text. No raw HTML (no rehype-raw). 2. Per-message Copy + Regenerate. New endpoint POST /api/sessions/:id/messages/:message_id/regenerate validates the target (404/400/409), atomically deletes the target plus any later messages in the session, inserts a fresh streaming assistant row, and enqueues a normal inference run. The DELETE bound uses a SQL subquery (`created_at >= (SELECT created_at FROM messages WHERE id = $1)`) instead of a JS round-trip so postgres TIMESTAMPTZ µs precision is preserved — otherwise sub-ms clock_timestamp() differences between the user row and the assistant row collapsed to the same JS Date, pulling the triggering user message into the >= bound. New `messages_deleted` WS frame so already-connected clients prune the stale tail without needing a full snapshot resend. 3. tok/s + ctx counter. Five new nullable message columns: tokens_used, ctx_used, ctx_max, started_at, finished_at. started_at is set right before the OpenAI call in services/inference.ts (not in the route, not in the frame handler); finished_at + tokens_used + ctx_used + ctx_max are committed in the same UPDATE that flips status to 'complete'. The inference request now opts into stream_options.include_usage so the final chunk carries usage; defensive parsing also picks up timings.n_ctx when llama.cpp emits it (currently absent for our llama-swap models, so ctx_max stays NULL and the UI just shows `<used> ctx`). message_complete frame extended with tokens_used / ctx_used / ctx_max / started_at / finished_at / model. Frontend StatsLine in MessageBubble computes tok/s client-side from the timestamps and renders muted mono text below the body of completed assistant messages. 4. AI chat naming after the first turn. Backend services/auto_name.ts runs via setImmediate after the top-level inference resolves; it checks that there is exactly one completed assistant message and that the session has not been user-renamed (`name IS NULL OR name = '' OR name = 'New session'`), then fires a single non-streaming chat completion with the spec prompt. Qwen3 chat templates emit chain-of- thought into reasoning_content and burn the entire max_tokens budget without producing visible output, so the request includes `chat_template_kwargs: { enable_thinking: false }` and max_tokens=30. Title is trimmed, quote-stripped, "Title:" prefix dropped, and truncated to 60 chars before a guarded UPDATE on sessions.name. New `session_renamed` WS frame propagates to the open session view directly and to the project's session list via a tiny module-scope event bus (apps/web/src/hooks/sessionEvents.ts) — kept dumb: one event type, two methods, no library. Cleanups: dropped the now-unused splitCodeBlocks export from CodeBlock.tsx (react-markdown supersedes it), and added a long-form NOTE in auto_name.ts documenting the enable_thinking + max_tokens pattern for any future Qwen- family non-streaming utility calls (planned: fork-message, agent-routing, web-search summarization). Schema bootstrap remains idempotent (ADD COLUMN IF NOT EXISTS). Auth, broker, clock_timestamp() conventions, and zod validation all unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 22:52:40 +00:00
parent a7f218e182
commit 2464d23bb6
18 changed files with 1559 additions and 94 deletions
--- a/apps/server/src/services/auto_name.ts
+++ b/apps/server/src/services/auto_name.ts
@@ -0,0 +1,157 @@
+import type { InferenceContext } from './inference.js';
+
+const NAMING_SYSTEM_PROMPT =
+  'You name chat sessions. Reply directly with no thinking, reasoning, or explanation. Output ONLY the title, 4 words max, no quotes, no punctuation, no prefix like "Title:".';
+
+const MAX_TITLE_CHARS = 60;
+
+// QWEN3 NON-STREAMING UTILITY-CALL PATTERN
+// ----------------------------------------
+// Qwen3-family chat templates default to chain-of-thought reasoning: the
+// model emits a long <think>…</think> block into `reasoning_content` and
+// only finalizes a real reply in `content`. For short utility calls
+// (naming, classification, routing, summarization) with a tight token
+// budget, the model burns the entire budget on reasoning and returns:
+//   - content: ""
+//   - reasoning_content: "Thinking Process: 1. ..." (mid-thought, truncated)
+//   - finish_reason: "length"
+// Fix: pass `chat_template_kwargs: { enable_thinking: false }` to skip the
+// thinking block, and keep `max_tokens` low (~30 is plenty for a 4-word
+// title). The kwarg is a no-op for non-Qwen chat templates, so it's safe
+// to apply unconditionally for any short non-streaming model call.
+// Apply this same pattern to: fork-message (planned), agent-routing
+// (planned), web-search summarization (planned).
+
+function cleanTitle(raw: string): string {
+  let name = raw.trim();
+  // Strip surrounding straight or smart quotes (one layer).
+  const quotes = ['"', "'", '`', '‘', '’', '“', '”'];
+  while (name.length >= 2 && quotes.includes(name[0]!) && quotes.includes(name[name.length - 1]!)) {
+    name = name.slice(1, -1).trim();
+  }
+  // Drop a leading "Title:" prefix if the model added one despite instructions.
+  name = name.replace(/^title\s*:\s*/i, '').trim();
+  if (name.length > MAX_TITLE_CHARS) {
+    name = name.slice(0, MAX_TITLE_CHARS).trim();
+  }
+  return name;
+}
+
+interface NamingResponse {
+  choices?: Array<{
+    message?: {
+      content?: string;
+      reasoning_content?: string;
+    };
+  }>;
+}
+
+// Some Qwen-family models emit "thinking" tokens into reasoning_content and
+// only finalize a real reply in content. Pull a sensible candidate string.
+function pickTitleSource(data: NamingResponse): string {
+  const choice = data.choices?.[0]?.message;
+  if (!choice) return '';
+  if (choice.content && choice.content.trim().length > 0) return choice.content;
+  // Fallback: try to extract a last-line title from reasoning, if present.
+  const reasoning = choice.reasoning_content ?? '';
+  if (reasoning.length === 0) return '';
+  const lines = reasoning
+    .split('\n')
+    .map((l) => l.trim())
+    .filter((l) => l.length > 0);
+  return lines[lines.length - 1] ?? '';
+}
+
+export async function maybeAutoNameSession(
+  ctx: InferenceContext,
+  sessionId: string
+): Promise<void> {
+  const counts = await ctx.sql<{ n: number }[]>`
+    SELECT COUNT(*)::int AS n
+    FROM messages
+    WHERE session_id = ${sessionId}
+      AND role = 'assistant'
+      AND status = 'complete'
+  `;
+  if (counts[0]?.n !== 1) return;
+
+  const sessionRows = await ctx.sql<
+    { id: string; name: string; model: string }[]
+  >`
+    SELECT id, name, model FROM sessions WHERE id = ${sessionId}
+  `;
+  const session = sessionRows[0];
+  if (!session) return;
+  const existingName = session.name ?? '';
+  if (existingName !== '' && existingName !== 'New session') return;
+
+  const userMsg = await ctx.sql<{ content: string }[]>`
+    SELECT content FROM messages
+    WHERE session_id = ${sessionId} AND role = 'user'
+    ORDER BY created_at ASC
+    LIMIT 1
+  `;
+  const assistantMsg = await ctx.sql<{ content: string }[]>`
+    SELECT content FROM messages
+    WHERE session_id = ${sessionId}
+      AND role = 'assistant'
+      AND status = 'complete'
+    ORDER BY created_at ASC
+    LIMIT 1
+  `;
+  if (!userMsg[0] || !assistantMsg[0]) return;
+
+  const userText = userMsg[0].content.slice(0, 2000);
+  const assistantText = assistantMsg[0].content.slice(0, 2000);
+
+  const body = {
+    model: session.model,
+    messages: [
+      { role: 'system', content: NAMING_SYSTEM_PROMPT },
+      {
+        role: 'user',
+        content: `First user message: ${userText}\nFirst assistant reply: ${assistantText}`,
+      },
+    ],
+    max_tokens: 30,
+    temperature: 0.3,
+    stream: false,
+    // Qwen-family models default to chain-of-thought; this template kwarg
+    // tells llama.cpp's chat template renderer to skip the thinking block.
+    // Harmless for non-Qwen models.
+    chat_template_kwargs: { enable_thinking: false },
+  };
+
+  const res = await fetch(`${ctx.config.LLAMA_SWAP_URL}/v1/chat/completions`, {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify(body),
+  });
+  if (!res.ok) {
+    const text = await res.text().catch(() => '');
+    throw new Error(`naming request failed: ${res.status} ${text.slice(0, 200)}`);
+  }
+  const data = (await res.json()) as NamingResponse;
+  const raw = pickTitleSource(data);
+  const name = cleanTitle(raw);
+  if (!name) {
+    ctx.log.warn({ sessionId, raw }, 'auto-name: empty title from model');
+    return;
+  }
+
+  const updated = await ctx.sql<{ id: string; name: string }[]>`
+    UPDATE sessions
+    SET name = ${name}, updated_at = NOW()
+    WHERE id = ${sessionId}
+      AND (name IS NULL OR name = '' OR name = 'New session')
+    RETURNING id, name
+  `;
+  if (updated.length === 0) return;
+
+  ctx.publish(sessionId, {
+    type: 'session_renamed',
+    session_id: sessionId,
+    name,
+  });
+  ctx.log.info({ sessionId, name }, 'session auto-named');
+}