# multi-llama-swap-providers-model-favorites — implementation analysis ## Scope compared - **Current state:** the shipped implementation in `apps/server`, `apps/coder`, `apps/web`, and `packages/contracts` - **Desired state:** the behavior described in `docs/research/2026-06-10-multi-llama-swap-providers-model-favorites.md` and the corresponding OpenSpec batch Purpose: determine the safest and most coherent implementation path before building the feature. ## Conclusion The best implementation path is to treat this as a **shared local-model routing subsystem**, not as a picker-only UI feature. That subsystem needs two interfaces: 1. **An in-process resolver** used directly by BooChat and native BooCoder paths. 2. **A gateway surface** for consumers that cannot call the resolver directly and still assume one OpenAI-compatible provider contract. Without that split, the feature looks straightforward in BooChat but stays architecturally broken in BooCoder because the existing opencode integration collapses provider identity back to one local llama-swap endpoint. ## Current-state findings ### F-001 — config authority is split - `apps/server` is driven by `LLAMA_SWAP_URL`, `LLAMA_SIDECAR_URL`, and `DEFAULT_MODEL`. - `apps/coder` reuses `LLAMA_SWAP_URL` for local models and has a separate `data/coder-providers.json` for ACP providers. Effect: there is no single source of truth for local model providers that both apps can consume. ### F-002 — model identity is still a raw string everywhere that matters - `sessions.model` is `TEXT NOT NULL`. - `chats.model` is `TEXT`. - `model-context.ts` caches by the raw model string. - multiple dispatchers treat the model as an opaque string and infer behavior from prefixes. Effect: duplicate model names across hosts cannot be represented safely without composite IDs. ### F-003 — routing logic is duplicated and heuristic-heavy - BooChat streaming uses `upstreamModel()` in `provider.ts`. - non-streaming calls use `resolveModelEndpoint()`. - context lookup bypasses both and fetches `LLAMA_SWAP_URL` directly. - arena local calls bypass both and hit `LLAMA_SWAP_URL` directly. Effect: even after adding a registry, call sites will diverge unless they all share one resolver. ### F-004 — favorites are a UI concern backed by shared settings, not a server catalog concern - The `settings` table is already the right persistence surface. - BooChat already reads/writes server state. - BooCoder currently keeps picker prefs in browser localStorage, but those are provider-specific UI prefs, not a shared favorite-model feature. Effect: favorites should be stored server-side and derived in the client from `/api/settings` + provider-aware model data. ### F-005 — BooCoder has a deeper coupling than the research initially surfaced The dangerous assumption is not only in `dispatcher.ts`. It is in the whole opencode local-model bridge: - the snapshot merges local llama models into the `opencode` provider by prefixing them as `llama-swap/` - the dispatcher treats bare IDs as `llama-swap/` - the opencode backend parses `provider/model` - current host opencode config points every local-model family at a single llama-swap base URL Effect: translating `embedding/qwen3.5-9b` back to `llama-swap/qwen3.5-9b` reintroduces the exact ambiguity this batch is trying to remove. ### F-006 — Arena is a separate local-model consumer, not just another caller Arena currently: - builds its "local model" set from one live llama-swap list - classifies local-vs-cloud contestants from that set - performs one-shot local calls directly against `LLAMA_SWAP_URL` Effect: arena needs the same provider-aware resolver as BooChat, but it does not need the full BooChat picker/favorites work. ## Gap summary ### G-001 — no shared local-provider registry What is missing: - one schema and one loader contract for named local providers consumed by both server and coder Why it matters: - every downstream fix becomes duplicated if config remains split ### G-002 — no canonical model-ref format and parser What is missing: - a shared `provider/model` identity format and parse/format helpers Why it matters: - caches, DB values, routing, and UI rendering cannot stay aligned otherwise ### G-003 — no single provider-aware resolver What is missing: - one shared resolver API for: - route selection - base URL selection - sidecar selection - wire-model extraction - context-props endpoint selection Why it matters: - keeping separate "streaming", "non-streaming", "context", and "arena" resolution paths will re-create subtle bugs ### G-004 — no neutral provider-aware catalog contract What is missing: - a provider-aware model catalog response that exposes providers and models without baking favorites into the server payload Why it matters: - BooChat and BooCoder both need provider metadata, but favorites are derived from user settings, not from upstream inventory ### G-005 — no safe path for opencode local-model parity What is missing: - either: - a generated/synced opencode-facing local-model config, or - a BooCoder-hosted OpenAI-compatible gateway that preserves provider identity under one provider namespace, or - a deliberate scope cut that removes multi-provider local models from the `opencode` provider until that bridge exists Why it matters: - without one of these, the feature is correct in BooChat but false-advertised in the `opencode` provider ## Recommended architecture ### 1. Shared local-provider registry Add a new shared config surface for local inference providers, separate from `data/coder-providers.json`. Recommendation: - schema in `packages/contracts` - live file such as `/data/llama-providers.json` - fallback synthesis from `LLAMA_SWAP_URL` and `LLAMA_SIDECAR_URL` while the file is absent This keeps ACP provider management and local model provider management as two separate concerns. ### 2. Shared model-ref and resolver helpers Add shared helpers for: - parsing `provider/model` - resolving legacy bare IDs to the default provider - deciding route type - selecting upstream base URL - extracting the wire model id All of these should be used by: - server streaming inference - server non-streaming calls - model-context lookup - arena one-shot local calls - any future control-plane or routing feature ### 3. Provider-aware catalog, client-derived favorites Do **not** make the server return a synthetic Favorites section. Instead: - `/api/models` (or a replacement contract) should return provider-grouped inventory only - `/api/settings` should hold `favorite_models: string[]` - BooChat and BooCoder should derive: - Favorites first - then provider sections - hide unavailable favorites without deleting them This keeps the server contract inventory-shaped and the favorite behavior user-shaped. ### 4. Treat BooCoder native and BooCoder external-agent paths differently There are two different BooCoder consumers: - **native `boocode` provider** - **external-agent providers like `opencode`** The native `boocode` provider can adopt the shared resolver directly. The `opencode` provider cannot safely adopt `provider/model` by simple string translation, because its current local-model bridge still assumes one local provider. Recommendation: - ship native `boocode` provider parity first - do **not** claim `opencode` parity until provider identity is preserved end-to-end there too ### 5. Preferred parity path for opencode: a BooCoder-hosted local-model gateway If full `opencode` parity is required in the same initiative, the cleanest path is a small OpenAI-compatible gateway inside `apps/coder`: - accepts model ids that still carry provider identity - strips provider prefix only at the final upstream boundary - routes to the correct local provider - becomes the single local-model base URL for `opencode` Why this is better than adding many direct opencode providers: - one stable provider contract for opencode - no duplicated base-URL registry in opencode config - the same gateway can serve arena/local utility calls later - it stays inside an existing always-on service, not a new third service If this gateway is not in scope now, the correct fallback is to remove or hide multi-provider local models from the `opencode` provider until the bridge is real. ## Recommended sequence ### Phase 1 — shared foundation - shared local-provider config schema - shared `provider/model` parsing helpers - shared resolver - legacy bare-id fallback ### Phase 2 — BooChat + native BooCoder - provider-aware model catalog - server inference routing updates - model-context cache-key fix - compaction and task-model endpoint resolution - BooChat picker grouping + server-side favorites - BooCoder `boocode` provider model list grouped by local provider ### Phase 3 — arena parity - local-model set built from the shared provider catalog, not one llama-swap - one-shot local calls use the shared resolver ### Phase 4 — opencode parity Choose one: - preferred: BooCoder-hosted local-model gateway plus opencode-facing model sync - fallback: temporarily stop advertising multi-provider local models under the `opencode` provider ### Phase 5 — boocontrol - build BooControl only after the local-provider registry and canonical model identity land ## What this changes in the existing OpenSpec batch 1. The design should treat favorites as **client-derived from settings**, not as a server-generated catalog section. 2. The design should explicitly separate **native BooCoder parity** from **opencode parity**. 3. The tasks should call out the `opencode` bridge as a dedicated risk area, not as a small dispatcher rename. ## Recommendation Implement the shared local-provider registry and resolver first, then ship BooChat plus native BooCoder on top of it. Treat `opencode` multi-provider support as a distinct integration seam that either gets a real gateway or stays out of scope for the first slice. That is the fastest path that is still architecturally honest.