boocode/openspec/changes/multi-llama-swap-providers-model-favorites/artifacts/implementation-analysis.md

multi-llama-swap-providers-model-favorites — implementation analysis

Scope compared

• Current state: the shipped implementation in apps/server, apps/coder, apps/web, and packages/contracts
• Desired state: the behavior described in docs/research/2026-06-10-multi-llama-swap-providers-model-favorites.md and the corresponding OpenSpec batch

Purpose: determine the safest and most coherent implementation path before building the feature.

Conclusion

The best implementation path is to treat this as a shared local-model routing subsystem, not as a picker-only UI feature.

That subsystem needs two interfaces:

1. An in-process resolver used directly by BooChat and native BooCoder paths.
2. A gateway surface for consumers that cannot call the resolver directly and still assume one OpenAI-compatible provider contract.

Without that split, the feature looks straightforward in BooChat but stays architecturally broken in BooCoder because the existing opencode integration collapses provider identity back to one local llama-swap endpoint.

Current-state findings

F-001 — config authority is split

• apps/server is driven by LLAMA_SWAP_URL, LLAMA_SIDECAR_URL, and DEFAULT_MODEL.
• apps/coder reuses LLAMA_SWAP_URL for local models and has a separate data/coder-providers.json for ACP providers.

Effect: there is no single source of truth for local model providers that both apps can consume.

F-002 — model identity is still a raw string everywhere that matters

• sessions.model is TEXT NOT NULL.
• chats.model is TEXT.
• model-context.ts caches by the raw model string.
• multiple dispatchers treat the model as an opaque string and infer behavior from prefixes.

Effect: duplicate model names across hosts cannot be represented safely without composite IDs.

F-003 — routing logic is duplicated and heuristic-heavy

• BooChat streaming uses upstreamModel() in provider.ts.
• non-streaming calls use resolveModelEndpoint().
• context lookup bypasses both and fetches LLAMA_SWAP_URL directly.
• arena local calls bypass both and hit LLAMA_SWAP_URL directly.

Effect: even after adding a registry, call sites will diverge unless they all share one resolver.

F-004 — favorites are a UI concern backed by shared settings, not a server catalog concern

• The settings table is already the right persistence surface.
• BooChat already reads/writes server state.
• BooCoder currently keeps picker prefs in browser localStorage, but those are provider-specific UI prefs, not a shared favorite-model feature.

Effect: favorites should be stored server-side and derived in the client from /api/settings + provider-aware model data.

F-005 — BooCoder has a deeper coupling than the research initially surfaced

The dangerous assumption is not only in dispatcher.ts. It is in the whole opencode local-model bridge:

• the snapshot merges local llama models into the opencode provider by prefixing them as llama-swap/<model>
• the dispatcher treats bare IDs as llama-swap/<model>
• the opencode backend parses provider/model
• current host opencode config points every local-model family at a single llama-swap base URL

Effect: translating embedding/qwen3.5-9b back to llama-swap/qwen3.5-9b reintroduces the exact ambiguity this batch is trying to remove.

F-006 — Arena is a separate local-model consumer, not just another caller

Arena currently:

• builds its "local model" set from one live llama-swap list
• classifies local-vs-cloud contestants from that set
• performs one-shot local calls directly against LLAMA_SWAP_URL

Effect: arena needs the same provider-aware resolver as BooChat, but it does not need the full BooChat picker/favorites work.

Gap summary

G-001 — no shared local-provider registry

What is missing:

• one schema and one loader contract for named local providers consumed by both server and coder

Why it matters:

• every downstream fix becomes duplicated if config remains split

G-002 — no canonical model-ref format and parser

What is missing:

• a shared provider/model identity format and parse/format helpers

Why it matters:

• caches, DB values, routing, and UI rendering cannot stay aligned otherwise

G-003 — no single provider-aware resolver

What is missing:

• one shared resolver API for:
  • route selection
  • base URL selection
  • sidecar selection
  • wire-model extraction
  • context-props endpoint selection

Why it matters:

• keeping separate "streaming", "non-streaming", "context", and "arena" resolution paths will re-create subtle bugs

G-004 — no neutral provider-aware catalog contract

What is missing:

• a provider-aware model catalog response that exposes providers and models without baking favorites into the server payload

Why it matters:

• BooChat and BooCoder both need provider metadata, but favorites are derived from user settings, not from upstream inventory

G-005 — no safe path for opencode local-model parity

What is missing:

• either:
  • a generated/synced opencode-facing local-model config, or
  • a BooCoder-hosted OpenAI-compatible gateway that preserves provider identity under one provider namespace, or
  • a deliberate scope cut that removes multi-provider local models from the opencode provider until that bridge exists

Why it matters:

• without one of these, the feature is correct in BooChat but false-advertised in the opencode provider

Recommended architecture

1. Shared local-provider registry

Add a new shared config surface for local inference providers, separate from data/coder-providers.json.

Recommendation:

• schema in packages/contracts
• live file such as /data/llama-providers.json
• fallback synthesis from LLAMA_SWAP_URL and LLAMA_SIDECAR_URL while the file is absent

This keeps ACP provider management and local model provider management as two separate concerns.

2. Shared model-ref and resolver helpers

Add shared helpers for:

• parsing provider/model
• resolving legacy bare IDs to the default provider
• deciding route type
• selecting upstream base URL
• extracting the wire model id

All of these should be used by:

• server streaming inference
• server non-streaming calls
• model-context lookup
• arena one-shot local calls
• any future control-plane or routing feature

3. Provider-aware catalog, client-derived favorites

Do not make the server return a synthetic Favorites section.

Instead:

• /api/models (or a replacement contract) should return provider-grouped inventory only
• /api/settings should hold favorite_models: string[]
• BooChat and BooCoder should derive:
  • Favorites first
  • then provider sections
  • hide unavailable favorites without deleting them

This keeps the server contract inventory-shaped and the favorite behavior user-shaped.

4. Treat BooCoder native and BooCoder external-agent paths differently

There are two different BooCoder consumers:

• native boocode provider
• external-agent providers like opencode

The native boocode provider can adopt the shared resolver directly.

The opencode provider cannot safely adopt provider/model by simple string translation, because its current local-model bridge still assumes one local provider.

Recommendation:

• ship native boocode provider parity first
• do not claim opencode parity until provider identity is preserved end-to-end there too

5. Preferred parity path for opencode: a BooCoder-hosted local-model gateway

If full opencode parity is required in the same initiative, the cleanest path is a small OpenAI-compatible gateway inside apps/coder:

• accepts model ids that still carry provider identity
• strips provider prefix only at the final upstream boundary
• routes to the correct local provider
• becomes the single local-model base URL for opencode

Why this is better than adding many direct opencode providers:

• one stable provider contract for opencode
• no duplicated base-URL registry in opencode config
• the same gateway can serve arena/local utility calls later
• it stays inside an existing always-on service, not a new third service

If this gateway is not in scope now, the correct fallback is to remove or hide multi-provider local models from the opencode provider until the bridge is real.

Recommended sequence

Phase 1 — shared foundation

• shared local-provider config schema
• shared provider/model parsing helpers
• shared resolver
• legacy bare-id fallback

Phase 2 — BooChat + native BooCoder

• provider-aware model catalog
• server inference routing updates
• model-context cache-key fix
• compaction and task-model endpoint resolution
• BooChat picker grouping + server-side favorites
• BooCoder boocode provider model list grouped by local provider

Phase 3 — arena parity

• local-model set built from the shared provider catalog, not one llama-swap
• one-shot local calls use the shared resolver

Phase 4 — opencode parity

Choose one:

• preferred: BooCoder-hosted local-model gateway plus opencode-facing model sync
• fallback: temporarily stop advertising multi-provider local models under the opencode provider

Phase 5 — boocontrol

• build BooControl only after the local-provider registry and canonical model identity land

What this changes in the existing OpenSpec batch

1. The design should treat favorites as client-derived from settings, not as a server-generated catalog section.
2. The design should explicitly separate native BooCoder parity from opencode parity.
3. The tasks should call out the opencode bridge as a dedicated risk area, not as a small dispatcher rename.

Recommendation

Implement the shared local-provider registry and resolver first, then ship BooChat plus native BooCoder on top of it. Treat opencode multi-provider support as a distinct integration seam that either gets a real gateway or stays out of scope for the first slice.

That is the fastest path that is still architecturally honest.