llama-sidecar v0.1.0: daemon + benchmarks + eval suite
Go daemon (cmd/llama-sidecar): per-agent llama-server process pool with LRU eviction, OpenAI-compatible proxy, flag validation (Unsloth port), deterministic hash-keyed sidecar reuse. Windows service support via schtasks/NSSM with DETACHED_PROCESS, stdout pipe drain, and request-ctx decoupled child lifetime. Bug fixes (3b.1–3b5): -c flag drop from StripShadowingFlags, UTF-8 BOM in JSON config, -fa → --flash-attn on default, child process exit after one request (stdin devnull, stdout pipe, CREATE_NO_WINDOW → DETACHED, context.Background for child lifetime, background reaper goroutine). bench/: MTP on/off throughput sweep across 8 GGUFs via SSH+schtasks automation to sam-desktop. Per-GGUF production flags from llama-swap config with --ctx-size 32768 override. eval/: accuracy benchmarks (MMLU 100q, GSM8K 50q, HumanEval 164) + A/B model comparison (14 agent-typed prompts × 8 models). All scripts resumable at individual question level. 94 Go tests, race detector clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
18
bench/prompts/p256.txt
Normal file
18
bench/prompts/p256.txt
Normal file
@@ -0,0 +1,18 @@
|
||||
You will rejoice to hear that no disaster has accompanied the
|
||||
commencement of an enterprise which you have regarded with such evil
|
||||
forebodings. I arrived here yesterday, and my first task is to assure
|
||||
my dear sister of my welfare and increasing confidence in the success
|
||||
of my undertaking.
|
||||
|
||||
I am already far north of London, and as I walk in the streets of
|
||||
Petersburgh, I feel a cold northern breeze play upon my cheeks, which
|
||||
braces my nerves and fills me with delight. Do you understand this
|
||||
feeling? This breeze, which has travelled from the regions towards
|
||||
which I am advancing, gives me a foretaste of those icy climes.
|
||||
Inspirited by this wind of promise, my daydreams become more fervent
|
||||
and vivid. I try in vain to be persuaded that the pole is the seat of
|
||||
frost and desolation; it ever presents itself to my imagination as the
|
||||
region of beauty and delight. There, Margaret, the sun is for ever
|
||||
visible, its broad disk just skirting the horizon and diffusing a
|
||||
perpetual splendour.
|
||||
Continue this passage in exactly 200 tokens of prose.
|
||||
Reference in New Issue
Block a user