v1.13.15-tools: tiered tool loading via BOOCODE_TOOLS env var
Pattern lift from eyaltoledano/claude-task-master (MIT + Commons Clause — pattern only, no code lift). Adds BOOCODE_TOOLS env var with three tiers: - core (4 tools): view_file, list_dir, grep, find_files. ~2k token schema cost. - standard (15 tools): core + web_search, web_fetch, git_status, all 8 codecontext_* tools. ~10k token schema cost. - all (default; current behavior): every tool in ALL_TOOLS (20). ~21k token schema cost. The env var is a CEILING — narrows agent whitelists, never expands. Default behavior unchanged when var is unset. resolveToolTier is case-insensitive and falls back to 'all' on unknown values. CORE_TOOL_NAMES + STANDARD_TOOL_NAMES validated at module load against TOOLS_BY_NAME via two top-level for-loops that throw on the first missing name. Module fails to import if a tier references a tool that doesn't exist in the registry — catches typos and stale tier definitions at boot rather than silently filtering valid tools out of agent whitelists. Wiring: agents.ts parseAgentBlock now reads BOOCODE_TOOLS from process.env per parse, intersects with the agent's declared frontmatter tools (or DEFAULT_TOOLS when frontmatter omits the field). Per-parse read is fine — agents are re-parsed on the existing 60s cache TTL. Tests: tools.test.ts grows from 1 to 10 tests. Covers resolveToolTier across tiers/case/unknown values + the CORE-subset-of-STANDARD invariant + TOOLS_BY_NAME existence for both tier sets. 204/204 pass (was 195; +9 new). Deviation from the brief: the codecontext tools in the actual registry have NO codecontext_* prefix (the brief's STANDARD list assumed it). Used the actual names (get_codebase_overview, search_symbols, etc.). Module-load validation would have failed boot with the prefixed names. Smoke: with BOOCODE_TOOLS unset, agents return their full 12-tool whitelists. With BOOCODE_TOOLS=core in .env + container restart, the same agents narrow to 4 tools (find_files, grep, list_dir, view_file) — intersection of declared whitelist ∩ core tier. Reverted after confirmation. CLAUDE.md updated with BOOCODE_TOOLS in the Environment section's Optional list. .env.example gained a commented BOOCODE_TOOLS=all line with the per-tier token-cost table. ~110 LoC across 5 files (4 modified + 1 test expansion). Under the brief's ~30 LoC estimate for code; the test suite expansion drove most of the growth.
This commit is contained in:
@@ -10,3 +10,12 @@ POSTGRES_PASSWORD=CHANGE_ME
|
||||
# Internal Tailscale address that bypasses Authelia. Override if you
|
||||
# point BooCode at a different SearXNG instance.
|
||||
SEARXNG_URL=http://100.114.205.53:8888
|
||||
|
||||
# v1.13.15-tools: BOOCODE_TOOLS narrows the tool whitelist sent to the LLM.
|
||||
# Unset (default) → all tools (~21k schema). Useful primarily for single-purpose
|
||||
# sessions where the model only needs read-only filesystem access.
|
||||
#
|
||||
# core → view_file, list_dir, grep, find_files (~2k)
|
||||
# standard → core + web_*, git_status, all 8 codecontext_* tools (~10k)
|
||||
# all → every tool in ALL_TOOLS (~21k)
|
||||
# BOOCODE_TOOLS=all
|
||||
|
||||
@@ -112,7 +112,7 @@ Schema CHECK migration order when renaming allowed values: (1) `ALTER TABLE ...
|
||||
|
||||
## Environment
|
||||
|
||||
Required: `DATABASE_URL`, `LLAMA_SWAP_URL`. Optional: `PORT` (3000), `HOST` (0.0.0.0), `PROJECT_ROOT_WHITELIST` (/opt, read-only scope for add-existing path resolution), `BOOTSTRAP_ROOT` (/opt/projects, writable scope for create-new-project bootstrap mkdir target — host must `mkdir -p /opt/projects` before container start), `DEFAULT_MODEL`, `LOG_LEVEL`, `SEARXNG_URL` (default `http://100.114.205.53:8888` — internal Tailscale Fathom; the public `search.indifferentketchup.com` is behind Authelia and unusable from server context).
|
||||
Required: `DATABASE_URL`, `LLAMA_SWAP_URL`. Optional: `PORT` (3000), `HOST` (0.0.0.0), `PROJECT_ROOT_WHITELIST` (/opt, read-only scope for add-existing path resolution), `BOOTSTRAP_ROOT` (/opt/projects, writable scope for create-new-project bootstrap mkdir target — host must `mkdir -p /opt/projects` before container start), `DEFAULT_MODEL`, `LOG_LEVEL`, `SEARXNG_URL` (default `http://100.114.205.53:8888` — internal Tailscale Fathom; the public `search.indifferentketchup.com` is behind Authelia and unusable from server context), `BOOCODE_TOOLS` (`core` | `standard` | `all`, default `all`; v1.13.15-tools tier filter — ceiling, never expands an agent's whitelist).
|
||||
|
||||
## Workflow
|
||||
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { ALL_TOOLS } from '../tools.js';
|
||||
import {
|
||||
ALL_TOOLS,
|
||||
CORE_TOOL_NAMES,
|
||||
STANDARD_TOOL_NAMES,
|
||||
TOOLS_BY_NAME,
|
||||
resolveToolTier,
|
||||
} from '../tools.js';
|
||||
|
||||
describe('ALL_TOOLS registry', () => {
|
||||
// v1.13.3: tools must be alpha-sorted at module load. llama.cpp's prompt
|
||||
@@ -12,3 +18,59 @@ describe('ALL_TOOLS registry', () => {
|
||||
expect(names).toEqual([...names].sort((a, b) => a.localeCompare(b)));
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveToolTier (v1.13.15-tools)', () => {
|
||||
it('returns CORE tools for tier=core', () => {
|
||||
expect(resolveToolTier('core')).toEqual(CORE_TOOL_NAMES);
|
||||
});
|
||||
|
||||
it('returns STANDARD tools for tier=standard', () => {
|
||||
const result = resolveToolTier('standard');
|
||||
expect(result.length).toBe(STANDARD_TOOL_NAMES.length);
|
||||
expect(result.length).toBeGreaterThan(CORE_TOOL_NAMES.length);
|
||||
// STANDARD is a strict superset of CORE.
|
||||
expect(result).toEqual(expect.arrayContaining([...CORE_TOOL_NAMES]));
|
||||
});
|
||||
|
||||
it('returns ALL tool names for tier=all', () => {
|
||||
expect(resolveToolTier('all').length).toBe(ALL_TOOLS.length);
|
||||
});
|
||||
|
||||
it('defaults to all when env var is undefined', () => {
|
||||
expect(resolveToolTier(undefined).length).toBe(ALL_TOOLS.length);
|
||||
});
|
||||
|
||||
it('is case-insensitive', () => {
|
||||
expect(resolveToolTier('CORE')).toEqual(CORE_TOOL_NAMES);
|
||||
expect(resolveToolTier('Standard').length).toBe(STANDARD_TOOL_NAMES.length);
|
||||
});
|
||||
|
||||
it('falls back to all for unknown tier strings', () => {
|
||||
expect(resolveToolTier('bogus').length).toBe(ALL_TOOLS.length);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CORE_TOOL_NAMES + STANDARD_TOOL_NAMES validation', () => {
|
||||
// The module-load validation in tools.ts throws if a tier references a
|
||||
// tool that doesn't exist in TOOLS_BY_NAME. These tests double-check that
|
||||
// invariant from the consumer side so a future tier-list edit can't smuggle
|
||||
// in a typo without a test failure.
|
||||
it('every CORE name exists in TOOLS_BY_NAME', () => {
|
||||
for (const name of CORE_TOOL_NAMES) {
|
||||
expect(TOOLS_BY_NAME[name], `CORE references unknown tool '${name}'`).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('every STANDARD name exists in TOOLS_BY_NAME', () => {
|
||||
for (const name of STANDARD_TOOL_NAMES) {
|
||||
expect(TOOLS_BY_NAME[name], `STANDARD references unknown tool '${name}'`).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it('CORE is a subset of STANDARD', () => {
|
||||
const standardSet = new Set<string>(STANDARD_TOOL_NAMES);
|
||||
for (const name of CORE_TOOL_NAMES) {
|
||||
expect(standardSet.has(name), `'${name}' is in CORE but not STANDARD`).toBe(true);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { promises as fs } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import type { Agent, AgentsResponse, AgentParseError } from '../types/api.js';
|
||||
import { ALL_TOOLS } from './tools.js';
|
||||
import { ALL_TOOLS, resolveToolTier } from './tools.js';
|
||||
|
||||
// v1.8.1: global agents live at /data/AGENTS.md inside the container
|
||||
// (./data:/data:ro mount on the host). Per-project AGENTS.md at the project
|
||||
@@ -186,11 +186,14 @@ function parseAgentSection(section: RawSection): Omit<Agent, 'source'> {
|
||||
throw new Error(fmErrors.join('; '));
|
||||
}
|
||||
|
||||
// v1.13.15-tools: intersect with BOOCODE_TOOLS tier (ceiling, not expansion).
|
||||
// Unset → resolveToolTier returns ALL tool names → no narrowing.
|
||||
const tierAllowed = new Set(resolveToolTier(process.env.BOOCODE_TOOLS));
|
||||
const filteredTools = Array.isArray(fm.tools)
|
||||
? fm.tools.filter((t): t is string =>
|
||||
(ALL_TOOL_NAMES as readonly string[]).includes(t),
|
||||
(ALL_TOOL_NAMES as readonly string[]).includes(t) && tierAllowed.has(t),
|
||||
)
|
||||
: DEFAULT_TOOLS;
|
||||
: DEFAULT_TOOLS.filter((t) => tierAllowed.has(t));
|
||||
|
||||
return {
|
||||
id: slugify(section.name),
|
||||
|
||||
@@ -700,6 +700,64 @@ export const TOOLS_BY_NAME: Record<string, ToolDef<unknown>> = Object.fromEntrie
|
||||
ALL_TOOLS.map((t) => [t.name, t])
|
||||
);
|
||||
|
||||
// v1.13.15-tools: tiered tool loading. BOOCODE_TOOLS env var (`core` |
|
||||
// `standard` | `all`) filters the agent's tool whitelist before LLM dispatch.
|
||||
// Daily-driver token win on qwen3.6-35b-a3b — the 35B-A3B MoE benefits from
|
||||
// any prompt-cache stability win (fewer tools = shorter, more stable tool
|
||||
// schemas in the system prompt). Pattern lift from eyaltoledano/claude-task-
|
||||
// master (MIT + Commons Clause — pattern only, no code lift).
|
||||
//
|
||||
// The env var is a CEILING. It only narrows; never expands an agent's
|
||||
// declared whitelist. Default behavior (var unset) is unchanged: all tools.
|
||||
export const CORE_TOOL_NAMES = [
|
||||
'view_file',
|
||||
'list_dir',
|
||||
'grep',
|
||||
'find_files',
|
||||
] as const;
|
||||
|
||||
export const STANDARD_TOOL_NAMES = [
|
||||
...CORE_TOOL_NAMES,
|
||||
'web_search',
|
||||
'web_fetch',
|
||||
'git_status',
|
||||
'get_codebase_overview',
|
||||
'get_file_analysis',
|
||||
'get_symbol_info',
|
||||
'search_symbols',
|
||||
'get_dependencies',
|
||||
'watch_changes',
|
||||
'get_semantic_neighborhoods',
|
||||
'get_framework_analysis',
|
||||
] as const;
|
||||
|
||||
// Module-load validation: every name in CORE / STANDARD must exist in
|
||||
// TOOLS_BY_NAME. Catches typos and stale tier definitions before they reach
|
||||
// production; server boot fails loudly rather than silently filtering valid
|
||||
// tools out of agent whitelists.
|
||||
for (const name of CORE_TOOL_NAMES) {
|
||||
if (!TOOLS_BY_NAME[name]) {
|
||||
throw new Error(`CORE_TOOL_NAMES references unknown tool: '${name}'`);
|
||||
}
|
||||
}
|
||||
for (const name of STANDARD_TOOL_NAMES) {
|
||||
if (!TOOLS_BY_NAME[name]) {
|
||||
throw new Error(`STANDARD_TOOL_NAMES references unknown tool: '${name}'`);
|
||||
}
|
||||
}
|
||||
|
||||
export function resolveToolTier(tier: string | undefined): readonly string[] {
|
||||
switch ((tier ?? 'all').toLowerCase()) {
|
||||
case 'core':
|
||||
return CORE_TOOL_NAMES;
|
||||
case 'standard':
|
||||
return STANDARD_TOOL_NAMES;
|
||||
case 'all':
|
||||
default:
|
||||
return ALL_TOOLS.map((t) => t.name);
|
||||
}
|
||||
}
|
||||
|
||||
export function toolJsonSchemas(): ToolJsonSchema[] {
|
||||
return ALL_TOOLS.map((t) => t.jsonSchema);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user