What they did

TRIAGE uses semantic similarity retrieval to route requests among three execution levels: direct reuse for identical queries, deterministic parameter substitution for similar queries with an extracted skill, and full ReAct reasoning for novel queries. Newly generated trajectories are stored and recurring patterns can be distilled automatically into skills.

The framework was evaluated on 1,007 security-monitoring queries using the all-MiniLM-L6-v2 semantic encoder. Cross-domain validation covered 345 ToolBench queries from 15 domains, while an online experiment measured how reuse developed during the first 100 requests.

Key findings

• On the security-monitoring workload, token consumption fell 62.3%, from 199,782 to 75,238 tokens.
• Skill substitution handled 56.0% of security queries without new LLM tokens, while exact trajectory reuse handled another 5.5%.
• On ToolBench, TRIAGE reduced token use by 76.3% across 15 domains and 345 queries.
• During online learning, the Level 2 hit rate rose from 0% to 57% within 100 queries, while average token cost declined from 198 to 74.7.

Why it matters

The results suggest that agent systems serving repetitive workloads need not invoke full reasoning for every request. Treating validated trajectories as parameterized skills could reduce inference costs and make frequently used tool workflows progressively more efficient.

Caveats

The reported evidence emphasizes token savings; the provided results do not establish whether reuse preserves task accuracy across changing tools, ambiguous requests, or distribution shifts. “Zero-token” execution refers to avoiding new LLM generation and does not eliminate retrieval, routing, tool-execution, storage, or skill-maintenance costs.