The authors represent inference choices as leaves in a tree, with shared prefixes or related candidates grouped into internal nodes.
Canopy learns when tree-based shortcuts fail in LLM inference
The method tests local smoothness with cheap probes, then concentrates costly evaluations near detected discontinuities.
Big Tech
Michael Jerge · Suman Jana
Amazon · Columbia University
Research Digest··3 min read
Thread:Agent Harness Optimization
Jerge and Jana present CANOPY, a multi-fidelity bandit algorithm for searching tree-structured candidate spaces without assuming that nearby branches always have similar value.
Why this paper
From Amazon and Columbia University · Part of Agent Harness Optimization, now 90 papers
In one line
CANOPY adapts multi-fidelity tree bandits by learning where smoothness is valid, improving LLM inference tasks.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (5 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§