The authors constructed FinHopper using a state-transition-based synthesis framework that composes verified atomic financial QA patterns into dependency-preserving chains, covering stocks, funds, and macroeconomic facts.
Financial multi-hop benchmark exposes sharp agent performance drop beyond four hops
FinHopper evaluates LLM agents on dependency-preserving financial queries with step-wise gold answers, revealing that even top models lag behind human experts and process-aware scoring uncovers hidden partial progress.
Chinese Tech
Yuxuan Wu · Miaopei Lin · Qiufang Ying · Lin Dan · Hao Zhao · Qirui Liu · +6 more
South China University of Technology · FiT, Tencent · Shenzhen Technology University
Research Digest··2 min read
The authors introduce FinHopper, a benchmark for LLM-based agents on complex financial queries requiring 1–6 reasoning hops, each conditioned on the exact answer from the previous hop.
§