The authors built a replayable forecasting environment from resolved prediction-market questions and a temporally restricted news corpus.
Replayable news timelines test how LLM agents update forecasts
Forecast-Dojo reconstructs historical prediction tasks across successive dates, enabling repeatable evaluation and training with immediate outcome feedback.
Big Tech
Liqin Ye · Haorui Wang · Fardin Ahmed · Rongzhi Zhang · Yuan He · Ziyuan Lin · +5 more
Georgia Institute of Technology · Amazon · University of Florida
Research Digest··2 min read
8 million dated news articles, allowing forecasting agents to research each question and revise their probabilities as historical evidence arrives.
Why this paper
From Amazon and 2 others
In one line
Replayable historical forecasting environments let LLM agents research dated news, lowering Brier scores across 12 models, though all still trail market forecasts.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§