WhatIf-Bench generates target series from mechanistic simulators of data centers (OpenDC), power grids (GridLAB-D), road traffic (SUMO), and water networks (EPANET), with controlled covariate-target relationships imposed on each.
Even top forecasters fail when future events set covariates to unseen values
WhatIf-Bench tests intervention-aware forecasting and finds every model's skill collapses on novel scenarios.
Big Tech
Arjun Ashok · Mihir Parmar · Palash Goyal · Bhavana Dalvi Mishra · Chun-Liang Li · Irina Rish · +3 more
Google · Mila - Quebec AI Institute · Université de Montréal · Université Laval
Research Digest··3 min read
The authors introduce WhatIf-Bench, a semi-synthetic benchmark of 37,665 tasks built from mechanistic simulators of data centers, power grids, road traffic, and water networks.
§