A was tested on four astrophysics and genomics tasks that required finding and correctly operating published specialist models. Five configuration dimensions were varied: supplied information, reasoning, self-verification instructions, runtime budget, and backbone model.
About 54% of outcome variance among genuine attempts came from repeated runs of the same configuration. Supplied information had the largest effect, exceeding runtime and model size while reducing cost and improving calibration. More runtime helped only with adequate information or a capable model, and a dedicated tool changed behavior more than prompting the agent to verify itself.
The results matter for agent evaluation because model choice and a single success rate do not characterize the deployed system. Configuration interactions and run-to-run noise must also be measured.