BOReFT introduces a method to treat the discrete proposal space of a language model as a continuous search domain for Bayesian optimization. It learns a compact, low-dimensional space of hidden-state interventions in a frozen LLM, then uses that space to steer the model's generations. This bridges the gap between iterative LLM prompting and principled continuous optimization.
Evaluated on the word game Semantle and three molecular property optimization tasks, BOReFT outperforms strong LLM baselines. It finds more hidden targets in Semantle and achieves higher property scores on two of three molecular objectives. The approach provides a principled way to explore an LLM's semantic space with guarantees from Bayesian optimization.