The paper introduces Dyad, an architecture that augments a pretrained large language model with an action encoder. This encoder processes each possible action description in parallel and scores them against the LLM's internal state to directly produce a distribution over typed actions, rather than generating actions as text tokens. The approach separates state representation from action semantics, allowing efficient scoring even with large action spaces.
Dyad was tested in two settings. With the LLM frozen, training only the action encoder improved performance across four unseen environments. Jointly training the LLM and action encoder outperformed conventional post-training methods, including a 3.80% average absolute gain on the ALFWorld benchmark with a 9B model, while also improving performance on general knowledge, reasoning, and coding tasks.