can generate text but users have little control over the style or content. This makes them unpredictable for practical applications.
CTRL is a 1.63 billion-parameter transformer model trained with control codes. These codes are derived from natural features of the training data, like the website or domain of the text. The model uses them to generate text matching a desired style, topic, or task.
The main finding is that these codes allow explicit control while preserving the strengths of unsupervised learning. This matters because it makes large language models more useful and predictable, and the codes also help trace generated text back to its training data for analysis.