Collaborative-filtering (CF) scores are personalized scores used to train (LLMs) to recommend items. But when used as rewards in , they cause "exposure collapse": the LLM recommends the same few items across users, even though the CF model itself ranks items diversely.
The researchers broke each CF score into two parts: the item's average score across users (item-mean) and the user's deviation from that average (residual). Training on the item-mean alone caused collapse; the residual alone did not. They then proposed per-item centering (subtracting the item-mean) and per-item standardization (centering plus dividing by the standard deviation).
Both corrections increased the number of distinct top-1 items by up to 9.8 times, with comparable accuracy. This shows the non-personalized part of the reward is responsible for collapse, and a simple removal can make LLM recommenders more diverse and fair.