Organizations use (LLMs) to make decisions about people, such as hiring or investing. To ensure these models are fair, they commission fairness audits. But these audits are usually done only once, assuming the model stays the same over time.
Researchers audited five LLMs from different providers over a year, using a venture capital screening task. They measured bias in investment decisions when founder profiles differed only by gender, origin, and university. They repeated the audit multiple times.
The bias pattern changed after model updates, even when the model name did not. Some models were retired or replaced with little notice. This means a single audit result expires when the provider changes the model. Regulators should require re-audits and version tracking to keep evidence valid.
