An application programming interface (API) for a language model usually returns only text, so customers cannot tell whether every request was answered by the model they paid for. But honest providers also change generation settings, which can make outputs look different without swapping models.
API-TAIL scores how unexpected each answer is under the advertised model. It adjusts for shifts affecting the whole batch, then counts answers that remain unusually unexpected. It compares that count with legitimate variations in how the service might be run.
On real commercial services, it detected substitution affecting one in ten requests in most tested settings, using a few hundred queries. This offers a way to audit hidden model swaps without treating ordinary serving differences as evidence of cheating.