Two independent research papers published this week argue that the metrics most commonly used to evaluate AI language models are fundamentally flawed, with one showing that a simple trick can make incoherent text appear state-of-the-art, and another demonstrating that standard cultural alignment tests measure knowledge rather than genuine values.
Researchers at institutions including Microsoft Research and McGill University have separately identified critical weaknesses in the benchmarks used to judge the quality of modern AI language models — raising questions about whether years of reported progress in certain areas of AI development is as meaningful as it appears.
The first study, by Antonio Franca and Alexander Tong, targets 'generative perplexity' (gen-PPL), the dominant metric for evaluating non-autoregressive language models such as diffusion and continuous-flow models. Gen-PPL works by asking a separate AI model — typically GPT-2 Large — to score how predictable a generated text is. A lower score is taken to mean better, more natural language.
The researchers argue this approach is fundamentally unsound. Predictability under a scoring model is not the same as grammatical correctness or semantic coherence. To prove the point, they constructed a set of deliberately naive text samplers — requiring no learned parameters — that achieved state-of-the-art gen-PPL scores on two standard benchmarks while producing text that was, by design, incoherent.
'The set of predictable but still low-quality sequences is combinatorially large,' the paper states, warning that the metric creates a large surface for inadvertent or deliberate gaming. The authors recommend replacing gen-PPL with distributional metrics that directly compare generated text to reference corpora, and they re-benchmarked several recent models using this approach, finding a markedly different picture of which models are actually performing well.
The second study addresses a different but related problem: how well do large language models reflect the cultural values of the people who use them? As AI systems are deployed globally, ensuring they do not impose a single cultural worldview has become a significant concern for both researchers and regulators.
The paper, by a team including researchers from Microsoft Research Asia and Singapore Management University, identifies what they call the 'Construct-Composition-Context' (C³) challenge in existing cultural alignment benchmarks. Most current tests use multiple-choice questions that reveal whether a model knows about cultural values — not whether it actually reflects them. They also tend to treat cultures as monolithic, ignoring internal diversity and subgroups.
In response, the team developed DOVE (Distributional Open-Ended Value Evaluation), a framework that compares the statistical distribution of text written by humans from a given culture against text generated by a language model in response to open-ended prompts. The system uses an 'optimal transport' mathematical technique to measure how closely the two distributions match, including their internal diversity.
Testing across 12 language models, DOVE achieved a 31.56% correlation with real-world downstream task performance — substantially higher than existing benchmarks — while remaining statistically reliable with as few as 500 sample documents per culture.
Taken together, the two papers point to a systemic issue in AI evaluation: metrics that are easy to compute and compare tend to become dominant, even when they measure something subtly different from what researchers intend. Both teams call for the field to adopt more rigorous, distribution-based evaluation methods before drawing conclusions about model quality or alignment.
Analysis
Why This Matters
- Progress in AI is largely communicated through benchmark scores — if those scores are misleading, funding decisions, safety assessments, and public trust may all be based on faulty foundations.
- The generative perplexity finding is particularly significant for non-autoregressive models, which are being developed as faster, more energy-efficient alternatives to systems like GPT; if their reported gains are illusory, years of research may need to be reassessed.
- Cultural alignment failures in globally deployed AI systems carry direct risks for non-Western users, who may receive outputs that subtly conflict with their values, norms, or communication styles.
Background
Evaluating AI language models has always been a contested problem. Early benchmarks focused on narrow tasks like translation or question-answering, but as models became more general-purpose, the field shifted toward broader measures of fluency and coherence. Perplexity — borrowed from information theory — became a standard tool because it is mathematically tractable and easy to compare across models.
The rise of diffusion and flow-based language models over the past two to three years introduced a new complication. Unlike autoregressive models that generate text one token at a time, these non-autoregressive approaches generate text in a fundamentally different way, making traditional perplexity scores harder to compute directly. Generative perplexity — using a separate model to score outputs — emerged as a workaround, and quickly became the de facto standard for the field.
Cultural alignment as a research area has grown alongside the globalisation of AI deployment. Early concerns focused primarily on bias and stereotyping, but researchers have increasingly recognised a deeper challenge: models trained predominantly on English-language, Western-origin data may embed implicit value systems that do not translate well across cultures. Major AI labs including Google, Meta, and Microsoft have published their own alignment frameworks, though critics have argued these often lack rigorous evaluation methods.
Key Perspectives
AI researchers developing non-autoregressive models: Those who have published results showing improvements in gen-PPL may dispute the severity of the critique, arguing that the metric, while imperfect, has still served as a useful relative signal for comparing models within the same research community.
AI safety and evaluation researchers: Likely to welcome both papers as overdue corrections. The broader field of 'evaluation science' has long warned that metrics which become targets — a phenomenon known as Goodhart's Law — tend to lose their validity over time.
Critics and skeptics: Some may note that the proposed alternatives, particularly optimal transport-based distributional metrics, are computationally more expensive and harder to standardise, which could slow down research cycles. Others may question whether any single metric can adequately capture language quality or cultural alignment across diverse contexts.
What to Watch
- Whether major AI benchmarking organisations such as EleutherAI, Hugging Face, or academic leaderboards adopt distributional metrics as a replacement or supplement to gen-PPL in the coming months.
- Responses from research teams whose published results relied heavily on gen-PPL improvements — any replication or rebuttal studies will be significant.
- Whether AI governance bodies or regulators, particularly in the EU under the AI Act, reference these findings when setting standards for model evaluation and cultural safety testing.