LLMs follow abstract rules even when they contradict surface statistics
Conlang-like tasks show models exhibit meaning-mediated abstraction, not just pattern matching
Academic
Julia Witte Zimmerman · Calla G. Beauregard · Tabia Tanzin Prama · Parisa Suchdev · Kathryn Cramer · Elisabeth Kollrack
University of Vermont
Research Digest··3 min read
The authors test the strong stochastic parrot hypothesis—that LLMs only do statistical pattern matching—using tasks where models must follow rules of invented languages that violate common patterns in training data.
Why this paper
From University of Vermont
In one line
Large language models follow meaning-mediated rules, not just statistical patterns, contradicting the strong stochastic parrot hypothesis.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§