StudentBench is a public evaluation suite and data collection platform that logged over 175,000 student-AI messages. In a randomized study, 2,383 participants prepared for Quantitative and Verbal GRE questions under AI tutoring, human tutoring, or no tutoring, with pre- and post-tests counterbalanced to control for form difficulty.
AI tutoring was statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and the best AI tutor beat the average human tutor in five of seven GRE domains. One AI tutor achieved equivalent gains at roughly 918 times lower cost ($0.0052 vs. $4.81 per percentage point gained). Expert human tutors also rated LLM-generated lesson plans and practice problems in 2,028 pairwise rubric evaluations. Faster AI replies correlated with more student messages, more correct practice, and larger learning gains in Quantitative sessions.
The findings suggest LLM-based tutors can match human tutors on standardized-test learning at far lower cost, but the correlations around latency and engagement are observational.