CypherTurn evaluates conversational Text-to-Cypher across 721 sessions, 5,927 turns, 7 , and 13 conversational phenomena. Fifteen models were tested both with guided oracle support and as fully autonomous agents.
The strongest model achieved 64.7% execution accuracy, while session-level correctness remained below 5%. Autonomous operation also reordered leading models, indicating that handling errors is partly distinct from generating good queries. Larger action budgets did not eliminate this gap, and single-turn Cypher fine-tuning harmed multi-turn instruction following.