Customer-service AI agents can improve from their own mistakes after deployment

A framework called CARE enables a medium-sized model to replace an expensive frontier model and keep learning from production failures without human relabeling.

Top University

Yanbei Jiang · Chao Lei · Rena Gao · Xiaochen Zhao · Yuchen Ge · Haohang Liu · +14 more

Meituan · University of Melbourne · Fudan University

Research Digest·3 November 2026·3 min read

Thread:Agent Self-Improvement

Illustration: Customer-service AI agents can improve from their own mistakes after deployment

In one line

An in-house mid-sized customer-service model that retrains on its own verified failures, using diagnoses as hints for itself, cut its error rate below the frontier model it replaced.

Before you read this

Three ideas this paper builds on

recursive self-improvementAn AI system repeatedly uses its own outputs to enhance its capabilities, creating a feedback loop for autonomous improvement.55 papers →on-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.115 papers →failure attributionIdentifying which component or step in a system caused an observed failure, often using structured trace representations.121 papers →

Read it as

Companies often run customer service chatbots on a large rented AI model (a large language model, or LLM): costly, and outside the operator's control. Swapping in a smaller in-house model is not one-off work: it fails on rare situations, and business policies keep changing.

A 31-billion-parameter open model was first trained on recorded conversations, then served live traffic alongside the incumbent. Reviewers judged its failures independently, deciding whether the fault lay with the model or with a tool or written procedure. Model faults were corrected and the model retrained to make the corrected choice, with the diagnosis supplied as a hint.

After five rounds its failure rate fell below the incumbent's and customer satisfaction matched it.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

recursive self-improvement on-policy distillation failure attribution agentic evaluation privileged information evidence blindness