What they did

CRAFT consists of two stages: first, schema-stripped PLAN supervised fine-tuning learns domain-structured plans and executable behaviors from validated trajectories that deliberately omit schema context; second, execution-shaped reinforcement learning aligns the policy for tool selection, code quality, plan-code consistency, and self-recovery from failed executions. Training data is curated via a Tri-Gate filter that combines execution validation, data-integrity checks, and an LLM-judge reasoning audit. The method is evaluated in an enterprise advertising analytics environment covering 25 schema-linked core entities and 30 agentic workflows, with beta APIs as the agent-facing tool surface.

Key findings

• CRAFT improves composite Agent Score by +9.6 percentage points over a schema-stuffed baseline.
• Consistency improves by +4.1 pp, multi-turn coherence by +4.2 pp.
• Input-token burden is reduced by approximately 9×.
• Schema-discovery loops (re-reading documentation) are reduced by up to 5×.

Why it matters

This work shows that enterprise coding agents can internalize complex schema knowledge and tool-use behavior through post-training, greatly reducing reliance on prompt engineering and long context windows. The approach addresses a key production bottleneck: high inference cost, brittle schema updates, and inconsistent multi-turn analysis in deployed analytic agents.

Caveats

The evaluation is limited to a single advertising analytics domain with a fixed set of 30 workflows and 25 entities. The training pipeline requires curated multi-turn trajectories with execution validation, which may not be straightforward to obtain in other domains. The paper also acknowledges reward-shaping limitations and infrastructure challenges for scaling multi-turn reinforcement learning in enterprise settings.