What they did

The authors represented an LLM serving workflow—including execution stages and the control decisions coordinating them—as a unified, dynamic graph composed of reusable simulator components.

They then used Synthesizer, a coding agent equipped with simulator-specific guardrails and fidelity checks, to translate natural-language feature requests into extensions of this shared simulator. They compared the resulting extensions with those produced using the same coding agent and harnesses on existing simulators, validating throughput against a vLLM-based real system and measuring simulation speed against LLMServingSim2.0 and Vidur.

Key findings

• Simthesizer-based extensions had 2.51% average throughput error relative to the vLLM-based system, compared with 6.03% for extensions built on existing simulators.
• On identical workloads, Simthesizer ran up to 284.96× faster than LLMServingSim2.0.
• It also ran up to 23.19× faster than Vidur.
• The framework accommodated new serving features by modifying one composable simulator rather than requiring a separate monolithic implementation for each mechanism.

Why it matters

LLM serving architectures are changing quickly, particularly with agentic workflows and disaggregated serving. A simulator designed for agent-driven extension could shorten the delay between the appearance of new serving mechanisms and researchers' ability to evaluate them without costly full-system deployments.

Caveats

The reported accuracy is based on throughput against a vLLM-based reference system; the abstract does not establish fidelity for latency tails, memory behavior, failures, network effects, or other serving stacks. The speed figures are maximum improvements rather than averages, and the provided material does not specify the breadth of feature requests, workloads, hardware configurations, or how often agent-generated extensions required human correction.