Simthesizer introduces an agent-driven framework for building and extending LLM serving simulators. It uses a composable dynamic DAG abstraction to represent the entire serving workflow, and a harnessed that translates natural language feature requests into executable simulation code under guardrails and validation.
The authors show that extensions built with Simthesizer follow a real vLLM-based system with 2.51% average throughput error, compared to 6.03% for existing simulators. Simthesizer also simulates up to 284.96x and 23.19x faster than LLMServingSim2.0 and Vidur, respectively.
This is interesting because it addresses the gap between rapidly evolving LLM serving systems and slower human-driven simulator development. By automating simulator extension, Simthesizer enables faster exploration of emerging mechanisms like agentic workflows and disaggregated serving without invasive rewrites.