When you ask an AI to draw a picture as an SVG file, you need to know if it actually did what you asked. Existing tests often just check if the right objects are present, but miss details like whether the objects are in the right place or have the right colors.
The researchers built a two-part test: one with pairs of SVGs where the better one is known by design, and another with pairs judged by humans. They also created a new automated judge that breaks the prompt into a list of requirements, checks each one with a visual question-answering model, and then adjusts scores when a missing object affects later requirements.
Their judge reached 0.880 accuracy on human-judged pairs, matching how often human annotators agreed with each other. The nine existing methods, including large language models acting as judges, were less accurate and often failed on specific requirements like spatial relationships.