The authors designed a fixed-artifact audit: they generated 300 scenes from six text-to-3D generators and froze them.
Evaluation scores for text-to-3D models shift more with configuration changes than model changes
Audit of 19 alignment evaluators on fixed 3D scenes shows that camera and caption choices can change point-estimate winners for 18 evaluators.
Top University
Anson Y. Lam · Shuqing Li · Michael R. Lyu
The Chinese University of Hong Kong
Research Digest··2 min read
Lam et al.
Why this paper
From The Chinese University of Hong Kong
In one line
Configuration variance exceeds generator variance for 17/19 alignment evaluators in fixed-scene text-to-3D evaluation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§