The authors constructed HumanPhys from eight real-world and rendered video sources, yielding 13 sub-tasks that probe six dimensions of human-centric physical perception: spatial state (depth, inter-person distance, gaze angle), kinematics (displacement, velocity, heading-step trajectory), blink events, deformation (contact-deformation strength), illumination (from body and face cues), and camera ego-motion (with and without a person in frame).
MLLMs struggle to perceive human physical states quantitatively from video
HumanPhys, a benchmark with programmatic labels across six dimensions of human-centric physical perception, shows current multimodal models score below 50% on most sub-tasks.
Chinese Tech
Yiming Zhang · Enzhi Zhou · Junan Lin · Dulan Zhou · Weiqiang Wang · Yuge Huang · +1 more
Nanyang Technological University · Ant Group
Research Digest··3 min read
The authors introduce HumanPhys, a benchmark and training corpus for evaluating multimodal large language models (MLLMs) on physically grounded human perception.
Why this paper
From Ant Group and Nanyang Technological University
In one line
HumanPhys tests MLLMs on six dimensions of human-centric physical perception and finds them challenging.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§