Foundation4D uses a shared visual backbone with task-specific decoders to predict camera poses, metric depth, 3D objects, semantic occupancy and Gaussian attributes, a representation used to render scenes from new viewpoints.
One model combines driving perception with scene reconstruction from camera video
Foundation4D shares features across geometry, object detection, occupancy and rendering tasks, with joint training improving selected detection and camera-calibration results.
Big Tech
NVIDIA · : · Sérgio Agostinho · Maximilian Apodaca · Emmanuel Attia · Guillem Braso · +22 more
NVIDIA
Research Digest··2 min read
The authors introduce Foundation4D, a model that processes multi-camera driving video to estimate scene geometry, detect objects, identify occupied space and predict a renderable scene representation.
§