Vision

Image and video understanding, multimodal perception, detection, segmentation, 3D understanding

201 articles · page 3 of 5

Vision

Video world model uses test-time training as persistent memory for long-range consistency

The authors introduce World-T3, a camera-controlled video world model that maintains persistent scene memory through test-time training (TTT). By writing pose-keyed visual features into a fixed-size fast-weight memory that is updated with a recursive least-squares rule, the model avoids spatial memory decay over long rollouts. On a long-horizon revisit benchmark, World-T3 raises PSNR from 13.52 to 18.31 dB and reduces absolute rotation error from 3.344 to 1.075 relative to the strongest baseline, while sustaining 20.6 FPS at 1280×704 on a single NVIDIA B200 GPU.

Vision

Bifocal pruning keeps fine-grained accuracy while using quarter of visual tokens

The authors evaluate ten visual token pruning methods across three multimodal LLM families and find that strong average performance masks severe drops on fine-grained tasks. They propose Bifocal, a training-free, two-stage pruner that allocates separate budgets to query-focused and image-intrinsic evidence plus a coarse pooled context. With roughly a quarter of the tokens, Bifocal is the only method to retain over 90 percent of unpruned performance in every ability group across all models.

Vision

Generative framework reconstructs complete large-scale 3D meshes from unposed photos

The authors introduce WorldRecon, a framework that reconstructs complete, large-scale 3D scene meshes with PBR materials from unposed multi-view RGB images. It uses coarse point clouds and camera poses from a reconstruction model as a scaffold, then generates each spatial block with a native 3D generative model conditioned on local point-cloud crops and image observations. On indoor benchmarks, it outperforms prior methods, reducing Chamfer distance by 14.9% and 43.1% while improving F-score by 17.3% and 12.0% on synthetic and real scenes.

Vision

MLLMs miss text readable by squinting; diagnosis and training method recovers it

The authors define Squint Text: text embedded in an image's coarse structure that humans can read by squinting but multimodal LLMs (MLLMs) often report as absent. They introduce SquintBench (1076 production moderation images) and SquintCtrl (synthetic suite) for diagnosis, identifying spatial selection and high-frequency interference as key bottlenecks. Their training method, Squint OPSD, uses a frozen teacher with a downscaled view and ground-truth transcription to supervise a student on the original image, achieving 88.61 ANLS versus 26.50 for the base model without requiring auxiliary views at inference.

Vision

Long skip connections help diffusion transformers learn cleaner visual representations

Meng and colleagues investigate why Diffusion Transformers, or DiTs, can generate strong images while producing weaker visual representations than U-Net diffusion models. They report that adding long skip connections improves the quality of intermediate activations used for vision tasks, and propose that these connections let diffusion noise bypass the representation-forming core of the network.

Vision

Generative and agentic pipeline reconstructs scene-aligned 3D objects from casual photos

GATOR reconstructs complete, textured 3D objects and their scene-relative pose from one or a few casual images. The authors combine a generative cascade with agentic refinement: the cascade produces an initial asset, which anchors a multimodal agent that edits geometry and texture against the input observations. Results show strong fidelity on synthetic objects, cluttered tabletops, and indoor scenes.

Vision

Visual attention sinks in VLM decoders are layer-dependent and can be steered

The authors analyze decoder attention in vision-language models and discover that early and late layers consistently attend to the same image regions regardless of the prompt, while middle layers shift attention meaningfully. They propose SAGE, a lightweight intervention that uses pretrained vision backbones to generate region-of-interest masks and steers attention away from these prompt-invariant sinks, improving visual grounding and reducing hallucinations across several VLM architectures.

Vision

Zero-shot framework uses visual geometry priors for multiview RGB-D registration

The authors introduce PointVGGT, a zero-shot framework for multiview RGB-D registration that replaces the conventional pairwise-then-global paradigm with a foundation-then-refinement approach. Using visual geometry foundation model priors, they directly recover metrically consistent global poses and achieve robust alignment without any training or finetuning, outperforming both classical and learning-based methods across indoor, object-centric, and outdoor datasets.

Vision

Training-free method repairs object hallucination in LVLMs instead of suppressing it

The authors propose ResOT, a training-free, inference-time method that mitigates object hallucination in large vision-language models by repairing, rather than suppressing, hallucination-related hidden representations. It aligns each token state toward a faithful distribution via Gaussian optimal transport in a localized residual subspace. Across three LVLMs, ResOT reduced hallucination while improving caption quality and multimodal performance.

Vision

Relaxing the Attention-FFN split in Vision Transformers can boost small model performance

The authors introduce AttenFeed, a module designed to subsume the functions of both Attention and Feed-Forward Network (FFN) layers, and use it to build uViT, a Vision Transformer with no strict Attention-FFN separation. Comparing uViT against standard ViTs across datasets and model scales, they find that the conventional alternating structure imposes a rigid parameter allocation that actively hinders performance at smaller model scales.

Vision

Discrete part concepts from frozen self-supervised backbones match base accuracy without labels

The authors introduce DisParQ, a method that learns spatially grounded, discrete part concepts from a frozen DINOv2 backbone without class labels or language supervision. On ImageNet linear probing, DisParQ achieves 83.2% top-1 accuracy, within 0.2 percentage points of its teacher, and its concepts are more consistent across images than language-aligned models. The method also enables cross-category part retrieval, outperforming self-supervised baselines on a new benchmark.

Vision

Theory and experiments show when splitting large images into views helps VLMs find small objects

The authors analyzed why vision-language models (VLMs) miss small objects, deriving a theory based on two quantities: S (visual tokens across an object's side) and L (content per call). They found fundamental token budget limits and proved that splitting an image into overlapping views is safe (no recall loss) if no view zooms out relative to the whole image and views overlap by at least one object. Their experiments, involving about 177,000 requests on 797 images across 11 models, largely confirmed the theoretical predictions, with only a few failures for certain OpenAI models given extra pixels.

Vision

Geometric decomposition of VLM representations improves visual grounding without retraining

The authors introduce GeoPID, a training-free framework that decomposes multimodal representations in vision-language models (VLMs) into redundant, modality-unique, and synergistic components using geometric relationships between visual and textual subspaces. By selectively amplifying vision-unique representations during inference, they achieve an average relative accuracy gain of 7.63% across 22 VLMs and 14 benchmarks. The method requires no parameter updates and improves visual grounding in tasks that demand visual evidence.

Vision

Vision-language models detect object rotation but misjudge its magnitude

Wang and colleagues evaluated 12 vision-language models on eight tasks requiring them to detect, estimate and reason about object rotations across views. Models recognized whether rotation occurred, but magnitude estimates remained near chance and exact-angle answers collapsed onto 0°, 90° and 180°. The authors show that some coarse rotation information is present inside model representations but is not effectively used during answer generation.

Vision

Model predicts full 6-DoF motion at every pixel from monocular video

The authors present MoSE3, the first feed-forward model to predict dense per-pixel SE(3) (3D rotation and translation) from monocular RGB video. By decomposing SE(3) into two easier intermediates — 3D point tracks and rigidity embeddings — and recovering the full transform via a differentiable closed-form fit, the model achieves state-of-the-art performance on SE(3) estimation and 3D point tracking benchmarks, despite being trained solely on synthetic data.

Vision

Confidence-guided complementary evidence pool improves multimodal retrieval-augmented generation

The authors propose CLIMB, a training-free inference-time framework for multimodal retrieval-augmented generation. It first builds a complementary evidence pool using an MMR-style objective to balance relevance and redundancy. Within this fixed pool, it iteratively refines answers using an R/E/C critic for scoring and a confidence estimator that accepts updates only when confidence rises. On Encyclopedic-VQA and InfoSeek, CLIMB consistently improves over baselines, with ablations confirming each component's contribution.

Vision

World model forecasts future 3D geometry directly without recursive rollouts

Noori et al. propose GeoWM, a world model that predicts future scene geometry directly in explicit 3D space rather than in pixel or latent space. The model extracts geometric features from observed RGB frames using a geometry foundation model, then uses a horizon-conditioned flow-matching diffusion transformer to forecast depth, camera pose, and 3D point clouds at a specified future time without autoregressive recursion. On four datasets spanning driving, aerial flight, and robotic manipulation, GeoWM outperforms existing world models by 4 to 58 percent in depth error at short horizons and 7 to 48 percent at long ones, while reducing inference time by about 98 percent.

Vision

Multi-perspective self-verification improves vision-language model answer reliability

Zhu et al. systematically analyze how verifier capability and prompt design affect self-verification in vision-language models (VLMs), finding that stronger verifiers are more reliable and no single prompt dominates across tasks. They propose MOTIVE, which evaluates candidate answers from multiple verification perspectives and learns a correctness-aligned reliability score to guide accept-or-rethink decisions, consistently outperforming strong self-verification and self-correction baselines.

Vision

Aligning partial point clouds to frozen global 3D encoders without labels

Adeniran-Lowe et al. show that state-of-the-art 3D encoders, pretrained on globally reconstructed scenes, fail catastrophically on partial viewpoint-dependent point clouds (drop from 72.47 to 2.57 mIoU), primarily due to coordinate-frame mismatch rather than missing geometry. They propose PAGER, a parameter-efficient, label-free adaptation that aligns partial-view features with a frozen global semantic representation using only paired partial/global geometry. On ScanNet partial views, PAGER achieves 65.46 mIoU without any semantic labels, outperforming supervised fine-tuning baselines, and in zero-shot transfer to ScanNet++ it surpasses a fully fine-tuned Sonata encoder (53.93 vs. 48.09 mIoU).

Vision

Content-based frame identities unify 3D reconstruction and point tracking

Fradlin et al. present ARROW, a feed-forward model that reconstructs scene geometry and tracks 3D points using arbitrary collections of RGB observations, including videos, static images and multiple camera streams. Its central mechanism assigns each observation a content-derived identity rather than a sequence position, allowing queries to address source, target and reference images without imposing temporal order. The authors report state-of-the-art results on two 3D tracking benchmarks while retaining competitive reconstruction performance.

Vision

Continuous preference fields improve aesthetic crops beyond rigid annotation grids

Zhang et al. argue that standard cropping datasets encode subjective judgments on rigid grids, encouraging models to reproduce a small set of coordinate templates rather than learn flexible composition. They introduce a continuous reward landscape, train a vision-language model against it, and recalibrate existing benchmark labels using the same framework. The authors report improved in-domain accuracy and out-of-domain generalization, although the supplied text does not include the underlying performance figures.

Vision

Confidence scores for generated video intervals enable controllable selection without external verification

Chen et al. introduce a method to add interval-level confidence scores to generative video temporal grounding models. A small scoring head, trained with offline verifier scores and online temporal-overlap labels, lets applications adjust recall-precision trade-offs without rerunning the model or invoking an external verifier. On OMTG-Bench, confidence-based selection raises query-macro Recall@0.5 from 9.95% to 14.42% at a 10% return budget and from 26.48% to 31.12% at a 25% budget, using the same candidate pool.

Vision

Radar rendering that captures Doppler in dynamic scenes by separating sensor blur from geometry

Keidar et al. present DyRAD, a radar novel-view synthesis method for dynamic driving scenes that renders full range-azimuth-Doppler (RAD) tensors. It reconstructs scenes as static background reflectors plus motion-tracked dynamic point reflectors, with velocities derived from object tracks and projected onto the line of sight. On RADIal, DyRAD recovers detections for 90.7% of reference-detected objects versus 26.9% for the strongest baseline.

Vision

Reusing single-view vision model attention yields scalable correspondence matching

Zhang et al. show that query-key projections inside single-view vision foundation models (VFMs) encode reusable cross-view matching priors. They introduce Flow Any Scene Transformer (FAST), built by zero-parameter rewiring of selected self-attention layers into cross-attention in a DINOv3 backbone. Trained on a newly assembled 6-million-pair Flow-6M dataset, FAST achieves state-of-the-art performance across diverse correspondence benchmarks and scales favorably with backbone size and data.

Vision

Grounding foundation model surpasses larger vision-language models on precise perception and robotics

The authors introduce GroundingPI, a 4-billion-parameter grounding foundation model that generates points and bounding boxes as quantized coordinates in a shared vocabulary. Evaluated across 34 benchmarks spanning 11 perceptual capabilities, it sets a new state of the art with an average of 73.68%, outperforming the larger GPT-6 Astra (71.54%). When used as a visual backbone for robotic manipulation and autonomous driving, GroundingPI consistently outperforms all evaluated baselines, achieving relative improvements of up to 24.8% on RoboTwin 2.0 and enabling data-efficient learning on RoboCasa-GR1.