Vision

Image and video understanding, multimodal perception, detection, segmentation, 3D understanding

201 articles · page 2 of 5

Vision

AI models can reuse layers while retaining image and text abilities

The authors introduce recursive transformer backbones for CLIP where a small set of blocks is applied multiple times, reducing distinct parameters while keeping computational depth constant. They show that such recursive models can be effectively trained via distillation from a conventional CLIP teacher, retaining much of its transferable ability on zero-shot classification, cross-modal retrieval, few-shot learning, and visual grounding.

·3 min
Vision

Brittle visual cues can mislead neural networks on unmodified images

Zhang and colleagues show that subtle visual patterns can hinder, rather than help, standard image classifiers on clean images. Comparing standard and robustly trained models on ImageNet’s validation set, they identify failures that depend on precise combinations of fine-scale image content and spatial arrangement. Small transformations disrupt individual failures, but leave their overall prevalence approximately unchanged.

·2 min
Vision

Camera software certifies best-fit position and orientation estimates under moderate noise

The authors propose PnP*, the first practical solver that directly optimizes the Maximum Likelihood Estimation (MLE) for Perspective-n-Point (PnP) and returns a certifiably globally optimal solution. By reformulating PnP as a quadratic program and deriving a convex semidefinite relaxation that is provably exact under moderate noise, they achieve a stable and scalable solver that runs at over 10 Hz.

·2 min
Vision

AI image classifiers can share combined updates with little accuracy loss

The authors sum low-rank adaptation updates trained separately for the same image classification task on different pretrained Vision Transformer backbones. They find that the resulting superposed LoRA improves all 56 backbone-task pairs by a mean 30.21 percentage points over the unadapted backbone, only 0.28 points below individually trained solo LoRAs. This suggests that each frozen backbone implicitly selects its own component from the sum, enabling adapter reuse without cross-model alignment.

·3 min
Vision

AI reconstructs hand movements more accurately from single-camera, first-person video

Chen et al. introduce EgoHand, a monocular 3D hand reconstruction model trained on 14 million images from synthetic, in-the-wild, and multi-camera motion capture sources. The model achieves state-of-the-art zero-shot performance on HOT3D, ARCTIC, and H2O benchmarks, with inference under 10 milliseconds, and is favored in a blinded user preference study over Apple EgoDex and Ropedia Xperience-10M.

·2 min
Vision

AI can trace objects’ outlines and flag when they’re absent

Tianming Liang, Junyi Pan, Yuxi Li and colleagues at Tencent present WeVisOne-VL, a vision-language model that unifies object-level and pixel-level perception in one interface. Built on Qwen3-VL at 2B and 4B scales and trained on roughly 100 million annotated images, the model performs detection, referring localization, counting, OCR, segmentation and matting. Across more than 30 benchmarks it leads on both discrete and dense prediction tasks while rejecting queries when no target object is present.

·2 min
Vision

BlazeVGGT speeds up VGGT 13x by selecting representative views without retraining

The authors introduce BlazeVGGT, a training-free plug-in for VGGT that sparsifies cross-view attention at the frame level. By analyzing camera tokens, they found that shallow layers capture local view topology, while deeper layers encode global geometry. Using a greedy determinantal point process to select complementary views, BlazeVGGT achieves up to 13x faster inference at 2000 frames with a lower Chamfer distance (0.41 to 0.32).

·3 min
Vision

OACap turns video captioning into active, tool-guided observation

Shi et al. propose OACap, an agent that treats an evolving video caption as a testable account, distinguishing gaps that can be revised from existing evidence from gaps that require targeted observation via tools such as cropping, dense sampling, and OCR. Across five backbones and three video-understanding benchmarks, OACap consistently improves caption-based question answering, and the authors introduce FSCap-Bench with two new metrics to diagnose factual and semantic preservation.

·2 min
Vision

VLMs mostly err by ignoring visual evidence they possess

The researchers propose a method to decompose VLM errors into two types: those where necessary visual information is missing (representation failure) and those where it is encoded but not used (utilization failure). Using a lightweight rank-based probe (VELens) on over 6,000 images, they estimate that utilization failures account for the vast majority of errors, suggesting that improving how VLMs use existing visual information is more urgent than improving visual encoding.

·2 min
Vision

Unified visual encoder delivers state, language, and reconstruction for world models

The authors pretrain a ViT-B/16 from scratch using language similarity, pixel reconstruction, and spatial correspondence objectives, then refine it with cross-view self-distillation. On fifteen-encoder benchmarks, ALPS achieves the lowest LIBERO state-probe error among ViT-B models, matching the 3.5x larger V-JEPA 2-L, while retaining strong language alignment and enabling pixel-perfect reconstruction. Frozen ALPS tokens further achieve 85.5% mean policy success on LIBERO with a native text tower.

·2 min
Vision

Balancing within-person and between-person variation improves clothes-changing re-identification

Researchers from Reichman University and Microsoft introduce MINER, a model-agnostic training framework that explicitly balances intra- and inter-person variance for Clothes-Changing Person Re-Identification (CC-ReID). By stochastically mining hard positive and hard negative examples and adaptively controlling the variance trade-off, MINER improves retrieval across four CC-ReID benchmarks and ranks among the top three in all three tracks of the TVRID 2026 challenge.

·2 min
Vision

Vision and language representations may never fully align regardless of model scale

The authors develop a theoretical framework showing that current vision and language representations are linearly unalignable due to fundamental differences in the underlying data, not model scale. They show that this persistent misalignment leads to a specific generalization failure and correlates with existing failure modes in vision-language models (VLMs). The findings challenge the strong version of the Platonic Representation Hypothesis, which predicts that representations should converge with sufficient scale.

·3 min
Vision

Vision-language alignment fails to improve text-only spatial reasoning

Singh et al. test whether vision-language models (VLMs) form truly multimodal text representations that aid tasks with latent visual structure. They construct a shape-counting task that is trivial with images but difficult from text coordinates. They find that VLMs perform no better on the text version than their language-only counterparts, but a self-distillation method that aligns text embeddings with image embeddings yields substantial gains.

·2 min
Vision

ReVision adapts frozen image generators to perception tasks with per-instance parameter updates

The authors introduce ReVision, which connects a frozen multimodal understanding model to a frozen image generator through a trainable parameter generator. For each image and natural-language instruction, it synthesizes transient low-rank weight updates that tailor the generator's behavior for dense prediction. ReVision matches state-of-the-art performance on segmentation, depth estimation, and surface-normal prediction without modifying the pretrained backbones.

·2 min
Vision

View-decoupled geometry states improve predictive 3D world modeling

The authors propose GeoState, a view-decoupled geometry representation that disentangles scene content from camera viewpoint when using pretrained geometry foundation model (GFM) features as predictive states. By aggregating multi-view GFM features into a shared scene state and querying it only at decoding time, the model allows a temporal predictor to focus on genuine scene evolution. GeoState outperforms existing 3D world models on Waymo in depth forecasting, supports geometrically consistent cross-view queries, and generalizes zero-shot to KITTI and nuScenes.

·3 min
Vision

Privileged motion and gravity cues improve feedforward 3D reconstruction without extra test-time cost.

Qian, Ren and Bai augment feedforward transformer models for multi-view 3D reconstruction by supplying optical flow and gravity direction as privileged information during training only. Evaluated on standard benchmarks, the models show consistent gains in accuracy and robustness, especially for dynamic objects, ambiguous structure and challenging camera motion. The optimal injection stage is architecture-dependent: middle-layer integration suits VGGT, while late-stage integration suits π3.

·2 min
Vision

WorldRoPE encodes patch tokens as 3D Gaussians to ground multi-view attention

Mijeong Kim, Kyujin Lim, and Bohyung Han introduce WorldRoPE, a positional encoding that models each image patch as volumetric Gaussian primitives in a shared world frame. At every Transformer layer, the rotary operator is integrated in closed form over where key tokens may appear in the query view, attenuating uncertain locations without rasterizing features. The method improves over image-space and ray-based encodings across three multi-view geometry tasks.

·3 min
Vision

Distribution-aware Gaussian labels enable training-free open-vocabulary 3D segmentation

Zeng et al. propose DIGS, a training-free framework for open-vocabulary segmentation of 3D Gaussian Splatting scenes. Instead of committing each Gaussian to a single semantic feature, DIGS stores a compact top-M label distribution updated by a tempered Dirichlet rule, and renders via late argmax. On LERF-OVS it reaches 71.95% mean mIoU in 1.0 to 3.2 minutes per scene, outperforming both the training-free SFS and the training-based LaGa baselines.

·3 min
Vision

Training-free framework turns single photos into interactive, movable 3D scenes

The authors propose Unravel, a training-free framework that converts a single image into an interactive 3D scene where each object is a separate movable asset. Instead of generating entangled geometry or predicting all objects at once, Unravel iteratively removes fully visible objects, restores occluded regions using image editing, and reconstructs each asset from its best-observed snapshot before reassembling the scene in reverse order. Experiments on diverse indoor scenes show it recovers more objects with more faithful placement and improved physical plausibility compared to existing feed-forward and agentic baselines.

·3 min
Vision

Anchor-based 3D Gaussians cut primitives tenfold without losing quality

The authors present Anchor3DGS, a pose-free feed-forward 3D Gaussian Splatting method that replaces the standard pixel-aligned prediction of one Gaussian per pixel with a compact anchor-based representation. Guided by information entropy, anchors are placed adaptively in 3D space, each decoding into a small set of local Gaussians. This decouples the Gaussian count from input resolution and view number, yielding state-of-the-art rendering quality on multiple benchmarks with roughly ten times fewer primitives.

·3 min
Vision

Different inference prompts recover rare classes suppressed during VLM finetuning

Ben Baruch and colleagues study vision-language models finetuned on datasets where a few classes dominate and others have scarce examples. They find that evaluating with the same descriptive prompt used during finetuning can suppress rare-class recognition, sometimes below zero-shot performance. Using a different descriptive prompt shifts the model toward knowledge retained from pretraining and can recover rare-class recall.

·2 min
Vision

Motion capture from video works with any target skeleton topology

The authors present CAST, a framework that recovers motion from a single monocular video onto a user-supplied target skeleton, even when that skeleton's topology—joint count, branching, and rest pose—differs from the subject in the video. It introduces GRACE, a module that corrects parent-to-child rotation propagation using reliability-gated alignment with predicted global bone directions. On matched-rig and cross-skeleton benchmarks, CAST reduces MPJPE by 27.6–62.5% and 54.7% respectively over a strong baseline, and a causal streaming variant exceeds 51 FPS.

·3 min
Vision

One model matches pixels across any pair of five sensor modalities

The authors present TartanMatch, a single network that predicts dense pixel correspondence for any ordered pair among RGB, event, thermal, depth, and projected LiDAR. On zero-shot real-world benchmarks it reduces endpoint error by 61.9% for cross-modal pairs and 49.6% for same-modal pairs relative to strong baselines. Joint training across all 25 modality pairs further improves accuracy over pair-specific models.

·2 min
Vision

Generative framework converts sparse panoramas into annotated multi-room 3D scenes

The authors present Holi-Spatial 360, a pipeline that transforms sparse 360-degree panoramas from real-estate listings into photorealistic, semantically annotated 3D environments. Key innovations include Correspondence-Guided Geometric Optimization to establish reliable cross-view geometry and adapted generative priors for view densification. The resulting Holi360 dataset, with over 1,000 scenes and 2 million spatial question-answer pairs, substantially improves the spatial reasoning capabilities of fine-tuned vision-language models.

·3 min
Vision

Mixed-supervision method improves mammography segmentation for rare lesions

Dai et al. propose a two-branch framework for mammographic lesion segmentation that jointly trains on pixel-level masks for a few base lesion categories and image-level labels for a broader set of categories. A Lesion-wise Attentive Classifier uses category-specific feature prototypes to focus on lesion-relevant regions, reducing background interference. On multiple mammography datasets, the method consistently outperforms open-vocabulary baselines on both base and novel categories, narrowing the gap to fully supervised models.

·3 min
Vision

Many contrastive training strategies for multimodal embeddings add little value

Li et al. systematically evaluate 12 contrastive training strategies for universal multimodal embedding models under a unified framework, finding that most offer no meaningful downstream gains and several degrade performance. Using a novel MLLM-as-a-Judge diagnostic, they identify three failure modes and show that jointly integrating hard-negative mining, false-negative mitigation, and hard-batch construction yields a 6.0% average improvement on the MMEB benchmark.

·2 min
Vision

Consumer XR headsets can drive personalized full-body avatars in real time

The authors present FlowAvatar, a system that reconstructs personalized look-alike avatars in real time on consumer XR hardware from sparse headset-mounted sensors. It tracks full-body pose, hands, face, and gaze, and achieves state-of-the-art temporal consistency on the AMASS benchmark. Two user studies show that tracking completeness and personalized appearance play complementary roles, and only their combination yields positive total embodiment.

·2 min
Vision

Multimodal models learn when to look and how long to reason

Chen et al. propose L2T, a framework for allocating multimodal reasoning between visual processing and textual chains of thought. It converts intermediate outputs from visual tools into discrete tokens, allowing a model to reproduce tool-like visual computation without executing external tools at inference. Across perception and reasoning tasks, L2T reportedly preserved competitive performance while delivering 1.7 to 3.6 times higher throughput than explicit tool-calling methods.

·2 min
Vision

Using a model's own past errors improves visual in-context learning

The authors propose SMILE, a visual in-context learning method that constructs counterfactual demonstrations from a vision-language model's own past prediction errors. By selecting hard negatives whose mistaken predictions are semantically similar to the query's likely error, SMILE outperforms state-of-the-art counterfactual selection across four benchmarks and four VLMs, with an average gain of +3.09 points and 5× lower latency.

·3 min
Vision

Progressive depth growth improves vision transformers with less compute

The authors systematically investigate progressive depth growth in self-supervised vision transformers (MAE and DINOv3). They show that inserting layers during pretraining improves downstream performance on classification, segmentation, and depth estimation while reducing training FLOPs by 13.9–24.8%. Grown models also exhibit altered attention patterns, with MAE developing recurring global-local alternations and DINOv3 showing earlier local-to-global progression.

·2 min