Vision

Image and video understanding, multimodal perception, detection, segmentation, 3D understanding

201 articles · page 1 of 5

Vision

Speculative decoding speeds up vision models by 3.58 times with visual-token weighting

Feng et al. introduce a metric of token-level visual dependence and discover that highly visual tokens are rejected much more often in speculative decoding (46% acceptance in the lowest quartile drops to 22% in the highest). They propose MMFlash, which reweights the draft training loss to prioritize these tokens and uses sparse attention for long videos. MMFlash achieves up to 3.58× lossless speedup on multimodal benchmarks, outperforming previous methods.

·3 min
Vision

Splitting 3D scenes into sub-regions boosts detail without extra training memory

The authors propose split-and-scale, a method that increases the detail captured by latent 3D representations at inference without retraining the model. By dividing a scene into sub-regions and applying the same pretrained tokenizer independently to each, the effective latent capacity grows proportionally to the scene's surface area. This allows reconstruction quality competitive with models that exceed single-GPU memory limits, while keeping training efficient.

·2 min
Vision

Training vision tokens for semantic consistency boosts multimodal AI on visual tasks

The authors define semantically inconsistent tokens (SITs) in vision encoders used by multimodal large language models (MLLMs) and show that removing SITs degrades performance less than removing consistent tokens, indicating MLLMs benefit from semantic consistency. They then introduce Semantically-Guided Training (SGT), which adds a loss term that minimizes semantic inconsistency while preserving language alignment, and find it outperforms existing models on vision-centric tasks such as spatial reasoning and object counting.

·3 min
Vision

Video models adapt reasoning to frame evidence, slashing output length 62%

The authors introduce EVAThink, a method that trains a vision-language model to adapt its response mode, direct answer or chain-of-thought reasoning, based on whether the sampled frames provide sufficient evidence. Using only 6,700 curated video-question pairs, the approach consistently improves over the base model and outperforms fixed strategies, achieving higher accuracy than always-reasoning while cutting average output length by roughly 62%.

·2 min
Vision

Weak models can improve image-text alignment with statistical calibration

The authors introduce Permute-then-Adapt (PTA), a framework that uses a weak model to identify missed positive pairs in contrastive learning. By employing permutation testing to estimate a dataset-specific random-pairing distribution, PTA selects auxiliary positives with controlled false positive rates, improving performance on zero-shot classification and cross-modal retrieval without additional validation data.

·2 min
Vision

Vision models fail false-belief tests because they cannot see, not because they cannot reason

The authors show that vision-language models (VLMs) drop over 20 percentage points on false-belief tasks when presented visually rather than as text. They argue this is a perceptual bottleneck: the models fail to extract structured scene state (who was present, where objects were) from pixels, not to reason about beliefs. Ground-truth scene descriptions close the gap across both abstract and naturalistic scenes, while automatic scene parsers, including frontier VLMs, do not.

·3 min
Vision

Image forensics protocol sorts AI edits into retouching, tampering, and full synthesis

Li et al. propose 2CAP, a dual-channel framework that moves beyond binary real/fake classification to distinguish camera-captured, AI-retouched, locally tampered, and fully synthetic images. On a multi-source benchmark, it achieves the best classification performance, notably detecting retouched images where existing methods fail. For evidence generation, it feeds per-channel alignment scores to a frozen vision-language model through a self-refinement loop, improving explanation quality without forensic fine-tuning.

·2 min
Vision

Flash decoding speeds visual planning for robots 25x

The authors introduce VFlash, a method that exploits the structured nature of visual primitives like bounding boxes and trajectories to decode them jointly, rather than token by token. A lightweight diffusion drafter generates an entire primitive in a single forward pass, and the system accepts structurally valid predictions without requiring exact token-level agreement. On 3D point motion forecasting, this yields roughly 25x faster generation with reduced prediction errors compared to autoregressive decoding.

·2 min
Vision

Normalizing Gaussian spacing improves 3D scene reconstruction across scales and viewpoints

The authors identify that existing feed-forward 3D Gaussian Splatting (3DGS) methods use radial distance normalization to canonicalize scene scale, which does not account for variations in spacing between Gaussian centers, causing ambiguous scale supervision across different scenes and camera configurations. They propose Gaussian Spacing Normalization (GSNorm), which uses the median spacing between neighboring Gaussian centers as a normalization factor, providing a consistent spatial reference for Gaussian prediction. The method achieves state-of-the-art performance across a systematic benchmark, with particularly strong gains when scene scale or viewpoint varies substantially.

·3 min
Vision

Video reconstruction can certify optimal estimates of camera and scene motion

Han and colleagues present a system for reconstructing a scene’s changing three-dimensional shape from video while estimating how the camera moves. Their central contribution is to make two geometric optimization stages separately certifiable, with experiments across seven benchmarks reporting accurate camera estimates, optimization over thousands of frames, and state-of-the-art rendering of new viewpoints.

·3 min
Vision

Visual puzzle solvers improve by building explicit problem states

The authors introduce Vis2State, a framework that abstracts visual puzzles into explicit, revisable problem states, then generates programs to solve them. By keeping the problem representation separate from the solving code, the system can independently correct errors in either perception or reasoning. On four visual reasoning benchmarks, Vis2State improves accuracy by 8.1 to 15.5 percentage points over direct chain-of-thought prompting.

·2 min
Vision

AI that identifies radiological findings first can diagnose unseen diseases

The authors introduce Diagnosis by Composition (DbC), a framework that first learns to recognize radiological findings from images by aligning features with text embeddings, then combines these findings to diagnose diseases per published guidelines. On multi-phase liver CT, DbC significantly outperformed all comparators in detecting and localizing diseases not seen during training, including rare diseases from external institutions.

·3 min
Vision

Training with deceptive visual traps makes AI agents correct their own perception errors

The authors propose MIRA, a training framework that improves multimodal reasoning by exposing vision-language models to deliberately deceptive visual contexts. By injecting perturbed visual descriptions and using a group-wise reflection trigger, the model learns to detect and correct its own perceptual mistakes, achieving a 7% average improvement on in-domain benchmarks without additional human data.

·3 min
Vision

Video AI answers 3D questions better when it predicts how objects move

Vision-language models answer questions about video from 2D pixels, which makes it hard to judge how objects move and relate in 3D over time, especially when the camera moves too. The authors train a model to predict an explicit, time-ordered 4D state sequence first and to use the changes between timesteps, alongside what it sees in the video, as evidence for its answer. It reaches the highest overall accuracy among evaluated models on the authors' held-out benchmark and improves over its base model on an existing 4D benchmark.

·3 min
Vision

AI reading 3D scenes gives steadier answers when the camera moves

The authors document a failure mode they call viewpoint-induced semantic drift: show a multimodal language model the same object in the same 3D scene from a different camera angle and its response about that object weakens, blurs or lands on the wrong region. Their fix keeps pretrained features untouched and adds a gated, geometry-predicted rotation to small groups of feature channels, which they report improves 3D captioning, grounding, detection and spatial reasoning over a matched baseline that simply adds geometry features.

·3 min
Vision

Grouping splats into boxes with stored order speeds up web 3D viewers 4.5 times

The authors present AbacusGS, a method for rendering 3D Gaussian Splatting scenes without per-frame per-splat sorting. They group splats into octree boxes of up to 128 splats, store the blending order per box along an axis derived from training cameras, and fine-tune the scene to adapt to this order. At render time, they sort only the visible boxes and use a dot product per box to decide whether to read forward or backward. In an OpenGL viewer with CPU ordering, AbacusGS achieves a 4.5 times speedup over per-splat sorting while losing only 0.13 dB PSNR compared to an equally fine-tuned baseline.

·3 min
Vision

Correct image correspondences often in top-k but missed by top-1

The authors show that for modern pretrained visual features, the correct semantic correspondence between images of the same object category frequently appears among the top-ranked candidates but is not selected by greedy top-1 matching. They propose a test-time reranking module that first adaptively selects a high-recall candidate set and then scores each candidate by how well it cycles back to the original keypoint. The method boosts three state-of-the-art baselines consistently across multiple datasets without retraining or modifying the feature extractor.

·3 min
Vision

Video super-resolution sharpens and runs 10x faster via sub-pixel frame fusion

The authors present a continuous video super-resolution method that fuses low-resolution frames directly onto a high-resolution grid at their predicted sub-pixel motion positions, unifying temporal interpolation and upscaling into a single operation. On GoPro and Adobe240 benchmarks, it improves reconstruction fidelity and perceptual quality over the state of the art while running 6 to 11 times faster.

·3 min
Vision

Language models shown red hearts can favor code that calls itself

Tartaglini et al. tested whether vision-language models (VLMs) share human visual communication abilities by probing how simple symbols affect model behavior on unrelated tasks. They found that a red heart strongly steers models toward recursive solutions on simple coding problems, and this influence is absent from the model's explicit reasoning. Further experiments in communication games revealed a gap between how models generate and interpret visual meaning, and their visual associations only partially align with human intuition.

·3 min
Vision

AI models can become more confident while relying less on video

The authors investigate training dynamics of video reinforcement learning with verifiable rewards (RLVR) and uncover a shortcut amplification phenomenon: models increasingly favor text-prior answers even when unsupported by visual evidence. Through analysis of predictive uncertainty, they identify a 'perceptual uncertainty collapse' where uncertainty reduction from visual input diminishes during training. They then propose BSPO, a self-distillation framework that reweights advantage signals to promote visually grounded reasoning, showing improvements in video reasoning and visual faithfulness.

·2 min
Vision

AI models reason better about images without enlarging their core networks

The authors introduce V-LUMEN, an external visual memory for vision-language models that uses a frozen codebook to discretize visual features, spatial aggregation for region-aware keys, and text-conditioned hashing for query-dependent retrieval. On five reasoning benchmarks, it improves a frozen Qwen3-VL-2B backbone and transfers zero-shot to unseen domains. At a matched parameter budget, it matches LoRA and MoE adapters while keeping computation constant.

·2 min