Robotics & embodied AI

Robot learning, manipulation, navigation, vision-language-action models, embodied agents in the physical world

191 articles · page 3 of 4

Robotics & embodied AI

Human demonstrations scale whole-body learning for humanoid loco-manipulation

The Lambda-humanoid Team presents HumanVerse, a 500-hour dataset pairing first-person video with synchronized body and reconstructed hand motion, plus λ₀, a vision-language-action policy trained to convert that experience into humanoid behavior. On Unitree G1 evaluations, increasing whole-body human data consistently improved task progress, and the full system reached 90.0% success on SIMPLE and 62.5% across four real-world loco-manipulation tasks.

·3 min
Robotics & embodied AI

Per-point affordance fields let equivariant policies focus on task-relevant regions

The authors introduce phase-conditioned affordance fields (PAF), per-point scores learned by an SE(3)-equivariant visuomotor policy. These scores serve both as a learned anchor for canonicalizing observations and actions, and as additional per-point features. Across MimicGen tasks and multi-object real-robot manipulation, PAF consistently improves success rates over the underlying policy and baselines.

·2 min
Robotics & embodied AI

Interactive toolkit speeds real-to-sim refinement for robot manipulation tasks

The authors present RoboSimStudio, a toolkit built on the Newton physics engine that makes solver, contact model, and physical parameters explicit and editable. They demonstrate 18 contact-rich manipulation simulations and a pilot study in which coding agents built a bimanual sponge-wiping simulation from a smartphone video in about one hour, then transferred an improved policy to a real robot.

·2 min
Robotics & embodied AI

Benchmark reveals when robotic agents truly learn and when they just retry

Wang et al. introduce SEAR, a framework and benchmark to evaluate whether LM-driven robotic agents genuinely learn from experience or merely benefit from repeated attempts. Across seven frontier LMs and over four years of agent runtime, they find that learning is real but conditional: harder tasks benefit from longer sessions, while easier tasks do not; and code-based skills transfer across tasks and simulators, but task-specific solutions often fail to generalize to novel scenes.

·3 min
Robotics & embodied AI

Large-scale benchmark and end-to-end model achieve robust dexterous grasping in cluttered scenes

The authors introduce OmniDex, a massive simulation benchmark for dexterous grasping in cluttered scenes, created via a scalable seed-and-filter strategy that avoids costly scene-level optimization. They also propose an end-to-end grasping model that uses Soft Winner-Takes-All learning and human-inspired physical constraints, achieving state-of-the-art performance on the benchmark without the latency of post-optimization.

·2 min
Robotics & embodied AI

Counterfactual post-training improves robot world model's action following and physical plausibility

The authors present DreamTrue, a multi-view robot world model that predicts future video frames conditioned on action sequences. They address two data-quality problems: imprecise camera calibration and a lack of recorded failed interactions. With offline geometric calibration and reward-guided counterfactual post-training, DreamTrue achieves state-of-the-art action following on AgiBot and cuts a human-assessed interaction defect rate from 48.12% to 6.25%.

·2 min
Robotics & embodied AI

Agent harness lets a pretrained multimodal LLM navigate any task without fine-tuning

The authors present SuperNav, a navigation framework that equips a frozen pretrained multimodal large language model with an agent harness for tool use, progress tracking, and context management. Without any navigation-specific fine-tuning of the MLLM, SuperNav achieves 78.00% success on single-object navigation and 59.50% on demand-driven tasks, outperforming four baselines. It also reaches 73.33% success on a category-level HM3D benchmark and was deployed on a real quadruped robot.

·2 min
Robotics & embodied AI

Adaptive sampling helps million-environment robot training solve harder tasks

Zhang et al. identify inefficient exploration as a bottleneck in massively parallel robot reinforcement learning: uniform sampling spends substantial experience on situations that are already mastered or currently impossible. Their adaptive sampler instead emphasizes the frontier of a policy’s abilities, enabling difficult locomotion and assembly behaviors without demonstrations or manually ordered curricula.

·3 min
Robotics & embodied AI

Autoregressive video pretraining unlocks the value of long context for real-time robot control

The authors introduce Long-WAM, a framework that scales the context window of causal world-action models for real-time robot control. They find that longer visual history improves success only when the video foundation is pretrained autoregressively: on RoboCasa GR-1, extending context from 0.0 to 19.2 seconds raises success from 63.3% to 78.7%, whereas bidirectionally pretrained models show no gain.

·3 min
Robotics & embodied AI

Latent world models for robotics scale predictably with compute

The authors introduce RoboJEPA, a latent world model based on the Joint Embedding Predictive Architecture (JEPA), and train it on a large-scale multi-embodiment robotic dataset. They demonstrate that the model's latent forward-prediction error obeys a scaling law with training compute, and that this error is strongly correlated with downstream planning performance on real and simulated robots, enabling predictable allocation of resources for model training.

·3 min
Robotics & embodied AI

Better video predictions improve robots’ performance on unfamiliar manipulation tasks

Guo et al. developed VPP2, a 14-billion-parameter world action model that predicts an intended visual future and converts that prediction into robot actions. On open-ended video tests, real-world ALOHA tasks and three post-trained benchmarks, the authors report that improving and preserving video prediction produced more reliable manipulation than competing video models and robot policies.

·2 min
Robotics & embodied AI

One action-conditioned JEPA improves vision-language-action policies across all lifecycle stages

The authors present Juno, a framework that repurposes a single action-conditioned joint-embedding predictive architecture (JEPA) as a control-aligned backbone, a predictive teacher, and an adaptable dynamics model across the vision-language-action (VLA) lifecycle. On SimplerEnv, Juno raises average success from 60.9% to 68.5% over the strongest baseline, and to 72.7% with test-time adaptation. On a real robot, it retains 70-75% success under background, height, and object shifts where the base policy collapses to 0%.

·2 min
Robotics & embodied AI

Depth supervision boosts RGB and geometric consistency in robot world models

The authors introduce a calibration pipeline that recalibrates the DROID dataset into a metric 3D corpus (DROID-3D) and train DepthWorld, a video diffusion world model that jointly predicts multi-view RGB and depth. The central finding is that depth supervision not only provides metric depth but also improves RGB prediction by 1.48 dB PSNR over an identical RGB-only baseline at equal training budget.

·2 min
Robotics & embodied AI

Portable RGB demonstrations can train mobile manipulation policies and reduce base-velocity errors

Yutian Zhang et al. propose VOMMI, a framework that collects portable demonstrations from RGB video of body and hand cameras for mobile manipulation. Their R2-VO module refines offline trajectories using sparse geometric anchors, and the resulting policy, post-trained only on portable demos, achieves 18.2% lower base-velocity error than a policy trained on robot-collected demonstrations, with comparable end-effector accuracy.

·2 min
Robotics & embodied AI

Retrieval replaces generation for building simulation-ready articulated objects from a single photo

The authors present AffordCraft, a method that constructs a physically valid articulated simulation asset from a single RGB image and a task instruction by retrieving and adapting a matching entry from a library of pre-built assets, rather than estimating parts and joints from scratch. On 2,000 photographs across 31 categories, AffordCraft yields a valid asset for 85% of inputs, compared to at most 45% for five generative methods, while using 10–78× less GPU time per valid asset.

·3 min
Robotics & embodied AI

Caching frozen model states makes forward-only robot adaptation much faster

Kim and colleagues present VLA-ZO, a forward-only method for adapting vision-language-action models to deployment shifts using one target-environment demonstration. On LIBERO tasks with altered camera viewpoints, the method made zeroth-order optimization up to 32.54 times faster than a naive implementation, while raising average task success from 48.27 percent without adaptation to as high as 63.58 percent.

·2 min
Robotics & embodied AI

Flexible block-causal models sustain efficient long-horizon action-conditioned rollouts

Khorrambakht and colleagues introduce FLEX-WAM, a joint generative model of states and actions designed for efficient, open-loop simulation over long horizons. By combining block-causal attention with training that explicitly preserves sensitivity to commanded actions, the model produces stable rollouts for thousands of steps and supports planning entirely through imagined trajectories.

·2 min
Robotics & embodied AI

Stereo module lifts 2D vision-language-action models to 3D for robot manipulation

The authors introduce ExStereo, a stereo module that adds 3D perception to pre-trained 2D Vision-Language-Action (VLA) models. By reconstructing metric depth from stereo images, rendering multi-view observations, and using an attention mechanism to condition actions on 3D features, ExStereo significantly boosts performance across simulation and real-world tasks, outperforming RGB, point-cloud, and implicit-stereo baselines.

·3 min
Robotics & embodied AI

Offline stitching of robot demos boosts policy success by 21 percentage points

The authors present NEEDLE, an offline dataset-augmentation algorithm that stitches together useful segments from multiple robot demonstrations by adding short, verified action bridges. Using only RGB images and proprioception, NEEDLE identifies connections that bypass detours, combine complementary behaviors, and recover from failures. On real-robot tasks, it improves success rate over the strongest baseline on each task by an average of 21 percentage points.

·3 min
Robotics & embodied AI

Human-like scripted demonstrations reproduce behavioral cloning’s puzzling failures

Park and Levine introduce OCBench, a simulated robotic manipulation benchmark designed to reproduce behavioral cloning phenomena that usually appear only with costly human-collected data. By generating narrow, history-dependent and smoothly varying trajectories, the benchmark reproduces four reported anomalies, including benefits from overfitting and action chunking, while allowing candidate explanations to be tested under controlled conditions.

·2 min
Robotics & embodied AI

Inverse dynamics helps visual world models retain unstable control modes

Toso and colleagues identify a control failure hidden by standard JEPA training: accurate next-step prediction and anti-collapse regularization can coexist with representations that erase unstable but controllable dynamics. They propose endpoint inverse dynamics, which reconstructs an action sequence from the initial and final latent states, and prove that exact reconstruction preserves every state direction reachable within the training horizon.

·2 min
Robotics & embodied AI

Synthetic views make robot policies more tolerant of pose errors

Wistreich and colleagues introduce an offline augmentation framework that converts demonstrations recorded at one canonical robot pose into physically consistent examples at perturbed poses. Across simulated humanoid and bimanual tasks, plus experiments on a real Galaxea R1 Pro, policies trained with these examples were more robust to unseen base-position and camera-viewpoint shifts than the tested baselines.

·3 min
Robotics & embodied AI

Filtered critic gradients speed reinforcement learning for flow-based robot policies

Kim and colleagues introduce QF3, an online, off-policy reinforcement learning algorithm for flow policies, which generate robot actions through an iterative learned transformation. The method combines standard flow-matching training with gradients from a critic, while filtering out action updates deemed unreliable. The authors report a 10-fold wall-clock speedup over the on-policy FPO++ method and zero-shot transfer of simulated humanoid policies to hardware.

·2 min
Robotics & embodied AI

One Model Predicts Robot Futures and Chooses Actions

Syed et al. present SUAVE, a transformer that represents language, video frames, and robot actions as discrete tokens in one sequence. By changing which tokens are masked during inference, the model can predict future video, select actions, or generate visual subgoals together with actions. Across simulated and physical manipulation tasks, it competed with specialized models while benefiting substantially from video without action labels.

·3 min
Robotics & embodied AI

One Contact Skill Transfers Dexterous Manipulation Across Robot Hands

Liu and colleagues introduce GOTT, a dexterous manipulation framework that executes robot-agnostic object trajectories using a reusable contact-acquisition skill. In simulation and real-world experiments, the authors report that this shared policy established stable contact across multiple objects and hands, including previously unseen hand morphologies, while improving task success over open-loop grasp execution.

·2 min
Robotics & embodied AI

Wiring details determine whether affordance heads help or hurt VLA policies

The authors systematically ablate how an affordance head is injected into a vision-language-action policy. They show that a stop-gradient is mandatory—without it, the policy collapses. With the backbone protected, the dominant factor is the bridge initialization: a randomly initialized residual bridge (allowing immediate head influence) yields 96.2% success on LIBERO, compared to 93.1% without the head.

·3 min
Robotics & embodied AI

Guided by failures, choosing initial states improves robot learning per supervised episode

The authors introduce Mulligan, a data-collection strategy for batch-online robot learning that picks each round's initial states from observed failures and untried regions. Across three real-world and two simulated tasks, Mulligan beats uniform initial-state sampling at matched budgets, and with a value function that reranks policy samples improves final real-task success by 10 to 34 percentage points. Operator-assisted collection completes 98% of episodes, so learning does not stall task completion.

·3 min
Robotics & embodied AI

Control objectives, not reconstruction quality, should guide representation design in world-action models

Through controlled comparisons, the authors demonstrate that reconstruction fidelity alone is insufficient for choosing representations in world-action models, and that perceptual features benefit from calibration for dynamics modeling. Their proposed method, ReWAM, achieves 93.6% success on RoboTwin 2.0 and an average score of 12.29 on RoboDojo without relying on generative video pre-training, by using frozen DINO features processed through feature calibration and a temporal representation bottleneck shaped by action-loss gradients.

·2 min
Robotics & embodied AI

Humanoid tracker expands executable manipulation skills from sparse human demonstrations

Zhang et al. combine heterogeneous human-object motion capture, robot-specific retargeting and physics-based imitation to train one humanoid controller for dexterous whole-body tasks. Their central result is a data flywheel: successful variations of existing interactions become new references, progressively expanding the range of executable motions without changing the underlying task.

·2 min
Robotics & embodied AI

Human demonstrations teach humanoids coordinated whole-body manipulation without teleoperation

Chen and colleagues present a pipeline for converting egocentric human demonstrations into training data for coordinated humanoid loco-manipulation, where locomotion, balance and manipulation must work together. With 200 human demonstrations per task, their policies achieved a mean score of 51.67% across four real-world tasks, without using robot demonstrations from those target tasks.

·2 min
Robotics & embodied AI

Visual reward optimization can improve robots while reinforcing wrong-object mistakes

Luo et al. tested learned visual rewards in a closed loop, optimizing diffusion-based robot policies while independently measuring requested-task success and wrong-object failures using simulator state. On a drawer-opening task, optimizing Robometer increased success by 10.2 percentage points but also increased wrong-object failures by 10.9 points, a defect hidden by otherwise strong aggregate reward-model performance.

·2 min
Robotics & embodied AI

Recurrent memory policy improves robot manipulation on tasks requiring spatial recall, counting, and timing

Vanjani et al. introduce ReCAT, a language-conditioned policy for memory-dependent robot manipulation. The policy uses a recurrent memory built from Mamba-2 layers and a causal attention layer, alongside separate pathways for current observations. On standard benchmarks, ReCAT achieves 95.3% average success on LIBERO and 62.4% on RMBench, with the best or tied-best result on six of nine tasks. On three real-robot tasks probing spatial recall, event counting, and interval timing, the best variant reaches 66.7% average success, compared to 8.3% for the strongest short-history baseline.

·3 min
Robotics & embodied AI

Robot policies generalize better when future tokens remain at inference

Zhou and colleagues compare world action models that either generate future video during inference or discard future representations to save computation. Although both approaches perform similarly on familiar tasks, removing future conditioning harms robustness, data efficiency and transfer to unseen tasks. Their Simple-WAM retains fully noised future tokens for one forward pass, recovering most of the benefit at substantially lower cost than iterative video generation.

·2 min