Robotics & embodied AI

Robot learning, manipulation, navigation, vision-language-action models, embodied agents in the physical world

191 articles · page 2 of 4

Robotics & embodied AI

Quadrotors learn to fly through moving gaps using differentiable simulation

Bianco et al. demonstrate that a quadrotor can learn to traverse a moving narrow gap end-to-end, from visual and inertial inputs to thrust and body-rate commands, using differentiable simulation without a teacher, planner, or distillation. In simulation, the policy reaches a 99.3% success rate after a few minutes of training on a single GPU, and it transfers directly to a real quadrotor with an 80% success rate over 10 physical trials.

Robotics & embodied AI

Future 3D motion prediction boosts robot learning with fewer demonstrations

The authors introduce Trace-WAM, a robot learning approach that augments future-video prediction with auxiliary 3D motion prediction to focus on manipulation-relevant dynamics. This yields higher success rates on standard benchmarks and real-world tasks, with the largest gains under data scarcity and distribution shifts. The inference cost remains identical to prior methods that do not predict future motion.

Robotics & embodied AI

Contact-preserving retargeting improves robot task success from 13% to 83%

The authors present CARD, a retargeting framework that explicitly preserves contact force targets from human demonstrations, bypassing the need for object models or simulators. By converting data from video, tactile gloves, and motion capture into a unified representation with contact forces, CARD maintains contact on 43–78% of frames, far surpassing kinematic methods. On hardware, a tactile tracking controller raises open-loop task success from 13% to 83% over position-only replay.

Robotics & embodied AI

Dexterous hands match grippers on many tasks, but precision tasks remain difficult

The authors present DexSuite, a simulation framework for dexterous manipulation built on Genesis. It includes 30 tasks, 10 arms, and 12 end-effectors, along with a benchmark of 16 task-embodiment pairs and a dataset of 4,700 successful demonstrations. Policies for dexterous hands matched or outperformed gripper policies on several tasks, but precise tool use and sequential manipulation remained challenging.

Robotics & embodied AI

Replacing language-style decoding cuts robot trajectory errors by 60%

The authors show that autoregressive token decoding is the primary bottleneck for trajectory prediction in compact vision-language models (VLMs) for robotic manipulation. They replace this with a geometry-compatible head that directly regresses continuous coordinate values, reducing trajectory error by over 60% and improving affordance mean average precision by 85% relative to the autoregressive baseline.

Robotics & embodied AI

Robots can anticipate physical changes without generating videos of the future

The authors introduce PDRVLA, a Vision-Language-Action (VLA) model with a Physical Dynamics Module that learns compact physical tokens from intermediate vision-language features. The module is trained by predicting future RGB, depth, and optical-flow features, with auxiliary supervision from a frozen pretrained video world model. On four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin 2.0, and 81.5% on SimplerEnv, demonstrating effective physical reasoning without running the video generator during control.

Robotics & embodied AI

Robots learn from human videos to handle unfamiliar tasks and environments

The authors introduce ThinkWorld, a framework for vision-language-action models that grounds action reasoning in world dynamics. By representing actions as a Chain-of-Dynamics (CoD) of discrete motion tokens and using reinforcement learning from predictions of future observations, they enable robots to learn from both labeled robot demos and abundant unlabeled human videos. This achieves effective human-to-robot skill transfer and strong generalization to unseen tasks.

Robotics & embodied AI

AI camera tracking halves position errors compared with motion sensors alone

The authors introduce Tight Learned Event-Inertial Odometry (TLEIO), a pipeline whose learned front-end, EventsFormer, predicts short-window relative translation and per-axis uncertainty from voxelized event streams in a single forward pass. These constraints are fused with IMU propagation in a stochastic-cloning extended Kalman filter, producing metric 6-DoF trajectories without explicit feature tracks or pose optimization at inference. On the TartanAir benchmark, TLEIO reduces average absolute trajectory error from 2.26 m to 1.10 m over the IMU-only TLIO baseline and achieves up to 48.9% lower runtime than optimization-based event-odometry methods.

Robotics & embodied AI

Robots can answer questions about changing scenes from previously unseen viewpoints

The authors introduce GEMS, a persistent episodic memory built on 3D Gaussian Splatting (3DGS) that efficiently maintains a scene's history through localized updates. An agentic reasoning pipeline renders novel views across space and time to answer questions about object states and motions. On a new benchmark (GEMS-Bench, 473 QA pairs across 15 scenes), GEMS outperforms baselines in 3D object grounding and spatiotemporal QA while matching dedicated scene-update methods in rendering quality.

Robotics & embodied AI

Hand-held skill interface enables portable data collection for dexterous robot hands

The authors present DexPort, a hand-held interface for collecting demonstrations for high-degree-of-freedom robot hands. Operators move the device to specify wrist motion and select from a small set of skills to coordinate fingers, while the interface records native commanded joint targets and measured states. Behavior cloning policies trained on these demonstrations completed four real-world dexterous tasks and outperformed a state-as-action baseline.

Robotics & embodied AI

Sensorimotor alignment predicts robot task success offline

The authors propose Sensorimotor Alignment (S^2), a training-free offline metric that quantifies the alignment between a model's sensory (vision and language) representations and end-effector trajectories. Across vision-language-action models, raw vision-language models, and world action models in simulation and on real robots, higher S^2 scores correlate positively with downstream task success, offering a low-cost proxy for model selection before expensive closed-loop evaluation.

Robotics & embodied AI

World models calibrated to hidden physics via short interaction clips

The authors propose PhysiCalWorld, a method that calibrates action-conditioned world models to instance-specific dynamics by prompting them with a short exploratory interaction clip. The clip is compressed into a compact physics descriptor token that grounds subsequent predictions in the actual physical behavior of the scene. Without requiring physics labels, the method substantially improves rollout prediction accuracy and visual planning success across diverse environments.

Robotics & embodied AI

Unified world action model jointly learns perception, prediction, and planning

The authors introduce P3-WAM, a unified world action model that trains 3D interaction reconstruction, action-conditioned world prediction, and joint action-future planning in one architecture. Unified training improves all three capabilities over single-mode training, and the model outperforms specialized baselines on hand reconstruction and action-conditioned video prediction across four datasets while producing goal-directed planning videos with stronger instruction adherence.

Robotics & embodied AI

Hierarchical skill-based framework achieves high precision in robotic surgery by separating semantic decisions from low-level control

The authors propose SurgiCLAW, a hierarchical agent that confines a multimodal large language model (MLLM) to intraoperative visual understanding and atomic-skill selection, delegating physical execution to verifiable deterministic control programs. On a brain-machine-interface implantation task, the system achieved 88.0% task success rate in 50 closed-loop real-robot trials, with 94.6% skill-selection accuracy and 93.0% macro-averaged F1 on a fixed test set.

Robotics & embodied AI

Humanoid repositions bulky objects using its arms and torso

Mc Gartoll and colleagues introduce Whole-Body Dexterity, a framework that treats a humanoid’s arms and torso as coordinated contact surfaces rather than relying only on its hands. The system generates feasible body-object grasps, trains reinforcement-learning experts to move between them, then distills those experts into a shared controller that achieved a reported 90 percent success rate across real-world tasks.

Robotics & embodied AI

LLM orchestrator persistently coordinates robot teams in dynamic disasters

Mendoza et al. propose a framework for using a large language model (LLM) to persistently orchestrate heterogeneous multi-robot teams in continuous, time-varying missions. They introduce three mechanisms that prevent prompt growth, reduce token usage, and maintain vigilance. In a simulated urban search-and-rescue with spreading fire and moving evacuees, their system rescues 98% of civilians at one-sixth the token cost of the strongest baseline.

Robotics & embodied AI

Separating exploration and exploitation policies with classifier-free guidance improves diffusion RL fine-tuning

The authors introduce CFG-Policy, a plug-and-play framework that decouples exploration and exploitation into two separate diffusion or flow policies combined via classifier-free guidance during action generation. For online RL, a frozen exploration branch sustains diversity while an exploitation branch learns from rewards; for offline-to-online RL, a frozen pretrained branch retains capabilities while a trainable branch steers behavior. The method improves final returns and convergence speed on MuJoCo control tasks and success rates on RoboTwin manipulation tasks.

Robotics & embodied AI

DexCast learns real-world dexterous control from human videos without teleoperation

Gao, Zhang, Wang and colleagues present DexCast, a framework that transfers dexterous manipulation skills to a five-fingered robot hand using only monocular human RGB-D videos. The authors show that combining 4D hand-object reconstructions with a diffusion policy conditioned on a pretrained hand-motion perception module removes the need for teleoperated demonstrations and supports zero-shot cross-scene deployment.

Robotics & embodied AI

Context scheduling improves long-horizon robot video prediction

Hao Wu and colleagues from multiple institutions introduce ReCAP, a training-free context-scheduling method for autoregressive robot video world models. By preserving the initial scene anchor and a contiguous window of recent visual-action blocks while evicting older generated history, ReCAP improves visual fidelity, action consistency, and computational efficiency across four robotics benchmarks, reducing LPIPS by 63.3% on BridgeV2 and KV cache by 73.3%.

Robotics & embodied AI

Cross-video human demonstration pairs improve robot world-action model transfer

The authors introduce WATCH, a cross-video pre-training objective that learns transferable world-action representations from task-paired human demonstrations. By conditioning on one demonstration and predicting another's actions and future frames, the model learns task-relevant interactions that transfer across scenes and embodiments. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy, with gains largest on unseen levels.

Robotics & embodied AI

Systematic study reveals what matters for world modeling in robot VLAs

The authors present the first systematic, controlled empirical investigation of world modeling in Vision-Language-Action models (VLAs). By categorizing existing approaches into four integration paradigms and evaluating them with diverse world representations and policy backbones, they find that 2D semantic representations (DINOv3, SigLIP) provide consistent improvements, the decoupled MoT Future Expert paradigm offers strong out-of-domain robustness, and world modeling yields clear benefits on long-horizon and dynamic tasks.

Robotics & embodied AI

Action-aware embeddings retrieve better robot experiences for future actions

Liu and colleagues trained a vision-language-action embedding model that represents both a robot’s present state and the future continuation of prior experience as searchable vectors. On a new retrieval benchmark, native actions substantially improved matching, while retrieved demonstrations and evolving task-specific memory increased success for several frozen or conditioned robot policies.