World Action Models (WAMs) for robot manipulation suffer from high latency due to multi-step action denoising and sequential video/action expert execution. RealtimeWAM introduces two techniques to address these bottlenecks. Teacher-Anchored Consistency Distillation (TACD) enables accurate one-step action generation by combining local consistency constraints with a teacher endpoint anchor. Cross-Expert Wavefront Pipelining (CEWP) overlaps video and action expert execution through block-wise KV cache sharing, eliminating unnecessary waiting.
On benchmarks like LIBERO and RoboTwin, RealtimeWAM achieves 12.2 ms inference latency and 14-25x speedup on H100 GPUs while maintaining less than 1% accuracy loss compared to multi-step counterparts. This makes real-time WAM deployment practical, previously requiring over 300 ms per inference call. The method works across model variants like Fast-WAM and Faster-WAM.