This paper introduces an interaction-centric framework for two-finger gripper manipulation that uses a parameterized universal gripper abstraction to create a canonical gripper-frame representation. The method combines a VLM for subtask inference, SAM 2.1 for mask tracking, and hybrid features from artificial potential fields and segmented point clouds, with a Flow-Matching Transformer to predict 7-DoF action chunks.
Using only 50 demonstrations and 30 million parameters, the approach is the first to achieve simultaneous zero-shot cross-embodiment and cross-viewpoint transfer under compounded perturbations like sim-to-real gap, dual-arm to single-arm transfer, camera shifts, and object variations. It matches in-domain performance of end-to-end baselines in simulation while generalizing to distinct, heterogeneous robot platforms.