Flying a small drone through a moving window is very hard. It needs to hit a precise spot at exactly the right time. Previous methods required extra tools like a separate planner or a training routine that knows the drone's exact position and orientation in space.
The researchers developed a system that uses a special kind of simulator, called a differentiable simulator. This simulator smoothly calculates how to improve the drone's commands, even from failed attempts. The drone's control policy was trained from scratch to use camera images and motion sensors, directly outputting thrust and rotation commands.
In simulation, the trained policy succeeded 99.3% of the time. When transferred to a real drone, it successfully flew through a moving gap in 80% of physical tests using only its onboard sensors. This shows that end-to-end learning without a separate planner is possible for high-precision flight.