Training a large model splits it across many graphics processors, which must regularly exchange data. The usual path first copies data into flat communication buffers; those copies cost time and use computation units the model needs. In one common setup, this overhead is noticeable in every step.
New operations read and write tensors where training already stores them, moving bytes with copy engines, remote direct memory access, and switch hardware instead of computation units. They also take compact half-precision gradients and accumulate sums in higher precision, which standard libraries lack.
On a large cluster, they cut training step time by up to 27.1%, and by 6.0% over an earlier optimized path. Memory use drops and interference with concurrent computation is far smaller, keeping large-model training faster.