The authors compared training algorithms across nine deep learning workloads, including a new decoder-only language-modeling task using FineWeb-Edu 10B.
Schedule-Free AdamW leads optimizer benchmark without workload-specific tuning
Across nine workloads on fixed hardware, one universal Schedule-Free AdamW configuration delivered the best aggregate wall-clock time-to-target.
Big Tech
Priya Kasimbeg (Alice) · Ahmed Khaled (Alice) · Aahlad Chandrabhatta (Alice) · Yifan Wang (Alice) · Andy Liu (Alice) · Niccolò Ajroldi (Alice) · +6 more
Research Digest··2 min read
Kasimbeg et al.
Why this paper
From Google and 5 others
In one line
Schedule-Free AdamW ranked first among 13 self-tuning submissions on AlgoPerf V1, without per-workload tuning.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§