The authors study Adam under L₀-Lₚ generalized smoothness, where changes in the true gradient can grow with the gradient’s magnitude.
Adam Converges Beyond Global Smoothness Using Only Second Moments
The authors prove high-probability convergence for generalized smooth objectives without assuming bounded or sub-Gaussian stochastic gradients.
Research Lab
Ruinan Jin · Difei Cheng · Ling Chen · Jun Luo · Hao Zhou · Youzhi Zhang
Mohamed bin Zayed University of Artificial Intelligence · Aerospace Information Technology University · The Ohio State University · JD.com, Inc. · Centre for Artificial Intelligence and Robotics
Research Digest··3 min read
Jin et al.
Why this paper
From Chinese Academy of Sciences and 6 others
In one line
Adam converges under generalized smoothness with only second-moment stochastic gradients, no strong tail assumptions needed.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§