State evolution for gradient descent beyond the mean-field scaling
Sourced from the work of Qiyang Han, Xiaocong Xu
§ Problem Statement
Setup
Consider high-dimensional empirical-risk minimization solved by gradient descent in random-design settings where the aspect ratio is allowed to deviate from the proportional mean-field regime. Existing non-asymptotic joint state-evolution guarantees in this line of work are proved under proportional scaling assumptions used in the source, together with its structured response-model conditions.
This setup follows Han & Xu (2025).
Unsolved Problem
Extend a comparable joint state-evolution theorem, with explicit finite-sample error control and time-uniform guarantees up to horizon , to non-mean-field regimes such as and , and identify the corresponding Onsager correction operators/matrices under clearly stated assumptions for those regimes.
§ Discussion
§ Significance & Implications
Interpreting the source discussion as motivation, this problem asks whether the Onsager-corrected dependency structure developed in the mean-field analysis persists, changes form, or breaks down outside that regime, which would help relate the paper's high-dimensional theory to more classical or extreme aspect-ratio limits.
§ Known Partial Results
Han et al. (2025): The paper proves a non-asymptotic joint state-evolution theorem with two Onsager correction matrices in its mean-field proportional regime under its specified modeling assumptions. This entry is treated as open beyond the scope established in the cited source.
§ References
Gradient descent inference in empirical risk minimization
Qiyang Han, Xiaocong Xu (2025)
Annals of Statistics
📍 Section 2.1 (Assumption A1: proportional/mean-field regime of main interest), and Section 2.3 discussion after Theorem 2.3 around Eqs. (2.14)-(2.15), where behavior beyond mean-field/approximate low-dimensional regimes is flagged as different and not covered by the proved theorem.
Primary source; 2025 online publication in Annals of Statistics (preprint available as arXiv:2412.09498).