Overparameterized optimal subsample size for infinite-ensemble subagging

Assume the homogeneous subagging setup studied in Koriyama et al.: i.i.d. data $(x_i,y_i)_{i=1}^n$ in proportional asymptotics ($n,p\to\infty$ with $p/n\to\gamma\in(0,\infty)$), with Gaussian design $x_i\sim N(0,I_p/p)$, linear signal-plus-noise response $y_i=x_i^\top\theta+z_i$, and finite second moments for signal/noise. Base learners are regularized M-estimators with convex differentiable loss and convex penalty, trained on uniform subsamples of size $k$, and the full-ensemble estimator is the $M\to\infty$ limit (conditional subsample expectation).

Let $\mathcal R_{n,p}(\lambda,k)$ denote squared prediction risk of the full-ensemble estimator, and let $k^\star_{n,p}(\lambda)\in\arg\min_{1\le k\le n}\mathcal R_{n,p}(\lambda,k)$.