Theory for Debiased PCR Under General Covariate Models

Let $(x_i,y_i)_{i=1}^n$ be independent observations from the linear model

where $\beta_0\in\mathbb R^p$ is unknown, $x_i\in\mathbb R^p$ satisfies $\mathbb E[x_i]=0$ and $\mathrm{Cov}(x_i)=\Sigma$ (not assumed to be right-rotationally invariant), and $\varepsilon_i$ is independent of $x_i$ with $\mathbb E[\varepsilon_i]=0$, $\mathrm{Var}(\varepsilon_i)=\sigma^2\in(0,\infty)$. Write $X\in\mathbb R^{n\times p}$ for the design matrix (rows $x_i^\top$) and $y\in\mathbb R^n$ for the response vector.

Let the singular value decomposition of $X$ be $X=UDV^\top$, with rank $r$, singular values $d_1\ge\cdots\ge d_r>0$, and right singular vectors $v_1,\dots,v_r$. For a fixed truncation level $k\le r$, define $V_k=(v_1,\dots,v_k)$, $U_k=(u_1,\dots,u_k)$, $D_k=\mathrm{diag}(d_1,\dots,d_k)$, and the rank-$k$ principal components regression estimator

equivalently the least-squares estimator constrained to $\mathrm{span}(V_k)$.

Consider a spectrum-aware debiased PCR estimator of the form

where $M_k\in\mathbb R^{p\times p}$ is a data-dependent matrix built from the empirical spectrum/eigenstructure of $X$ to correct truncation and regularization bias.

For a contrast vector $a\in\mathbb R^p$, define the conditional centering and scale

The