Quantify and characterize the efficiency gap induced by convex-loss restriction for non-log-concave errors

Let $p \in \mathbb{N}$ be fixed, and suppose we observe i.i.d. pairs $(X_i,Y_i) \in \mathbb{R}^p \times \mathbb{R}$, $i=1,\dots,n$, from the linear model

where $\beta_0 \in \mathbb{R}^p$ is unknown, $X_i$ is independent of $\varepsilon_i$, $\Sigma_X := \mathbb{E}[X_iX_i^\top]$ is positive definite, and $\varepsilon_i$ has density $f$ on $\mathbb{R}$. Assume regularity conditions ensuring standard $M$-estimation asymptotics (e.g., $\mathbb{E}\|X_i\|^{2+\delta}<\infty$ for some $\delta>0$, integrability and stochastic equicontinuity conditions, and nondegeneracy of curvature terms).

For a convex loss $\rho:\mathbb{R}\to\mathbb{R}$, write $\psi=\rho'$. Assume $\psi$ is absolutely continuous (so $\psi'$ exists a.e.), $\mathbb{E}[\psi'(\varepsilon_i)]\in(0,\infty)$, and $\mathbb{E}[\psi(\varepsilon_i)^2]<\infty$. For Fisher consistency/identification, assume $\mathbb{E}[\psi(\varepsilon_i)]=0$ and that $\beta_0$ is the unique root of $\mathbb{E}[\psi(Y-X^\top b)X]=0$. Define the convex $M$-estimator

Its asymptotic covariance matrix is

Assume $f$ is known. Let $s_f(u):=\frac{d}{du}\log f(u)$ and $I_f:=\mathbb{E}[s_f(\varepsilon)^2]$ (finite, positive Fisher information for location). The oracle maximum-likelihood estimator

has asymptotic covariance

For the class $\mathcal{C}_{\mathrm{cvx}}(f)$ of convex losses $\rho$ satisfying the above conditions, define the best convex relative efficiency

which, when $\rho$ is twice differentiable with integrable $\rho''$, is equivalently

The matrix ratio is interpreted in Loewner order (equivalently here as a scalar ratio since both covariances are multiples of $\Sigma_X^{-1}$).