The fourth-root thing

This post is partly because I think the result is interesting and partly to see if anyone will tell me an original reference.

---

Suppose we get $\hat{\beta }$ by solving $U(\beta ;\alpha )=0$ and that $\alpha$ is a nuisance parameter we plug into the equation. Assume that for any fixed $\alpha$, $$E\left[U\right({\beta }_0;\alpha \left)\right]=0.$$ Assume $$U(\beta ,\alpha )=\frac{1}{n}\sum _{i=1}^n{U}_i(\beta ,\alpha )$$ and that $U$ converges pointwise (and in mean, assuming finite moments) to its expected value. Also assume enough other regularity that this leads to $$\sqrt{n}(\hat{\beta }-\beta )\stackrel{d}{\to }N(0,{\sigma }^2(\alpha \left)\right).$$

Examples include GEE with $\alpha$ as the working correlation parameters, and raking with $\alpha$ as the imputation model and calibration parameters, and stabilised weights with $\alpha$ as the stabilising model parameters.

Now, suppose we have an estimator $\hat{\alpha }$ whose limit in probability exists; we’ll call it ${\alpha }^{\ast }$. With enough regularity to differentiate under the expectation $$\frac{\partial }{\partial \alpha }{E\left[U\right({\beta }_0;\alpha \left)\right]|}_{{\alpha }^{\ast }}=0=E\left[{\frac{\partial }{\partial \alpha }U({\beta }_0;\alpha )|}_{{\alpha }^{\ast }}\right]$$ As the derivative has zero mean, the law of large numbers says $${\frac{\partial }{\partial \alpha }U({\beta }_0;\alpha )|}_{{\alpha }^{\ast }}=o_p\left(1\right)$$ and the central limit theorem says $${\frac{\partial }{\partial \alpha }U({\beta }_0;\alpha )|}_{{\alpha }^{\ast }}=O_p\left(n^{-1/2}\right)$$ On the other hand, the derivative with respect to $\beta$ does not have mean zero, so it is $O_p\left(1\right)$. In a parametric model it would be the average per-observation observed Fisher information.

A Taylor series expansion about $({\beta }_0,{\alpha }^{\ast })$ gives $$\begin{array}{cc}U(\hat{\beta },\hat{\alpha })=U({\beta }_0,{\alpha }^{\ast })=& U({\beta }_0,{\alpha }^{\ast })+(\hat{\alpha }-{\alpha }^{\ast })\frac{\partial }{\partial \alpha }U({\beta }_0;{\alpha }^{\ast })\\ & +(\hat{\beta }-{\beta }_0)\frac{\partial }{\partial \beta }U({\beta }_0;{\alpha }^{\ast })\\ & +O_p(\parallel \hat{\alpha }-{\alpha }^{\ast }{\parallel }_2^2)+O_p(\parallel \hat{\beta }-{\beta }_0{\parallel }_2^2)\end{array}$$ If $\hat{\alpha }-{\alpha }^{\ast }=o_p\left(n^{-1/4}\right)$ then the second, fourth, and fifth terms are $o_p\left(n^{-1/2}\right)$ so $$U(\hat{\beta },\hat{\alpha })=U({\beta }_0,{\alpha }^{\ast })=U({\beta }_0,{\alpha }^{\ast })+(\hat{\beta }-{\beta }_0)\frac{\partial }{\partial \beta }U({\beta }_0;{\alpha }^{\ast })+o_p\left(n^{-1/2}\right)$$ Under the standard smoothness/moment assumptions we can rearrange to $$\hat{\beta }-{\beta }_0={\left[\frac{\partial }{\partial \beta }U\right({\beta }_0;{\alpha }^{\ast }\left)\right]}^{-1}U({\beta }_0,{\alpha }^{\ast })+o_p\left(n^{-1/2}\right)$$ so the distribution of $\hat{\beta }$ depends on $\hat{\alpha }$ only through ${\alpha }^{\ast }$. ◼️

For most purposes the fourth-root condition doesn’t really matter: if you have a fixed finite-dimensional parameter that you can estimate at all, you can probably estimate it at root-$n$ rate, and if your parameters are infinite-dimensional or growing in size with $n$ you need to worry about more than just powers of $n$ in remainders. However, if you needed root-$n$ convergence you’d worry that low efficiency would be a problem in sub-asymptotic settings, which is less of a worry if you know fourth-root consistency is enough.

I worked this argument out for the GEE case, back when I was a PhD student, but I certainly wasn’t the first person to do so. I have been told that the first person to come up with the fourth-root part of it was Whitney Newey, which would make sense, but I don’t have a reference. If you know that reference or any early (mid 90s or earlier) reference, I’d like to hear about it.

The Biometrika GEE paper in 1986 has the essential idea that ${\partial }_{\alpha }E\left[U\right({\beta }_0,\alpha \left)\right]=0$, but it assumes $n^{1/2}$ consistency for $\alpha$. Also, some people at the time (and since) have been confused by its using ‘consistency’ both for the assumption that $\hat{\beta }$ converges to its true value ${\beta }_0$ and the assumption that $\hat{\alpha }$ converges to something.