The sandwich and the t-test

As every schoolchild know, you can derive the Student $t$-test as a linear regression with a single binary predictor. How about the Welch/Satterthwaite unequal-variance $t$-test?

We have a technique for handling linear regression with unequal variances in the responses, the ‘model-agnostic’¹ or ‘model-robust’ sandwich estimator. You might wonder what happens if you use the sandwich estimator on a linear regression with a single binary predictor.

Let $X$ be binary, coded so it has zero mean (so that it’s orthogonal to the intercept) and fit a linear model with $Y$ as the outcome and $X$ as the predictor: $$E\left[Y\right]=\alpha +\beta X.$$

We know $\hat{\beta }$ is the difference in mean between the two groups. The sandwich variance estimator for $(\hat{\alpha },\hat{\beta })$ is $$\left(X^{T}X{)}^{-1}\left(\sum _{i=1}^n{x}_i{x}_i^T\right(y_i-{\hat{\mu }}_i{)}^2)\right(X^{T}X{)}^{-1}$$ First, note that the two outer matrices are diagonal, because of the centering of $X$, so that we need only consider the $(\beta ,\beta )$ component.

We can break the inner sum into sums over the two groups. Within each group, $x_i{x}_i^T$ is constant, so it can be taken out of the sum. Write $x_{\left(0\right)}$, $x_{\left(1\right)}$ for the two values of $X$; $n_0,n_1$ for the two sample sizes; and $S_0$, $S_1$ for the standard deviations of $Y$ in the two groups. Then $$\left(\sum _{i=1}^n{x}_i{x}_i^T\right(y_i-{\hat{\mu }}_i{)}^2)=x_{\left(0\right)}^2(n_0-1)S_0^2+x_{\left(1\right)}^2(n_1-1)S_0^2$$

Next, note that $x_{\left(0\right)}$ and $x_{\left(1\right)}$ can be determined from $n_0$ and $n_1$: we have $x_{\left(1\right)}-x_{\left(0\right)}=1$ and $n_1{x}_{\left(1\right)}+n_0{x}_{\left(0\right)}=0$, giving $x_{\left(1\right)}=n_0/n$ and $x_{\left(0\right)}=-n_1/n$, so the middle term is $$\frac{n_1^2(n_0-1)}{n^2}{S}_0^2+\frac{n_0^2(n_1-1)}{n^2}{S}_1^2.$$

In the outside of the sandwich the $(\beta ,\beta )$ element is just ${\sum }_i{x}_i^2$, which is $$n_1{n}_0^2/n^2+n_0{n}_1^2/n^2=n_0{n}_1/n.$$ Putting these together, the variance is $$\frac{n}{n_0{n}_1}(\frac{n_1^2(n_0-1)}{n^2}{S}_0^2+\frac{n_0^2(n_1-1)}{n^2}{S}_1^2)\frac{n}{n_0{n}_1}=\frac{1}{n_0}\left(\frac{n_0-1}{n_0}{S}_0^2\right)+\frac{1}{n_1}\left(\frac{n_1-1}{n_1}{S}_1^2\right).$$ This is almost exactly the variance for the Welch-Satterthwaite $t$-test, except that it uses $n_i$ rather than $n_i-1$ in the denominator of the individual group variances. Or, writing ${\hat{\sigma }}_i^2$ for the variance estimator in group $i$ using $n_i$ in the denominator it’s just ${\hat{\sigma }}_0^2/n_0+{\hat{\sigma }}_1^2/n_1$.

So, the Welch-Satterthwaite $t$-statistic is basically just a linear regression with a binary predictor and the sandwich variance estimator, just as Student’s $t$-test is a linear regression with a binary predictor and the Fisher-information variance estimator.

We don’t get the degrees of freedom that way. Improving on the Normal reference distribution for $t$-statistics with the sandwich estimator is a bit more complicated.