2 min read

Simulations and modes of convergence

We often have theory that says n(θ^nθ)dN(0,σ2),
and then do simulations to see how well the asymptotic approximation applies. After doing so, we often present tables of the empirical mean and standard deviation of θ^n. This doesn’t make a lot of sense.

Knowing that n(θ^nθ)dN(0,σ2) doesn’t tell us anything about the moments of θ^n for any finite n. Convergence in distribution does not imply convergence in mean. For example,  θ^n could be maximum likelihood estimates in a logistic regression model. These  have no finite moments for any finite n, because they are infinite with positive probability.

However, the asymptotic result does tell us that the quantiles of θ^n, suitably scaled, should converge to those of the approximating Normal distribution. The median of θ^n should converge to θ; the MAD of θ^n (after scaling by n) should converge to σ, and the probability that (θ^n1.96×se^[θ^n],θ^n+1.96×se^[θ^n]) includes θ should converge to 95%.

Knowing how good the asymptotic approximation is for these quantile-based statistics is usually sufficient to tell us if the approximation is useful in practice. We usually don’t care about the mean of θ^n in any substantive way, since if we did, we’d be more worried when it didn’t have one.

I think the usefulness of ‘robust statistics’ gets oversold a lot, but this really is a case where it’s meaningful and true to say that the median of an approximately-Normal variable is a better summary of the location parameter than the mean is.  We should be presenting robust (ie, weakly continuous) summaries of the results of simulations motivated by asymptotics unless there’s a specific reason to care about moments.