Bayesian Surprise — the Shiny app

I wrote a while back about a toy case of the Bayesian surprise problem: what does Bayes Theorem tell you to believe when you get really surprising data. The one-dimensional case is a nice math-stat problem, if you like that sort of thing, but maybe you’d rather have the calculations done for you.

Here’s an app

The mathematical setup is that you have a prior distribution for a location parameter $\theta$ centered at zero, and you see a data point $x$ that’s a long way from zero. If $\pi \left(\theta \right)$ and $f(x-\theta )$ are the prior and likelihood, the posterior is proportional to $\pi \left(\theta \right)f(x-\theta )$.

When the prior is heavy-tailed and the data distribution isn’t, you’re willing to believe $\theta$ can be weird, so a very large $x$ means your posterior for $\theta$ will be near $x$. When the data distribution is heavy-tailed and the prior isn’t, you’re willing to believe $x$ can be a long way from $\theta$, but not that $\theta$ can be a long way from zero, so the prior ends up pretty much like the posterior – you ‘reject’ the data.

The details, though, depend on how heavy-tailed things are, and the app lets you play around with a range of possibilities. Laplace–Laplace and $t_{30}$–$t_{30}$ and $t_{30}$–Normal might be interesting.

The code is here