3 min read

Some failure modes of statistics research talks

Written before #JSM2013 actually starts, so it’s not about your talk there.

Also, this is about deliberate choices by the presenter, and specifically about statistics research talks. 

  1. “The Overgeneralized Beta Distribution”. There is a place for new parametric distributions, but it’s a fairly small place and mostly occupied by distributions derived from underlying substantive knowledge.

  2. “Asymptotics of an uninteresting estimator”. If there were a novel mathematical idea this would be fine, but otherwise we know its asymptotic behavior and roughly why it happens, and we can’t read your notation fast enough anyway.

  3. “A simple mathematical solution to a complex non-mathematical problem” Includes, but is not limited to, straw-man Bayesian/Frequentist talks.

  4. “Small improvements from heroic assumptions”. Yes, you can do second-order Cornish-Fisher expansions, but do you believe the distributional assumptions hold that accurately?

  5. “My model takes five pages!” Predominantly, but not exclusively, a Bayesian problem.  If you’re solving a real problem don’t fill all your slides with model and proposal distributions. If you’re not? Eh.

  6. “Implausible results from inadequate data.” You battled strong confounding, non-classical measurement error, and 90% missing data, and used clever statistical techniques to demonstrate that the conventional wisdom on health and exercise was completely wrong.

  7. “Uninteresting results from inadequate data” You battled strong confounding, non-classical measurement error, and 90% missing data, and and used clever statistical techniques to demonstrate that the conventional wisdom on health and exercise was completely correct.

  8. “I did an analysis.” That’s good for your clients or collaborators, but unless it helps us do one, this isn’t the right venue. 

  9. “Mine is faster than yours” Useful if it’s true and the problem is computation-constrained, but it’s not, and it’s not.

  10. “Small-sample efficiency comparisons” These can’t be comprehensive, so they are only useful when the scope of the real question is very narrow. Is there a reason you know the treatment has exactly the same effect on everyone?

  11. “You need little teeny eyes for reading little teeny print” And I left my opera glasses behind.

  12. “It worked for Dr Ishihara” He was actually trying to make his slides into vision test.

“I did an analysis” is the least annoying of these, since the background is often interesting and the analysis sensible. It’s also one of the few that would be a good talk in the right setting.

My own contribution to #3 is here, but in partial defense (a) it was on a web page, not at a conference, (b) I was a student, and (c) it’s less over-the-top and less incorrect than typical for the genre.

It’s possible that my JSM poster will be a #9 failure, but I think it’s a setting where users actually are computationally constrained and there isn’t an easier way.