As you know, rank tests can be non-transitive: they can have the rock-paper-scissors property. Tests that are for a single real-valued summary statistic (eg a test comparing means or medians or variance) are always transitive, because they are just comparing a single number, and ordering on numbers is transitive.
The converse is almost obviously almost true: if you have a transitive test, it almost has to be a test for a single real-valued summary statistic. The test gives you an ordering on distributions, so there’s a one-dimensional summary statistic in some ordered set saying where you are in the ordering. In any sane situation that one-dimensional summary statistic can be chosen to be real-valued.
What gets a little tricky is the definition of ‘sane’. Econometricians have been looking at this problem since at least the 1960s, and there’s one simple pathological case. Suppose your test statistic uses the lower quartile, and then breaks ties using the upper quartile. That is, you have a two-dimensional summary statistic and you flatten it into one dimension by using the second value just to break ties. This doesn’t fit into the real numbers: the ordering has uncountably many copies of the real numbers, because the second number can be anything, for any value of the first number. You can’t replace the pair of numbers with a single number.
A relatively recent (2002) paper by Alan Beardon and co-workers characterises all the things that can go wrong. There are four of them
- The two-dimensional ordering, as above
- Just Too Long: when you keep finding bigger and bigger elements you have to go on not just a countably infinite number of times, but an uncountably infinite number of times before you bound the whole set.
- A weird thing called Aronszajn chains
- A truly weird thing called a Souslin line
The first two are standard counterexamples in topology: the unit square under the dictionary ordering on points, and the long line.
Aronszajn chains need a delicate construction involving orderings on infinite sets of rational numbers. Not only is the whole ordering not representable by real numbers, not even any uncountable subset is representable.
It’s impossible to say anything about useful what Souslin chains look like; their existence or otherwise is not decidable by basic axioms of set theory (ZFC\(\pm\)CH)
Because the counterexamples are weird, it’s hard to come up with intuitive conditions that rule them out. However, I found a book by Bridges & Mehta (and found that Douglas Bridges is just down the road in Hamilton,ok, at least in the same country), with basically the result I need (Theorem 4.6.3)
If \(\preceq\) is a continuous, countably bounded, total preorder on a path-connected topological space, there is a continuous order isomorphism into \(\mathbb{R}\).
Mathematicians can be useful, sometimes, even if it’s just to solve problems basically created by mathematical formalism.