ESTIMATING
DIFFERENCES in RATING

Sections Introduction Likelihood and sprt Rating Distribution Uniform Prior Use of the Distribution Outcome Models Appendix

§ Introduction

to do Five outcomes of a game pair

For a random variable with five possible outcomes, where ${\mathrm{p}}_{\mathrm{i}}$ is the probability of observing outcome i and $\sum {\mathrm{p}}_{\mathrm{i}}=1$, the pentanomial distribution gives the prob­ability $\mathrm{P}(⟨{\mathrm{p}}_1\text{,}\text{. . .}\text{,}{\mathrm{p}}_5⟩\text{,}⟨{\mathrm{n}}_1\text{,}\text{. . .}\text{,}{\mathrm{n}}_5⟩)$ of sampling the random variable n times and observing each of the outcomes ${\mathrm{n}}_{\mathrm{i}}$ times:

where $\mathrm{n}=\sum {\mathrm{n}}_{\mathrm{i}}$.

Given some observations $\mathrm{obs}=⟨{\mathrm{n}}_1\text{,}\text{. . .}\text{,}{\mathrm{n}}_5⟩$ and a model $\mathrm{M}\left(\mathrm{diff}\right)=⟨{\mathrm{p}}_1\text{,}\text{. . .}\text{,}{\mathrm{p}}_5⟩$ that is a function of the rating difference diff, we can consider

§ Likelihood and sprt

The quantity $\mathrm{P}\left(\mathrm{obs}\right|\mathrm{diff})$ is the likelihood, ...

to do Likelihood ratio

to do Sequential probability ratio testing

Sprt requires two hypotheses to compare – that the rating difference is ${𝜗}_0=0$ or some particular $𝜗$ – but you may only care that a patch is better at all.

There is implicitly a uniform prior?

§ Rating Distribution

We may also be interested in finding

where

leaving only the prior $\mathrm{P}\left(\mathrm{diff}\right)$ to be determined.

This is a distribution, where $\mathrm{P}(\mathrm{diff}=\mathrm{x}|\mathrm{obs})$ is the probability that the difference in rating is x given the observed outcomes obs.

§ Uniform Prior

We can use as a prior a uniform distribution

over some interval $\left[\mathrm{a}\text{,}\mathrm{b}\right]$, and in particular, we can set $\mathrm{a}=-\mathrm{L}/2$ and $\mathrm{b}=+\mathrm{L}/2$ for some L so that $\mathrm{P}\left(\mathrm{diff}\right)=1/\mathrm{L}$ over the support.

Then

and L cancels, giving us

which is simply $\mathrm{P}\left(\mathrm{obs}\right|\mathrm{diff})$ normalized to have unit area over the support.

We can then consider the limit

which converges. Then ${\text{lim}}_{\mathrm{L}\to \infty }\mathrm{P}\left(\mathrm{diff}\right|\mathrm{obs})$ converges, and in this sense it is reasonable to talk about a “uniform prior over all the integers”.

It is, of course, not necessary to use a uniform prior, but in the absence of any reason to assume a different prior it is perhaps the best default.

§ Use of the Distribution

to do Conditional distribution function

to do Stopping when the cdf at zero is less than some threshold

The cdf at t is

and you might say “stop when the probability that the rating difference is negative is less than 5%” or whatever.

§ Outcome Models

to do Equivalence of exit advantage and rating difference

to do Trinomial game outcome

to do Pentanomial game pair outcome

Model is perhaps more properly probability of pair outcome given rating difference, book bias, and book drawishness.

§ Appendix

to do Calculating multinomials