Statistics

Basic Properties

1. $E(X)=\sum xp(x)$
2. $Var(X)=\sum (x-\mu {)}^{2}f(x)$
3. X is around $E(X)$, give or take $SD(X)$
4. $E(aX+bY)=aE(X)+bE(Y)$
5. $Var(aX+bY)=a^{2}Var(X)+b^{2}Var(Y)$
6. $Var(X)=E(X^2)-[E(X){]}^2$
7. $Cov(X_1,X_2)=E(X_1{X}_2)-E(X_1)E(X_2)$
8. if $X$, $Y$ are independent:
   1. $M_{X+Y}(t)=M_X(t)M_Y(t)$
   2. $E(XY)=E(X)E(Y)$, converse is true if $X$ and $Y$ are bivariate normal, extends to multivariate normal

Approximations

Law of Large Numbers

Let $X_1,X_2,\dots ,X_n$ be IID, with expectation $\mu$ and variance ${\sigma }^2$. $\overline{X_n}=\frac{1}{n}\sum _{i=1}^n{X}_i\underset{n}{\overset{\mathrm{\infty }}{\to }}\mu$. Let $x_1,x_2,\dots ,x_n$ be realisations of the random variable $X_1,X_2,\dots ,X_n$, then $\overline{x_n}=\frac{1}{n}\sum _{i=1}^n{x}_n\underset{n}{\overset{\mathrm{\infty }}{\to }}\mu$

Central Limit Theorem

Let $S_n=\sum _{i=1}^n{X}_i$ where $X_1,X_2,\dots ,X_n$ IID. $\frac{S_n-n\mu }{\sqrt{n}\sigma }\underset{n}{\overset{\mathrm{\infty }}{\to }}\mathcal{N}(0,1)$

Distributions

Poisson($\lambda$)

$E(X)=Var(X)=\lambda$

Normal $X\sim \mathcal{N}(\mu ,{\sigma }^2)$

https://kfrankc.com/posts/2018/10/19/normal-dist-derivation

$f(x)=\frac{1}{\sqrt{2\pi }\sigma }exp(-\frac{(x-\mu {)}^2}{2{\sigma }^2}),-\mathrm{\infty }<x<\mathrm{\infty }$

1. When $\mu =0$, $f(x)$ is an even function, and $E(X^k)=0$ where $k$ is odd
2. $Y=\frac{X-E(X)}{SD(X)}$ is the standard normal

Gamma $\mathrm{\Gamma }$

$g(t)=\frac{{\lambda }^{\alpha }}{\mathrm{\Gamma }(\alpha )}{t}^{\alpha -1}{e}^{-\lambda t},t\ge 0$

${\mu }_1=\frac{\alpha }{\lambda },{\mu }_2=\frac{\alpha (\alpha +1)}{{\lambda }^2}$

${\chi }^2$ Distribution

Let $\mathcal{Z}\sim \mathcal{N}(0,1)$, $\mathcal{U}={\mathcal{Z}}^2$ has a ${\chi }^2$ distribution with 1 d.f.

$f_{\mathcal{U}}(u)=\frac{1}{\sqrt{2\pi }}{u}^{-\frac{1}{2}}{e}^{-\frac{u}{2}},u\ge 0$

${\chi }_1^2\sim \mathrm{\Gamma }(\alpha =\frac{1}{2},\lambda =\frac{1}{2})$

Let $U_1,U_2,\dots ,U_n$ be ${\chi }_1^2$ IID, then $V=\sum _{i=1}^n{U}_i$ is ${\chi }_n^2$ with n degree freedom, $V\sim \mathrm{\Gamma }(\alpha =\frac{n}{2},\lambda =\frac{1}{2})$

$E({\chi }_n^2)=n,Var({\chi }_n^2)=2n$

$M(t)={(1-2t)}^{-\frac{n}{2}}$

t-distribution

Let $\mathcal{Z}\sim \mathcal{N}(0,1)$, ${\mathcal{U}}_n\sim {\chi }_n^2$ be independent, $t_n=\frac{\mathcal{Z}}{\sqrt{U_n/n}}$ has a t-distribution with n d.f.

$f(t)=\frac{\mathrm{\Gamma }([(n+1)/2])}{\sqrt{n}\pi \mathrm{\Gamma }(n/2)}{(1+\frac{t^2}{n})}^{-\frac{n+1}{2}}$

1. t is symmetric about 0
2. $t_n\underset{n}{\overset{\mathrm{\infty }}{\to }}\mathcal{Z}$

F-distribution

Let $U\sim {\chi }_m^2,V\sim {\chi }_n^2$ be independent, $W=\frac{U/m}{V/n}$ has an F distribution with (m,n) d.f.

If $X\sim t_n$, $X^2=\frac{\mathcal{Z}/1}{U_n/n}$ is an F distribution with (1,n) d.f, with $w\ge 0$:

For $n>2$, $E(W)=\frac{n}{n-2}$

Sampling

Let $X_1,X_2,\dots ,X_n$ be IID $\mathcal{N}(\mu ,{\sigma }^2)$.

$\text{sample mean, }\bar{X}=\frac{1}{n}\sum _{i=1}^n{X}_i$

$\text{sample variance, }{S}^2=\frac{1}{n-1}\sum _{i=1}^n{(X_i-\bar{X})}^2$

Properties of $\bar{X}$ and $S^2$

1. $\bar{X}$ and $S^2$ are independent
2. $\bar{X}\sim \mathcal{N}(\mu ,\frac{{\sigma }^2}{n})$
3. $\frac{(n-1)S^2}{{\sigma }^2}\sim {\chi }_{n-1}^2$
4. $\frac{\bar{X}-\mu }{S/\sqrt{n}}\sim t_{n-1}$

Simple Random Sampling (SRS)

Assume $n$ random draws are made without replacement. (Not SRS, will be corrected for later).

Summary of Lemmas

• $P(X_i={\xi }_j)=\frac{n_j}{N}$: Lemma A
• For $i\ne j$, $Cov(X_i,X_j)=-\frac{{\sigma }^2}{N-1}$: Lemma B

Estimation Problem

Let $X_1,X_2,\dots ,X_n$ be random draws with replacement. Then $\bar{X}$ is an estimator of $\mu$. and the observed value of $\bar{X}$, $\bar{x}$ is an estimate of $\mu$.

Standard Error (SE)

SE of an $\bar{X}$ is defined to be $SD(\bar{X})$.

Without Replacement

SE is multiplied by $\frac{N-n}{N-1}$, because $s^2$ is biased for ${\sigma }^2$: $E(\frac{N-1}{N}{s}^2)={\sigma }^2$, but N is normally large.

Confidence Interval

An approximate $1-\alpha$ CI for $\mu$ is

$(\bar{x}-z_{\alpha /2}\frac{s}{\sqrt{n}},\bar{x}+z_{\alpha /2}\frac{s}{\sqrt{n}})$

Biased Measurements

Let $X=\mu +ϵ$, where $E(ϵ)=0$, $Var(ϵ)={\sigma }^2$

Suppose X is used to measure an unknown constant a, $a\ne \mu$. $X=a+(\mu -a)+ϵ$, where $\mu -a$ is the bias.

Mean square error (MSE) is $E((X-a{)}^2)={\sigma }^2+(\mu -a{)}^2$

with n IID measurements, $\bar{x}=\mu +\overline{ϵ}$

$E((x-a{)}^2)=\frac{{\sigma }^2}{n}+{(\mu -a)}^2$

$\text{MSE}={\text{SE}}^2+{\text{bias}}^2$, hence $\sqrt{\text{MSE}}$ is a good measure of the accuracy of the estimate $\bar{x}$ of a.

Estimation of a Ratio

Consider a population of $N$ members, and two characteristics are recorded: $(X_1,Y_1),(X_2,Y_2),\dots ,(X_n,Y_n)$, $r=\frac{{\mu }_y}{{\mu }_x}$.

An obvious estimator of r is $R=\frac{\bar{Y}}{\bar{X}}$

$Cov(\bar{X},\bar{Y})=\frac{{\sigma }_{xy}}{n}$, where

${\sigma }_{xy}:=\frac{1}{N}\sum _{i=1}^N(x_i-{\mu }_x)(x_i-{\mu }_y)$ is the population covariance.

Properties

$Var(R)\approx \frac{1}{{\mu }_x^2}(r^2{\sigma }_{\bar{X}}^2+{\sigma }_{\bar{Y}}^2-2r{\sigma }_{\bar{X}\bar{Y}})$

Population coefficient $\rho =\frac{{\sigma }_{xy}}{{\sigma }_x{\sigma }_y}$

$E(R)\approx r+\frac{1}{n}\left(\frac{N-n}{N-1}\right)\frac{1}{{\mu }_x^2}(r{\sigma }_x^2-\rho {\sigma }_x{\sigma }_y)$

$s_{xy}=\frac{1}{n-1}\sum _{i=1}^n(X_i-\bar{X})(Y_i-\bar{Y})$

Ratio Estimates

${\bar{Y}}_R=\frac{{\mu }_x}{\bar{X}}\bar{Y}={\mu }_{x}R$

$Var({\bar{Y}}_R)\approx \frac{1}{n}\frac{N-n}{N-1}(r^2{\sigma }_x^2+{\sigma }_y^2-2r\rho {\sigma }_x{\sigma }_y)$

$E({\bar{Y}}_R)-{\mu }_y\approx \frac{1}{n}\frac{N-n}{N-1}\frac{1}{{\mu }_x}(r{\sigma }_x^2-\rho {\sigma }_x{\sigma }_y)$

The bias is of order $\frac{1}{n}$, small compared to its standard error.

${\bar{Y}}_R$ is better than $\bar{Y}$, having smaller variance, when $\rho >\frac{1}{2}\left(\frac{C_x}{C_y}\right)$, where $C_i={\sigma }_i/{\mu }_i$

Variance of ${\bar{Y}}_R$ can be estimated by

$s_{{\bar{Y}}_R}^2=\frac{1}{n}\frac{N-n}{N-1}(R^2{s}_x^2+s_y^2-2R{s}_{xy})$

An approximate $1-\alpha$ C.I. for ${\mu }_y$ is ${\bar{Y}}_R\pm z_{\alpha /2}{s}_{{\bar{Y}}_R}$

Method of Moments

To estimate $\theta$, express it as a function of moments $g({\hat{\mu }}_1,{\hat{\mu }}_2,\dots )$

Monte Carlo

Monte Carlo is used to generate many realisations of random variable.

$\bar{X}\underset{n}{\overset{\mathrm{\infty }}{\to }}\alpha /\lambda ,{\hat{\sigma }}^2\underset{n}{\overset{\mathrm{\infty }}{\to }}\alpha /{\lambda }^2$, MOM estimators are consistent (asymptotically unbiased).

$\text{Poisson}(\lambda )$: $\text{bias}=0,SE\approx \sqrt{\frac{\bar{x}}{n}}$

$N(\mu ,{\sigma }^2)$: $\mu ={\mu }_1$, ${\sigma }^2={\mu }_2-{\mu }_1^2$

$\mathrm{\Gamma }(\lambda ,\alpha )$: $\hat{\lambda }=\frac{{\hat{\mu }}_1}{{\hat{\mu }}_2-{\hat{\mu }}_1^2}=\frac{\bar{X}}{{\hat{\sigma }}^2},\hat{\alpha }=\frac{{\hat{\mu }}_1^2}{{\hat{\mu }}_2-{\hat{\mu }}_1^2}=\frac{{\bar{X}}^2}{{\hat{\sigma }}^2}$

Maximum Likelihood Estimator (MLE)

Poisson Case

$L(\lambda )=\prod _{i=1}^n\frac{{\lambda }^{x_i}{e}^{-\lambda }}{x_i!}=\frac{\lambda \sum _{i=1}^n{x}_i{e}^{-n\lambda }}{\prod _{i=1}^n{x}_i!}$

$l(\lambda )=\sum _{i=1}^n{x}_i\log \lambda -n\lambda -\sum _{i=1}^n\log x_i!$

ML estimate of ${\lambda }_0$ is $\bar{x}$. ML estimator is ${\hat{\lambda }}_0=\bar{X}$

Normal case

$l(\mu ,\sigma )=-n\log \sigma -\frac{n\log 2\pi }{2}-\frac{\sum _{i=1}^n{(X_i-\mu )}^2}{2{\sigma }^2}$

$\frac{\partial l}{\partial \mu }=\frac{\sum (X_i-\mu )}{{\sigma }^2}⟹\hat{\mu }=\bar{x}$

$\frac{\partial l}{\partial \sigma }=\frac{\sum _{i=1}^n{(X_i-\mu )}^2}{{\sigma }^3}-\frac{n}{\sigma }⟹\widehat{{\sigma }^2}=\frac{1}{n}\sum _{i=1}^n{(X_i-\bar{X})}^2$

Gamma case

$l(\theta )=n\alpha \log \lambda +(\alpha -1)\sum _{i=1}^n\log X_i-\lambda \sum _{i=1}^n{X}_i-n\log \mathrm{\Gamma }(\alpha )$

$\frac{\partial l}{\partial \alpha }=n\log \alpha +\sum _{i=1}^n\log X_i-\sum _{i=1}^n{X}_i-\frac{n}{\mathrm{\Gamma }(\alpha )}\mathrm{\Gamma }‘(\alpha )$

$\frac{\partial l}{\partial \lambda }=\frac{n\alpha }{\lambda }-\sum _{i=1}^n{X}_i$

$\hat{\lambda }=\frac{\hat{\alpha }}{\hat{x}}$

Multinomial Case

$f(x_1,\dots ,x_r)=(\genfrac{}{}{0ex}{}{n}{x_1,x_2,\dots x_r})\prod _{i=1}^n{p}_i^{X_i}$

where $X_i$ is the number of times the value occurs, and not the number of trials. and $x_1,x_2,\dots x_r$ are non-negative integers summing to $n$. $\mathrm{\forall }i$:

$E(X_i)=n{p}_i,Var(X_i)=n{p}_i(1-p_i)$

$Cov(X_i,X_j)=-n{p}_i{p}_j,\mathrm{\forall }i\ne j$

$l(p)=\text{Kappa}+\sum _{i=1}^{r-1}{x}_i\log p_i+x_r\log (1-p_1-\dots -p_{r-1})$

$\frac{\partial l}{\partial p_i}=\frac{x_i}{p_i}-\frac{x_r}{p_r}=0\text{ assuming MLE exists}$

$\frac{x_i}{{\hat{p}}_i}=\frac{x_r}{{\hat{p}}_r}⟹{\hat{p}}_i=\frac{x_i}{c},c=\frac{x_r}{{\hat{p}}_r}$

$\sum _{i=1}^r{\hat{p}}_i=\sum _{i=1}^r\frac{x_i}{c}=1⟹c=\sum _{i=1}^r{x}_i=n⟹{\hat{p}}_i=\frac{{\bar{x}}_i}{n}$

same as MOM estimator.

CIs in MLE

$\frac{\hat{X}-\mu }{s/\sqrt{n}}\sim t_{n-1}$

Given the realisations $\bar{x}$ and $s$, $\bar{x}\pm t_{n-1,\alpha /2}\frac{s}{\sqrt{n}},\bar{x}+t_{n-1,\alpha /2}\frac{s}{\sqrt{n}}$ is the exact $1-\alpha$ CI for $\mu$.

$\text{Missing superscript or subscript argument}$, $\frac{n{\hat{\sigma }}^2}{{\chi }_{n-1,\alpha /2}^2},\frac{n{\hat{\sigma }}^2}{{\chi }_{n-1,1-\alpha /2}^2}$ is the exact $1-\alpha$ CI for $\sigma$.

Fisher Information

$I\left(\theta \right)=-E(\frac{\partial }{\partial {\theta }^2}\log f\left(x|\theta \right))$

General trinomial: $(\frac{X_1}{n},\frac{X_2}{n})$

$$\left[\begin{array}{ccc}{p}_1(1-p_1)& -p_1{p}_2-p_1{p}_2& p_2(1-p_2)\end{array}\right]\frac{1}{n}$$

In all the above cases, $\text{var}(\hat{\theta })=I(\theta {)}^{-1}$.

Asymptotic Normality of MLE

As $n\to \mathrm{\infty }$, $\sqrt{nI(\theta )}(\hat{\theta }-\theta )\to N(0,1)$ in distribution, and hence $\hat{\theta }\sim N(\theta ,\frac{I{\left(\theta \right)}^{-1}}{n})$

As $\hat{\theta }\underset{n}{\overset{\mathrm{\infty }}{\to }}\theta$, MLE is consistent.

SE of an estimate of $\theta$ is the SD of the estimator $\hat{\theta }$, hence $SE=SD(\hat{\theta })=\sqrt{\frac{I(\theta {)}^{-1}}{n}}\approx \sqrt{\frac{I(\hat{\theta }{)}^{-1}}{n}}$

$1-\alpha \text{ CI }\approx \hat{\theta }\pm z_{\alpha /2}\sqrt{\frac{I(\theta {)}^{-1}}{n}}$

Efficiency

Cramer-Rao Inequality: if $\theta$ is unbiased, then $\mathrm{\forall }\theta \in \mathrm{\Theta }$ , $var(\hat{\theta })\ge I(\hat{\theta }{)}^{-1}/n$, if = then $\hat{\theta }$ is efficient.

$eff(\hat{\theta })=\frac{I(\hat{\theta }{)}^{-1}/n}{var(\hat{\theta })}<1$

Sufficiency

Characterisation

Let $S_t=x:T(x)=t$. The sample space of $X$, $S$ is the disjoint union of $S_t$ across all possible values of $T$.

$T$ is sufficient for $\theta$ if $\mathrm{\exists }q()\text{ s.t. }\mathrm{\forall }x\in S_t,f_{\theta }(X\text{x}|T=t)=q(x)$.

Factorisation Theorem

$T$ is sufficient for $\theta$ iff $\mathrm{\exists }g(t,\theta ),h(x)\text{ s.t. }\mathrm{\forall }\theta \in \mathrm{\Theta },f_{\theta }(x)=g(T(x),\theta )h(x)\mathrm{\forall }x$

Rao-Blackwell Theorem

Let $\hat{\theta }$ be an estimator of $\theta$ with finite variance, $T$ be sufficient for $\theta$. Let $\tilde{\theta }=E[\hat{\theta }|T]$. Then for every $\theta \in \mathrm{\Theta }$, $E{(\hat{\theta }-\theta )}^2\le E{(\hat{\theta }-\theta )}^2$. Equality holds iff $\hat{\theta }$ is a function of $T$.

Random Conditional Expectation

1. $E(X)=E(E(X|T))$
2. $var(X)=var(E(X|T))+E(var(X|T))$
3. $var(Y|X)=E(Y^2|X)-E(Y|X{)}^2$
4. $E(Y)=Y,var(Y)=0$ iff $Y$ is a constant

Hypothesis Testing

Let $X_1\dots X_n$ be IID with density $f(x|\theta )$. null $H_0:\theta ={\theta }_0$, $H-1:\theta ={\theta }_1$. Critical region is $R\subset R_n$. $size=P_0(X\in R)$ and $power=P_1(X\in R)$.

$\mathrm{\Lambda }(x)=\frac{f_0(x_1)\dots f_0(x_n)}{f_1(x_1)\dots f_1(x_n)}$. Critical region $x:\mathrm{\Lambda }(x)<c_{\alpha }$, and among all tests with this size, it has the maximum power (Neyman-Pearson Lemma).

A hypothesis is simple if it completely specifies the distibution of the data.

$H_1:\mu >{\mu }_0$: Critical region $\{\overline{x}>{\mu }_0+z_{\alpha }\frac{\sigma }{\sqrt{n}}\}$, the power is a function of $\mu$, and this is uniformly the most powerful test for size $\le \alpha$.

$H_1:\mu \ne {\mu }_0$: Critical region $\{|\overline{x}-{\mu }_0|>c\},c=z_{\frac{\alpha }{2}}\frac{\sigma }{\sqrt{n}}$, but not uniformly most powerful.

The $(1-\alpha )$ CI for $\mu$ consists of precisely the values ${\mu }_0$ for which $H_0:\mu ={\mu }_0$ is not rejected against $H_1:\mu \ne {\mu }_0$. Exact for normal with known variance, approx. in others.

p-value

the probability under $H_0$ that the test statistic is more extreme than the realisation. (A, B): $p=p_0(\overline{X}>\overline{x})=P(Z>\frac{\overline{x}-{\mu }_0}{\sigma /\sqrt{n}})$. (C): $p=P_0(|\overline{X}-{\mu }_0|>|\overline{x}-{\mu }_0|)$. The smaller the p-value, the more suspicious one should be about $H_0$. If size is smaller than p-value, do not reject $H_0$.

Generalized Likelihood Ratio

${\mathrm{\Lambda }}^{\ast }=\frac{{\text{max}}_{\theta \in {\omega }_0}L(\theta )}{{\text{max}}_{\theta \in \mathrm{\Omega }}L(\theta )}$, $\mathrm{\Omega }={\omega }_0\cup {\omega }_1$. The closer $\mathrm{\Lambda }$ is to 0, the stronger the evidence for $H_1$.

Large-sample null distribution of $\mathrm{\Lambda }$

Under $H_0$, when n is large, $-2\log \mathrm{\Lambda }={\chi }_k^2$, where $k=\text{dim}(\mathrm{\Omega })-\text{dim}({\omega }_0)$.

Normal (C): $p=P({\chi }_1^2>\frac{(\overline{x}-{\mu }_0{)}^2}{{\sigma }^2/n})$

Multinomial: $\mathrm{\Lambda }=\prod _{i=1}^r{\left(\frac{E_i}{X_i}\right)}^{X_i}$ where $E_i=n{p}_i(\hat{\theta })$ is the expected frequency of the ith event under $H_0$. $-2\log \mathrm{\Lambda }\approx \sum _{i=1}^r\frac{(X_i-E_i{)}^2}{E_i}$, which is the Pearson chi-square statistic, written as $X^2$.

Poisson Dispersion Test

For $i=1\dots n$ let $X_i\sim Poisson({\lambda }_i)$ are independent.

$w_0=\{\tilde{\lambda }|{\lambda }_1={\lambda }_2=\dots ={\lambda }_n\}$

$w_1=\{\tilde{\lambda }|{\lambda }_i\ne {\lambda }_j\text{ for some }i,j\}$

$-2\log \mathrm{\Lambda }\approx \frac{\sum _{i=1}^n(X_i-\overline{X}{)}^2}{\overline{X}}$. For large n, the null distribution of $-2\log \mathrm{\Lambda }$ is approximately ${\chi }_{n-1}^2$

Comparing 2 samples

Normal Theory: Same Variance

$X_1,\dots ,X_n$ be i.i.d $N({\mu }_X,{\sigma }^2)$ and $Y_1,\dots ,Y_m$ be i.i.d $N({\mu }_Y,{\sigma }^2)$, independent. $H_0:{\mu }_X-{\mu }_Y=d$

Known Variance

$Z:=\frac{\overline{X}-\overline{Y}-({\mu }_X-{\mu }_Y)}{\sigma \sqrt{\frac{1}{n}+\frac{1}{m}}}$ and reject $H_0$ when $|Z|>z_{\alpha /2}$

Unknown Variance

$s_p^2=\frac{(n-1)s_X^2+(m-1)s_Y^2}{m+n-2}$ where $s_X^2=\frac{1}{n-1}\sum _{i=1}^n(X_i-\overline{X}{)}^2$. $s_p^2$ is an unbiased estimator of ${\sigma }^2$. $s_X$ within factor of 2 from $s_Y$.

$t:=\frac{\overline{X}-\overline{Y}-({\mu }_X-{\mu }_Y)}{s_p\sqrt{\frac{1}{n}+\frac{1}{m}}}$ follows a t distribution with $m+n-2$ d.f.

If two-sided: reject $H_0$ when $|t|>t_{n+m-2,\alpha /2}$. If one-sided, e.g $H_1:{\mu }_X>{\mu }_Y$, reject $H_0$ when $t>t_{n+m-2,\alpha }$.

CI

$\frac{\overline{X}-\overline{Y}}{\pm }{z}_{\alpha /2}\cdot \sigma \sqrt{\frac{1}{n}+\frac{1}{m}}$ if $\sigma$ is known, or $\frac{\overline{X}-\overline{Y}}{\pm }{t}_{m+n-2,\alpha /2}\cdot s_p\sqrt{\frac{1}{n}+\frac{1}{m}}$ if $\sigma$ is unknown.

Unequal Variance

$Z:=\frac{\overline{X}-\overline{Y}-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{{\sigma }_X^2}{n}+\frac{{\sigma }_Y^2}{m}}}$

$t:=\frac{\overline{X}-\overline{Y}-({\mu }_X-{\mu }_Y)}{\sqrt{\frac{s_X^2}{n}+\frac{s_Y^2}{m}}}$, with $df=\frac{(a+b{)}^2}{\frac{a^2}{n-1}+\frac{b^2}{m-1}}$ where $a=\frac{s_X^2}{n}$ and $b=\frac{s_Y^2}{m}$

Mann-Whitney Test

We take the smaller sample of size $n_1$, and sum the ranks in that sample. $R^{\prime }=n_1(m+n+1)-R$, and $R\ast =min(R^{\prime },R)$, we reject $H_0:F=G$ if $R\ast$ is too small.

Test works for all distributions, and is robust to outliers.

Paired Samples

$(X_i,Y_i)$ are paired and related to the same individual. $(X_i,Y_i)$ is independent from $(X_j,Y_j)$. Compute $D_i=Y_i-X_i$, To test $H_0:{\mu }_D=d$, $t=\frac{\overline{D}-{\mu }_D}{s_D/\sqrt{n}}$.

$1-\alpha$ CI: $\overline{D}\pm t_{n-1,\alpha /2}{S}_D/\sqrt{n}$

Ranked Test

$W_{+}$ is the sum of ranks among all positive $D_i$ and $W_i$ is the sum of ranks among all negative $D_i$. We want to reject $H_0$ if $W=min(W_{+},W_{-})$ is too large.