Central Limit Theorem (CLT)

The most prolific result in statistics.

Statement of CLT

The CLT states that given $S_n = X_1 + \cdots + X_n$

\begin{align} Z_n = \frac{S_n - E[S_n]}{\sigma_{S_n}} = \frac{S_n - n E[X]}{\sqrt{n}\sigma} \end{align}

\begin{align} \implies E[Z_n] = 0 \ \land \ Var(Z_n) = 1 \end{align}

Let $Z$ be standard normal. Then the CLT also says:

\begin{align} P(Z_n \leq c) \overset{\plim}\to P(Z \leq c) \end{align}

Mnemonic

Sometimes to remember these results it's easier to think of the results at different "levels"

\begin{align} \underbrace{X}_\text{Level 1} \overset{+}\to \underbrace{S_n}_\text{Level 2} \overset{\text{normalize}}\to \underbrace{Z_n}_\text{Level 3} \end{align}

We can't say much about level 1 directly (because no aggregation has taken place).

After aggregation, we have our level 2 result:

\begin{align} S_n \sim N(n\mu, n\sigma^2) \end{align}

Which is easy to see from $E[S_n] = E[X_1 + \dots +X_n]$ and $Var(S_n) = Var(X_1 + \dots +X_n)$ .

Dividing the result by $n$ :

\begin{align} \bar X \sim N(\mu, \sigma^2/n) \end{align}

And if we demean and divide by the standard deviation we get our level 3 result:

\begin{align} Z_n \sim N(0,1) \end{align}

Note that this comes from a non-aggregative transformation on the data.

More Forms

We can also consider the distribution of $\sqrt{n}(\bar X-\mu)$ . This is the demeaned random variable multiplied by the square-root of the sample size. This is an easy result to get through

\begin{align} \bar X &\sim N(\mu, \sigma^2/n) \\ \implies \bar X - \mu &\sim N(0, \sigma^2/n) \\ \implies \sqrt{n}(\bar X - \mu) &\sim N(0, \sigma^2) \end{align}

Sometimes, the variance term is written in terms of $\bar X.$ We know that $Var(\bar X) = \sigma^2/n$ . Multiplying by $n$ we have $\bar \sigma^2 = n\sigma^2_{\bar X}$ so we can write the last claim as:

\begin{align} \sqrt{n}(\bar X - \mu) \sim N(0, n\sigma^2_{\bar X}) \end{align}

The CLT is about distribution, not variance

The last take away, that should be obvious, is that the CLT does not have a magical effect on variance of the random variable. The power of it comes from what it says about the distribution. The variance of the variable is easy to derive:

Given $Var(X) = \sigma^2$ we have:

\begin{align} Var(\bar X) &= Var(X_1/n + \dots + X_n/n) \\ &= Var(X_1 + \dots + X_n)/n^2\\&= \sigma^2/n \end{align}

So to reiterate, the CLT isn't special because of what it says about the variance of an aggregated random variable, it's cool because of what it says about that aggregated random variable's distribution.