Newey-West Covariance Matrix (Part 1)

A covariance matrix that incorporates heteroskedasticity and autocorrelation.

Univariate Case

A well behaved random variable.

Let XX represent a time-varying variable that doesn't exhibit heteroskedasticity nor autocorrelation:

Var(Xˉ)=Var[1T(X1+X2+⋯+XT]=1T2Var(X1+X2+⋯+XT)=1T2∑i=1TVar(X)=Var(X)T​\begin{align} Var(\bar X) &= Var\left[\frac{1}{T}(X_1 + X_2 + \cdots + X_T \right] \\ &= \frac{1}{T^2}Var(X_1 + X_2 + \cdots + X_T) \\ &= \frac{1}{T^2}\sum_{i=1}^TVar(X) \\&= \frac{Var(X)}{T} ​\end{align}

Thus,

Var(Xˉ)=1TVar(X)=1T∑i=1T(Xi−Xˉ)2T\begin{align} Var(\bar X) = \frac{1}{T}Var(X) = \frac{1}{T}\sum_{i=1}^T\frac{(X_i-\bar X)^2}{T} \end{align}

A random variable that is heteroskedastic and autocorrelated.

If xx exhibits heteroskedasticity and autocorrealtion, then a more appropriate approximation is:

Var(Xˉ)=Var[1T(X1+X2+⋯+XT]=1T2Var(X1+X2+⋯+XT)=1T2∑i=1T∑j=1TCov(Xi,Xj)\begin{align} Var(\bar X) &= Var \left[ \frac{1}{T}(X_1 + X_2 + \cdots + X_T \right] \\ &=\frac{1}{T^2} Var(X_1 + X_2 + \cdots + X_T) \\ &= \frac{1}{T^2} \sum_{i=1}^T \sum_{j=1}^T Cov(X_i, X_j) \end{align}

We can break this expression into what should be reminiscent of "rows".

Var(Xˉ)=1T2[ Cov(X1,X1+X2+⋯+XT)⏟Row 1Cov(X2,X1+X2+⋯+XT)⏟Row 2⋮Cov(XN,X1+X2+⋯+XT)⏟Row N ]\begin{align} Var(\bar X) = \frac{1}{T^2}\huge[\ & \underbrace{Cov(X_1, X_1 + X_2 + \cdots +X_T)}_\text{Row 1} \\ & \underbrace{Cov(X_2, X_1 + X_2 + \cdots +X_T)}_\text{Row 2} \\ & \qquad \qquad \qquad \vdots \\ & \underbrace{Cov(X_N, X_1 + X_2 + \cdots +X_T)}_\text{Row N}\ \huge] \end{align}

Imposing an assumption

Let's assume that as time between observations increases, they become less correlated. Precisely, let's assume that when the time between observations is greater than qq that covariance goes to zero. In math, ∣t−s∣>q  ⟹  Cov(xt,xs)=0|t-s|> q \implies Cov(x_t, x_s) = 0.

Imposing this assumption:

Var(Xˉ)=1T2[ ∑t=1TVar(Xt)⏟Diagonal∑t=2TCov(Xt,Xt−1)+Cov(Xt,Xt+1)⏟Off-1 Diagonal⋮∑t=q+1TCov(Xt,Xt−q)+Cov(Xt,Xt+q)⏟Off-q Diagonal ]\begin{align} Var(\bar X) = \frac{1}{T^2}\huge[\ & \underbrace{\sum_{t=1}^T Var(X_t)}_\text{Diagonal} \\ & \underbrace{\sum_{t=2}^T Cov(X_t, X_{t-1}) + Cov(X_t, X_{t+1})}_\text{Off-1 Diagonal} \\ & \qquad \qquad \qquad \vdots \\ & \underbrace{\sum^T_{t=q+1} Cov(X_t, X_{t-q}) + Cov(X_t, X_{t+q})}_\text{Off-q Diagonal}\ \huge] \end{align}

Notice that if we only have one observation at each point in time for a random variable then,

Cov(Xt,Xt−q)≈Cov^(Xt,Xt−q)=(Xt−Xˉ)(Xt−q−Xˉ)\begin{align} Cov(X_t, X_{t-q}) \approx \widehat{Cov}(X_t, X_{t-q}) = (X_t - \bar X)(X_{t-q}-\bar X) \end{align}

What happened to dividing by TT? In this case, we're technically summing over a set containing a single variable so we're dividing by 11.

Though our long formula for Var(Xˉ)Var(\bar X) is easy to visualize with the matrix analogy, we don't have a concise way of writing this down yet. We define a couple variables to help us rewrite our verbose formula in a more compact form.

γ^v=(1/T)∑t=v+1T(Xt−Xˉ)(Xt−v−Xˉ)\begin{align} \hat \gamma_v = (1/T) \sum_{t=v+1}^T (X_t - \bar X)(X_{t-v} - \bar X) \end{align}

See that vv determines how far off the diagonal we're summing up the elements, and that the numbers from v+1v+1 to TT count the elements. This formula would get one side of the off-diagonal while the following formula will get us the other side.

γ^−v=(1/T)∑t=v+1T(Xt−v−Xˉ)(Xt−Xˉ)\begin{align} \hat \gamma_{-v} = (1/T) \sum_{t=v+1}^T (X_{t-v} - \bar X)(X_t - \bar X) \end{align}

Note that the special case v=0  ⟺  γ^0v=0 \iff \hat \gamma_0 will sum up all the diagonal elements (the variance terms).

Using our large expression for Var(Xˉ)Var(\bar X) and our definition for γ^\hat \gamma we can finally rewrite Var(Xˉ)Var(\bar X).

Var^(Xˉ)=(1/T)[γ^0+∑v=1q(γ^v+γ^−v)]\begin{align} \widehat {Var}(\bar X) = (1/T) \left[\hat \gamma_0 + \sum_{v=1}^q (\hat\gamma_{v} + \hat \gamma_{-v})\right] \end{align}

Correcting for undesirable small sample problems

Though our formula for σ^Xˉ2\hat \sigma_{\bar X}^2 is consistent (converges to the truth as n→∞n \to \infty), it is not always positive for small sample sizes. The following formula preserves the asymptotic properties and allows variance to always be positive

σ^Xˉ2=(1/T)(γ^0+∑v=1q[1−vq+1](γ^v+γ^−v))\begin{align} \hat \sigma_{\bar X}^2 = (1/T)\left(\hat \gamma_0 + \sum_{v=1}^q \left[ 1 - \frac{v}{q+1}\right](\hat \gamma_v + \hat \gamma_{-v})\right) \end{align}