Normal Equations for Partitioned Least Squares

Given the linear model:

y=X1b1+X2b2+e\begin{align} \pmb y = X_1 \pmb b_1 + X_2 \pmb b_2 + \pmb e \end{align}

We want to find the first order conditions of the problem:

arg min⁡b1,b2e′e\begin{align} \argmin_{\pmb b_1,\pmb b_2} \pmb e'\pmb e \end{align}
arg min⁡b1,b2(y−y^)′(y−y^)\begin{align} \argmin_{\pmb b_1,\pmb b_2} (\pmb y - \pmb{\hat y})'(\pmb y - \pmb{\hat y}) \end{align}

The objective function of the problem is equivalent to:

e′e=(y−y^)′(y−y^)=y′y−2y′y^+y^′y^=y′y−2y′(X1b1+X2b2)+(X1b1+X2b2)′(X1b1+X2b2)=y′y−2y′X1b1−2y′X2b2+b1′X1′X1b1+2b1′X1′X2b2+b2′X2′X2b2\begin{align} \pmb e'\pmb e &= (\pmb y - \pmb{\hat y})'(\pmb y - \pmb{\hat y}) \\&= \pmb y' \pmb y - 2 \pmb y' \pmb{\hat y} + \pmb{\hat y}' \pmb{\hat y} \\ &= \pmb y'\pmb y - 2\pmb y'(X_1 \pmb b_1 + X_2 \pmb b_2) + (X_1 \pmb b_1 + X_2 \pmb b_2)'(X_1 \pmb b_1 + X_2 \pmb b_2) \\ &= \pmb y' \pmb y - 2\pmb y'X_1 \pmb b_1 - 2 \pmb y' X_2 \pmb b_2 + \pmb b_1' X_1'X_1 \pmb b_1 + 2\pmb b_1' X_1'X_2 \pmb b_2 + \pmb b_2' X_2'X_2 \pmb b_2 \end{align}

Now we take the gradient of the function with respect to b1\pmb b_1 and b2\pmb b_2:

∂f∂b1=0  ⟹  −2X1′y+2X1′X1b1+2X1′X2b2=0  ⟹  X1′X1b1+X1′X2b2=X1′y\begin{align} \frac{\partial f }{\partial \pmb b_1} = 0 &\implies -2 X_1'\pmb y + 2 X_1'X_1 \pmb b_1 + 2 X_1'X_2 \pmb b_2 = 0 \\ &\implies X_1'X_1 \pmb b_1 + X_1'X_2 \pmb b_2 = X_1'\pmb y \end{align}
∂f∂b2=0  ⟹  −2X2′y+2X2′X2b2+2X2′X1b1=0  ⟹  X2′X2b2+X2′X1b1=X2′y\begin{align} \frac{\partial f }{\partial \pmb b_2} = 0 &\implies - 2 X_2'\pmb y + 2 X_2'X_2 \pmb b_2 + 2 X_2'X_1 \pmb b_1 = 0 \\ &\implies X_2'X_2 \pmb b_2 + X_2'X_1 \pmb b_1 = X_2'\pmb y\end{align}

Which we can package into a block of conditions like so:

[X1′X1X1′X2X2′X1X2′X2][b1b2]=[X1′yX2′y]\begin{align} \begin{bmatrix} X_1' X_1 & X_1'X_2 \\ X_2'X_1 & X_2'X_2\end{bmatrix}\begin{bmatrix}\pmb b_1 \\ \pmb b_2\end{bmatrix} = \begin{bmatrix}X_1'\pmb y \\ X_2' \pmb y\end{bmatrix} \end{align}