Solution for Constrained Weighted Least Squares

Full setup and solving for CWLS in the context of factor models.

Formally, the problem being solved is:

arg max⁡ft (rt−B~tft)′Ω−1(rt−B~tft)  s.t.  Cft=d\begin{align} \argmax_{\pmb f_t} \ (\pmb r_t - \tilde B_t \pmb f_t)^\prime \Omega^{-1}(\pmb r_t - \tilde B_t \pmb f_t)\ \ s.t.\ \ C \pmb f_t = \pmb d \end{align}

Where ft∈RK×1\pmb f_t \in \mathbb{R}^{K \times 1}, rt∈RN×1\pmb r_t \in \mathbb R^{N \times 1}, Bt∈RN×KB_t \in \mathbb R^{N \times K}, C∈RP×KC \in \mathbb R^{P \times K}, d∈RP×1\pmb d \in \mathbb R^{P \times 1}, and Ω∈RN×N\Omega \in \mathbb R^{N \times N}. Note that K=1+J+SK=1+J+S where JJ is the number of industries and SS is the number of style factors.

The associate Lagrangian is:

L=(rt−B~tft)′Ω−1(rt−B~tft)+2λ′(Cft−d)=rt′Ω−1rt−2rt′Ω−1B~tft+ft′B~tΩ−1B~tft+2λ′Cft−2λ′d\begin{align} L &= (\pmb r_t - \tilde B_t \pmb f_t)'\Omega^{-1} (\pmb r_t -\tilde B_t \pmb f_t) + 2 \lambda'(C\pmb f_t-\pmb d) \\ &= \pmb r_t'\Omega^{-1}\pmb r_t- 2\pmb r_t'\Omega^{-1}\tilde B_t \pmb f_t +\pmb f_t'\tilde B_t\Omega^{-1}\tilde B_t \pmb f_t + 2\pmb \lambda' C\pmb f_t -2\pmb \lambda'\pmb d \end{align}

Taking first order conditions:

∂F∂ft=0  ⟹  −2B~t′Ω−1rt+2B~t′Ω−1B~tft+2C′λ=0−B~t′Ω−1rt+B~t′Ω−1B~tft+C′λ=0  ⟹  B~t′Ω−1B~tft+C′λ=Bt′Ω−1rt∂F∂λ=0  ⟹  2Cft−2d=0  ⟹  Cft′=d\begin{align} \frac{ \partial F }{ \partial \pmb f_t } = 0 \implies - 2\tilde B'_t\Omega^{-1}\pmb r_t + 2\tilde B_t'\Omega^{-1}\tilde B_t\pmb f_t + 2C'\pmb \lambda &= \pmb 0 \\ -\tilde B'_t\Omega^{-1}\pmb r_t + \tilde B_t'\Omega^{-1}\tilde B_t\pmb f_t + C'\pmb \lambda &= \pmb 0 \\ \implies \tilde B_t'\Omega^{-1}\tilde B_t\pmb f_t + C'\pmb \lambda = B_t'\Omega^{-1}\pmb r_t \\ \frac{\partial F}{\partial \pmb \lambda} = 0 \implies 2C\pmb f_t - 2\pmb d = 0 \implies C\pmb f_t' = \pmb d \end{align}

We wrote the first order conditions with ftf_t and λ\lambda on one side so that we could factor our system of equations into block matrices.

[B~t′Ω−1B~tC′C0][ftλ]=[Bt′Ω−1rtd]  ⟹  [ftλ]=[B~t′Ω−1B~tC′C0]−1[Bt′Ω−1rtd]\begin{align} \begin{bmatrix} \tilde B_t'\Omega^{-1}\tilde B_t & C'\\ C & 0 \end{bmatrix} \begin{bmatrix} \pmb f_t \\ \pmb \lambda \end{bmatrix} = \begin{bmatrix} B_t'\Omega^{-1}\pmb r_t \\ \pmb d \end{bmatrix} \end{align} \\[5pt] \implies \begin{bmatrix} \pmb f_t \\ \pmb \lambda \end{bmatrix} = \begin{bmatrix} \tilde B_t'\Omega^{-1}\tilde B_t & C'\\ C & 0 \end{bmatrix}^{-1} \begin{bmatrix} B_t'\Omega^{-1}\pmb r_t \\ \pmb d \end{bmatrix}

Following the formula for 2x2 block matrix inversion. We have A=B~t′Ω−1B~tA = \tilde B_t' \Omega^{-1}\tilde B_t, B=C′B = C', C=CC = C, and D=0D = 0

Therefore we have our inverted matrix equal to:

[(B~t′Ω−1B~t)−1−(B~t′Ω−1B~t)−1C′(C(B~t′Ω−1B~t)−1C′)−1C(B~t′Ω−1B~t)−1(B~t′Ω−1B~t)−1C′(C(B~t′Ω−1B~t)−1C′−(0−C(B~t′Ω−1B~t)−1C′)−1C(B~t′Ω−1B~t)−1(0−C(B~t′Ω−1B~t)−1C′)−1]\begin{align} \begin{bmatrix} (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1} - (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C'(C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C')^{-1}C (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1} & (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C'(C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C' \\ -(0-C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C')^{-1}C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1} & (0 - C (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1} C')^{-1} \end{bmatrix} \end{align}

Consider multiplying the first row of the inverse matrix with the column vector to solve for ft\pmb f_t.In our case d=0\pmb d = \pmb 0 which will annihilate the second term in our equation. (In fact, it will set all terms in the second column of our block matrix to 00.) Therefore when solving for ft\pmb f_t we only need to consider the top left element multiplied with B~t′Ω−1rt\tilde B_t'\Omega^{-1} \pmb r_t. (Also, λ\pmb \lambda will simply equal CC multiplied by B~t′Ω−1rt\tilde B_t'\Omega^{-1} \pmb r_t.)

Thus we have:

ft=[(B~t′Ω−1B~t)−1−(B~t′Ω−1B~t)−1C′(C(B~t′Ω−1B~t)−1C′)−1C(B~t′Ω−1B~t)−1]B~t′Ω−1rt=(I−(B~t′Ω−1B~t)−1C′(C(B~t′Ω−1B~t)−1C′)−1C)(B~t′Ω−1B~t)−1B~t′Ω−1rt\begin{align} \pmb f_t &= \Big[(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1} - (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C'(C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C')^{-1}C (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}\Big]\tilde B_t'\Omega^{-1} \pmb r_t \\ &= \Big(I - (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C'(C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C')^{-1}C\Big)(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}\tilde B_t'\Omega^{-1} \pmb r_t \end{align}

If we want to break this up to make it a little easier on the eyes we can write:

ft=Wtrt\begin{align} \pmb f_t = W_t \pmb r_t \end{align}

Where WtW_t, our linear estimator, is equal to:

Wt=(I−Q)(B~t′Ω−1B~t)−1B~t′Ω−1\begin{align} W_t= (I - Q)(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}\tilde B_t'\Omega^{-1} \end{align}

and

Q=(B~t′Ω−1B~t)−1C′(C(B~t′Ω−1B~t)−1C′)−1C\begin{align} Q = (\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C'(C(\tilde B_t' \Omega^{-1}\tilde B_t)^{-1}C')^{-1}C \end{align}