跳转至

线性回归的普通最小二乘估计

ols

本文推导了线性回归的普通最小二乘估计量的矩阵形式,并在一元线性回归的情境下给出了求和形式的表达式。 $$ Y=X \widehat{\beta}+e $$

\[ \beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y \]

在一元线性回归的情境下:

\[ \beta_1^{O L S} =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \]
\[ \beta_0^{O L S} =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \]

线性模型

若我们想要用\(X_k \left(k=1,\ldots,K\right)\)来解释\(Y\),且\(Y\)关于\(X\)是线性的,即: $$ Y_t=\beta_0+\beta_1 X_{t 1}+\beta_2 X_{t 2} +\cdots+\beta_K X_{t K} $$

Note

这里加入了截距项(或偏置项),因此需要加入\(\beta_0\)

将各变量以矩阵形式表示如下:

\[ Y=\underbrace{\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{ccccc} 1 & X_{11} & X_{12} & \ldots & X_{1 K} \\ 1 & X_{21} & X_{22} & \ldots & X_{2 K} \\ \ldots & \ldots & \ldots & \ldots & \ldots \\ 1 & X_{T 1} & X_{T 2} & \ldots & X_{T K} \end{array}\right]}_{T \times(K+1)}, \beta=\underbrace{\left[\begin{array}{c} \beta_0 \\ \beta_1 \\ \ldots \\ \beta_K \end{array}\right]}_{(K+1) \times 1}, u=\underbrace{\left[\begin{array}{c} u_1 \\ u_2 \\ \ldots \\ u_T \end{array}\right]}_{T \times 1} \]

矩阵\(X\)中的每一行对应一个观测点,每一列对应一个解释变量。

简化为: $$ Y=X \beta+u\label{1}\tag{1} $$ 公式\(\eqref{1}\)是真实的线性模型,其中的\(\beta\)是真实存在但未知的。我们需要构造估计量来估计\(\beta\): $$ Y=X \widehat{\beta}+e\tag{2}\label{2} $$

回归系数的普通最小二乘估计量

“普通最小二乘”的含义是:使得误差项的平方和最小。普通最小二乘估计量是如下最小化问题的解: $$ \underset{\widehat{\beta}}{\operatorname{argmin}} \sum_{t=1}^T e_{t} ^{2} = e^{\prime} e=(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta}) $$ 其中

\[ e=\underbrace{\left[\begin{array}{c} e_1 \\ e_2 \\ \ldots \\ e_T \end{array}\right]}_{T \times 1} \]

解这个最小化问题的推导过程在 普通最小二乘法的矩阵形式推导 中已有介绍。这里简单回顾如下: $$ \min (Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta}) $$ 由于 \(\frac{d\left(A^{\prime} A\right)}{d A}=2 A, \frac{d(C B)}{d B}=C^{\prime}\) $$ \begin{aligned} \frac{d(Y-X \widehat{\beta})^{\prime}(Y-X \widehat{\beta})}{d \widehat{\beta}} & =0 \\ (-X)^{\prime} 2(Y-X \widehat{\beta}) & =0 \\ X^{\prime}(Y-X \widehat{\beta}) & =0 \\ X^{\prime} Y-X^{\prime} X \widehat{\beta} & =0 \\ X^{\prime} X \widehat{\beta} & =X^{\prime} Y \end{aligned} $$ 若 \(\left(X^{\prime} X\right)^{-1}\) 存在,则 $$ \left(X^{\prime} X\right)^{-1} X^{\prime} X \widehat{\beta}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y $$

\[ \Rightarrow \beta^{O L S}=\left(X^{\prime} X\right)^{-1} X^{\prime} Y \]

一元线性回归

若只有一个解释变量,则

\[ Y=\underbrace{\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{T \times 1}, X=\underbrace{\left[\begin{array}{cc} 1 & X_1 \\ 1 & X_2 \\ \ldots & \ldots \\ 1 & X_T \end{array}\right]}_{T \times 2}, u=\underbrace{\left[\begin{array}{c} u_1 \\ u_2 \\ \ldots \\ u_T \end{array}\right]}_{T \times 1}, \beta=\underbrace{\left[\begin{array}{c} \beta_0 \\ \beta_1 \end{array}\right]}_{2 \times 1} \]
\[ \begin{aligned} \underbrace{\left[\begin{array}{c} \beta_0^{O L S} \\ \beta_1^{O L S} \end{array}\right]}_{2 \times 1}& =\underbrace{\left(\left[\begin{array}{cccc} 1 & 1 & \ldots & 1 \\ X_1 & X_2 & \ldots & X_T \end{array}\right]\left[\begin{array}{cc} 1 & X_1 \\ 1 & X_2 \\ \ldots & \ldots \\ 1 & X_T \end{array}\right]\right)^{-1}}_{2 \times 2} \underbrace{\left[\begin{array}{cccc} 1 & 1 & \ldots & 1 \\ X_1 & X_2 & \ldots & X_T \end{array}\right]\left[\begin{array}{c} Y_1 \\ Y_2 \\ \ldots \\ Y_T \end{array}\right]}_{2 \times 1} \\ & =\left(\left[\begin{array}{cc} T & \sum X_t \\ \sum X_t & \sum X_t^2 \end{array}\right]\right)^{-1}\left[\begin{array}{c} \sum Y_t \\ \sum X_t Y_t \end{array}\right] \\ & =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{cc} \sum X_t^2 & -\sum X_t \\ -\sum X_t & T \end{array}\right]\left[\begin{array}{c} \sum Y_t \\ \sum X_t Y_t \end{array}\right] \\ & =\frac{1}{T \sum X_t^2-\left(\sum X_t\right)^2}\left[\begin{array}{c} \sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t \\ -\sum X_t \sum Y_t+T \sum X_t Y_t \end{array}\right] \\ & =\left[\begin{array}{c} \frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ \frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \end{array}\right] \\ & \end{aligned} \]

\(\beta_1\)的估计

\[ \begin{aligned} \beta_1^{O L S} & =\frac{T \sum X_t Y_t-\sum X_t \sum Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ & =\frac{\frac{1}{T^2}\left\{T \sum X_t Y_t-\sum X_t \sum Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\ & =\frac{\frac{\sum X_t Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\ & =\frac{\overline{X Y}-\overline{X} * \overline{Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \end{aligned} \]

\(\beta_0\)的估计

\[ \begin{aligned} \beta_0^{O L S} & =\frac{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t}{T \sum X_t^2-\left(\sum X_t\right)^2} \\ & =\frac{\frac{1}{T^2}\left\{\sum X_t^2 \sum Y_t-\sum X_t \sum X_t Y_t\right\}}{\frac{1}{T^2}\left\{T \sum X_t^2-\left(\sum X_t\right)^2\right\}} \\ & =\frac{\frac{\sum X_t^2 \sum Y_t}{T}-\frac{\sum X_t}{T} \frac{\sum X_t Y_t}{T}}{\frac{\sum X_t^2}{T}-\left(\frac{\sum X_t}{T}\right)^2} \\ & =\frac{\overline{X^2} * \overline{Y}-\overline{X} * \overline{X Y}}{\overline{X^2}-\left(\overline{X}\right)^2} \end{aligned} \]

评论