Linear Regression: Standard Normal Linear Model

The first multiple regression model examined in almost any statistics text takes the form

\begin{equation} y_t = X_t \beta + u_t \label{eq:linreg_basereg} \end{equation}

with

\begin{equation} u_t \sim N(0,\sigma^2) \text{ i.i.d.} \label{eq:linreg_snlm} \end{equation}

Assumption \eqref{eq:linreg_snlm} is usually made conditional upon the full \(\bf{X}\) matrix, or, even stronger, \(\bf{X}\) is assumed to be independent of \(u\) entirely. The least squares estimator is

\begin{equation} \hat \beta = \left( {{\bf{X'X}}} \right)^{ - 1} {\bf{X'y}} \end{equation}

where \(\bf{X}\) and \(\bf{y}\) are the matrices formed by stacking the individual observations. The residual for observation \(t\) is

\begin{equation} \hat u_t = y_t - X_t \hat \beta \end{equation}

Under those assumptions, least squares has all kinds of optimal properties: it’s the best linear unbiased estimator (BLUE), it’s the maximum likelihood estimator. Under very weak assumptions, it is consistent, and the least squares estimator is Normally distributed, even in finite samples. The covariance matrix of \({\hat \beta }\), if \(\sigma^2\) is known, is

\begin{equation} \sigma ^2 \left( {{\bf{X'X}}} \right)^{ - 1} \end{equation}

When (as is typical) \(\sigma^2\) is not known, it can be estimated by either

\begin{equation} \hat \sigma ^2 = \frac{1}{T}\sum {\hat u_t^2 } \end{equation}

which is the maximum likelihood estimator, or

\begin{equation} s^2 = \frac{1}{{T - K}}\sum {\hat u_t^2 } \end{equation}

where \(K\) is the number of regressors. This is the unbiased estimator, typically called the (Squared) Standard Error of Estimate. (\(s\) itself is the Standard Error of Estimate). This is, by far, the most common choice, and is what shown in regression output if you just use the default options for the estimator.

The LINREG Instruction

The primary RATS instruction for estimating linear regressions is LINREG. In its simplest form, it estimates the standard regression model described above. We discuss LINREG in some detail in the discussion of Example Three of the Introduction. Linear Regression: A General Framework is devoted to cases where some of the assumptions described above do not hold.

As a brief reminder, a simple linear regression can be handled using an instruction like this:

linreg rate

# constant ip m1diff ppisum

This regresses the series RATE on a constant and three regressors. You can find the full example in the file ExampleThree.rpf. The output produced by this LINREG is described in detail in Regression Output.

Here is another example, also taken from the ExampleThree.rpf example program:

linreg rate

# constant ip grm2 grppi{1}

Here, we use the lag notation “{1}” to include the one-period lag of GRPPI as an explanatory variable. Note not grppi(-1) as is used in some other software.

The Regressions Wizard

You can use the Statistics—Linear Regressions wizard to estimate linear regression models.