CHAPTER 7: MULTIPLE REGRESSION ANALYSIS: THE PROBLEM OF ESTIMATION

ü  The Three-Variable Model Notation and Assumptions

Y_i = β₁ + β₂X2_i + β₃X_3i + u_i

β2 and β3 are called the partial regression coefficients.

Assumptions:

1.      Zero mean value of u_i or

       E(u_i  X_2i, X_3i) = 0 for each i

2.      No serial correlation, or

cov(u_i, u_j) = 0         i   j

3.      Homoscedasticity, or

var(u_i) = 

4.      Zero covariance between u_i and each X variable, or

cov(u_i, X_2i) = cov(u_i, X_3i) = 0

5.      No specification bias, or

The model is correctly specified

6.      No exact collinearity between the X variables, or

No exact linear relationship between X₂ and X₃.

ü  Interpretation of Multiple Regression Equation

E(Y_i  X_2i, X_3i) = β₁ + β₂X_2i + β_3iX_3i

The conditional mean or expected value of Y conditional upon the given or fixed values of X₂ and X₃.

ü  The Meaning of Partial Regression Coefficients

β₂ and β₃ are known as partial regression or partial slope coefficients. β₂ measures the change in the mean value of Y, E(Y), per unit change in X₂, holding the value of X₃ constant. Put differently, it gives the “direct” or “net” effect of a unit change in X₂ on the mean value of Y, net of any effect that X₃ may have on mean Y. Likewise, β₃ measures the change in the mean value of Y per unit change in X₃, holding the value of X₂ constant. That is, it gives the “direct” or “net” effect of a unit change in X₃ on the mean value of Y, net of any effect that X₂ may have on mean Y.

ü  OLS and ML Estimation of the Partial Regression Coefficients

ü  OLS Estimators

OLS  estimator of the population intercept β₁

β₁ = Y – β₂X₂ – β₃X₃

OLS  estimator of the population intercept β₂

β₂ = (∑Y_iX_2i)(∑X_3i) – (∑Y_iX_3i)(∑X_2iX_3i)

                    (∑X_2i)(∑X_3i) – (∑X_2i3i)²

OLS  estimator of the population intercept β₃

β₃ = (∑_YiX_3i)(∑X_2i) – (∑Y_iX_2i)(∑X_2iX_3i)

                     (∑X_2i)(∑X_3i) – (∑X_2i3i)²

Notation:

1)      Equations of β₂ and β₃ are symmetrical in nature because one can be obtained from the other by interchanging the roles of X₂ and X₃.

2)      The denominators of these two equations are identical.

3)      The three-variable case is a natural extension of the two-variable case.

ü  Variances and Standard Errors of OLS Estimators

var(β₁) =  1   +     X₂∑X_3i + X₃∑X_2i - 2X₂X₃∑X_2iX_3i

                   n               ∑X_2i∑X_3i – (∑X_2iX_3i)2

se(β₁) =   +   var(β₁)

var(β₂) =                ∑X_3i

                  (∑X_2i)(∑X_3i) – (∑X_2iX_3i)2

or equivalently,

var(β₂) =     

                 ∑X_2i(1 – r₂₃)

var(β₃) =         ∑X_2i

                 (∑X_2i)(∑X_3i) – (∑X_2iX_3i)2

se(β₂) =   +    var(β₂)

se(β₃) =   +    var(β₃)

cov(β₂, β₃) =        -r₂₃

                        (1 – r₂₃)  X_2i   X_3i

ü  Properties of OLS Estimators

1)      The three-variable regression line passes through the means Y, X₂ and X₃.

2)      The mean value of the estimated Y_i(= Y_i) is equal to the mean value of the actual Y_i.

3)      ∑u_i = u =0

4)      The residuals u_i are uncorrelated with X_2i and X_3i, that is, ∑u_iX_2i = ∑u_iX_3i = 0

5)      The residuals u_i are uncorrelated with Y_i, that is, ∑u_iY_i = 0

6)      The correlation coefficient between X₂ and X₃ increases toward 1, the variances of β₂ and β₃ increase for given values of        and ∑X_2i or ∑X_3i.

ü  The Multiple Coefficient of Determination R² and the Multiple Coefficient of Correlation R

r² = measure the goodness of fit of the regression equation; that is, it gives the proportion or percentage, of the total variation in the dependent variable Y explained by the explanatory variable X.

R² = RSS

        TSS

R² = β₂∑Y_iX_2i + β₃Y_iX_3i

        ∑Y_i²

ü  Simple Regression in the Context of Multiple Regression: Introduction to Specification Bias

Observe several things about this regression compared to the “true” multiple regression.

1.      In absolute terms, the PGNP coefficient has increased.

2.      The standard errors are different.

3.      The intercept values are different.

4.      The r² values are dramatically different.

ü  Interpretation of Simple and Partial Correlation Coefficients

1.      Even if r₁₂ = 0, r₁₂₃ will not be zero unless r₁₃ or r₂₃ or both are zero.

2.      If r₁₂ = 0 and r₁₃ and r₂₃ are nonzero and are of the same sign, r₁₂₃ will be negative, whereas if they are of the opposite signs, it will be positive.

3.      In terms of r₁₂₃ and r₁₂ need not have the same sign.

4.      In the two-variable case we have seen that r² lies between 0 and 1.

5.      The coefficient of partial determination may be interpreted as the proportion of the variation in Y not explained by the variable X₃ that has been explained by the inclusion of X₃ into the model.