CHAPTER 13: ECONOMETRIC MODELING: MODEL SPECIFICATION AND DIAGNOSTIC TESTING

ü  Model Selection Criteria

1.      Be data admissible.

2.      Be consistent with theory.

3.      Have weekly exogenous regressors.

4.      Exhibit parameter constancy.

5.      Exhibit data coherency.

6.      Be encompassing.

ü  Types of Specification Errors

1.      Omission of relevant variable(s).

2.      Inclusion of an unnecessary variable(s).

3.      Adopting the wrong functional form.

4.      Errors of measurement.

5.      Incorrect specification of the stochastic error term.

ü  Consequences of Model Specification Errors

·         Underfitting a Model(Omitting a Relevant Variable)

The consequences of omitting variable X₃ are as follows:

1.      If the left-out, or omitted, variable X₃ is correlated with the included variable X₂, that is, r₂₃, the correlation coefficient between the two variables, is nonzero, α₁ and α₂ are biased as well as inconsistent.

2.      Even if X₂ and X₃ are not correlated, α₁ is biased, although α₂ is now unbiased.

3.      The disturbance variance         is incorrectly estimated.

4.      The conventionally measured variance of α₂(=         /∑X_2i ) is a biased estimator of the variance of the true estimator β₂.

5.      In consequence, the usual confidence interval and hypothesis testing procedures are likely to give misleading conclusions about the statistical significance of the estimated parameters.

6.      The forecasts based on the incorrect model and forecast intervals will be unreliable.

·         Inclusion of an Irrelevant Variable(Overfitting a Model)

The consequences of this specification error are as follows:

1.      The OLS estimators of the parameters of the “incorrect” model are all unbiased and consistent, that is, E(α₁) = β₁, E(α₂) = β₂ and E(α₃) = β₃ = 0.

2.      The error variance           is correctly estimated.

3.      The usual confidence interval and hypothesis testing procedures remain valid.

4.      However, the estimated α’s will be generally inefficient, that is, their variances will be generally larger than those of the β’s of the true model.

ü  Test of Specification Errors

·         Detecting the Presence of Unnecessary Variables(Overfitting a Model)

Bottom-up approach(data mining) – starting with a smaller model and expanding it as one goes along.

·         Tests for Omitted Variables and Incorrect Functional Form

-          Examination of Residuals

-          The Durbin-Watson d Statistic Once Again

To use the Durbin-Watson test for detecting model specification error(s) we proceed as follows:

1.      From this assumed model, obtain the OLS residuals.

2.      If it is believed that the assumed model is mis-specified because it excludes a relevant explanatory variable, say, Z from the model, order the residuals obtained in Step 1, according to increasing values of Z.

3.      Compute the d statistic.

4.      From the Durbin-Watson tables, if the estimated d value is significant, then one can accept the hypothesis of model mis-specification.

-          Ramsey’s RESET Test

Steps involved in RESET are as follows:

1.      From the chosen model, obtained the estimated Y_i, that is, Y_i.

2.      Rerun introducing Y_i in some form as an additional regressor(s).

3.      Use F test namely

F = (R²new – R²old)/number of new regressors

                               (1 – R²new)/(n – number of parameters in the new model)

4.      If the computed F value is significant, say, at the 5 percent level, one can accept the hypothesis that the model is mis-specified.

-          Lagrange Multiplier (LM) Test for Adding Variables

1.      Estimate the restricted regression by OLS and obtain the residuals, ui.

2.      If in fact the unrestricted regression

Y_i = β₁ + β₂X_i + β3X_i² + β₄X_i³ + u_i

             is the true regression, the residuals obtained in:

Y_i = λ₁ + λ₂X_i + u_3i

             should be related to the squared and cubed output terms, that is, X_i² and X_i³.

3.      This suggests that we regress the u_i obtained in step 1 on all the regressors, which in the present case means

u_i = α₁ + α₂X_i + α₃X_i² + α₄X_i³ + v_i

             where vi is an error term with the usual properties.

4.      Symbolically, we write

nR² _asy X²(number of restrictions)

5.      If the chi-square value obtained in nR² _asy X² exceeds the critical chi-square value at the chosen level of significance, we reject the restricted regression.

ü  Errors of Measurement

·         Errors of Measurement in the Dependent Variable Y

Although the errors of measurement in the dependent variable still give unbiased estimates of the parameters and their variances, the estimated variances are now larger in the case where there are no such errors of measurement.

ü  Tests of Non-nested Hypothesis

1.      Discrimination Approach – where given two or more competing models, one chooses a model based on some criteria of goodness of fit.

2.      Discerning Approach – where, in investigating one model, we take into account information provided by other models.

ü  Model Selection Criteria

·         The R² Criterion

R² = ESS = 1 – RSS

         TSS           TSS

·         Adjusted R²

R² = 1 – RSS/ (n – k) = 1 – (1 – R²)n – 1

               TSS/ (n – 1)                         n – k

·         Alkalke Information Criterion (AIC)

AIC = e^2kln ∑u_i² = e^2kln RSS

                       n                    n

ln AIC =     2k      +  ln    RSS

                   n                      n

·         Schwarz Information Criterion (SIC)

SIC = n^kln ∑u_i² = n^kln RSS

                     n                  n

ln SIC = k ln n   +    ln   RSS

              n                         n

·         Mallows’s Cp Criterion

Cp = RSSp – (n – 2p)

E(Cp)        (n – p)        - (n – 2p)     p

ü  Additional Topics in Econometric Modeling

·         Outliers, Leverage and Influence

Defined as an observation with a “large residual”

·         Recursive Least Squares

·         Chow’s Prediction Failure Test

ü  A Word to the Practitioner

“Ten Commandments of Applied Econometrics”

1.      Thou shalt use common sense and economic theory.

2.      Thou shalt ask the right questions.

3.      Thou shalt know the context.

4.      Thou shalt inspect the data.

5.      Thou shalt not worship complexity.

6.      Thou shalt look long and hard at thy results.

7.      Thou shalt beware the costs of data mining.

8.      Thou shalt be willing to compromise.

9.      Thou shalt not confuse significance with substance.

10.  Thou shalt confess in the presence of sensitivity.