Clark-McCracken test

[juffa]

The interpretation of the Clark-McCracken test:

The null hypothesis is that there is no difference between the forecasting ability of the restricted and the unrestricted models. If the null is rejected, the unrestricted model yields better forecasts. Right?

But what if the forecasts of the unrestricted model are considerably worse than the forecasts of the restricted model, as in the example below? Is it possible to infer, for example on the basis of the MSF-F test statistic, that the forecast of the restricted model are significantly better than the forecast of the unrestricted model.

***Restricted model
In-sample 1988:01 - 2007:04
Out-sample 2008:01 - 2011:02
ME MAE RMSE MSE
-------- -------- -------- --------
-2.22828 3.64396 5.18879 26.92357
***Unrestricted model
In-sample 1988:01 - 2007:04
Out-sample 2008:01 - 2011:02
ME MAE RMSE MSE
-------- -------- -------- --------
-0.69805 5.15850 6.57663 43.25203
*****
Actual pi val = 0.17500
:nearest CM pi val = 0.20000
:second nearest CM pi val = 0.10000
*****
number of restrictions in test = 4
**************************** test results ****************************


test test value asymptotic critical values (1-sided)
10% 5% 1%

MSF-F -5.285 1.184 1.864 3.258
MSF-t -1.241 0.781 1.142 1.766
ENC-F 0.358 0.990 1.410 2.287
ENC-t 0.690 1.186 1.595 2.208

[tclark]

The null is that the models are equally accurate in population (i.e., as the estimation and forecasting samples limit to infinity, and the parameters converge to true values). Because the models are nested, if the models are equally accurate in population, it has to be that the true model is the restricted model. So the tests are one-sided: we only reject the smaller model in favor of the larger. If the null is not rejected, it implies the true model is the smaller one. In that situation, in population, the models will be equally accurate. But when the models are estimated with finite samples of data, if the smaller model is the true one, we would expect it to be more accurate in the finite sample, and we should use that one for forecasting.

[alberta123]

Is the "ENC-F" test from the program output the same as "ENC-NEW" in Clark and McCracken (2001 Journal of Econometrics)?
"MSE-t" the same as the Diebold and Mariano (1995) test?

Is the program code of Clark and West (2007) "Approximately normal tests for equal predictive accuracy in nested models" Journal of Econometrics available?

Thanks.

[tclark]

The answer to each of your three questions is "yes." I have attached a .zip file with data and programs for producing the application results in the 2007 Clark-West paper.

[e1983]

Am I correct that in the @clarkforetest program that the critical values the program generates for the for the OOS-t and OOS-F are from "Asymptotics for Out of Sample Tests of Granger Causality" McCracken 2007?


test test value asymptotic critical values (1-sided)
10% 5% 1%

MSF-F -2.169 0.522 1.770 4.721
MSF-t -0.229 0.147 0.461 1.084
ENC-F 14.124 2.310 3.249 5.080
ENC-t 3.113 1.150 1.491 2.200

[tclark]

Yes, you are correct. There was some documentation in a PDF file in some version of this that said that, although it referred to the working paper version of the published McCracken paper.

[e1983]

Thanks for the reply. Much appreciated.