The 'Davies' Test

alexecon · Unread post by **alexecon** » Tue Jan 24, 2017 6:52 am

I was going through Chapter 7 of Time Series Econometrics Analysis by W Enders (4th ed.), as I was trying to find out more about how to test for the presence or not of non-linearity in a VAR. Apparently, the usual LR test (~chi-sq) is not recommended because under the null there is (are) unidentified parameter(s). Terasvirta (2006) also discusses this.

The solution (?) is to use Davies's (1987) sup test. Enders outlines in his book how to construct critical values but does not accompany the discussion with a relevant program. I have seen implementations of Davies's test both in a univariate context (Garcia and Perron, 1996) http://www.jstor.org/stable/2109851, and a multivariate one (Balcilar et al, 2015). https://doi.org/10.1016/j.eneco.2015.01.026

Would it be worth to add this functionality to the wealth of procedures already available in RATS?

alexecon · Unread post by **alexecon** » Thu Jan 26, 2017 6:15 am

Assuming that the likelihood ratio has a single peak, the upper bound of this test can be calculated as follows:

Pr[χ2 >M]+2(M/2)^(q/2) exp(−M/2)/Γ(q/2)

where M=2(lnL1-lnL0) is the observed likelihood, q are parameters that appear only under the alternative hypothesis, and Γ is the Gamma function. [Davies (1987) and Garcia and Perron (1996)]
Any suggestions as to how this can be written in RATS? Many thanks in advance for any help.

PS: This test is also relevant to non-VAR settings of course so I may have posted in a less-than-ideal topic. Please move it if you think that's the case!

TomDoan · Unread post by **TomDoan** » Thu Jan 26, 2017 11:04 am

You're missing a 2 on the LR statistic, but isn't everything in that directly computable?

alexecon · Unread post by **alexecon** » Thu Jan 26, 2017 11:26 am

TomDoan wrote:You're missing a 2 on the LR statistic, but isn't everything in that directly computable?

Yes (I will edit), -and yes. What I wasn't sure was how to invoke the Gamma distribution but I have now found it in the documentation. Thanks!

alexecon · Unread post by **alexecon** » Thu Jan 26, 2017 4:35 pm

I am trying to apply this test to the MS-VECM model in the Balcilar et al (2015) paper. (The authors report that they reject the restricted linear model using the Davies upper bound.) Following estimation of the non-switching and switching VECMs using the code here viewtopic.php?f=8&t=2536&hilit=msvecm, is it possible to obtain the two log likelihoods?

TomDoan · Unread post by **TomDoan** » Thu Jan 26, 2017 4:47 pm

Sure the ESTIMATE (which does a one-regime VECM) produces %LOGL and the MAXIMIZE (which does the MS VECM) does also. Save the values into other variables, and compute the LR test statistic by 2 x their difference.

TomDoan · Unread post by **TomDoan** » Mon Nov 20, 2017 1:20 pm

I'm not convinced that this is a proper use of Davies(1987). Davies never mentions applicability to asymptotic chi-squared distributions---only actual finite sample chi-squared distributions (from linear models with normal residuals). He also allows for only a single unidentified parameter under the null (the adjustment is based upon an approximation to the integral over that), while a MS model will have two in testing two regimes against one (the two transition probabilities). The other references all go back to Garcia and Perron, which has an incorrect summary of the result from Davies.

alexecon · Unread post by **alexecon** » Wed Nov 22, 2017 9:52 am

Thanks for the nudge. Davies (1987) is a 30-year old reference now so the literature has indeed moved on. Interesting, however, that it's still used in recent MS papers, e.g. as in Balcilar et al and discussed at some length in Enders' book. Looking at more recent contributions, the following claim to offer tangible improvements:

HANSEN, B. (1996): “Inference When a Nuisance Parameter Is Not Identified Under the Null Hypothesis,” Econometrica, 64, 413–430.
GARCIA, R. (1998): “Asymptotic Null Distribution of the Likelihood Ratio Test in Markov Switching Models,” International Economic Review, 39, 763–788.
CHO, J. S., AND H. WHITE (2007): “Testing for Regime Switching,” Econometrica, 75, 1671–1720.
CARRASCO, M., HU, L. AND PLOBERGER, W. (2014): “Optimal Test for Markov Switching Parameters. Econometrica,” 82: 765–784. http://onlinelibrary.wiley.com/doi/10.3 ... A8609/full

Clearly there's value in testing for constant means and variances across regimes instead of presuming the validity of a regime-switching model. E.g. Carrasco et al cannot reject the null of a constant mean in Hamilton's model. I'm working in a VAR/VECM setting and allow for shifts in the process but cannot say with any confidence whether this is the appropriate modelling decision. I can only rely on the statistical significance of the estimated parameters across regimes and the regime-dependent impulse responses, but the lack of a formal testing procedure places a serious limitation on this kind of work.

What do you think of the above papers? Carrasco supplies the GAUSS code here: https://www.webdepot.umontreal.ca/Usage ... /index.htm. The data and programme needed to replicate Table 3 in their paper follow.

Data:

Code: Select all

Programme:

Code: Select all

/* This program applies CHP to test for Markov Switching in Hamilton's AR model on GNP */
/* test for switching in the mean only */
/* Hamilton's original series: t = 135 for gnp82.dat */
/* Extended series: t = 239 for gnpc96.txt */
/* critical value and p-values are reported */
/* Table 3 in the resubmission */

new;

rndseed 2938438;

t = 239;
rho_b = 0.7;
itnb = 3000;
load gnp[]=c:/paper/werner/revision/application/hamilton/gnpc96.txt;

gnp = 100*ln(gnp);
dat = gnp[2:t+1] - gnp[1:t];

nar = 4;         

nn = t-nar;

y = dat[1+nar:t];
x = dat[nar:t-1];

it = 2; 
do while it <= nar;
  x = x~dat[nar-it+1:t-it];
  it = it + 1; 
endo;

/* first estimate the AR under H0 */

xx = ones(nn,1)~x;

b0 = inv(xx'xx)*(xx'y);
u = y - xx*b0;
v0 = sqrt((u'u)/rows(u));
var0 = inv(xx'xx)*((xx.*u)'(xx.*u))*inv(xx'xx);
se0 = sqrt(diag(var0));

phi = b0[2:1+nar];      @AR cofficients@
mu0 = b0[1]/(1-sumc(phi));  @ mean of GNP growth @

/* get the 1st derivative of the log likelihood, use lt as prefix */

ltmu = u*(1-sumc(phi))/v0^2;

ltphi = u.*(x[.,1]-mu0)/v0^2;

for i(2,nar,1);
    ltphi = ltphi~(u.*(x[.,i]-mu0)/v0^2);
endfor;

ltsig2 = u^2/(2*v0^4)-1/(2*v0^2);

ltmx = ltmu~ltphi~ltsig2;

/* get the 2nd derivative of the log likelihood, use mt as prefix */
/* In this case, we only need the derivative wrt mu */

mtmu = -(1-sumc(phi))^2/v0^2;   @scalar@


/* calculate our test statistic supTS and expTS */

rho = -rho_b;   @lower bound of rho@
ir = 1;           @iterations for rho@

cv = zeros(200,1); @stores supTS critical values for each rho@

cv2 = zeros(200,1);    @stores expTS critical values for each rho@


do while rho<=rho_b + 0.01;

mu2t = zeros(nn,1);
xs = zeros(nn,1);

xs[2] = rho * ltmu[1];
mu2t[2] = ltmu[2] * xs[2];

for i(3,nn,1);
    xs[i] = rho * (xs[i-1] + ltmu[i-1]);
    mu2t[i] = ltmu[i] * xs[i];

endfor;

mu2t = (mtmu+ltmu^2)/2+mu2t;
gamma_e = 1/sqrt(t)*sumc(mu2t);

epsilont = mu2t-ltmx*inv(ltmx'ltmx)*ltmx'mu2t; @projection of mu2t on lt1@
esqe = meanc(epsilont^2);

tspe = gamma_e/sqrt(esqe);

if esqe<1e-5;
    cv[ir] = 0;
    cv2[ir] = 1;
else;
    cv[ir] = 1/2*maxc(tspe|0)^2;    @supTS test statistic@ 
    cv2[ir] = sqrt(2*pi)*exp((tspe-1)^2/2)*cdfn(tspe-1);  @expTS test statistic@
endif;

rho = rho+0.01;
ir = ir+1;
endo;

cv = cv[1:ir-1];

cv2 = cv2[1:ir-1];

supts = maxc(cv);
expts = meanc(cv2);
print /flush cv~cv2;
/* bootstrap the critical values */
supb = zeros(itnb,1);   @stores the bootstrapped supTS critical value@
expb = zeros(itnb,1);   @stores the bootstrapped expTS critical value@


for itb(1,itnb,1);

/* first simulate the series itnb(e.g., 3000) times according to ML estimators */

ys = zeros(t,1);
ys[1:nar] = meanc(y)+stdc(y)*rndn(nar,1);   @start from stationary distribution@

for i(nar+1,t,1);
    ys[i] = b0'(1|ys[i-1:i-nar])+v0*rndn(1,1);
endfor;

yb = ys[1+nar:t];
xb = ys[nar:t-1];

it = 2; 
do while it <= nar;
  xb = xb~ys[nar-it+1:t-it];
  it = it + 1; 
endo;

/* first estimate the AR under H0 */

xxb = ones(nn,1)~xb;

b0b = inv(xxb'xxb)*(xxb'yb);
ub = yb - xxb*b0b;
v0b = sqrt((ub'ub)/rows(ub));

phib = b0b[2:1+nar];      @AR cofficients@
mu0b = b0b[1]/(1-sumc(phib));  @ mean of GNP growth @

/* get the 1st derivative of the log likelihood, use lt as prefix */

ltmub = ub*(1-sumc(phib))/v0b^2;

ltphib = ub.*(xb[.,1]-mu0b)/v0b^2;

for i(2,nar,1);
    ltphib = ltphib~(ub.*(xb[.,i]-mu0b)/v0b^2);
endfor;

ltsig2b = ub^2/(2*v0b^4)-1/(2*v0b^2);

ltmxb = ltmub~ltphib~ltsig2b;

/* get the 2nd derivative of the log likelihood, use mt as prefix */

mtmub = -(1-sumc(phib))^2/v0b^2;


/* calculate our test statistic supTS and expTS */

rho = -rho_b;   @lower bound of rho@
ir = 1;           @iterations for rho@

cv = zeros(200,1); @stores supTS critical values for each rho@

cv2 = zeros(200,1);    @stores expTS critical values for each rho@


do while rho<=rho_b + 0.01;


mu2tb = zeros(nn,1);
xsb = zeros(nn,1);

xsb[2] = rho * ltmub[1];
mu2tb[2] = ltmub[2] * xsb[2];

for i(3,nn,1);
    xsb[i] = rho * (xsb[i-1] + ltmub[i-1]);
    mu2tb[i] = ltmub[i] * xsb[i];

endfor;

mu2tb = (mtmub+ltmub^2)/2+mu2tb;
gamma_e = 1/sqrt(t)*sumc(mu2tb);


epsilont = mu2tb-ltmxb*inv(ltmxb'ltmxb)*ltmxb'mu2tb; @projection of mu2t on lt1@
esqe = meanc(epsilont^2);
tspe = gamma_e/sqrt(esqe);  @test statistic process@

if esqe<1e-5;
    cv[ir] = 0;
    cv2[ir] = 1;
else;
    cv[ir] = 1/2*maxc(tspe|0)^2;    @supTS test statistic@ 
    cv2[ir] = sqrt(2*pi)*exp((tspe-1)^2/2)*cdfn(tspe-1);  @expTS test statistic@
endif;

rho = rho+0.01;
ir = ir+1;
endo;

cv = cv[1:ir-1];

cv2 = cv2[1:ir-1];

supb[itb] = maxc(cv);
expb[itb] = meanc(cv2);

endfor;

/*empirical critical values */

supb = sortc(supb,1);
expb = sortc(expb,1);

crtsup99=supb[floor(itnb*0.99)];
crtsup95=supb[floor(itnb*0.95)];
crtsup90=supb[floor(itnb*0.90)];   

crtexp99=expb[floor(itnb*0.99)];
crtexp95=expb[floor(itnb*0.95)];
crtexp90=expb[floor(itnb*0.90)];   

"Estimation Under Null of Linearity";
" Regression Parameters   Standard Error"
b0~se0;
"";
"Standard Deviation : " v0;
"";
"our test statstic ";
"supTS is " supts;
"expTS is " expts;

"Iteration of simulation is " itnb;
"supTS critical values are " crtsup99 crtsup95 crtsup90;
"expTS critical values are " crtexp99 crtexp95 crtexp90;
"supTS p-value is " meanc(supts.<supb);
"expTS p-value is " meanc(expts.<expb);

end;

TomDoan · Unread post by **TomDoan** » Wed Nov 22, 2017 10:26 am

Hansen and Garcia both involve a staggering amount of number-crunching.

Is there anything wrong with using SBC? It's a lot simpler than doing bootstrapping.

alexecon · Unread post by **alexecon** » Wed Nov 22, 2017 11:01 am

I don’t know. Is testing with the SBC not subject to the issue of nuisance parameters? It seems surprising that so much work has gone into resolving this, if the SBC were to be the answer.

TomDoan · Unread post by **TomDoan** » Wed Nov 22, 2017 11:18 am

alexecon wrote:I don’t know. Is testing with the SBC not subject to the issue of nuisance parameters? It seems surprising that so much work has gone into resolving this, if the SBC were to be the answer.

You don't get papers published in Econometrica by using well-known methods.

There are certain regularity conditions in the derivation of the SBC that aren't met by nested Markov Switching models. However, in practice it appears to give effectively the same results as the very complicated calculations of the actual posterior odds of the models (SBC is based upon asymptotic posterior odds). See the discussion in Fruhwirth-Schnatter's(2006) "Finite Mixture and Markov Switching Models".

alexecon · Unread post by **alexecon** » Wed Nov 22, 2017 1:05 pm

Thank you for the reference. I can see that Fruhwirth-Schnatter uses the SBC to select the number of regimes in chapter 11. I have to read a couple of the references therein to understand more, as I got confused with the "bridge", "importance", and "reciprocal importance" sampling estimators. Still, I take it that these aren't necessary and that the log-likelihoods obtained from RATS estimation of Ehrmann et al (2003) and Balcilar et al (2015) can be used to construct the BIC given on page 347? Or am I being naive here?

TomDoan · Unread post by **TomDoan** » Wed Nov 22, 2017 1:32 pm

alexecon wrote:Thank you for the reference. I can see that Fruhwirth-Schnatter uses the SBC to select the number of regimes in chapter 11. I have to read a couple of the references therein to understand more, as I got confused with the "bridge", "importance", and "reciprocal importance" sampling estimators.

Those are all MCMC techniques for computing the posterior odds.

alexecon wrote: Still, I take it that these aren't necessary and that the log-likelihoods obtained from RATS estimation of Ehrmann et al (2003) and Balcilar et al (2015) can be used to construct the BIC given on page 347? Or am I being naive here?

EEV should be fine. Balcilar et al is a different situation because of the unbounded likelihood function due to the prolonged flat spot in the oil price. The randomized values allow the model to be estimated, but the peak log likelihood is still misleadingly high.

alexecon · Unread post by **alexecon** » Wed Nov 22, 2017 1:46 pm

I'm not using their (Balcilar et al's) data and 'my' likelihood is better behaved. I guess it should be OK.

The RATS Software Forum

The 'Davies' Test

The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test

Re: The 'Davies' Test