GARCHBACKTEST.RPF

GARCHBACKTEST.RPF provides an example of rolling estimates for a GARCH model, with analysis of the performance of the model's out-of-sample estimates of Value-at-Risk.

This uses the entire sample up to a given point to estimate the GARCH model, which is the standard procedure for simulated out-of-sample analysis. It’s possible to adjust this for a moving fixed-width window, but you have to be very careful about making an estimation window too narrow. Remember that with any type of rolling non-linear model, you are hoping that the estimator will work successfully every period because you’re not estimating one “model” but possibly hundreds. If you have a data set with long quiet stretches interrupted by some noisy patches, the moving window GARCH estimates can get very unstable if the window moves out of a quiet range. It’s also not clear what the point is of using the fixed-width window rather than using all of the previous data—given a set of estimated parameters, the calculated value of the GARCH variance rarely depends on more than about the last 100 data points anyway, so the only effect of the rolling sample is to change the parameters, not (necessarily) for the better.

This is the tail probability to evaluate. This is higher than would be typical, but at least for this sample, the Markov test done later doesn’t work if this is smaller (there are no back-to-back failures at .05 (or smaller), so the empirical transition probability from failure to failure is zero).

compute alpha=.10

Period of back test (roughly the last two years of the data, excluding the last period).

compute tstart=%allocend()-520

compute tend =%allocend()-1

TRIGGER will be a dummy variable which is 1 if the observed data is below the model’s estimate for the VaR.

set trigger tstart tend = 0.0

INFOBOX is useful here because it takes a while to do over 500 GARCH estimates on 6000 data points. It puts up a progress bar which notifies you of the progress (and approximate time until completion).

infobox(action=define,lower=tstart,upper=tend,progress) $

"GARCH Model Backtest"

This estimates the GARCH model over the full sample and saves the estimates (into FULLBETA) and the inverse Hessian (into FULLXX) to be fed back into GARCH for the other samples. Note the PRINT option. This is to contrast with the NOPRINT inside the loop. Make sure before you go on that you actually have a model which you can fit successfully. (In practice, you shouldn’t even write the rest of this until you can get a GARCH model which fits for the full sample).

garch(p=1,q=1,resids=u,hseries=h,print) / x

compute fullbeta=%beta,fullxx=%xx

This is the start of the loop:

do end=tstart,tend

This is the GARCH instruction inside the loop. The end parameter on the range is the END that’s being used for the DO index. This puts the NOPRINT option on and increases the iteration limit to 400 (just in case). If you’re having problems with a loop like this, cut the loop range to a more manageable length and take the NOPRINT option off.

garch(p=1,q=1,resids=u,hseries=h,noprint,$

initial=fullbeta,hessian=fullxx,iters=400) * end x

This computes the one-step forecast for the variance (for END+1 given END).

compute hhat=%beta(2)+%beta(3)*u(end)^2+%beta(4)*h(end)

And this computes the model’s prediction for the VaR. Because the model allows for a non-zero mean, the one-step prediction is Normal with mean %BETA(1) and variance given by HHAT. This computes the lower ALPHA tail value for that distribution.

compute limit=%beta(1)+%invnormal(alpha)*sqrt(hhat)

This computes the observed value at END+1 with the model’s VaR calculation. If it’s smaller, TRIGGER is set to 1 (otherwise 0).

compute trigger(end)=(x(end+1)<limit)

This updates the progress bar and also puts a 1 (for good) or 0 (for bad) on the second line. You should check that that’s staying at 1. If not, you should take the advice above, and cut down the loop range and get rid the NOPRINT to see what’s happening.

infobox(current=end) "Model converged "+%converged

end do end

infobox(action=remove)

The remainder of the program is just number-crunching the values of the TRIGGER dummy. The first computes Kupiec’s likelihood ratio test for the observed alpha matching the chosen one:

sstats(mean) tstart tend trigger>>alphahat

compute pof=2*%nobs*((1-alphahat)*log((1-alphahat)/(1-alpha))+$

alphahat*log(alphahat/alpha))

disp "POF measure" pof

This has a $\chi _1^2 $ distribution under the null and rejects if the percentage of exceedances is either too large or too small.

This computes Christofferson’s Markov measure of independence. This also has a $\chi _1^2 $ under the null.

sstats tstart+1 tend .not.trigger.and..not.trigger{1}>>n1 $

.not.trigger.and.trigger{1}>>n2 $

trigger.and..not.trigger{1}>>n3 $

trigger.and.trigger{1}>>n4

compute p00=n1/(n1+n3),p10=n2/(n2+n4),p0=(n1+n2)/(n1+n2+n3+n4)

compute markov=n1*log(p00)+n3*log(1-p00)+n2*log(p10)+$

n4*log(1-p10)-((n1+n2)*log(p0)+(n3+n4)*log(1-p0))

disp "Markov test" markov

Full Program

all 6237
open data g10xrate.xls
data(format=xls,org=columns) / usxjpn
*
* Convert to percent daily returns
*
set x = 100.0*log(usxjpn/usxjpn{1})
*
* Tail probability to evaluate
*
compute alpha=.10
*
* Period of back test (roughly the last two years of the data, excluding
* the last period).
*
compute tstart=%allocend()-520
compute tend =%allocend()-1
*
set trigger tstart tend = 0.0
garch(p=1,q=1,resids=u,hseries=h,print) / x
compute fullbeta=%beta,fullxx=%xx
infobox(action=define,lower=tstart,upper=tend,progress) $
   "GARCH Model Backtest"
do end=tstart,tend
   garch(p=1,q=1,resids=u,hseries=h,noprint,$
      initial=fullbeta,hessian=fullxx,iters=400) * end x
   *
   * Compute the one-step forecast for the variance
   *
   compute hhat=%beta(2)+%beta(3)*u(end)^2+%beta(4)*h(end)
   *
   * Compute the model's prediction for the VaR
   *
   compute limit=%beta(1)+%invnormal(alpha)*sqrt(hhat)
   *
   compute trigger(end)=(x(end+1)<limit)
   infobox(current=end) "Model converged "+%converged
end do end
infobox(action=remove)
*
* Kupiec measure
*
sstats(mean) tstart tend trigger>>alphahat
compute pof=2*%nobs*((1-alphahat)*log((1-alphahat)/(1-alpha))+$
    alphahat*log(alphahat/alpha))
disp "POF measure" pof
*
* Christofferson Markov measure
*
sstats tstart+1 tend .not.trigger.and..not.trigger{1}>>n1 $
                     .not.trigger.and.trigger{1}>>n2 $
                     trigger.and..not.trigger{1}>>n3 $
                     trigger.and.trigger{1}>>n4
compute p00=n1/(n1+n3),p10=n2/(n2+n4),p0=(n1+n2)/(n1+n2+n3+n4)
*
compute markov=n1*log(p00)+n3*log(1-p00)+n2*log(p10)+$
   n4*log(1-p10)-((n1+n2)*log(p0)+(n3+n4)*log(1-p0))
disp "Markov test" markov