goodness of fit measures for different garch models

[anozman]

Hi Tom,

I have been using the following code to find the number of iterations needed for Simplex before using BFGS. For example, when I run the code, it tells me that it is converged at i=4. But when I use 4 to replace i in the first max (method=Simplex, iters=4,...) and re-run it, it tells me it is not converged. I was wondering whether you know where the problem is.

Thanks
anozman

LINGREG(NOPRINT) V0 / res
# Constant V1 V2 V3 V4…… V23

frml(lastreg,vector=beta) meanf
nonlin(parmset=meanparms) beta

set u = %resids
set uu = u^2
set h = %seesq
*
nonlin(parmset=garchparms) c shift
compute c=0.2,shift=5.0
frml varf = c+shift*V2
compute nu=6.0
*
source logskewtdensity.src
nonlin(parmset=distparms) lambda eta
compute eta=nu,lambda=0.0
*
do i=0,10
frml loglsst = (h=varf),(u=V0-meanf),(uu=u^2),(-0.5*log(h)+%logskewtdensity(u/sqrt(h),eta,lambda))
maximize(noprint,method=SIMPLEX,iters=i,parmset=meanparms+garchparms+distparms) loglsst
maximiz(print,method=bfgs,iters=2000,subiterations=1000000000,cvcrit=0.000000001,robust,LAGS=3,LWINDOW=NEWEYWEST,parmset=meanparms+garchparms+distparms) loglsst
if %converged==1;dis 'Theoretical Model Specification 1 is converged at' i
end do i
dis 'Total number of runs is' i

[moderator]

First, I don't think an "automated" search over a range of simplex iterations really makes much sense. You certainly may need to try different numbers of preliminary simplex iterations, and for a given model and a given set of initial parameter values, you might find that there is a range of simplex iterations that works best (i.e. that fewer or more simplex iterations results in a failure to converge or convergence to a lower maximum). But I don't think there's anything to be gained over a formal search on this quantity, and it is highly unlikely that you would find a "magic" number of simplex iterations that works where others fail.

Also, assuming you do get the model to converge with a given set of initial values and estimation settings, you should still experiment with other initial values and settings, to try to make sure you aren't arriving at a local, rather than a global, max.

Second, setting a convergence criteria that tight is almost certainly a bad idea. If you are having trouble getting convergence, tightening up the convergence criteria is about the last thing you'd want to do.

Finally, the subiterations limit you are using is almost certainly several orders of magnitude larger than would be desirable. I would suggest trying values on the order of 100 or 200 at most. If you are still hitting the subiterations limit at that point, you need to try something else (different initial values, different estimation method, more or fewer simplex iterations, different model, etc.). Any further increase in the subiterations limit probably won't help, and just wastes time.

Regards,
Tom Maycock
Estima

[TomDoan]

Yes Tom. The garch effect is not strong. in fact, a single shift paramter in the variance equation is sufficient to make the standardised residuals behave well. i am just not sure whether the violation of the skewness will have an impact on my results as t and GED can deal with fat tails, but not the skewness. unfortunately, almost all financial series related models suffer from this little problem. i was wondering how other people adjust their models to deal with this problem using RATS. can LOGSKEWTDENSITY be used in this case?

thanks Tom
anozman

If you don't have much of a GARCH effect, using a complicated density for fine-tuning a GARCH fit isn't a good idea. You are very unlikely to suddenly discover GARCH behavior with fat tails and skewness that wasn't there without them.

[anozman]

Thanks Tom for your advice!

Unfortunately, the ARCH effect does exist, but it can be dealt with easily. The problem is the raw residuals that have fat tails and are also skewed. This is the only reason why I have decided not to use the RATS pre-programmed garch routine but code it using the skewed t. However, it seems it is a bit difficult to achieve the convergence for some model specifications. I have been thinking about how to improve this in a systematic way using RATS. Do you have any other suggestions that can deal with the residual distributional issues in an easier way with available RATS codes (or I can modify them to achieve the goal)?

Thanks for your help!

anozman

[anozman]

Thanks Tom Maycock for your suggestions!

I can not understand the logic behind the convergence and non-convergence result based on my few lines of code. There could be potentially two reasons:

1. My little code has problems imbedded in it, so it does not run according to what I expect it to do;
2. Maybe the re-starting values are not the same based on certain choice of i for simplex. However, if I set i=4, it should give me the same result as it shows convergence when i was searched and the littlie code indicates convergence at 4 even with the model result shown when "noprint" is removed.

As I am fairly new to RATS, if anyone can see the problem in the code, please let me know.

Thanks in advance!

anozman

[moderator]

It doesn't appear that you are doing anything to reset the parameter values back to their original initial values before each trip through the loop. So, the second "simplex" estimation (for i=1) is starting off from the final estimates produced by the first (i=0) BFGS estimation, which is why the behavior isn't what you expect.

For this to work as intended, you would need to add some COMPUTE and SET instructions to save the initial parameter values, residuals, and variances, and reset the relevant variables back to those variables at the start of each loop iteration.

Again, though, the whole idea seems a bit flawed to me, but yes, for it to make any sense at all, you need to be starting each SIMPLX/BFGS sequence from the same set of starting values.

Regards,
Tom Maycock

[TomDoan]

Thanks Tom for your advice!

Unfortunately, the ARCH effect does exist, but it can be dealt with easily. The problem is the raw residuals that have fat tails and are also skewed. This is the only reason why I have decided not to use the RATS pre-programmed garch routine but code it using the skewed t. However, it seems it is a bit difficult to achieve the convergence for some model specifications. I have been thinking about how to improve this in a systematic way using RATS. Do you have any other suggestions that can deal with the residual distributional issues in an easier way with available RATS codes (or I can modify them to achieve the goal)?

Thanks for your help!

anozman

The raw residuals should have fat tails, even if the shocks are conditionally Normal. ARCH/GARCH models can explain both fat tails and clustering. There are almost no interesting diagnostics on the raw residuals from a GARCH model.