Hi Tom,
The expected value of the violation ratio (VR) is 1 for h=1 step ahead.
Kupiec and Christofferson tests https://estima.com/webhelp/topics/garchbacktestrpf.html are designed for 1-step ahead forecasts only.
To generate multi-step ahead forecasts for various risk algo's e.g. HS, (G)ARCH-family, I have used either sqrt-of-time-rule (former) or the dynamic recursive nature of the forecast equation (latter) i.e. variance's are summable.
The expected value of the VR is 1 for multi-step h=2 steps ahead.
.
The expected value of the VR is 1 for multi-step h=T steps ahead.
How do I test the significance of muli-step ahead risk algo forecasts i.e.
- if VR=1 for those, or
- modifications of Kupiec and Christofferson, or
- otherwise?
Amarjit
Significance of multi-step ahead forecasts from risk algo's
Re: Significance of multi-step ahead forecasts from risk algo's
One-step out violation calculations are (theoretically) independent of each other. Multiple step are not---in fact, they are likely to be very strongly correlated with each other. So Christofferson is pointless---it's a test of serial independence.
Kupiec is a likelihood ratio test of whether the chosen tail probability seems to be met in practice. It's basically an LR test on a binomial process using the 0-1 dummies of violations as data. With the assumption of independence, all you need to do that is to count the number of violations/non-violations to construct that. In the GARCHBACKTEST example, that 0-1 dummy is never even created as a series, as an SSTATS instruction is used to count the number of violations. An alternative way to look at that is that it's a probit model with just an intercept as an explanatory variable. The advantage of that is that you can use robust standard error calculations to allow for the serial correlation. Generate the 0-1 dummy as a series, do a DDV on CONSTANT only with ROBUSTERRORS, LWINDOW=NEWEY and LAGS=horizon-1 and test whether the coefficient is equal to the chosen tail probability. Note that the test on multiple steps is likely to be substantially less powerful than on a single step because of the serial correlation. (It's more likely that you will randomly stray farther from the chosen probability).
Kupiec is a likelihood ratio test of whether the chosen tail probability seems to be met in practice. It's basically an LR test on a binomial process using the 0-1 dummies of violations as data. With the assumption of independence, all you need to do that is to count the number of violations/non-violations to construct that. In the GARCHBACKTEST example, that 0-1 dummy is never even created as a series, as an SSTATS instruction is used to count the number of violations. An alternative way to look at that is that it's a probit model with just an intercept as an explanatory variable. The advantage of that is that you can use robust standard error calculations to allow for the serial correlation. Generate the 0-1 dummy as a series, do a DDV on CONSTANT only with ROBUSTERRORS, LWINDOW=NEWEY and LAGS=horizon-1 and test whether the coefficient is equal to the chosen tail probability. Note that the test on multiple steps is likely to be substantially less powerful than on a single step because of the serial correlation. (It's more likely that you will randomly stray farther from the chosen probability).
Re: Significance of multi-step ahead forecasts from risk algo's
For some of the methods, at h=1 (numerically, and statistically via Kupiec), and on occasion at h=2, VR's are not far from expected value VR=1. At h>2, they tend towards being close to 0, but not always, which I can see from the exceedance plots. Given your comments, it's good enough. Thanks!