garchmvbootstrap.rpf

[ac_1]

Randomly rearranging (block bootstrapping) independent random variables as in your S4 does not produce correlated random variables.

Do you have an alternative solution as a procedure in RATS?

[TomDoan]

In viewtopic.php?p=19882#p19882, I describe a valid test for PoF in overlapping samples. If you aren't satisfied with the Wald test based on QMLE asymptotics, you can block bootstrap the exceedance data and get a bootstrapped distribution for the alphahat. (You don't have to do anything fancy to estimate alphahat since the point estimate is just the fraction of exceedances in each bootstrapped replication).

[ac_1]

In viewtopic.php?p=19882#p19882, I describe a valid test for PoF in overlapping samples. If you aren't satisfied with the Wald test based on QMLE asymptotics, you can block bootstrap the exceedance data and get a bootstrapped distribution for the alphahat. (You don't have to do anything fancy to estimate alphahat since the point estimate is just the fraction of exceedances in each bootstrapped replication).

As an example
https://estima.com/RATSBetaHelp/topics/ ... aprpf.html

calculating p-value: is it always > rather than >=, i.e. not to include the equality, in bootstrapping

*
* Figure out the bootstrapped p-value
*
sstats(mean) 1 ndraws (stats>%cdstat)>>pvalue

[TomDoan]

No. That's a different type of bootstrap (which is bootstrapping data under the null). You need to compare the results with the assumed value of alpha.

[ac_1]

No. That's a different type of bootstrap (which is bootstrapping data under the null). You need to compare the results with the assumed value of alpha.

I can block bootstrap trigger within a loop, ndraws times, and calculate the means. That's straightforward.

The question is what’s the p-value?
Something like (compare the results with the target value of alpha)

sstats(mean) 1 ndraws %if( abs( Tstar ) > abs( T ),1.0,0.0)>>pvalue

Tstar = (bootstrapping trigger as a mean in a loop) - (actual violation rate e.g. 0.007 in the sample)
T = (actual violation rate e.g. 0.007 in the sample) - (target alpha e.g. 0.01)

And with large ndraws either using > or >= the difference is negligible.

[TomDoan]

No. You're not bootstrapping to get an estimate of alpha; the mean across bootstrap samples should be roughly the same as the mean of the data. Instead, you're bootstrapping to get the sampling distribution and to compare that to the assumed value of alpha. So you want to count the percentage of times the bootstrapped sample has an alphahat that is on the other side of alpha. (A two-sided test is not easy to construct in this situation). The closest example is bootsimple.rpf (which actually does generate a two-sided test, but that's when doing a sample mean, which symmetrizes more easily than a binomial).

[ac_1]

No. You're not bootstrapping to get an estimate of alpha; the mean across bootstrap samples should be roughly the same as the mean of the data.

Yes, the mean across bootstrap samples is approximately the same as the mean of the data.

Instead, you're bootstrapping to get the sampling distribution and to compare that to the assumed value of alpha.

I've plotted alphahatDist.

So you want to count the percentage of times the bootstrapped sample has an alphahat that is on the other side of alpha. (A two-sided test is not easy to construct in this situation). The closest example is bootsimple.rpf (which actually does generate a two-sided test, but that's when doing a sample mean, which symmetrizes more easily than a binomial).

I don't know how to proceed?

Here's where I'm at (it's straightforward to do the bootstrap loop and plot the distribution) - I am aiming for a p-value to test for frequency.

Code: Select all

procedure acAlphaBlockBoot trigger start end

   type series                trigger
   type integer               start end

   local integer              startl endl
   local integer              draw N
   local real                 alphahatStar
   local series               alphahatDist
   local series               boottrigger
   local series[integer]      bootentries

   option real                alpha    0.01
   option integer             horizon  1
   option integer             ndraws    10000


   * Sample range
   inquire(reglist) startl<<start endl<<end
   # trigger

   * Compute observed violation rate
   sstats(mean) startl endl trigger>>%%alphahatObs
   compute N = %nobs

   * Create series to store bootstrap results
   set alphahatDist 1 ndraws = 0.0

   * Bootstrap loop
   do draw = 1, ndraws

      * Generate bootstrap block indices from the actual data range
      BOOT(block=horizon) bootentries startl endl startl endl

      * Resample the ACTUAL trigger series using those indices
      set boottrigger startl endl = trigger(bootentries(t))

      * Compute violation rate for this bootstrap sample
      sstats(mean) startl endl boottrigger>>alphahatStar
      compute alphahatDist(draw) = alphahatStar

   end do draw

   * Compute bootstrap statistics
   stats(fractiles) alphahatDist 1 ndraws

   compute %%bootmean = %mean
   compute %%bootse   = sqrt(%variance)


* --- Plot density function for alphabootdist ---
   density(smoothing=2.0) alphahatDist 1 ndraws galphaboot falphaboot

   * 10 horizons, 10 colours
   compute col = %if(horizon==1,2,$
                     %if(horizon==2,4,$
                     %if(horizon==3,5,$
                     %if(horizon==4,6,$
                     %if(horizon==5,7,$
                     %if(horizon==6,8,$
                     %if(horizon==7,9,$
                     %if(horizon==8,10,$
                     %if(horizon==9,11,12)))))))))
   spgraph(hea="",vfi=1,hfi=1)
   scatter(key=upright,head="A non-parametric empirical approximation to the sampling distribution",$
            hgrid=||%%bootmean||,style=line) 1
   # galphaboot falphaboot 1 ndraws col
   spgraph(done)


* --- Output Results ---
   disp "=================================================="
   disp " Alpha Block Bootstrap Unconditional Coverage Test"
   disp "=================================================="
   disp "Horizon (horizon):     " horizon
   disp "Sample size (N):       " N
   disp "Target alpha:          " alpha
   disp "Observed alphahat:     " %%alphahatObs
   disp "Bootstrap mean:        " %%bootmean
   disp "Bootstrap SE:          " %%bootse
   disp "=================================================="

end procedure

[TomDoan]

Compute the percentage of draws that are on the wrong side of the hypothesized alpha.

sstats(mean) 1 ndraws (alphahatDist>alpha)>>pctover (alphadataDist<alpha)>>pctunder

whichever of pctover and pctunder is appropriate. See the first example in the description of SSTATS.

[ac_1]

Compute the percentage of draws that are on the wrong side of the hypothesized alpha.

sstats(mean) 1 ndraws (alphahatDist>alpha)>>pctover (alphadataDist<alpha)>>pctunder

whichever of pctover and pctunder is appropriate. See the first example in the description of SSTATS.

[1]
* --- Calculate p-value ---
sstats(mean) 1 ndraws (alphahatDist>alpha)>>pctover (alphahatDist<alpha)>>pctunder

compute p_value_boot = 2.0 * %min(pctover,pctunder)
compute p_value_boot = %min(p_value_boot,1.0)
compute %%palphahat_boot = p_value_boot


Here's acAlphahatBlockBoot.src

Code: Select all

procedure acAlphahatBlockBoot trigger start end

   type series                trigger
   type integer               start end

   local integer              startl endl
   local real                 k_real
   local integer              k_obs N
   local series               alphahatDist
   local integer              draw
   local series[integer]      bootentries
   local series               boottrigger
   local real                 alphahatStar bootmean bootse pctunder pctover p_value_boot
   local integer              col galphahatboot
   local series               falphahatboot

   option real                alpha    0.01
   option integer             horizon  1
   option integer             ndraws   10000


   * --- Sample range ---
   inquire(reglist) startl<<start endl<<end
   # trigger

* --- Calculate observed statistics ---
   sstats startl endl trigger>>k_real
   compute k_obs = fix(k_real)

   * --- Compute observed violation rate ---
   sstats(mean) startl endl trigger>>alphahatObs
   compute N = %nobs

   * --- Create series to store bootstrap results ---
   set alphahatDist 1 ndraws = 0.0

   * --- Bootstrap loop ---
   do draw = 1, ndraws

      * Generate bootstrap block indices from the actual data range
      BOOT(block=horizon) bootentries startl endl startl endl

      * Resample the ACTUAL trigger series using those indices
      set boottrigger startl endl = trigger(bootentries(t))

      * Compute violation rate for this bootstrap sample
      sstats(mean) startl endl boottrigger>>alphahatStar
      compute alphahatDist(draw) = alphahatStar

   end do draw

   * --- Compute bootstrap statistics ---
   stats(fractiles) alphahatDist 1 ndraws

   compute bootmean = %mean
   compute bootse   = sqrt(%variance)

   *  --- Calculate p-value ---
   sstats(mean) 1 ndraws (alphahatDist>alpha)>>pctover (alphahatDist<alpha)>>pctunder

   compute p_value_boot = 2.0 * %min(pctover,pctunder)
   compute p_value_boot = %min(p_value_boot,1.0)
   compute %%palphahat_boot = p_value_boot


   * --- Plot density function for alphahatDist ---
   density(smoothing=2.0) alphahatDist 1 ndraws galphahatboot falphahatboot

   * 10 horizons, 10 colours
   compute col = %if(horizon==1,2,$
                     %if(horizon==2,4,$
                     %if(horizon==3,5,$
                     %if(horizon==4,6,$
                     %if(horizon==5,7,$
                     %if(horizon==6,8,$
                     %if(horizon==7,9,$
                     %if(horizon==8,10,$
                     %if(horizon==9,11,12)))))))))
   spgraph(hea="",vfi=1,hfi=1)
   scatter(key=upright,head="A non-parametric empirical approximation to the sampling distribution\\of alphahat with serial dependence",$
            footer="horizon="+%strval(horizon,"#")+",   "+$
                   "Observed alphahat:"+%strval(alphahatObs,".####")+",   "+$
                   "Target alpha (fixed,regardless of N):"+%strval(alpha,".##")+",   "+$
                   "Bootstrap mean:"+%strval(bootmean,".####")+",   "+$
                   "p-value (bootstrap):"+%strval(%%palphahat_boot,".####"),$
            hgrid=||alpha,bootmean||,style=line) 1
   # galphahatboot falphahatboot 1 ndraws col
   spgraph(done)


   * --- Output Results ---
   display "====================================================="
   display " Alphahat Block Bootstrap Unconditional Coverage Test"
   display "====================================================="
   display "Horizon (horizon):                     " horizon
   display "Sample size (N):                       " N
   display "Expected exceedances E[K]=N*alpha:     " N*alpha
   display "Observed exceedances (k):              " k_obs
   display "Target alpha (fixed,regardless of N):  " alpha
   display "Observed alphahat:                     " alphahatObs
   display "Bootstrap mean:                        " bootmean
   display "Bootstrap SE:                          " bootse
   display "pctover:                               " pctover
   display "pctunder:                              " pctunder
   display "p-value (bootstrap):                   " %%palphahat_boot
   display "=================================================="

end procedure

Please check carefully (especially type mismatch: integer vs. real, fix vs. %round(), arithmetic). Is there anything else I may include or correct?


[2]
To confirm, with staggered multi-step forecasts, block-bootstrapping: N=1000, 999,...,991, alpha=0.01

(a)
alphahat
Horizon=any: observed alphahat: e.g. 0.002, Target (or expected) alpha (fixed, regardless of N):0.01

(b)
If I was to block-bootstrap exceedances
Horizon=1 observed exceedances: e.g. 9, expected exceedances: 10
...
Horizon=6 observed exceedances: e.g. 6, expected exceedances: 9.95
...
Horizon=10 observed exceedances: e.g. 3, expected exceedances: 9.91


[3]
acKupiecBlockBoot.src is wrong, can it be fixed?

[TomDoan]

acKupiecBlockBoot.src is wrong, can it be fixed?

11.1 fixed a problem with parmeters or options inside PARMSETS used in SUMMARIZE.