APBREAKTEST—General test for breaks in linear regression

Use this forum to post complete RATS "procedures". Please be sure to include instructions on using the procedure and detailed references where applicable.

APBREAKTEST—General test for breaks in linear regression

Postby TomDoan » Thu May 28, 2015 9:24 pm

@APBreakTest performs Andrews-Ploberger and Andrews-Quandt structural break tests for a linear regression, with p-values using Hansen's approximations. These are used to test for a single structural break at an unknown point within the sample. A series of LM statistics are generated for breaks at each of the points in the middle range of the data set. The Andrews-Quandt test uses as the test statistic the maximum of the LM statistics, while Andrews-Ploberger uses the geometric mean. These both have highly non-standard distributions. Asymptotic p-values are computed.

References
Andrews, Donald W. K. and Werner Ploberger, "Optimal Tests When a Nuisance Parameter is Present Only Under the Alternative", Econometrica, 1994, pp 1383-1414

Hansen, Bruce E., "Approximate Asymptotic P-Values for Structural Change Tests", Journal of Business and Economic Statistics, 1997, pp 60-67.

Fix on 29 June 2011 to fix (clearly) wrong p-values on the Andrews-Ploberger form for k=2 and k=3 for values well in the tail. (These were present in Hansen's Gauss code.)



Detailed description


@APBREAKTEST is used in the ONEBREAK.RPF example file distributed with RATS.

The related procedure @APGradientTest can be applied to the series of gradients from a non-linear estimation such as one done with GARCH or MAXIMIZE.
Last edited by TomDoan on Thu Sep 20, 2018 4:18 pm, edited 7 times in total.
Reason: Switch to description from HTML help
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST - General test for breaks in linear regressi

Postby asmith05 » Thu Mar 31, 2016 11:57 am

Hi Tom,
I noticed that APBreakTest and RegHBreak don't produce critical values for the AP and AQ break test statistics, only p-values. I tried to remedy this by editing RegHBreak to produce critical values (renamed to RegHBreakCV) as an output of the bootstrapping process.

Code: Select all
*
* @RegHBreakCV( options )
*
* Regression post-processor which tests the regression just run for a
* full structural break with fixed regressor bootstrapping of the
* significance level. The test statistics are the Andrews-Quandt
* (maximum) and Andrews-Ploberger (geometric mean) of the F-statistics
* for the break.
*
* Options:
*   REPS=# of residual bootstrap replications [100]
*   PI1=fraction of early entries not examined as break points [.15]
*   PI2=fraction of late entries not examined as break points [PI1]
*   [PRINT]/NOPRINT
*   PERCENTILES=||percentiles for lower and upper bounds|| [||.05,.10||]
*
* Variables Defined:
*   %%BREAKPOINT    Entry with largest break test
*   %%AQTEST      Andrews-Quandt test statistic
*   %%APTEST      Andrews-Ploberger test statistic
*   %%AQSIGNIF    Bootstrapped significance value for AQ test
*   %%APSIGNIF    Bootstrapped significance value for AP test
*   %%AQL         Bootstrapped Lower % critical value for AQ test
*   %%AQU         Bootstrapped Upper % critical value for AQ test
*   %%APL             Bootstrapped Lower % critical value for AP test
*   %%APU             Bootstrapped Upper % critical value for AP test
*
* References:
*   Bruce Hansen(2000), "Testing for Structural Change in Conditional
*   Models", Journal of Econometrics, vol. 97, no. 1, 93-115, July.
*
*   Andrews and Ploberger(1994), "Optimal Tests When a Nuisance Parameter
*   is Present Only Under the Alternative", Econometrica, vol 62, no 6,
*   1383-1414.
*
* Revision Schedule:
*   08/2010 Written by Tom Doan, Estima
*   05/2015 %%BREAKPOINT changed from REAL to INTEGER
*   03/2016 Added Critical Value Calculation
*
procedure RegHBreakCV depvar start end
type series    depvar
type integer    start end
*
option real      pi1    .15
option real      pi2    pi1
option integer reps    100
option switch  print  1
option vector  percentiles  ||.05,.10||
*
local vect[int] reghold
local integer    piStart piEnd rnobs rep time bestbreak
local series     yb wstat u
local vect       x
local symm       dxx dxeex ss
local vect       dxe
local real       rssdiff fstat bestdiff apcount aqcount aqvalue apvalue ci
local string   ltitle
local vect[real] aqteststorage apteststorage
decl  real     %%aqtest %%aptest %%aqsignif %%apsignif %%aql %%aqu %%apl %%apu
decl  integer  %%breakpoint
*
compute reghold=%regsave()
*
compute rnobs  =%regend()-%regstart()+1
compute piStart=%regstart()+fix((rnobs-1)*pi1)
compute piEnd  =%regstart()+fix((rnobs-1)*(1-pi2))
set u = %resids
*
dim dxx(%nreg,%nreg) dxeex(%nreg,%nreg) dxe(%nreg) aqteststorage(reps) apteststorage(reps)
compute percentiles(1) = 1.00-percentiles(1)
compute percentiles(2) = 1.00-percentiles(2)
*
compute bestdiff=0.0
compute apcount=aqcount=0.0
if reps>0
   infobox(action=define,progress,lower=0,upper=reps) $
     "Bootstrapping P-Values and Confidence Intervals"
do rep=0,reps
   infobox(current=rep)
   if rep==0
      set yb = u
   else
      set yb = %ran(1.0)
   *
   * Run the regression on the residuals or bootstrapped residuals
   *
   linreg(lastreg,noprint) yb
   *
   compute dxx  =%zeros(%nreg,%nreg)
   compute dxeex=%zeros(%nreg,%nreg)
   compute dxe  =%zeros(%nreg,1)
   set wstat piStart piEnd = 0.0
   do time=%regstart(),%regend()
      compute x=%eqnxvector(0,time)
      compute dxx  =dxx  +%outerxx(x)
      compute dxeex=dxeex+%outerxx(x*%resids(time))
      compute dxe  =dxe  +x*%resids(time)
      if time<piStart.or.time>piEnd
         next
      compute ss=(dxx-dxx*%xx*dxx)
      compute rssdiff=%qform(inv(ss),dxe)
      *
      * This follows Hansen's example code. It's actually F/number of
      * restrictions.
      *
      compute fstat=(rssdiff)/((%rss-rssdiff)/(%ndf-%nreg))
      compute wstat(time)=fstat
      if rep==0.and.rssdiff>bestdiff
         compute bestbreak=time,bestdiff=rssdiff
   end do time
   *
   * Compute sup F and overflow-safe exp F statistic
   *
   sstats(max)  piStart piEnd wstat>>aqvalue
   sstats(mean) piStart piEnd exp(.5*(wstat-aqvalue))>>apvalue
   compute apvalue=log(apvalue)+.5*aqvalue
   *
   * Save the test statistics for rep==0; count the number above the
   * saved statistics for the bootstrapped data.
   *
   if rep==0
      compute %%aqtest=aqvalue,%%aptest=apvalue
   else
      compute aqcount=aqcount+(aqvalue>%%aqtest),apcount=apcount+(apvalue>%%aptest),aqteststorage(rep)=aqvalue,apteststorage(rep)=apvalue
end do rep
infobox(action=remove)
*
compute %%apsignif=%if(reps>0,apcount/reps,%na)
compute %%aqsignif=%if(reps>0,aqcount/reps,%na)
compute %%breakpoint=bestbreak
compute fracaq=%fractiles(aqteststorage,percentiles)
compute fracap=%fractiles(apteststorage,percentiles)
compute %%aql=fracaq(1)
compute %%aqu=fracaq(2)
compute %%apl=fracap(1)
compute %%apu=fracap(2)
*
compute ltitle="AQ-AP Break Tests/Bootstrapped P-Values and Critical Values"
report(use=hreport,action=define,title=ltitle)
report(use=hreport,atrow=1,atcol=1,span) ltitle
report(use=hreport,atrow=2,atcol=1) "Estimated Break"          %datelabel(bestbreak)
report(use=hreport,atrow=3,atcol=2)                            "Statistic" "Signif"     1-percentiles(2) "Crit Val"  1-percentiles(1) "Crit Val"
report(use=hreport,atrow=4,atcol=1) "Andrews-Quandt Test"      %%aqtest    %%aqsignif   %%aqu                              %%aql
report(use=hreport,atrow=5,atcol=1) "Andrews-Ploberger Test"   %%aptest    %%apsignif   %%apu                              %%apl
report(use=hreport,atrow=3,torow=3,action=format,picture="#.####")
report(use=hreport,atcol=2,tocol=2,action=format,picture="#.###")
report(use=hreport,atcol=3,tocol=3,action=format,picture="#.#####")
if print
   report(use=hreport,action=show)

compute %regload(reghold)

end HBreakTest




I am trying to replicate Table 4 in Stock and Watson, 1998, Median Unbiased Estimation of Coefficient Variance in a Time-Varying Parameter Model, JASA (the data is from Mark Watson's website). I can generate very similar estimates of their AQ and AP test statistics along with very similar p-values, but I can't replicate their 90% Confidence Intervals for lambda (which is directly found from Table 3: Lookup Table for Constructing Median-Unbiased Estimator of lambda using Various Test Statistics). I think they are using the critical values for the AP and AQ test statistics to calculate what they call "equal-tailed confidence intervals" for lambda from lookup table 3.

Do you have any insight into what I may have missed here or if there is an easier way to generate critical values for the AP and AQ test statistics?

Thanks for your time,

Lee
Attachments
REGHBREAKCV.SRC
(5.69 KiB) Downloaded 119 times
GDP.RAT
(7.25 KiB) Downloaded 112 times
StockWatson1998.RPF
(766 Bytes) Downloaded 113 times
Last edited by asmith05 on Thu Mar 31, 2016 4:55 pm, edited 4 times in total.
asmith05
 
Posts: 14
Joined: Thu Mar 24, 2011 8:33 pm

Re: APBREAKTEST - General test for breaks in linear regressi

Postby TomDoan » Thu Mar 31, 2016 2:34 pm

I'm confused. Confidence intervals are used for estimators, not for test statistics. It's one thing to want to look at a comparison of bootstrapped vs asymptotic p-values (@APBREAKTEST does the asymptotics, while @REGHBREAK does bootstrapping), but in each case, you're looking for p-values.
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST - General test for breaks in linear regressi

Postby asmith05 » Thu Mar 31, 2016 3:56 pm

Sorry Tom, I would like to generate critical values for the test statistics. I believe these are used to calculate confidence intervals for the median unbiased estimators (lambdas). Sorry for mixing the language across these steps (the AP & AQ test statistics become estimators via Stock and Watson's methodology). I have edited the post above to correct this.
Thanks
asmith05
 
Posts: 14
Joined: Thu Mar 24, 2011 8:33 pm

Re: APBREAKTEST - General test for breaks in linear regressi

Postby TomDoan » Thu Mar 31, 2016 6:54 pm

They use interpolation off table 3 which I assume is generated using simulations of the model under gaussianity
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST - General test for breaks in linear regressi

Postby asmith05 » Thu Mar 31, 2016 11:01 pm

Thanks for the reply Tom. Yes, I understand how they use the AP and AQ test statistics to generate point estimates of lambda via interpolation from Table 3 , but i thought they were also generating the confidence intervals for lambda using critical values or p-values from the AP and AQ test statistics. It looks like that in their Gauss code at least. I can come very close to replicating their point estimates of lambda, it's the confidence intervals I can't replicate.
asmith05
 
Posts: 14
Joined: Thu Mar 24, 2011 8:33 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby TomDoan » Fri Apr 01, 2016 11:06 am

Their confidence intervals use extensive tables of simulated values (the LCDF.DAT file and the like). They don't use bootstrapping that I can see and I don't think that would work anyway.
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby asmith05 » Tue Nov 07, 2017 11:55 am

Hi Tom,
I have a question about the output of @APBREAKTEST.

I noticed that in some applications the %%breakpoint doesn’t align with the maximum of the breakpoint test statistic. However, in other applications the two align.

In the detailed description of @APBREAKTEST, %%breakpoint is described as “Entry with largest break test.” However, in the source code for @APBREAKTEST I can see that %%breakpoint is calculated as the point that minimizes the sum of squared residuals.

I guess I’m envisioning a vertical line in the graphed output of @APBREAKTEST not at:
Code: Select all
extremum(noprint) rss(i) piStart piEnd
comp brkdates(i) = %minent


But instead at:
Code: Select all
extremum(noprint) wstat(i) piStart piEnd
comp brkdates(i) = %maxent


I would note that this seems to be a function of the financial crisis with the minimum of rss equaling 0.0000000000000 in 2009:01. That seems implausible to me to begin with. In this same application, the maximum of wstat occurs in 2010:05. (When I remove the financial crisis from the sample I get a %%breakpoint that aligns with the maximum of wstat at 2010:05.)

Is the fact that these two dates are different telling me something from an econometric standpoint or should I be focused on the entry with the largest break test statistic?

Thanks,
Lee
asmith05
 
Posts: 14
Joined: Thu Mar 24, 2011 8:33 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby TomDoan » Tue Nov 07, 2017 12:43 pm

That actually is taken out of Hansen's program (the "Andrews date"). The entry with the largest test statistic is the same as the one with the smallest RSS if you don't use ROBUSTERRORS as the two are 1-1 related. They will not necessarily be the same if you use ROBUSTERRORS. As to why you're getting an RSS of zero, I would have to see the program and data, but offhand I can guess that either the model is too big for the PI value, or you have some dummies in the model which are conflicting with the sample breaks.
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby alvarezcc » Tue Nov 14, 2017 6:15 pm

Hi Tom! Is it possible for you to change the code for performing the AP-test in order to also show (or replace) the mean of the LM statistic by the exponential statistic?

Thanks in advance for your help.
alvarezcc
 
Posts: 25
Joined: Wed Mar 25, 2009 8:59 am

Re: APBREAKTEST—General test for breaks in linear regression

Postby TomDoan » Tue Nov 14, 2017 6:31 pm

alvarezcc wrote:Hi Tom! Is it possible for you to change the code for performing the AP-test in order to also show (or replace) the mean of the LM statistic by the exponential statistic?


I'm not sure what you mean. The Andrews-Ploberger statistic is the geometric mean. Andrews-Quandt is the sup. The procedure doesn't compute or report the arithmetic mean.
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby alvarezcc » Wed Nov 15, 2017 12:09 pm

Hi Tom! Thanks for your kind reply. I'm trying to estimate the variance in the state equation for the unemployment rate using the kalman filter. I'm trying to follow the procedure suggested by Leong Szeto and Guy (2004) in their paper Estimating a New Zealand NAIRU (http://www.treasury.govt.nz/publication ... p04-10.pdf) . This procedure implies using a result from Stock and Watson (1998) in order to obtain an unbiased estimate of the lambda parameter. According to the first authors, this procedure implies obtaining the exponential Wald statistic of the Andrews and Ploberger (1994) test for an intercept shift at an unknown date. Then they use the data from table 3 in Stock and Watson's (https://www.princeton.edu/~mwatson/pape ... A_1998.pdf) and the exponential Wald statistic to obtain the estimate of lambda. As it can be seen from table 3, one can obtain a different estimates of lambda if you use the mean Wald test (second column) or the exponential (third column). However, as the AP-test performed in RATS provide the geometric mean then I would like to obtain also the exponential statistic.
alvarezcc
 
Posts: 25
Joined: Wed Mar 25, 2009 8:59 am

Re: APBREAKTEST—General test for breaks in linear regression

Postby TomDoan » Wed Nov 15, 2017 12:16 pm

%%APTEST is the Andrews-Ploberger test statistic.
TomDoan
 
Posts: 6452
Joined: Wed Nov 01, 2006 5:36 pm

Re: APBREAKTEST—General test for breaks in linear regression

Postby asmith05 » Thu Sep 20, 2018 4:19 pm

Hi Tom,
The problem I was encountering with the breakdate selection was due to a missing value I had in my data set.

To make the apbreaktest program robust to this I replaced:
Code: Select all
set wstat(i) piStart piEnd = 0.0
set rss(i)   piStart piEnd = 0.0


with:
Code: Select all
set wstat(i) piStart piEnd = %na
set rss(i)   piStart piEnd = %na


And similarly for the full coefficient vector and variance breaktest portions of the code.

Thanks for your reply,
Lee


Last bumped by TomDoan on Thu Sep 20, 2018 4:19 pm.
asmith05
 
Posts: 14
Joined: Thu Mar 24, 2011 8:33 pm


Return to RATS Procedures

Who is online

Users browsing this forum: Bing [Bot] and 0 guests