Page 1 of 1
NAs generated when bootstrapping forecasts from an LSTAR mod
Posted: Fri May 23, 2014 6:13 am
by nybammer
I am trying to bootstrap the forecasts (at 4, 8 and 12 steps ahead) from an LSTAR model using the following code:
Code: Select all
set lsforesBoot = 0.
set fcst4_lsforesBoot = 0.
set fcst8_lsforesBoot = 0.
set fcst12_lsforesBoot = 0.
com start = 1975:01 + 133;
com end = 1997:06-12;
do cap_T = start, end
nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar
stats(noprint) y
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean,d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
group lstarmodel lstareq>>lsforesBoot
set meanBoot cap_T+1 cap_T+12 = 0.0
compute ndraws = 2500
do draws = 1,ndraws
boot entries * cap_T
set path = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot cap_T+1 cap_T+12 = meanBoot + bootsim(1)
end do draws
set meanBoot cap_T+1 cap_T+12 = meanBoot/ndraws
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
end do cap_T
For some reason several of the draws for meanBoot series are filled with NAs, even when the bootsim(1) series are not. This then doesn't allow me to average the results. I have tried some dirty tricks to discard those instances where the meanBoot series contains NAs:
Code: Select all
set valid_draw = 0.0
.....
set meanBoot cap_T+1 cap_T+12 = %if(%valid(meanBoot), bootsim(1) + meanBoot, 0)
SET valid_draw = %if(%valid(meanBoot), 1, 0) + valid_draw
*or
*if %valid(meanBoot(t)) == 1; set meanBoot cap_T+1 cap_T+12 = bootsim(1) + meanBoot
*and then write
set meanBoot4 = meanBoot(cap_T+4)/valid_draw(cap_T+4)
set meanBoot8 = meanBoot(cap_T+8)/valid_draw(cap_T+8)
set meanBoot12 = meanBoot(cap_T+12)/valid_draw(cap_T+12)
The code above partially solves the problem by getting rid of the draws with NAs, but the forecasts are way off.
Anybody have any suggestions of what am I doing wrong? The reason I am puzzled is that when I Monte Carlo simulate the forecasts everything is working well.
Thanks in advance,
R.P.
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Fri May 23, 2014 8:38 am
by TomDoan
The problem is that the idea of * in
boot entries * cap_T
is different from what it is in the NLLS and LINREG instructions. For BOOT, * will be 1, while for LINREG and NLLS, it will depend upon the variables (formulas) involved. Since you have lags, you're losing some data points at the start.
Instead, after
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
put
compute bstart=%regstart(),bend=%regend()
and then do a BOOT as
boot entries bstart bend
That way it will sample only from the entries where RESIDS is defined.
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Sun May 25, 2014 6:20 am
by nybammer
Dear Tom,
Thank you very much for the reply. That solved the problem about the NAs in the meanBoot series. However, there is an additional question I have about the code that I hope you can help me with. The end purpose of the code is to get the forecasts at various steps ahead (4, 8, and 12). The way I am trying to get them is to write this (see in bold below):
set lsforesBoot = 0.
set fcst4_lsforesBoot = 0.
set fcst8_lsforesBoot = 0.
set fcst12_lsforesBoot = 0.
com start = 1975:01 + 133;
com end = 1997:06-12;
do cap_T = start, end
nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar
stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend
group lstarmodel lstareq>>lsforesBoot
set meanBoot = 0.0
compute ndraws = 2500
do draws = 1,ndraws
boot entries / bstart bend
set path = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot = meanBoot + bootsim(1)
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
end do draws
set meanBoot = meanBoot/ndraws
end do cap_T
pri start+1 end meanBoot bootsim(1) y path
pri start+1 end fcst4_lsforesBoot fcst8_lsforesBoot fcst12_lsforesBoot
I am puzzled that even though the meanBoot series looks alright, the fcst4_lsforesBoot, fcst8_lsforesBoot, and fcst12_lsforesBoot are filled with NAs. I tried several tricks (including the bold part of the code outside the "do draws" part of the code, or write:
Code: Select all
forecast(model=lstarmodel,paths,results=bootsim,steps=k,from=cap_T+1)
where k = 4, 8, or 12 and have three different meanBoot series).
However, none seemed to the job (i.e., in the second case the three meanBootk, where k=4,8,12 series are the same. What do you think I am doing wrong?
Thanks so much again...
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Sun May 25, 2014 9:59 am
by TomDoan
The problem here is it's not clear that you're getting what you want for the bootstrapping. You want to draw entries numbers from bstart to bend for the range from CAP_T+1 to CAPT+12.
boot entries CAP_T+1 CAP_T+12 bstart bend
set path CAP_T+1 CAP_T+12 = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
I think you may also want to take the >>... off
group lstarmodel lstareq>>lsforesBoot
It doesn't look like you use the redirect anywhere (since you have a RESULTS option on FORECAST).
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Tue May 27, 2014 11:08 am
by nybammer
Thank you so much for getting back to me on this issue. So looks like I am almost there. In the code below everything looks good for the "meanBoot" series, but I am still unable to gather the forecasts at the 4, 8, and 12 steps ahead. Following your advice to do:
Code: Select all
boot entries cap_T+1 cap_T+12 bstart bend
set path cap_T+1 cap_T+12 = resids(entries)
does not seem to help in this regard. The code I am using is below:
Code: Select all
do cap_T = start, end
nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar
stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend
group lstarmodel lstareq
set meanBoot = 0.0; *if I say "set meanBoot cap_T+1 cap_T+12 = 0.0" then meanBoot is filled with NAs
compute ndraws = 2500
do draws = 1,ndraws
boot entries cap_T+1 cap_T+12 bstart bend
set path cap_T+1 cap_T+12 = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot = meanBoot + bootsim(1)
end do draws
set meanBoot = meanBoot/ndraws
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
end do cap_T
pri start+1 end meanBoot bootsim(1) y path
What is weird is that when I define the meanBoot series and if I were to write: "set meanBoot cap_T+1 cap_T+12 = 0.0" then meanBoot is filled with NAs. So for the code to work i simply say:
.
However, then this part:
Code: Select all
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
does not lead to anywhere (i.e., fcstk_lsforesBoot series, where k=4,8, 12, are filled with NAs.) Do you have an insight what is leading to this issue and what could I do to get this working?
I appreciate all the help...
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Tue May 27, 2014 12:09 pm
by TomDoan
You'll have to post your full program and data set.
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Tue May 27, 2014 3:55 pm
by nybammer
The full program is actually the one in my last post. The data in question are listed below:
Code: Select all
3.357754218
3.337047362
3.312938592
3.342597789
3.330792125
3.319342037
3.367836145
3.404086059
3.414586294
3.399204437
3.39030879
3.398258762
3.386368139
3.371749804
3.36690916
3.353590391
3.356715381
3.363781785
3.355946118
3.3235835
3.310513563
3.283956875
3.269813527
3.252165453
3.24857697
3.254430389
3.247143424
3.243790144
3.241611161
3.233527629
3.194729312
3.202631214
3.205201341
3.184890552
3.169466887
3.135803478
3.122395381
3.105908129
3.088688589
3.097357976
3.130791758
3.120815691
3.10387908
3.072730853
3.058002241
3.001554394
3.03293442
3.019944319
3.001652429
3.00332297
3.005085411
3.022800992
3.026846216
3.005331605
2.972796647
2.96887336
2.94592255
2.941179486
2.928337669
2.905523679
2.902809674
2.912252138
2.964964215
2.976023232
2.933138488
2.915262075
2.905966459
2.928470151
2.921374951
2.956390656
3.000370761
3.0262569
3.052294048
3.115743717
3.102581032
3.133047297
3.189432072
3.218910694
3.241887443
3.257840802
3.192220511
3.138524172
3.124636285
3.136587627
3.151735202
3.168892708
3.163928207
3.16584358
3.126268289
3.171152438
3.178434016
3.171799064
3.169771331
3.17567097
3.183707765
3.129087156
3.108417789
3.11860678
3.102113016
3.101802361
3.110967817
3.143901699
3.155736163
3.191878758
3.189935196
3.161823595
3.18834737
3.211550987
3.232845818
3.196215271
3.15853989
3.170796354
3.204295856
3.192637726
3.227355666
3.233006323
3.271698592
3.284931788
3.259740221
3.290647876
3.309414995
3.339534458
3.335016371
3.256000875
3.254543628
3.229996931
3.174542159
3.120529774
3.13062568
3.064856281
3.045724882
3.011683325
2.970567309
2.904380874
2.864471289
2.868630681
2.851686074
2.847042767
2.795325609
2.742514771
2.72043779
2.696774578
2.70162583
2.679922377
2.611299191
2.59346291
2.594171208
2.582957251
2.572821483
2.59128546
2.605856305
2.606224069
2.577095158
2.570763833
2.494178986
2.460182742
2.46459466
2.483429523
2.467887387
2.467698737
2.481441602
2.522210453
2.574600819
2.561697265
2.544853874
2.514765064
2.460898263
2.467437878
2.563729837
2.568283594
2.57498216
2.578287433
2.615062245
2.624512307
2.573198057
2.581552268
2.596600442
2.550706612
2.522743252
2.474676631
2.457807171
2.440231022
2.445309355
2.427047057
2.414809395
2.430929163
2.401733986
2.368593609
2.369255055
2.350659065
2.307299535
2.306072525
2.329675841
2.29907427
2.37588517
2.4368191
2.445642338
2.4798705
2.478199727
2.457118221
2.428566257
2.426744289
2.382512299
2.34545452
2.348352961
2.376871417
2.403192838
2.395306027
2.381596013
2.355758315
2.296140287
2.263581923
2.26053972
2.280543516
2.348755235
2.344256108
2.365222199
2.382393033
2.392758216
2.366068458
2.372394637
2.393773689
2.42701097
2.412590488
2.363131584
2.374947296
2.414009185
2.41916962
2.44572228
2.446188529
2.420280712
2.421028967
2.395711778
2.380908071
2.345364611
2.345864119
2.334687371
2.310439162
2.324003208
2.347634792
2.32172071
2.299145734
2.23378488
2.216350236
2.23367891
2.230248863
2.218266734
2.256438811
2.27380345
2.243621642
2.239758703
2.255944495
2.274320314
2.275188565
2.286244407
2.313818204
2.337730867
2.338091689
2.322651913
2.308022481
2.322117701
2.333148043
2.322770785
2.355098647
2.390942644
2.42102769
2.425954712
2.433318852
2.429090183
2.440310968
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Tue May 27, 2014 4:04 pm
by nybammer
I am posting the
full program here as well since the top part was missing:
Code: Select all
set lsforesBoot = 0.
set fcst4_lsforesBoot = 0.
set fcst8_lsforesBoot = 0.
set fcst12_lsforesBoot = 0.
com start = 1975:01 + 133;
com end = 1997:06-12;
do cap_T = start, end
nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar
stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend
group lstarmodel lstareq
set meanBoot cap_T+1 cap_T+12 = 0.0; *if I say set "meanBoot cap_T+1 cap_T+12 = 0.0" meanBoot is filled with NAs
compute ndraws = 2500
do draws = 1,ndraws
boot entries cap_T+1 cap_T+12 bstart bend
set path cap_T+1 cap_T+12 = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot = meanBoot + bootsim(1)
end do draws
set meanBoot = meanBoot/ndraws
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
end do cap_T
pri start+1 end meanBoot bootsim(1) y path
pri start+1 end fcst4_lsforesBoot fcst8_lsforesBoot fcst12_lsforesBoot
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Tue May 27, 2014 6:36 pm
by TomDoan
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
needs to be written using %LOGISTIC as is described in the Users' Guide. The exp is overflowing because of the large values for d0 and d1.
If the posted data is your Y, it's not clear why you would want to do a STAR model on it. Your series is clearly non-stationary and is very close to being monotone declining, so any transition on Y{1} is basically going to partition the data based upon time.
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Wed May 28, 2014 8:16 am
by nybammer
Yes, I need to take the first difference and make the data stationary. Thanks for pointing out the need to use the %logistic as well. However, the problem still remains that I cannot collect the forecasts at steps 4, 8, and 12. I should point out that the last code was causing NAs in the "meanBoot" series because for some reason when I write "set meanBoot cap_T+1 cap_T+12 = 0.0;" NAs are generated. However, the code below works well to get the meanBoot series but not the "fcstk_lsforesBoot" series. I'm afraid that changing the "frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))" expression won't solve this issue. This is really puzzling...
Code: Select all
set lsforesBoot = 0.
set fcst4_lsforesBoot = 0.
set fcst8_lsforesBoot = 0.
set fcst12_lsforesBoot = 0.
com start = 1975:01 + 133;
com end = 1997:06-12;
do cap_T = start, end
nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
*frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstarwt = %logistic(d0+d1*y{1},1.0)
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar
stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend
group lstarmodel lstareq
set meanBoot = 0.0; *if I say set "meanBoot cap_T+1 cap_T+12 = 0.0" meanBoot is filled with NAs
compute ndraws = 2500
do draws = 1,ndraws
boot entries cap_T+1 cap_T+12 bstart bend
set path cap_T+1 cap_T+12 = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot = meanBoot + bootsim(1)
end do draws
set meanBoot = meanBoot/ndraws
compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)
end do cap_T
pri start+1 end meanBoot bootsim(1) y path
pri start+1 end fcst4_lsforesBoot fcst8_lsforesBoot fcst12_lsforesBoot
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Wed May 28, 2014 11:52 am
by TomDoan
You used * as the front end for the NLLS. The way that NLLS works, it will consider the start to be 1 and will knock that out with an internal SMPL. So BSTART is 1 which makes it possible for the bootstrap to pick an NA. (Won't happen on more than about 10% of the draws). If you change that to explicitly give 2 as the start period, it will work:
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y 2 cap_T
When you have a complex looping procedure like this, you should *never* start with the full range and with the NOPRINT on the estimation instruction. If you had NLLS(PRINT...) you would be able to see that it skipped a data point, which would be a sign that something wasn't correct.
Re: NAs generated when bootstrapping forecasts from an LSTAR
Posted: Thu May 29, 2014 5:44 am
by nybammer
Well that was the problem indeed! Works great now! Thank you very much for all the help!