NAs generated when bootstrapping forecasts from an LSTAR mod

[nybammer]

I am trying to bootstrap the forecasts (at 4, 8 and 12 steps ahead) from an LSTAR model using the following code:

Code: Select all

set lsforesBoot           = 0.
set fcst4_lsforesBoot     = 0.
set fcst8_lsforesBoot     = 0.
set fcst12_lsforesBoot    = 0.

com start = 1975:01 + 133; 
com end = 1997:06-12; 

do cap_T = start, end

nonlin a0 a1 b0 b1  d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar

stats(noprint) y
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean,d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;

group lstarmodel lstareq>>lsforesBoot

set meanBoot cap_T+1 cap_T+12 = 0.0
compute ndraws = 2500
do draws = 1,ndraws
  
 boot entries * cap_T
 set path = resids(entries)
 forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
 # path
 set meanBoot cap_T+1 cap_T+12 = meanBoot + bootsim(1)
end do draws

set meanBoot cap_T+1 cap_T+12 = meanBoot/ndraws

compute fcst4_lsforesBoot(cap_T+4)    = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8)    = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12)  = meanBoot(cap_T+12)

end do cap_T

For some reason several of the draws for meanBoot series are filled with NAs, even when the bootsim(1) series are not. This then doesn't allow me to average the results. I have tried some dirty tricks to discard those instances where the meanBoot series contains NAs:

Code: Select all

set valid_draw = 0.0

.....

set meanBoot cap_T+1 cap_T+12 = %if(%valid(meanBoot), bootsim(1) + meanBoot, 0)
SET valid_draw = %if(%valid(meanBoot), 1, 0) + valid_draw

*or
*if  %valid(meanBoot(t)) == 1; set meanBoot cap_T+1 cap_T+12 = bootsim(1) + meanBoot

*and then write

set meanBoot4 = meanBoot(cap_T+4)/valid_draw(cap_T+4)
set meanBoot8 = meanBoot(cap_T+8)/valid_draw(cap_T+8)
set meanBoot12 = meanBoot(cap_T+12)/valid_draw(cap_T+12)

The code above partially solves the problem by getting rid of the draws with NAs, but the forecasts are way off.

Anybody have any suggestions of what am I doing wrong? The reason I am puzzled is that when I Monte Carlo simulate the forecasts everything is working well.

Thanks in advance,

R.P.

[TomDoan]

The problem is that the idea of * in

boot entries * cap_T

is different from what it is in the NLLS and LINREG instructions. For BOOT, * will be 1, while for LINREG and NLLS, it will depend upon the variables (formulas) involved. Since you have lags, you're losing some data points at the start.

Instead, after

nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;

put

compute bstart=%regstart(),bend=%regend()

and then do a BOOT as

boot entries bstart bend

That way it will sample only from the entries where RESIDS is defined.

[nybammer]

Dear Tom,

Thank you very much for the reply. That solved the problem about the NAs in the meanBoot series. However, there is an additional question I have about the code that I hope you can help me with. The end purpose of the code is to get the forecasts at various steps ahead (4, 8, and 12). The way I am trying to get them is to write this (see in bold below):

set lsforesBoot = 0.
set fcst4_lsforesBoot = 0.
set fcst8_lsforesBoot = 0.
set fcst12_lsforesBoot = 0.

com start = 1975:01 + 133;
com end = 1997:06-12;

do cap_T = start, end

nonlin a0 a1 b0 b1 d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar

stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend

group lstarmodel lstareq>>lsforesBoot

set meanBoot = 0.0
compute ndraws = 2500
do draws = 1,ndraws

boot entries / bstart bend
set path = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path
set meanBoot = meanBoot + bootsim(1)


compute fcst4_lsforesBoot(cap_T+4) = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8) = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12) = meanBoot(cap_T+12)

end do draws

set meanBoot = meanBoot/ndraws

end do cap_T

pri start+1 end meanBoot bootsim(1) y path
pri start+1 end fcst4_lsforesBoot fcst8_lsforesBoot fcst12_lsforesBoot

I am puzzled that even though the meanBoot series looks alright, the fcst4_lsforesBoot, fcst8_lsforesBoot, and fcst12_lsforesBoot are filled with NAs. I tried several tricks (including the bold part of the code outside the "do draws" part of the code, or write:

Code: Select all

forecast(model=lstarmodel,paths,results=bootsim,steps=k,from=cap_T+1)

where k = 4, 8, or 12 and have three different meanBoot series).

However, none seemed to the job (i.e., in the second case the three meanBootk, where k=4,8,12 series are the same. What do you think I am doing wrong?

Thanks so much again...

[TomDoan]

The problem here is it's not clear that you're getting what you want for the bootstrapping. You want to draw entries numbers from bstart to bend for the range from CAP_T+1 to CAPT+12.

boot entries CAP_T+1 CAP_T+12 bstart bend
set path CAP_T+1 CAP_T+12 = resids(entries)
forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
# path


I think you may also want to take the >>... off

group lstarmodel lstareq>>lsforesBoot

It doesn't look like you use the redirect anywhere (since you have a RESULTS option on FORECAST).

[nybammer]

Thank you so much for getting back to me on this issue. So looks like I am almost there. In the code below everything looks good for the "meanBoot" series, but I am still unable to gather the forecasts at the 4, 8, and 12 steps ahead. Following your advice to do:

Code: Select all

boot entries cap_T+1 cap_T+12 bstart bend
 set path cap_T+1 cap_T+12 = resids(entries)
 

does not seem to help in this regard. The code I am using is below:

Code: Select all

do cap_T = start, end

nonlin a0 a1 b0 b1  d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar

stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend

group lstarmodel lstareq

 set meanBoot = 0.0;  *if I say "set meanBoot cap_T+1 cap_T+12 = 0.0" then meanBoot is filled with NAs
 
compute ndraws = 2500
do draws = 1,ndraws

 boot entries cap_T+1 cap_T+12 bstart bend
 set path cap_T+1 cap_T+12 = resids(entries)
 
 forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
 # path
 set meanBoot = meanBoot + bootsim(1)
 
end do draws

set meanBoot = meanBoot/ndraws

compute fcst4_lsforesBoot(cap_T+4)    = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8)    = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12)  = meanBoot(cap_T+12)

end do cap_T

pri start+1 end meanBoot bootsim(1) y  path

What is weird is that when I define the meanBoot series and if I were to write: "set meanBoot cap_T+1 cap_T+12 = 0.0" then meanBoot is filled with NAs. So for the code to work i simply say:

Code: Select all

set meanBoot = 0.0

.

However, then this part:

Code: Select all

compute fcst4_lsforesBoot(cap_T+4)    = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8)    = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12)  = meanBoot(cap_T+12)

does not lead to anywhere (i.e., fcstk_lsforesBoot series, where k=4,8, 12, are filled with NAs.) Do you have an insight what is leading to this issue and what could I do to get this working?

I appreciate all the help...

[TomDoan]

You'll have to post your full program and data set.

[nybammer]

The full program is actually the one in my last post. The data in question are listed below:

Code: Select all

3.357754218
3.337047362
3.312938592
3.342597789
3.330792125
3.319342037
3.367836145
3.404086059
3.414586294
3.399204437
3.39030879
3.398258762
3.386368139
3.371749804
3.36690916
3.353590391
3.356715381
3.363781785
3.355946118
3.3235835
3.310513563
3.283956875
3.269813527
3.252165453
3.24857697
3.254430389
3.247143424
3.243790144
3.241611161
3.233527629
3.194729312
3.202631214
3.205201341
3.184890552
3.169466887
3.135803478
3.122395381
3.105908129
3.088688589
3.097357976
3.130791758
3.120815691
3.10387908
3.072730853
3.058002241
3.001554394
3.03293442
3.019944319
3.001652429
3.00332297
3.005085411
3.022800992
3.026846216
3.005331605
2.972796647
2.96887336
2.94592255
2.941179486
2.928337669
2.905523679
2.902809674
2.912252138
2.964964215
2.976023232
2.933138488
2.915262075
2.905966459
2.928470151
2.921374951
2.956390656
3.000370761
3.0262569
3.052294048
3.115743717
3.102581032
3.133047297
3.189432072
3.218910694
3.241887443
3.257840802
3.192220511
3.138524172
3.124636285
3.136587627
3.151735202
3.168892708
3.163928207
3.16584358
3.126268289
3.171152438
3.178434016
3.171799064
3.169771331
3.17567097
3.183707765
3.129087156
3.108417789
3.11860678
3.102113016
3.101802361
3.110967817
3.143901699
3.155736163
3.191878758
3.189935196
3.161823595
3.18834737
3.211550987
3.232845818
3.196215271
3.15853989
3.170796354
3.204295856
3.192637726
3.227355666
3.233006323
3.271698592
3.284931788
3.259740221
3.290647876
3.309414995
3.339534458
3.335016371
3.256000875
3.254543628
3.229996931
3.174542159
3.120529774
3.13062568
3.064856281
3.045724882
3.011683325
2.970567309
2.904380874
2.864471289
2.868630681
2.851686074
2.847042767
2.795325609
2.742514771
2.72043779
2.696774578
2.70162583
2.679922377
2.611299191
2.59346291
2.594171208
2.582957251
2.572821483
2.59128546
2.605856305
2.606224069
2.577095158
2.570763833
2.494178986
2.460182742
2.46459466
2.483429523
2.467887387
2.467698737
2.481441602
2.522210453
2.574600819
2.561697265
2.544853874
2.514765064
2.460898263
2.467437878
2.563729837
2.568283594
2.57498216
2.578287433
2.615062245
2.624512307
2.573198057
2.581552268
2.596600442
2.550706612
2.522743252
2.474676631
2.457807171
2.440231022
2.445309355
2.427047057
2.414809395
2.430929163
2.401733986
2.368593609
2.369255055
2.350659065
2.307299535
2.306072525
2.329675841
2.29907427
2.37588517
2.4368191
2.445642338
2.4798705
2.478199727
2.457118221
2.428566257
2.426744289
2.382512299
2.34545452
2.348352961
2.376871417
2.403192838
2.395306027
2.381596013
2.355758315
2.296140287
2.263581923
2.26053972
2.280543516
2.348755235
2.344256108
2.365222199
2.382393033
2.392758216
2.366068458
2.372394637
2.393773689
2.42701097
2.412590488
2.363131584
2.374947296
2.414009185
2.41916962
2.44572228
2.446188529
2.420280712
2.421028967
2.395711778
2.380908071
2.345364611
2.345864119
2.334687371
2.310439162
2.324003208
2.347634792
2.32172071
2.299145734
2.23378488
2.216350236
2.23367891
2.230248863
2.218266734
2.256438811
2.27380345
2.243621642
2.239758703
2.255944495
2.274320314
2.275188565
2.286244407
2.313818204
2.337730867
2.338091689
2.322651913
2.308022481
2.322117701
2.333148043
2.322770785
2.355098647
2.390942644
2.42102769
2.425954712
2.433318852
2.429090183
2.440310968

[nybammer]

I am posting the full program here as well since the top part was missing:

Code: Select all

set lsforesBoot               = 0.
set fcst4_lsforesBoot      = 0.
set fcst8_lsforesBoot      = 0.
set fcst12_lsforesBoot     = 0.

com start = 1975:01 + 133;
com end = 1997:06-12;

do cap_T = start, end

nonlin a0 a1 b0 b1  d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar

stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend

group lstarmodel lstareq

 set meanBoot cap_T+1 cap_T+12 = 0.0;  *if I say set "meanBoot cap_T+1 cap_T+12 = 0.0" meanBoot is filled with NAs
 
compute ndraws = 2500
do draws = 1,ndraws

 boot entries cap_T+1 cap_T+12 bstart bend
 set path cap_T+1 cap_T+12 = resids(entries)
 
 forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
 # path
 set meanBoot = meanBoot + bootsim(1)
 
end do draws

set meanBoot = meanBoot/ndraws

compute fcst4_lsforesBoot(cap_T+4)    = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8)    = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12)  = meanBoot(cap_T+12)

end do cap_T

pri start+1 end meanBoot bootsim(1) y  path
pri start+1 end fcst4_lsforesBoot  fcst8_lsforesBoot  fcst12_lsforesBoot

[TomDoan]

frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))

needs to be written using %LOGISTIC as is described in the Users' Guide. The exp is overflowing because of the large values for d0 and d1.

If the posted data is your Y, it's not clear why you would want to do a STAR model on it. Your series is clearly non-stationary and is very close to being monotone declining, so any transition on Y{1} is basically going to partition the data based upon time.

[nybammer]

Yes, I need to take the first difference and make the data stationary. Thanks for pointing out the need to use the %logistic as well. However, the problem still remains that I cannot collect the forecasts at steps 4, 8, and 12. I should point out that the last code was causing NAs in the "meanBoot" series because for some reason when I write "set meanBoot cap_T+1 cap_T+12 = 0.0;" NAs are generated. However, the code below works well to get the meanBoot series but not the "fcstk_lsforesBoot" series. I'm afraid that changing the "frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))" expression won't solve this issue. This is really puzzling...

Code: Select all

set lsforesBoot               = 0.
set fcst4_lsforesBoot      = 0.
set fcst8_lsforesBoot      = 0.
set fcst12_lsforesBoot     = 0.

com start = 1975:01 + 133;
com end = 1997:06-12;

do cap_T = start, end

nonlin a0 a1 b0 b1  d0 d1
frml abranch = a0+a1*y{1}
frml bbranch = b0+b1*y{1}
*frml lstarwt = 1.0/(1+exp(d0+d1*y{1}))
frml lstarwt = %logistic(d0+d1*y{1},1.0)

frml lstareq y = fstar=lstarwt,abranch*(1-fstar)+ bbranch*fstar

stats(noprint) y * cap_T
compute ymean=%mean
linreg(noprint,smpl=(y{1}<ymean)) y * cap_T
# constant y{1}
compute a0=%beta(1),a1=%beta(2)
linreg(noprint,smpl=(y{1}>ymean)) y * cap_T
# constant y{1}
compute b0=%beta(1),b1=%beta(2)
compute d0=50.0*ymean, d1=-50.0
nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y * cap_T
set resids = %resids ;
compute bstart=%regstart(), bend=%regend()
*dis bstart; dis bend

group lstarmodel lstareq

 set meanBoot = 0.0;  *if I say set "meanBoot cap_T+1 cap_T+12 = 0.0" meanBoot is filled with NAs
 
compute ndraws = 2500
do draws = 1,ndraws

 boot entries cap_T+1 cap_T+12 bstart bend
 set path cap_T+1 cap_T+12 = resids(entries)
 
 forecast(model=lstarmodel,paths,results=bootsim,steps=12,from=cap_T+1)
 # path
 set meanBoot = meanBoot + bootsim(1)
 
end do draws

set meanBoot = meanBoot/ndraws

compute fcst4_lsforesBoot(cap_T+4)    = meanBoot(cap_T+4)
compute fcst8_lsforesBoot(cap_T+8)    = meanBoot(cap_T+8)
compute fcst12_lsforesBoot(cap_T+12)  = meanBoot(cap_T+12)

end do cap_T

pri start+1 end meanBoot bootsim(1) y  path
pri start+1 end fcst4_lsforesBoot  fcst8_lsforesBoot  fcst12_lsforesBoot

[TomDoan]

You used * as the front end for the NLLS. The way that NLLS works, it will consider the start to be 1 and will knock that out with an internal SMPL. So BSTART is 1 which makes it possible for the bootstrap to pick an NA. (Won't happen on more than about 10% of the draws). If you change that to explicitly give 2 as the start period, it will work:

nlls(noprint,frml=lstareq,trace,pmethod=simplex,method=evaluate,piters=20,cvcrit=0.01,iters=40) y 2 cap_T

When you have a complex looping procedure like this, you should *never* start with the full range and with the NOPRINT on the estimation instruction. If you had NLLS(PRINT...) you would be able to see that it skipped a data point, which would be a sign that something wasn't correct.

[nybammer]

Well that was the problem indeed! Works great now! Thank you very much for all the help!