simulate, estimate and forecast OOS univariate models incorporating a linear trend term

[ac_1]

Hi Tom,

I want to simulate, estimate and forecast OOS the following univariate models incorporating a linear trend term.
[1] Linear Trend stationary: y(t) = constant + beta*t + e(t)
[2] RWD Plus Linear trend: y(t) = constant + phi1*y(t-1) + beta*t + e(t), where phi=1
[3] AR(1) with constant Plus Linear trend: y(t) = constant + phi1*y(t-1) + beta*t + e(t), where phi=0.8 (say)

I think they can be handled either as Deterministic or Stochastic?
Specifically for:
[1]
(a) Linear Deterministic trend: y(t) = beta0 + beta(1)*trend + eta, eta being an ARMA process.
(b) Linear Stochastic trend: y(t) = beta0 + beta(1)*trend + eta, eta being an ARIMA process with diffs = 1
They are regression with time series errors.

Depending on how modelled it will make a big difference to the residual plots, and to the fans being wider.

I have estimated using either BOXJENK(GLS) or BOXJENK(REGRESSORS), with
(i) a trend and diffs=0, or
(ii) no trend and diffs=1?

Questions
- I do not understand the difference between the BOXJENK GLS and REGRESSORS options?
- Are the 3 processes simulated and modelled correctly?

[1] Linear Trend stationary: y(t) = constant + beta*t + e(t)

Code: Select all

*===============================
seed 1

compute sigma=3.0

compute nburn=100
compute nobs=500

set e 1 nburn+nobs = %ran(sigma)

* Linear Trend-Stationary model
set trend 1 nburn+nobs = t
set(first=0.0) y 1 nburn+nobs = 1.0 + 0.4*trend + e

prin / y

graph(key=upleft,head="Linear Trend-Stationary Process\\y = constant + beta*t + e(t)") 1
# y nburn+1 nburn+nobs


*===============================
* Linear Trend-Stationary model

* deterministic
* -------------
boxjenk(REGRESSORS,ar=0,diffs=0,ma=0,define=req,MAXL) y nburn+1 nburn+nobs resids
# constant trend

boxjenk(GLS,ar=0,diffs=0,ma=0,define=req,MAXL) y nburn+1 nburn+nobs resids
# constant trend

* stochastic
* ----------
boxjenk(REGRESSORS,ar=0,diffs=1,ma=0,define=req,MAXL) y nburn+1 nburn+nobs resids
# constant

boxjenk(GLS,ar=0,diffs=1,ma=0,define=req,MAXL) y nburn+1 nburn+nobs resids
# constant

[2] RWD with Linear trend: y(t) = constant + phi1*y(t-1) + beta*t + e(t), where phi=1

Code: Select all

*===============================
seed 1

compute nburn=100
compute nobs=500

set u 1 nburn+nobs = %ran(1.0)

* RWD Plus Linear Trend process: Level (Exponential pattern) and Change
set trend 1 nburn+nobs = t
set(first=0.0) y 1 nburn+nobs = 1.0 + 1.0*y{1} + 0.03*trend + u
set d = y-y{1}

graph(key=upleft,head=$
  "RWD Plus Linear Trend: Level and change",overlay=line) 2
# y nburn+1 nburn+nobs
# d nburn+1 nburn+nobs


*===============================
* RWD Plus Linear trend model

* stochastic
* ----------
boxjenk(REGRESSORS,ar=1,diffs=1,ma=0,define=req,MAXL,METHOD=GAUSS) y nburn+1 nburn+nobs resids
# constant

boxjenk(GLS,ar=1,diffs=1,ma=0,define=req,MAXL,METHOD=GAUSS) y nburn+1 nburn+nobs resids
# constant

[3] AR(1) with constant and Linear trend: : y(t) = constant + phi1*y(t-1) + beta*t + e(t), where e.g. phi=0.8

Same as [2] with e.g. phi1=0.8


thanks,
Amarjit

[TomDoan]

You will find it easier to separate the trend and stationary parts of the process. y(t)=trend+stationary part.

The deterministic trend is just trend(t)=a+bt+0
The stochastic trend is trend(t)=b+trend(t-1)+e

Note that the trend rate shifts to the "constant" in the stochastic trend and there is no overall "intercept" (as a random walk has no constant mean).

Simulate the stationary part separately and add it to the trend.

As you've written the formulas, your RWD + linear trend will actually have a quadratic trend.

[TomDoan]

- I do not understand the difference between the BOXJENK GLS and REGRESSORS options?

See https://estima.com/webhelp/topics/unifore-armax.html.

[ac_1]

You will find it easier to separate the trend and stationary parts of the process. y(t)=trend+stationary part.

The deterministic trend is just trend(t)=a+bt+0
The stochastic trend is trend(t)=b+trend(t-1)+e

Note that the trend rate shifts to the "constant" in the stochastic trend and there is no overall "intercept" (as a random walk has no constant mean).

Simulate the stationary part separately and add it to the trend.

As you've written the formulas, your RWD + linear trend will actually have a quadratic trend.

Thanks. Attached simulations please check.

Also, with these trend models I am confused with the definitions: constant and mu and drift?

And, in RWD + linear trend (i.e. quadratic trend) y(t) = a + phi1*y(t-1) + beta*t + e(t), where phi1=1. And theoretically
E(y(t)) = t*a + [(t*(t+1))/2]
Var(y(t)) = t*(sigma^2) - t^2*(t+1)*a*beta

However, what are the E(y(t)) and Var(y(t)) where |phi1|<1?

[TomDoan]

With |phi|<1, the trend goes to beta/(1-phi) x t + a messy constant (you should be running enough burn-in's that the asymptotics apply). The variance is the same as you would have without the trend. (Your variance for the RW is wrong).

[ac_1]

With |phi|<1, the trend goes to beta/(1-phi) x t + a messy constant (you should be running enough burn-in's that the asymptotics apply). The variance is the same as you would have without the trend. (Your variance for the RW is wrong).

|phi1|<1
E(y(t)) = (beta/(1-phi))*t + a messy constant.
Var(y(t)) = (sigma^2)/(1-phi^2)

What's the messy constant? Is there a derivation anywhere?

phi1=1
E(y(t)) = t*a + [(t*(t+1))/2]
Var(y(t)) = t*(sigma^2)

I have redone the algebra for variance, it's the same not including the trend.


Is actrends.rpf correct, I've increased nburn=200?

Also, how do I estimate the 5 trend models for OOS forecasting - BOXJENK and which options?

[TomDoan]

The messy constant can be computed if you know how to manipulate power series. But does it really matter? It's just the intercept. Note that you asked basically the same question earlier this year:

viewtopic.php?p=19352#p19352

Your last two are in the proper form for BOXJENK. The first three can be done with LINREG; of those the first two can trivially be done with BOXJENK as well. The third, however, while you can do it with BOXJENK, you will get completely different coefficients as it would be trying to estimate the trending process mean that's a complicated function of the a and beta. However, the reduced form generated by BOXJENK with DEFINE should be relatively close.

[ac_1]

The messy constant can be computed if you know how to manipulate power series. But does it really matter? It's just the intercept. Note that you asked basically the same question earlier this year:

viewtopic.php?p=19352#p19352

Yes it does. Sorry, but I didn't fully understand.

Also, with these trend models I am confused with the definitions: constant and mu and drift?

I want to be certain regarding the definitions - that's important.

y(t) = a + b*trend + e(t)
Is just a simple linear regression, so a = intercept, regardless of the value of b.

y(t) = a + phi1*y(t-1) + e(t)
if phi=1, a = drift
if |phi|<1, a = constant or intercept

y(t) = a + phi1*y(t-1) + b*trend + e(t)
if phi=1, a = drift
if |phi|<1, a = constant or intercept

I am interpreting the CONSTANT in BOXJENK as the mean (I know I have mentioned this before), but as you have said https://estima.com/forum/viewtopic.php?t=3775
" Because the Box-Jenkins model is a structural model, while the LINREG is the equivalent reduced form model. The estimated coefficients have different meanings. ... "

The LINREG CONSTANT in simple linear regression as the y-intercept.

a's name and interpretation depend's on the model, AND if estimated either via BOXJENK or LINREG. Here I need guidance with!

Your last two are in the proper form for BOXJENK. The first three can be done with LINREG; of those the first two can trivially be done with BOXJENK as well. The third, however, while you can do it with BOXJENK, you will get completely different coefficients as it would be trying to estimate the trending process mean that's a complicated function of the a and beta. However, the reduced form generated by BOXJENK with DEFINE should be relatively close.

I have attempted to estimate, and depending on how estimated that will affect the residuals (are they expected to be white-noise?) and shapes of the OOS fans.

Estimations w.r.t. the pre-defined parameters:
The 1st looks good.
The 2nd and 3rd BOXJENK looks good, but not LINREG.
The 4th and 5th look good for the AR(1) coeff, but how do I interpret the constant and trend? Also the SE's on the coeffs are different depending on BOXJENK REGRESSORS or GLS options (which one?), especially for the quadratic trend.

Attached actrends.rpf

There are 5 trend models, I have seen enders4p186.rpf, are there any more 'obvious' specifications?

[TomDoan]

This shows the conversion of the AR+trend from the BOXJENK form (first line) to the LINREG form (last). As you can see, the coefficients on the trend terms are completely different in the two. You can't add shifts to a dynamic model and think that they have the same effect as they would in a static model.

arconvert.gif (2.45 KiB) Viewed 28629 times

Note that the two are modeling exactly the same thing, just in a different form. The BOXJENK form is more "structural" in the sense that the a+bt is actually the process mean while the coefficients in the LINREG form are a complicated non-linear interaction of the structural parameters. It is not as handy for forecasting as it breaks the information into two pieces while the reduced form at the end has a single expression, which is why that is what is produced by BOXJENK(DEFINE=xxx).

[ac_1]

This shows the conversion of the AR+trend from the BOXJENK form (first line) to the LINREG form (last). As you can see, the coefficients on the trend terms are completely different in the two. You can't add shifts to a dynamic model and think that they have the same effect as they would in a static model.

arconvert.gif

Note that the two are modeling exactly the same thing, just in a different form. The BOXJENK form is more "structural" in the sense that the a+bt is actually the process mean while the coefficients in the LINREG form are a complicated non-linear interaction of the structural parameters. It is not as handy for forecasting as it breaks the information into two pieces while the reduced form at the end has a single expression, which is why that is what is produced by BOXJENK(DEFINE=xxx).

Thanks.

Therefore, BOXJENK is better than LINREG - and I should be using BOXJENK REGRESSORS for models y2 to y5 as I am estimating ARMAX models, and GLS for model y1. Right?

I have included a TRENDSQ term to the 4th, y4: quadratic trend process and removed the constant.

Code: Select all

    Variable                        Coeff      Std Error      T-Stat      Signif
************************************************************************************
1.  AR{1}                         0.982196673  0.008053562    121.95805  0.00000000
2.  TREND                        10.036218055  0.018950239    529.60905  0.00000000
3.  TRENDSQ                       0.025009592  0.000032593    767.33484  0.00000000

The AR{1} term is near 1.0, however TREND is "a" and TRENDSQ is "b", why is that?

In the 5 trend models:
All residuals appear to be well-behaved w.r.t Time Plot, ACF, and histogram with Gaussian overlay.
All OOS fans are different shapes across the models (estimation has not affected the overall shape per model), the quadratic having PI's at 95% almost identical to forecasts, and likewise the forecast SE's for y2 and y5.

I will do the same for enders4p186.rpf, and are there any more 'obvious' trend specifications?

[TomDoan]

I have included a TRENDSQ term to the 4th, y4: quadratic trend process and removed the constant.

Code: Select all

    Variable                        Coeff      Std Error      T-Stat      Signif
************************************************************************************
1.  AR{1}                         0.982196673  0.008053562    121.95805  0.00000000
2.  TREND                        10.036218055  0.018950239    529.60905  0.00000000
3.  TRENDSQ                       0.025009592  0.000032593    767.33484  0.00000000

The AR{1} term is near 1.0, however TREND is "a" and TRENDSQ is "b", why is that?

I think it would help you to figure that out yourself. Solve the recursion that you used to create the data.

[ac_1]

I have also estimated trends via differencing, however with BOXJENK diffs=1 there are (on occasion) significant lags in the residuals ACF plot. Estimation has affected the fans shapes per simulated model.

y1: linear trend, estimated with LINREG, BOXJENK(GLS) diffs=0, and BOXJENK diffs=1 here the residuals have a significant 1st lag in the ACF plot - which is ARMA errors?

y2: deterministic trend, estimated with LINREG, BOXJENK(REGRESSORS) diffs=0, and BOXJENK diffs=1.

y3: stochastic trend, estimated with LINREG, BOXJENK(REGRESSORS) diffs=0, and BOXJENK diffs=1.

Is y4(t) = a + 1.0*y4(t-1) + b*t + u(t), resembling a quadratic trend (but is not a pure quadratic trend as in y(t) = const + beta*t + gamma*t^2), the correct way to simulate

set(first=0.0) y4 1 nburn+nobs = a + 1.0*y4{1} + b*trend + u

or does the equation need to be simulated separately? If simulated correctly with BOXJENK diffs=1 the residuals have a significant 1st lag in the ACF plot.

I have removed y5 as I should be simulating with y2 instead.

Are these correct?

Attached actrends.rpf

[TomDoan]

If you first difference an equation that doesn't need it, you induce a moving average term of the form (1-L)eps which will show high negative autocorrelation. (Just take the equation and difference it to see). It won't be as big a problem if the process had a AR root relatively near 1; it should show some negative residual autocorrelation, but it might not be large enough to show in contrast to the usual residual autocorrelation.


y(t)=y(t-1)+a+bt+eps(t)

generates a quadratic trend with RW deviations which of course is not the same thing as a quadratic trend with i.i.d deviations, but it's still a quadratic trend. However, note that no one actually believes that a quadratic trend is likely a reasonable model for a trend process with economic data.


You seem to be spending an inordinate of time figuring out empirically that which can be determined algebraically. It you are interested in trending behavior, you might want to read

Nelson, C. and C. Plosser(1982), "Trends and Random Walks in Macroeconomic Time Series: Some Evidence and Implications." Journal of Monetary Economics, vol 10, pp 139-162.

That's generally considered to be the source for the standard assumption that drifting random walks on logs are generally the best representation of trends in economic data.

[ac_1]

If you first difference an equation that doesn't not need it, you induce a moving average term of the form (1-L)eps which will show high negative autocorrelation. (Just take the equation and difference it to see). It won't be as big a problem if the process had a AR root relatively near 1; it should show some negative residual autocorrelation, but it might not be large enough to show in contrast to the usual residual autocorrelation.

Thanks.

1st-differencing y1: linear trend (i.e. a stationary process around a deterministic trend or trend-stationary process) produces a high negative autocorrelation in the residuals at lag 1. Can that not be handled w.r.t. fit and forecasts? TSay(2010) p.588 models linear regression with ARMA errors.

y(t)=y(t-1)+a+bt+eps(t)

generates a quadratic trend with RW deviations which of course is not the same thing as a quadratic trend with i.i.d deviations, but it's still a quadratic trend. However, note that no one actually believes that a quadratic trend is likely a reasonable model for a trend process with economic data.

I agree. I have removed y4.


I've read

Nelson, C. and C. Plosser(1982), "Trends and Random Walks in Macroeconomic Time Series: Some Evidence and Implications." Journal of Monetary Economics, vol 10, pp 139-162.

it is a seminal/watershed paper in how to handle modelling trends, and highly recommended!


Given economic series, modelling as a univariate, I naturally use @BJTRANS and @BJDIFF. y1 to y3 are constructed series.
If I try:
- @BJTRANS the criterion (the higher value is better) suggests NONE, all series.
- @BJDIFF t/f=NONE, Seas Diff = 1, y1, y2 and y3. Which is strange.

As it is, I should not 1st-difference y1 and y2. Could I also model y1 and y2 as in Rothman(1998) https://estima.com/forum/viewtopic.php? ... man#p17979 ?


I have removed all 1st differencing models, so I have modelled:

y1
linreg(define=eq1lreg) y1 nburn+1 nburn+nobs resids_eq1lreg
# constant trend

boxjenk(print,ar=0,diffs=0,ma=0,GLS,define=eq1bjgls,MAXL) y1 nburn+1 nburn+nobs resids_eq1bjgls
# constant trend

y2
linreg(define=eq2lreg) y2 nburn+1 nburn+nobs resids_eq2lreg
# constant y2{1} trend

boxjenk(print,ar=||1||,diffs=0,ma=0,REGRESSORS,define=eq2bjreg,MAXL) y2 nburn+1 nburn+nobs resids_eq2bjreg
# constant trend

y3
linreg(define=eq3lreg) y3 nburn+1 nburn+nobs resids_eq3lreg
# constant y3{1} trend

boxjenk(print,ar=||1||,diffs=0,ma=0,REGRESSORS,define=eq3bjreg,MAXL) y3 nburn+1 nburn+nobs resids_eq3bjreg
# constant trend


Let me know if I have made error(s) and/or any other appropriate ways.

[TomDoan]

My suggestion is that you go back to Nelson and Plosser and read the mathematical background. They go over the difference between trend stationary and difference stationary series (basically what you are simulating here) and how the wrong choice can cause problems (what you are discovering here).

Note that @BJDIFF is not designed to allow for a trend stationary series.