Transformations, Prediction Intervals, Bias adjusted mean

[ac_1]

Hi Tom,

I am trying to get the same results as https://otexts.com/fpp2/transformations.html, Figure 3.4: Forecasts of egg prices using a random walk with drift applied to the logged data.

The back-transformed point forecast, considered the median of the forecast density, of the forecast distribution is simply achieved exp'ing the logged transformed data exp(lrfore), i.e. Simple back transformation.

Code: Select all

set leggs = log(eggs)

boxjenk(print,constant,ar=0,diffs=1,ma=0,define=eq) leggs 1900 1993
uforecast(equation=eq,from=1993+1,steps=50,stderrs=lstderrs) lrfore

set lL80 1993+1 1993+50 = lrfore+%invnormal(.1)*lstderrs
set lU80 1993+1 1993+50 = lrfore+%invnormal(.9)*lstderrs

set rfore 1993+1 1993+50 = exp(lrfore); * I can EXP(MEAN) i.e. considered the MEDIAN in the original scale


But, I cannot simply exp the transformed PI's: exp(lL80) and exp(lU80), to get back to the original scale - Figure 3.4 's PI's are slightly wider, than simply exp'ing the PI's.
From https://otexts.com/fpp2/prediction-intervals.html , "If a transformation has been used, then the prediction interval should be computed on the transformed scale, and the end points back-transformed to give a prediction interval on the original scale."

How do I calculate
(a) the back-transformed PI's, assuming positive bounds?


Also,
(b) the back-transformed bias-adjusted mean point forecasts - similar to this, but this is not correct?

Code: Select all

set fvar 1993+1 1993+50 = ((%BoxCox(lU80,0) - %BoxCox(lL80,0))/1.28/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)


thanks,
Amarjit

[TomDoan]

I'm not sure what you're saying. The recommendation in your source is to use the backtransformed lower and upper bounds. They will be asymmetrical, being slightly wider above the forecast than below. What is it that you think is wrong with exping the lower and upper bounds?

[ac_1]

On second thoughts, exp'ing the transformed PI's is correct. It's the meaning regarding "end points" in "end points back-transformed" which is puzzling?

However, if I exp the transformed PI's, I cannot achieve the exact results as in Figure 3.4. The PI's I get are narrower, but similarly asymmetrical in shape.

On running the example in R:

Point Forecast, Lo 80, Hi 80
1994, 61.27912, 51.663779, 72.68400
...
2043, 27.92292 , 6.284652, 124.06248

and from exp'ing the transformed series and transformed PI's in R or in RATS:

ENTRY, RFORE, L80, U80
1994, 61.2791179130, 51.6637789043, 72.6840036064
...
2043, 27.9229204377, 8.3523462489, 93.3497561682

Here https://otexts.com/fpp2/prediction-intervals.html says the back-transformation of prediction intervals is done automatically.


The biasadj_mean calculation is

Code: Select all

set fvar 1993+1 1993+50 = ( (%BoxCox(U80,0) - %BoxCox(L80,0) )/%invnormal(.9)/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)

The question is which forecast to use for loss-measures and correct directional accuracy, etc: median (i.e. back-transformed) or biasadj_mean?

For a square-root transformation, to back-transform I would just square the forecast mean and square the PI's. Correct?
Would the forecast mean still be interpreted as the median, and also the biasadj_mean calculated as above?

I use @BJTRANS to choose a preliminary transformation. How to calculate the "optimal" lambda for BoxCox?

[TomDoan]

On second thoughts, exp'ing the transformed PI's is correct. It's the meaning regarding "end points" in "end points back-transformed" which is puzzling?

However, if I exp the transformed PI's, I cannot achieve the exact results as in Figure 3.4. The PI's I get are narrower, but similarly asymmetrical in shape.

On running the example in R:

Point Forecast, Lo 80, Hi 80
1994, 61.27912, 51.663779, 72.68400
...
2043, 27.92292 , 6.284652, 124.06248

and from exp'ing the transformed series and transformed PI's in R or in RATS:

ENTRY, RFORE, L80, U80
1994, 61.2791179130, 51.6637789043, 72.6840036064
...
2043, 27.9229204377, 8.3523462489, 93.3497561682

Here https://otexts.com/fpp2/prediction-intervals.html says the back-transformation of prediction intervals is done automatically.

The R code is producing incorrect results if it's supposed to be backtransforming the calculations in log form.

The biasadj_mean calculation is

Code: Select all

set fvar 1993+1 1993+50 = ( (%BoxCox(U80,0) - %BoxCox(L80,0) )/%invnormal(.9)/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)

The question is which forecast to use for loss-measures and correct directional accuracy, etc: median (i.e. back-transformed) or biasadj_mean?

That depends upon your loss function. To look at the most important transformation (log), the non-adjusted forecast treats over and under percentage errors the same, while the bias-adjusted forecast treats over and under absolute errors the same. You have to decide which is more realistic.

For a square-root transformation, to back-transform I would just square the forecast mean and square the PI's. Correct?
Would the forecast mean still be interpreted as the median, and also the biasadj_mean calculated as above?

A monotonic transformation preserves quantiles such as the median.

I use @BJTRANS to choose a preliminary transformation. How to calculate the "optimal" lambda for BoxCox?

That's generally a bad idea. If a transformation is clearly necessary (typically when log is required), you should just do it. If there is relatively little difference, all you're doing is adding an extra (poorly estimated) parameter.

[ac_1]

Many Thanks.