Transformations, Prediction Intervals, Bias adjusted mean

Questions and discussions on Time Series Analysis

Transformations, Prediction Intervals, Bias adjusted mean

Unread postby ac_1 » Thu Apr 21, 2022 2:57 am

Hi Tom,

I am trying to get the same results as https://otexts.com/fpp2/transformations.html, Figure 3.4: Forecasts of egg prices using a random walk with drift applied to the logged data.

The back-transformed point forecast, considered the median of the forecast density, of the forecast distribution is simply achieved exp'ing the logged transformed data exp(lrfore), i.e. Simple back transformation.

Code: Select all
set leggs = log(eggs)

boxjenk(print,constant,ar=0,diffs=1,ma=0,define=eq) leggs 1900 1993
uforecast(equation=eq,from=1993+1,steps=50,stderrs=lstderrs) lrfore

set lL80 1993+1 1993+50 = lrfore+%invnormal(.1)*lstderrs
set lU80 1993+1 1993+50 = lrfore+%invnormal(.9)*lstderrs

set rfore 1993+1 1993+50 = exp(lrfore); * I can EXP(MEAN) i.e. considered the MEDIAN in the original scale

But, I cannot simply exp the transformed PI's: exp(lL80) and exp(lU80), to get back to the original scale - Figure 3.4 's PI's are slightly wider, than simply exp'ing the PI's.
From https://otexts.com/fpp2/prediction-intervals.html , "If a transformation has been used, then the prediction interval should be computed on the transformed scale, and the end points back-transformed to give a prediction interval on the original scale."

How do I calculate
(a) the back-transformed PI's, assuming positive bounds?


Also,
(b) the back-transformed bias-adjusted mean point forecasts - similar to this, but this is not correct?

Code: Select all
set fvar 1993+1 1993+50 = ((%BoxCox(lU80,0) - %BoxCox(lL80,0))/1.28/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)


thanks,
Amarjit
ac_1
 
Posts: 195
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: Transformations, Prediction Intervals, Bias adjusted mea

Unread postby TomDoan » Thu Apr 21, 2022 10:05 am

I'm not sure what you're saying. The recommendation in your source is to use the backtransformed lower and upper bounds. They will be asymmetrical, being slightly wider above the forecast than below. What is it that you think is wrong with exping the lower and upper bounds?
TomDoan
 
Posts: 7223
Joined: Wed Nov 01, 2006 5:36 pm

Re: Transformations, Prediction Intervals, Bias adjusted mea

Unread postby ac_1 » Fri Apr 22, 2022 1:56 am

On second thoughts, exp'ing the transformed PI's is correct. It's the meaning regarding "end points" in "end points back-transformed" which is puzzling?

However, if I exp the transformed PI's, I cannot achieve the exact results as in Figure 3.4. The PI's I get are narrower, but similarly asymmetrical in shape.

On running the example in R:

Point Forecast, Lo 80, Hi 80
1994, 61.27912, 51.663779, 72.68400
...
2043, 27.92292 , 6.284652, 124.06248

and from exp'ing the transformed series and transformed PI's in R or in RATS:

ENTRY, RFORE, L80, U80
1994, 61.2791179130, 51.6637789043, 72.6840036064
...
2043, 27.9229204377, 8.3523462489, 93.3497561682

Here https://otexts.com/fpp2/prediction-intervals.html says the back-transformation of prediction intervals is done automatically.


The biasadj_mean calculation is

Code: Select all
set fvar 1993+1 1993+50 = ( (%BoxCox(U80,0) - %BoxCox(L80,0) )/%invnormal(.9)/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)

The question is which forecast to use for loss-measures and correct directional accuracy, etc: median (i.e. back-transformed) or biasadj_mean?

For a square-root transformation, to back-transform I would just square the forecast mean and square the PI's. Correct?
Would the forecast mean still be interpreted as the median, and also the biasadj_mean calculated as above?

I use @BJTRANS to choose a preliminary transformation. How to calculate the "optimal" lambda for BoxCox?
ac_1
 
Posts: 195
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: Transformations, Prediction Intervals, Bias adjusted mea

Unread postby TomDoan » Thu Apr 28, 2022 8:46 am

ac_1 wrote:On second thoughts, exp'ing the transformed PI's is correct. It's the meaning regarding "end points" in "end points back-transformed" which is puzzling?

However, if I exp the transformed PI's, I cannot achieve the exact results as in Figure 3.4. The PI's I get are narrower, but similarly asymmetrical in shape.

On running the example in R:

Point Forecast, Lo 80, Hi 80
1994, 61.27912, 51.663779, 72.68400
...
2043, 27.92292 , 6.284652, 124.06248

and from exp'ing the transformed series and transformed PI's in R or in RATS:

ENTRY, RFORE, L80, U80
1994, 61.2791179130, 51.6637789043, 72.6840036064
...
2043, 27.9229204377, 8.3523462489, 93.3497561682

Here https://otexts.com/fpp2/prediction-intervals.html says the back-transformation of prediction intervals is done automatically.


The R code is producing incorrect results if it's supposed to be backtransforming the calculations in log form.


ac_1 wrote:The biasadj_mean calculation is

Code: Select all
set fvar 1993+1 1993+50 = ( (%BoxCox(U80,0) - %BoxCox(L80,0) )/%invnormal(.9)/2)**2
set biasadj_mean 1993+1 1993+50 = exp(lrfore) * (1 + 0.5*fvar)

The question is which forecast to use for loss-measures and correct directional accuracy, etc: median (i.e. back-transformed) or biasadj_mean?


That depends upon your loss function. To look at the most important transformation (log), the non-adjusted forecast treats over and under percentage errors the same, while the bias-adjusted forecast treats over and under absolute errors the same. You have to decide which is more realistic.

ac_1 wrote:For a square-root transformation, to back-transform I would just square the forecast mean and square the PI's. Correct?
Would the forecast mean still be interpreted as the median, and also the biasadj_mean calculated as above?


A monotonic transformation preserves quantiles such as the median.

ac_1 wrote:I use @BJTRANS to choose a preliminary transformation. How to calculate the "optimal" lambda for BoxCox?


That's generally a bad idea. If a transformation is clearly necessary (typically when log is required), you should just do it. If there is relatively little difference, all you're doing is adding an extra (poorly estimated) parameter.
TomDoan
 
Posts: 7223
Joined: Wed Nov 01, 2006 5:36 pm

Re: Transformations, Prediction Intervals, Bias adjusted mea

Unread postby ac_1 » Fri Apr 29, 2022 3:53 am

Many Thanks.
ac_1
 
Posts: 195
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK


Return to Other Time Series Analysis

Who is online

Users browsing this forum: No registered users and 2 guests