Evaluating Distributional Forecasts

Questions and discussions on Time Series Analysis

Re: Evaluating Distributional Forecasts

Unread postby TomDoan » Tue Jul 05, 2022 7:53 am

If your simulations are coming in with similar results to the analytical forecasts, then the "directional accuracy" should also have similar results. If they don't, then you are probably doing the calculation wrong when inferring directional accuracy from the analytical results.
Posts: 7385
Joined: Wed Nov 01, 2006 5:36 pm

Re: Evaluating Distributional Forecasts

Unread postby TomDoan » Thu Jul 14, 2022 10:46 pm

I would note that one of the methods you are using is Normal simulations. Given a high enough number of simulations, that should converge to the "analytical" solutions. Normal simulations are useful if you are doing something that requires calculations across forecast horizons (such as, P(min value < x over range [t0,t1]), but for looking only at single forecasts will differ from analytical values only by experimental errors.
Posts: 7385
Joined: Wed Nov 01, 2006 5:36 pm

Re: Evaluating Distributional Forecasts

Unread postby ac_1 » Sun Feb 12, 2023 9:11 am


TomDoan wrote:Why are you using two DO loops rather than two SET instructions? Neither one of those is saving the calculations into different slots in a time series. You can use this for the LPDS:

set scores_rwwd iend+1 iend+steps = %LOGDENSITY(stderrs_rwwd^2,y-yhat_rwwd)
sstats(mean) iend+1 iend+steps scores_rwwd>>lpds_rwwd
disp 'lpds_rwwd' lpds_rwwd

Note, BTW, that the LPDS measure is likely to favor forecasting procedures which get a better value of the variance, than a better value of the mean.

The above is for a normal distribution.

Using the transformations from @BJTRANS, what would be the LPDS for
(a) a log-normal distribution?
(b) a non-central chi-squared distribution?

LPDS is just an average of the densities of each forecast distribution. For (b) I have tried the following:

Code: Select all
set ncsq iend+1 iend+12 = yhat_D^2.0
sstats iend+1 iend+12 ncsq>>nc
set scores iend+1 iend+12 = %chisqrncdensity(stderrs_D,%ndf,nc)
sstats(mean) iend+1 iend+12 scores>>lpds
disp *.############ 'lpds' lpds

yhat_D and stderrs_D are the forecasts and forecast SE's in the square-root scale, and non-centrailty parameter nc is the sum of the squared forecasts; but get 0 :?: Please advise.
Posts: 295
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK


Return to Other Time Series Analysis

Who is online

Users browsing this forum: No registered users and 6 guests