The RATS Software Forum

Posted: **Fri Jun 30, 2023 6:19 am**

Before I write a main program and include in acLPDS.src, here's the recursive one-step ahead:

*===============================
seed 100

set(first=0.0) x 1 100 = .9*x{1}+%ran(1.0)
set x = x+6.0


*===============================
clear stdf

do regend = 80, 99
   boxjenk(noprin,const,ar=1,define=lineareq) x 1 regend
   set linearstdf regend+1 regend+1 = %sigmasq
   uforecast(equation=lineareq,from=regend+1,steps=1) linearf
end do regend

prin / linearstdf linearf

sstats 81 100 %logdensity(linearstdf,x-linearf)>>linlpds

disp linlpds


*===============================
clear logstdf

set logx = log(x)
do regend = 80, 99
   boxjenk(noprin,const,ar=1,define=logeq) logx 1 regend
   set logstdf regend+1 regend+1 = %sigmasq
   uforecast(equation=logeq,from=regend+1,steps=1) logf
end do regend

prin / logstdf logf

sstats 81 100 %logdensity(logstdf,log(x)-logf)-log(x)>>loglpds

disp loglpds


*===============================
clear sqrtstdf

set sqrtx = sqrt(x)
do regend = 80, 99
   boxjenk(noprin,const,ar=1,define=sqrteq) sqrtx 1 regend
   set sqrtstdf regend+1 regend+1 = %sigmasq
   uforecast(equation=sqrteq,from=regend+1,steps=1) sqrtf
end do regend

prin / sqrtstdf sqrtf

sstats 81 100 %logdensity(sqrtstdf,sqrt(x)-sqrtf)-log(2*sqrt(x))>>sqrtlpds

disp sqrtlpds

Questions:

(i) The LPDS for LOG and SQRT are not calculated on the back-transformed forecasts, rather can only be calculated on the transformed scale. Right?

(ii) In acLPDS.src I have

procedure acLPDS actual fcast start end stdsq
*
type series actual fcast
type integer start end
type real stdsq

For multi-step ahead forecasts I only need 1 %sigmasq (and it's a REAL).

I would like acLPDS.src to be able to handle both forecasting schemes. For one-step ahead forecasts I need all %sigmasq's as a series and aligned to actual and fcast. But stdsq cannot both be a (single) REAL and a SERIES. So if I try for one-step ahead's

procedure acLPDSS actual fcast stdsq start end
*
type series actual fcast stdsq

this won't work for multi-step ahead forecasts, the %sigmasq (stdsq) is a REAL not a SERIES (and doesn't have to be aligned). And I have to include all parameters in order when running except for the start and end.

Obviously, I can write a seperate procedure for one-step ahead, but that's not ideal.

(iii) %sigmasq is not in the output at the top of BOXJENK. What is the formula for %sigmasq?

(iv) I have included

inquire(reglist) startl<<start endl<<end
# actual fcast

in all of the procedures I've written so I don't have to explicitly put in start and end value's/date's. I have always wondered what does this exactly do i.e. why the reglist even though there are no regressions to be estimated?

Posted: **Fri Jun 30, 2023 11:11 am**

ac_1 wrote: Questions:

(i) The LPDS for LOG and SQRT are not calculated on the back-transformed forecasts, rather can only be calculated on the transformed scale. Right?

I'm not sure what you mean by that, but the calculations above are designed to measure the LPDS for the levels, regardless of what you actually use in the model.

ac_1 wrote: (ii) In acLPDS.src I have

procedure acLPDS actual fcast start end stdsq
*
type series actual fcast
type integer start end
type real stdsq

For multi-step ahead forecasts I only need 1 %sigmasq (and it's a REAL).

I would like acLPDS.src to be able to handle both forecasting schemes. For one-step ahead forecasts I need all %sigmasq's as a series and aligned to actual and fcast. But stdsq cannot both be a (single) REAL and a SERIES. So if I try for one-step ahead's

procedure acLPDSS actual fcast stdsq start end
*
type series actual fcast stdsq

this won't work for multi-step ahead forecasts, the %sigmasq (stdsq) is a REAL not a SERIES (and doesn't have to be aligned). And I have to include all parameters in order when running except for the start and end.

Obviously, I can write a seperate procedure for one-step ahead, but that's not ideal.

You can always have a SERIES all of whose members are the same. However, I'm not at all sure what you intend to do with multi-step ahead forecasts. You can't compute LPDS for forecasts across different numbers of steps. You can do one calculation for one-step and another for two-step, etc, but nothing for a combination of one and two and ...

ac_1 wrote: (iii) %sigmasq is not in the output at the top of BOXJENK. What is the formula for %sigmasq?

Maximum likelihood estimate of sigmasq, i.e. with a T divisor rather than degrees of freedom.

ac_1 wrote: (iv) I have included

inquire(reglist) startl<<start endl<<end
# actual fcast

in all of the procedures I've written so I don't have to explicitly put in start and end value's/date's. I have always wondered what does this exactly do i.e. why the reglist even though there are no regressions to be estimated?

REGRESSORLIST is a general term for a list of series which could possibly include lag/lead notation. The list does not have to include lags and leads, and in this case it doesn't.

Posted: **Sun Jul 02, 2023 11:15 am**

TomDoan wrote:
ac_1 wrote:
(ii) In acLPDS.src I have

procedure acLPDS actual fcast start end stdsq
*
type series actual fcast
type integer start end
type real stdsq

For multi-step ahead forecasts I only need 1 %sigmasq (and it's a REAL).

I would like acLPDS.src to be able to handle both forecasting schemes. For one-step ahead forecasts I need all %sigmasq's as a series and aligned to actual and fcast. But stdsq cannot both be a (single) REAL and a SERIES. So if I try for one-step ahead's

procedure acLPDSS actual fcast stdsq start end
*
type series actual fcast stdsq

this won't work for multi-step ahead forecasts, the %sigmasq (stdsq) is a REAL not a SERIES (and doesn't have to be aligned). And I have to include all parameters in order when running except for the start and end.

Obviously, I can write a seperate procedure for one-step ahead, but that's not ideal.
You can always have a SERIES all of whose members are the same. However, I'm not at all sure what you intend to do with multi-step ahead forecasts. You can't compute LPDS for forecasts across different numbers of steps. You can do one calculation for one-step and another for two-step, etc, but nothing for a combination of one and two and ...

My aim is to calculate LPDS, as a MAIN program and a procedure, for both:
(a) multi-step ahead forecasts (%sigmasq is a REAL and has not been aligned to the 1st OOS forecast); based on your example.
(b) recursive one-step ahead forecasts (all %sigmasq's as a SERIES have been aligned to the one-step head forecasts).
(a) and (b) cannot both be correct? And, also are you saying LPDS is not applicable for recursive one-step ahead?

TomDoan wrote:
ac_1 wrote: (iii) %sigmasq is not in the output at the top of BOXJENK. What is the formula for %sigmasq?
Maximum likelihood estimate of sigmasq, i.e. with a T divisor rather than degrees of freedom.

The residual SE squared or variance: (summation(resids^2))/T

set residsq 1 80 = %resids^2.0
stats residsq
disp %mean

Posted: **Sun Jul 02, 2023 3:59 pm**

The UFORECASTS in my (quick and dirty example to show you what you were doing wrong) should have had STATIC options. (I've corrected it above).

No, you can't aggregate fit statistics across different forecast horizons.

Posted: **Mon Jul 03, 2023 4:04 am**

TomDoan wrote:The UFORECASTS in my (quick and dirty example to show you what you were doing wrong) should have had STATIC options. (I've corrected it above).

No, you can't aggregate fit statistics across different forecast horizons.

Why can't the forecast standard errors STDERRS be used to calculate LPDS?

Posted: **Mon Jul 03, 2023 8:22 am**

The forecast errors at different horizons have VERY different statistical properties, and trying to aggregate any type of analysis across them is a bad idea.

I'm not sure where you are going with this. If you are trying to pick the "best" model for forecasting based upon simulated ex post forecasting performance, the LPDS isn't a good measure to use (as I have explained several times). Figure out what the most appropriate loss function is for your application (depending upon the situation, this is usually either the RMSE or RMS percentage error) and use that.

The RATS Software Forum

Plot various Normal's

Re: Plot various Normal's

Re: Plot various Normal's

Re: Plot various Normal's

Re: Plot various Normal's

Re: Plot various Normal's

Re: Plot various Normal's