graphing a large number of series on a single graph

[ac_1]

Hi Tom,

I want to plot a large number of series on a single graph with a supplementary card list. They are as a VECTOR of SERIES fcast. Plotting in the normal manner works fine, however using LIST and CARDS

Code: Select all

comp NCount = 30
list ieqn = 1 to NCount
graph NCount+4
cards fcast(ieqn) fstart fend
# lsales 1968:1 1993:12 1
# res_M fstart fend
# L95 fstart fend 3
# U95 fstart fend 3

results in

## SC7. LIST Needs At Least 31 Entries, Had Only 30


Additionally, is there a way to have all the fcast as black & white patterns, and the supplementary series as colours?

Cheers,
Amarjit

[TomDoan]

There is nothing that will let you do a combination of sources for series that way. (BTW, the SERIES and SYMBOLS options are easier to handle for a VECT[SERIES] than LIST and CARDS). You would have to add the supplementary series to the VECT[SERIES].

You can define line colors as black and white by using an RGB "color" that has equal parts of R, G and B. (000000 is black, FFFFFF is white, CCCCCC is a medium gray).

Code: Select all

dec vect[series] x(15)
do i=1,15
   set x(i) 1 30 = 2*i+%ran(1.0)
end do i
grparm(define="LINE_COLOR_06=1,000000,1")
grparm(define="LINE_COLOR_07=2,000000,1")
grparm(define="LINE_COLOR_08=3,000000,1")
grparm(define="LINE_COLOR_09=4,000000,1")
grparm(define="LINE_COLOR_10=5,000000,1")
grparm(define="LINE_COLOR_11=6,000000,1")
grparm(define="LINE_COLOR_12=1,888888,1")
grparm(define="LINE_COLOR_13=1,CCCCCC,1")
grparm(define="LINE_COLOR_14=1,444444,1")
grparm(define="LINE_COLOR_15=1,222222,1")
graph(series=x)

[ac_1]

I have added series: lsales, res_M, L95, U95 to the VECT[SERIES] pfcast.

I would like lsales as black (colour 1), res_M as blue (colour 2), the PI's L95, U95 as green (colour 5) and all fcast as grey's (colour 48).

Code: Select all

grparm(define="LINE_COLOR_01=1,CCCCCC,1"); * grey
grparm(define="LINE_COLOR_31=1,000000,1"); * black
grparm(define="LINE_COLOR_32=1,0000FF,1"); * blue
grparm(define="LINE_COLOR_33=1,00FF00,1"); * green
grparm(define="LINE_COLOR_34=1,00FF00,1"); * green

declare vector[series] pfcast(34)
do j = 1, 30
   set pfcast(j) fstart fend = fcast(j)
end do j
set pfcast(31) 1990:1 1993:12 = lsales
set pfcast(32) fstart fend = res_M
set pfcast(33) fstart fend = L80
set pfcast(34) fstart fend = U80

graph(series=pfcast)

NB: fcast is actually large (e.g. 1000), as I am generating SIMULATE fcast's from UFORECAST.

But I cannot get the correct colours for all the series?

[TomDoan]

There are only 30 styles---by default, when you graph more than 30 series they start to repeat (31 is then the same as 1, 32 as 2, ...). See how the HISTORY.RPF example uses the SYMBOLS option to control the colors.

Offhand, though, I'm thinking that this will end up being a tangle of lines that will be impossible to read.

[ac_1]

The idea is to generate a tangle of lines, to see the probability forecast distribution.

[TomDoan]

You won't see anything with 1000 lines. Given the resolution of any device currently in standard use, you'll just get a blob through about 2.5 standard deviations. This is not like a scatter plot where you represent single pixels on a grid which might have 1000000 intersections so 1000 points will show you clustering. Instead, you have only 1000 or so vertical pixels available at each time series point, and probably half of those are in the sparse regions (the 3+ standard deviations that you'll get with that many simulations). Plus, you get the lines connecting the time series points. Every spot anywhere near the center of the distribution will get covered and it won't matter that the ones closer to the center get covered more than one time, as all that will happen is that the pixel will show the last color written to it.

Note that in the example above where I showed defining black and white styles, I shifted the values of each sample by adding 2*i to series i. If I hadn't done that, it would have been almost impossible to pick out any of the styles.

[ac_1]

The question then is if there are several thousand paths generated from SIMULATE, how to plot them to have an idea of the area or size of the forecast probability distribution?

[TomDoan]

Probably the best way is with a fan chart. This is an example which does percentiles of a cloud of simulations in the VECT[SERIES] called TREND.

Code: Select all

set trend025  = work=%xt(trends,t),%fractiles(work,||.025||)(1)
set trend050  = work=%xt(trends,t),%fractiles(work,||.050||)(1)
set trendmean = work=%xt(trends,t),%avg(work)
set trend950  = work=%xt(trends,t),%fractiles(work,||.950||)(1)
set trend975  = work=%xt(trends,t),%fractiles(work,||.975||)(1)
*
graph(footer="MC Estimation of Trend",overlay=line,ovcount=2,ovsamescale,style=fan) 6
# trend025
# trend050
# trend950
# trend975
# trendmean / 1
# lur / 2

[ac_1]

Yes - great - many thanks!

[ac_1]

Further to the above:

(i) The simulated fcast's not within the PI's are not visible using a fan chart (I did say however I would like a plot to have an idea of the area or size of the forecast probability distribution), but they are using the normal graph and listing the fcasts individually for NCount<<a very large number, although I could list them...

(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?

(iii)
Also, I have noticed from an MC simulation:

The PI's (standard calculation, assuming normality)

Code: Select all

set res_M fstart fend = sum/NCount; * sum is calculated from fcast
set res_SE fstart fend = sqrt(res_MSS-(res_M)^2)

set L95 fstart fend = res_M+%invnormal(.025)*res_SE; * 95%
set L80 fstart fend = res_M+%invnormal(.1)*res_SE; * 80%
set U80 fstart fend = res_M+%invnormal(.9)*res_SE; * 80%
set U95 fstart fend = res_M+%invnormal(.975)*res_SE; * 95%

are different to these PI's

Code: Select all

set fcast025  fstart fend = work=%xt(fcast,t),%fractiles(work,||.025||)(1); * 95%
set fcast100  fstart fend = work=%xt(fcast,t),%fractiles(work,||.100||)(1); * 80%
set fcastmean fstart fend = work=%xt(fcast,t),%avg(work)
set fcast900  fstart fend = work=%xt(fcast,t),%fractiles(work,||.900||)(1); * 80%
set fcast975  fstart fend = work=%xt(fcast,t),%fractiles(work,||.975||)(1); * 95%

whilst the averages, res_M and fcastmean are the same.

Which method to use?

For the above calculations, the series is in logs, and I would like to back-transform.

[TomDoan]

Please note that I offered an example. If 90% and 95% bands aren't what you want, you can change the percentile numbers to more extreme values. Or you can add an extra band or two.

Percentiles are based upon the sample simulations; they will usually be fairly close to the Normal CI's if the simulation method actually produces Normally distributed results, but generally won't in the tails because the subsample is going to be fairly small out there. The mean will usually be fairly close if the shocks are mean zero since the law of large numbers will apply there.

[ac_1]

Thanks!

I will use the sample simulation method to calculate PI's for MC, and the standard normal method to calculate PI's for non-simulations (generally speaking, and the former specifically for forecasting non-linear models).

To back-transform from logs, for simulations:
I should not exp the individual fcast and calculate the avg and PI's, it should be as for non-simulations i.e. having generated the forecast (avg) and PI's in logs, then exp them. Correct?

How about?

(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?

[TomDoan]

Percentiles map to the same percentiles in a monotonic transformation. Moments (like mean, standard deviation) don't. You asked about transformations of the mean in a different thread; there are arguments that can be made either way about whether the adjusted transformed mean or the simple transformed mean is somehow more representative of the results.

[ac_1]

Hi Tom,

I want to plot the forecast densities as either a kernel or histogram (conditional PDF's), at each time step.

I have generated NCount fcast's: fcast(1),...,fcast(NCount), but need to gather NCount of them, at each time step, to be able to plot each forecast density for the specific time step.

Probably using %xt to extract the series, gather somehow, and then DENSITY?

thanks,
Amarjit

[ac_1]

I think the following plots a forecast density. But just for the first time step. Correct?

Code: Select all

* plot forecast density at first time-step
make array1 1994:01 1994:01
# fcast
disp array1

set f1 1 NCount = array1(1,t)

density(smoothing=1.5) f1 1 NCount xf ff
scatter(footer="Density Estimate for forecast 1994:01 1994:01",style=line)
# xf ff