Page 1 of 1
graphing a large number of series on a single graph
Posted: Tue May 24, 2022 4:18 am
by ac_1
Hi Tom,
I want to plot a large number of series on a single graph with a supplementary card list. They are as a VECTOR of SERIES fcast. Plotting in the normal manner works fine, however using LIST and CARDS
Code: Select all
comp NCount = 30
list ieqn = 1 to NCount
graph NCount+4
cards fcast(ieqn) fstart fend
# lsales 1968:1 1993:12 1
# res_M fstart fend
# L95 fstart fend 3
# U95 fstart fend 3
results in
## SC7. LIST Needs At Least 31 Entries, Had Only 30
Additionally, is there a way to have all the fcast as black & white patterns, and the supplementary series as colours?
Cheers,
Amarjit
Re: graphing a large number of series on a single graph
Posted: Tue May 24, 2022 12:41 pm
by TomDoan
There is nothing that will let you do a combination of sources for series that way. (BTW, the SERIES and SYMBOLS options are easier to handle for a VECT[SERIES] than LIST and CARDS). You would have to add the supplementary series to the VECT[SERIES].
You can define line colors as black and white by using an RGB "color" that has equal parts of R, G and B. (000000 is black, FFFFFF is white, CCCCCC is a medium gray).
Code: Select all
dec vect[series] x(15)
do i=1,15
set x(i) 1 30 = 2*i+%ran(1.0)
end do i
grparm(define="LINE_COLOR_06=1,000000,1")
grparm(define="LINE_COLOR_07=2,000000,1")
grparm(define="LINE_COLOR_08=3,000000,1")
grparm(define="LINE_COLOR_09=4,000000,1")
grparm(define="LINE_COLOR_10=5,000000,1")
grparm(define="LINE_COLOR_11=6,000000,1")
grparm(define="LINE_COLOR_12=1,888888,1")
grparm(define="LINE_COLOR_13=1,CCCCCC,1")
grparm(define="LINE_COLOR_14=1,444444,1")
grparm(define="LINE_COLOR_15=1,222222,1")
graph(series=x)
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 7:17 am
by ac_1
I have added series: lsales, res_M, L95, U95 to the VECT[SERIES] pfcast.
I would like lsales as black (colour 1), res_M as blue (colour 2), the PI's L95, U95 as green (colour 5) and all fcast as grey's (colour 48).
Code: Select all
grparm(define="LINE_COLOR_01=1,CCCCCC,1"); * grey
grparm(define="LINE_COLOR_31=1,000000,1"); * black
grparm(define="LINE_COLOR_32=1,0000FF,1"); * blue
grparm(define="LINE_COLOR_33=1,00FF00,1"); * green
grparm(define="LINE_COLOR_34=1,00FF00,1"); * green
declare vector[series] pfcast(34)
do j = 1, 30
set pfcast(j) fstart fend = fcast(j)
end do j
set pfcast(31) 1990:1 1993:12 = lsales
set pfcast(32) fstart fend = res_M
set pfcast(33) fstart fend = L80
set pfcast(34) fstart fend = U80
graph(series=pfcast)
NB: fcast is actually large (e.g. 1000), as I am generating SIMULATE fcast's from UFORECAST.
But I cannot get the correct colours for all the series?
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 8:34 am
by TomDoan
There are only 30 styles---by default, when you graph more than 30 series they start to repeat (31 is then the same as 1, 32 as 2, ...). See how the
HISTORY.RPF example uses the SYMBOLS option to control the colors.
Offhand, though, I'm thinking that this will end up being a tangle of lines that will be impossible to read.
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 9:16 am
by ac_1
The idea is to generate a tangle of lines, to see the probability forecast distribution.
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 11:41 am
by TomDoan
You won't see anything with 1000 lines. Given the resolution of any device currently in standard use, you'll just get a blob through about 2.5 standard deviations. This is not like a scatter plot where you represent single pixels on a grid which might have 1000000 intersections so 1000 points will show you clustering. Instead, you have only 1000 or so vertical pixels available at each time series point, and probably half of those are in the sparse regions (the 3+ standard deviations that you'll get with that many simulations). Plus, you get the lines connecting the time series points. Every spot anywhere near the center of the distribution will get covered and it won't matter that the ones closer to the center get covered more than one time, as all that will happen is that the pixel will show the last color written to it.
Note that in the example above where I showed defining black and white styles, I shifted the values of each sample by adding 2*i to series i. If I hadn't done that, it would have been almost impossible to pick out any of the styles.
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 1:49 pm
by ac_1
The question then is if there are several thousand paths generated from SIMULATE, how to plot them to have an idea of the area or size of the forecast probability distribution?
Re: graphing a large number of series on a single graph
Posted: Wed May 25, 2022 3:59 pm
by TomDoan
Probably the best way is with a fan chart. This is an example which does percentiles of a cloud of simulations in the VECT[SERIES] called TREND.
Code: Select all
set trend025 = work=%xt(trends,t),%fractiles(work,||.025||)(1)
set trend050 = work=%xt(trends,t),%fractiles(work,||.050||)(1)
set trendmean = work=%xt(trends,t),%avg(work)
set trend950 = work=%xt(trends,t),%fractiles(work,||.950||)(1)
set trend975 = work=%xt(trends,t),%fractiles(work,||.975||)(1)
*
graph(footer="MC Estimation of Trend",overlay=line,ovcount=2,ovsamescale,style=fan) 6
# trend025
# trend050
# trend950
# trend975
# trendmean / 1
# lur / 2
Re: graphing a large number of series on a single graph
Posted: Thu May 26, 2022 12:52 am
by ac_1
Yes - great - many thanks!
Re: graphing a large number of series on a single graph
Posted: Thu May 26, 2022 9:09 am
by ac_1
Further to the above:
(i) The simulated fcast's not within the PI's are not visible using a fan chart (I did say however I would like a plot to have an idea of the area or size of the forecast probability distribution), but they are using the normal graph and listing the fcasts individually for NCount<<a very large number, although I could list them...
(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?
(iii)
Also, I have noticed from an MC simulation:
The PI's (standard calculation, assuming normality)
Code: Select all
set res_M fstart fend = sum/NCount; * sum is calculated from fcast
set res_SE fstart fend = sqrt(res_MSS-(res_M)^2)
set L95 fstart fend = res_M+%invnormal(.025)*res_SE; * 95%
set L80 fstart fend = res_M+%invnormal(.1)*res_SE; * 80%
set U80 fstart fend = res_M+%invnormal(.9)*res_SE; * 80%
set U95 fstart fend = res_M+%invnormal(.975)*res_SE; * 95%
are different to these PI's
Code: Select all
set fcast025 fstart fend = work=%xt(fcast,t),%fractiles(work,||.025||)(1); * 95%
set fcast100 fstart fend = work=%xt(fcast,t),%fractiles(work,||.100||)(1); * 80%
set fcastmean fstart fend = work=%xt(fcast,t),%avg(work)
set fcast900 fstart fend = work=%xt(fcast,t),%fractiles(work,||.900||)(1); * 80%
set fcast975 fstart fend = work=%xt(fcast,t),%fractiles(work,||.975||)(1); * 95%
whilst the averages, res_M and fcastmean are the same.
Which method to use?
For the above calculations, the series is in logs, and I would like to back-transform.
Re: graphing a large number of series on a single graph
Posted: Thu May 26, 2022 5:15 pm
by TomDoan
Please note that I offered an example. If 90% and 95% bands aren't what you want, you can change the percentile numbers to more extreme values. Or you can add an extra band or two.
Percentiles are based upon the sample simulations; they will usually be fairly close to the Normal CI's if the simulation method actually produces Normally distributed results, but generally won't in the tails because the subsample is going to be fairly small out there. The mean will usually be fairly close if the shocks are mean zero since the law of large numbers will apply there.
Re: graphing a large number of series on a single graph
Posted: Fri May 27, 2022 3:18 am
by ac_1
Thanks!
I will use the sample simulation method to calculate PI's for MC, and the standard normal method to calculate PI's for non-simulations (generally speaking, and the former specifically for forecasting non-linear models).
To back-transform from logs, for simulations:
I should not exp the individual fcast and calculate the avg and PI's, it should be as for non-simulations i.e. having generated the forecast (avg) and PI's in logs, then exp them. Correct?
How about?
ac_1 wrote:
(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?
Re: graphing a large number of series on a single graph
Posted: Fri May 27, 2022 10:36 am
by TomDoan
Percentiles map to the same percentiles in a monotonic transformation. Moments (like mean, standard deviation) don't. You asked about transformations of the mean in a different thread; there are arguments that can be made either way about whether the adjusted transformed mean or the simple transformed mean is somehow more representative of the results.
Re: graphing a large number of series on a single graph
Posted: Sat Jun 11, 2022 1:45 am
by ac_1
Hi Tom,
I want to plot the forecast densities as either a kernel or histogram (conditional PDF's), at each time step.
I have generated NCount fcast's: fcast(1),...,fcast(NCount), but need to gather NCount of them, at each time step, to be able to plot each forecast density for the specific time step.
Probably using %xt to extract the series, gather somehow, and then DENSITY?
thanks,
Amarjit
Re: graphing a large number of series on a single graph
Posted: Tue Jun 14, 2022 7:40 am
by ac_1
I think the following plots a forecast density. But just for the first time step. Correct?
Code: Select all
* plot forecast density at first time-step
make array1 1994:01 1994:01
# fcast
disp array1
set f1 1 NCount = array1(1,t)
density(smoothing=1.5) f1 1 NCount xf ff
scatter(footer="Density Estimate for forecast 1994:01 1994:01",style=line)
# xf ff