graphing a large number of series on a single graph

For questions and discussion related to graphs, reports, and other output, including issues related to presenting or publishing results.

graphing a large number of series on a single graph

Unread postby ac_1 » Tue May 24, 2022 4:18 am

Hi Tom,

I want to plot a large number of series on a single graph with a supplementary card list. They are as a VECTOR of SERIES fcast. Plotting in the normal manner works fine, however using LIST and CARDS

Code: Select all
comp NCount = 30
list ieqn = 1 to NCount
graph NCount+4
cards fcast(ieqn) fstart fend
# lsales 1968:1 1993:12 1
# res_M fstart fend
# L95 fstart fend 3
# U95 fstart fend 3

results in

## SC7. LIST Needs At Least 31 Entries, Had Only 30


Additionally, is there a way to have all the fcast as black & white patterns, and the supplementary series as colours?

Cheers,
Amarjit
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Tue May 24, 2022 12:41 pm

There is nothing that will let you do a combination of sources for series that way. (BTW, the SERIES and SYMBOLS options are easier to handle for a VECT[SERIES] than LIST and CARDS). You would have to add the supplementary series to the VECT[SERIES].

You can define line colors as black and white by using an RGB "color" that has equal parts of R, G and B. (000000 is black, FFFFFF is white, CCCCCC is a medium gray).

Code: Select all
dec vect[series] x(15)
do i=1,15
   set x(i) 1 30 = 2*i+%ran(1.0)
end do i
grparm(define="LINE_COLOR_06=1,000000,1")
grparm(define="LINE_COLOR_07=2,000000,1")
grparm(define="LINE_COLOR_08=3,000000,1")
grparm(define="LINE_COLOR_09=4,000000,1")
grparm(define="LINE_COLOR_10=5,000000,1")
grparm(define="LINE_COLOR_11=6,000000,1")
grparm(define="LINE_COLOR_12=1,888888,1")
grparm(define="LINE_COLOR_13=1,CCCCCC,1")
grparm(define="LINE_COLOR_14=1,444444,1")
grparm(define="LINE_COLOR_15=1,222222,1")
graph(series=x)
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Wed May 25, 2022 7:17 am

I have added series: lsales, res_M, L95, U95 to the VECT[SERIES] pfcast.

I would like lsales as black (colour 1), res_M as blue (colour 2), the PI's L95, U95 as green (colour 5) and all fcast as grey's (colour 48).

Code: Select all
grparm(define="LINE_COLOR_01=1,CCCCCC,1"); * grey
grparm(define="LINE_COLOR_31=1,000000,1"); * black
grparm(define="LINE_COLOR_32=1,0000FF,1"); * blue
grparm(define="LINE_COLOR_33=1,00FF00,1"); * green
grparm(define="LINE_COLOR_34=1,00FF00,1"); * green

declare vector[series] pfcast(34)
do j = 1, 30
   set pfcast(j) fstart fend = fcast(j)
end do j
set pfcast(31) 1990:1 1993:12 = lsales
set pfcast(32) fstart fend = res_M
set pfcast(33) fstart fend = L80
set pfcast(34) fstart fend = U80

graph(series=pfcast)

NB: fcast is actually large (e.g. 1000), as I am generating SIMULATE fcast's from UFORECAST.

But I cannot get the correct colours for all the series?
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Wed May 25, 2022 8:34 am

There are only 30 styles---by default, when you graph more than 30 series they start to repeat (31 is then the same as 1, 32 as 2, ...). See how the HISTORY.RPF example uses the SYMBOLS option to control the colors.

Offhand, though, I'm thinking that this will end up being a tangle of lines that will be impossible to read.
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Wed May 25, 2022 9:16 am

The idea is to generate a tangle of lines, to see the probability forecast distribution.
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Wed May 25, 2022 11:41 am

You won't see anything with 1000 lines. Given the resolution of any device currently in standard use, you'll just get a blob through about 2.5 standard deviations. This is not like a scatter plot where you represent single pixels on a grid which might have 1000000 intersections so 1000 points will show you clustering. Instead, you have only 1000 or so vertical pixels available at each time series point, and probably half of those are in the sparse regions (the 3+ standard deviations that you'll get with that many simulations). Plus, you get the lines connecting the time series points. Every spot anywhere near the center of the distribution will get covered and it won't matter that the ones closer to the center get covered more than one time, as all that will happen is that the pixel will show the last color written to it.

Note that in the example above where I showed defining black and white styles, I shifted the values of each sample by adding 2*i to series i. If I hadn't done that, it would have been almost impossible to pick out any of the styles.
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Wed May 25, 2022 1:49 pm

The question then is if there are several thousand paths generated from SIMULATE, how to plot them to have an idea of the area or size of the forecast probability distribution?
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Wed May 25, 2022 3:59 pm

Probably the best way is with a fan chart. This is an example which does percentiles of a cloud of simulations in the VECT[SERIES] called TREND.

Code: Select all
set trend025  = work=%xt(trends,t),%fractiles(work,||.025||)(1)
set trend050  = work=%xt(trends,t),%fractiles(work,||.050||)(1)
set trendmean = work=%xt(trends,t),%avg(work)
set trend950  = work=%xt(trends,t),%fractiles(work,||.950||)(1)
set trend975  = work=%xt(trends,t),%fractiles(work,||.975||)(1)
*
graph(footer="MC Estimation of Trend",overlay=line,ovcount=2,ovsamescale,style=fan) 6
# trend025
# trend050
# trend950
# trend975
# trendmean / 1
# lur / 2
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Thu May 26, 2022 12:52 am

Yes - great - many thanks! :D
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Thu May 26, 2022 9:09 am

Further to the above:

(i) The simulated fcast's not within the PI's are not visible using a fan chart (I did say however I would like a plot to have an idea of the area or size of the forecast probability distribution), but they are using the normal graph and listing the fcasts individually for NCount<<a very large number, although I could list them...

(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?

(iii)
Also, I have noticed from an MC simulation:

The PI's (standard calculation, assuming normality)
Code: Select all
set res_M fstart fend = sum/NCount; * sum is calculated from fcast
set res_SE fstart fend = sqrt(res_MSS-(res_M)^2)

set L95 fstart fend = res_M+%invnormal(.025)*res_SE; * 95%
set L80 fstart fend = res_M+%invnormal(.1)*res_SE; * 80%
set U80 fstart fend = res_M+%invnormal(.9)*res_SE; * 80%
set U95 fstart fend = res_M+%invnormal(.975)*res_SE; * 95%

are different to these PI's
Code: Select all
set fcast025  fstart fend = work=%xt(fcast,t),%fractiles(work,||.025||)(1); * 95%
set fcast100  fstart fend = work=%xt(fcast,t),%fractiles(work,||.100||)(1); * 80%
set fcastmean fstart fend = work=%xt(fcast,t),%avg(work)
set fcast900  fstart fend = work=%xt(fcast,t),%fractiles(work,||.900||)(1); * 80%
set fcast975  fstart fend = work=%xt(fcast,t),%fractiles(work,||.975||)(1); * 95%

whilst the averages, res_M and fcastmean are the same.

Which method to use?

For the above calculations, the series is in logs, and I would like to back-transform.
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Thu May 26, 2022 5:15 pm

Please note that I offered an example. If 90% and 95% bands aren't what you want, you can change the percentile numbers to more extreme values. Or you can add an extra band or two.

Percentiles are based upon the sample simulations; they will usually be fairly close to the Normal CI's if the simulation method actually produces Normally distributed results, but generally won't in the tails because the subsample is going to be fairly small out there. The mean will usually be fairly close if the shocks are mean zero since the law of large numbers will apply there.
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Fri May 27, 2022 3:18 am

Thanks!

I will use the sample simulation method to calculate PI's for MC, and the standard normal method to calculate PI's for non-simulations (generally speaking, and the former specifically for forecasting non-linear models).

To back-transform from logs, for simulations:
I should not exp the individual fcast and calculate the avg and PI's, it should be as for non-simulations i.e. having generated the forecast (avg) and PI's in logs, then exp them. Correct?

How about?

ac_1 wrote:
(ii) If the actual ur had been transformed to logs, lur, exp'ing trendmean would result in median forecasts rather than mean forecasts. Correct? If so why? And therefore how to calculate the mean forecasts after back-transformation?
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby TomDoan » Fri May 27, 2022 10:36 am

Percentiles map to the same percentiles in a monotonic transformation. Moments (like mean, standard deviation) don't. You asked about transformations of the mean in a different thread; there are arguments that can be made either way about whether the adjusted transformed mean or the simple transformed mean is somehow more representative of the results.
TomDoan
 
Posts: 7240
Joined: Wed Nov 01, 2006 5:36 pm

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Sat Jun 11, 2022 1:45 am

Hi Tom,

I want to plot the forecast densities as either a kernel or histogram (conditional PDF's), at each time step.

I have generated NCount fcast's: fcast(1),...,fcast(NCount), but need to gather NCount of them, at each time step, to be able to plot each forecast density for the specific time step.

Probably using %xt to extract the series, gather somehow, and then DENSITY?

thanks,
Amarjit
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK

Re: graphing a large number of series on a single graph

Unread postby ac_1 » Tue Jun 14, 2022 7:40 am

I think the following plots a forecast density. But just for the first time step. Correct?

Code: Select all
* plot forecast density at first time-step
make array1 1994:01 1994:01
# fcast
disp array1

set f1 1 NCount = array1(1,t)

density(smoothing=1.5) f1 1 NCount xf ff
scatter(footer="Density Estimate for forecast 1994:01 1994:01",style=line)
# xf ff
ac_1
 
Posts: 201
Joined: Thu Apr 15, 2010 6:30 am
Location: London, UK


Return to Graphics, Reports, and Other Output

Who is online

Users browsing this forum: No registered users and 2 guests