ROLLINGCAUSALITY.RPF—Rolling sample Granger causality tests

[TomDoan]

ROLLINGCAUSALITY.RPF performs a "rolling sample" Granger causality test. While there are examples of this in the literature, it would probably be best described as statistically suspect—for instance, in this case, there is no causality (by construction), yet one could easily conclude that the "causality" has changed substantially across the sample, when it's just sampling error.

Detailed description

[bok1234]

Dear Mr.Doan,

This is the code for what I want to test. Thank you very much. But this code makes me more curious.
Let me ask you 2 questions about this. Please see the attached graph file.

Question 1.
As you can see, some parts of the graph (2 circles in the initial part) have significant F values, which you referred to 'changes in causality'. In testing with real data, unlike your virtual and spurious causality example, this kind of result - that is to say, some parts are significant, but another parts are not - could be given to us. In this case, should we judge whether there is causality or not 'by the number of tips (F values over critical value) of entire iceberg (total number of F values)?

Question 2.
As in usual full sample Granger causality test, I think that the rolling causality shows only the causality direction. So, if I want to know how much varible X influence variable Y, I need to test simple regression, VAR, etc., right?

[TomDoan]

I think you're missing the point. There is no causality at all in this example---the two series are generated independently. What you observe in the graph is all sampling error in the F statistic. The problem with most "rolling sample" analyses is that they treat the estimates like they're data, and interpret them without regard to how they're affected by sampling error.

[bok1234]

Your explanation sounds like that rolling regression has serious problem in itself as a econometric tool. So, your opinion is that I'd rather do usual Granger tests with total data, not with rolling (divisional) sample. Right?

[TomDoan]

There's nothing wrong with testing for stability in relationships. The problem is that rolling sample estimates are being done for no obvious reason.

[bok1234]

I see. Let' suppose that 2 variables, for example, inflation expectations and real inflation rates. We know that they have relationship theoretically and empirically. I want to analyze recent relationship changes between them. In this case can I think that 'Granger test with total periods' is main test and Rolling Causality is supplementary test? (Of course, I know that ROLLINGCAUSALITY.RPF generates both results at the same time). More specifically, considering your opinion, the result of Rolling Causality has meaning only if economic theory support relationship between variables and 'Granger test with total periods' confirms significant causal relationship. In this case we can accept causal relationship and see time varying relationship through Rolling Causality test. If we find no relationship through 'Granger test with total periods', then the result of Rolling Causality test might be meaningless. Please let me know whether my understanding is right or not.

[TomDoan]

No. I could have set up that example with a stable VAR that exhibits Granger causality and the GC tests could be very different on one end of the data set from the other. They might all be significant, but they could look quite different in how strong the rejection of non-causality is---EVEN THOUGH THERE IS NO ACTUAL DIFFERENCE.

[bok1234]

Dear Mr.Doan,
My last words were question based on my own new example (inflation expectations and real inflation rate), not your virtual model in ROLLINGCAUSALITY.RPF. I doubt that you are still talking about only your example. And what is the meaning of 'end' in the below sentence. It means that causal direction, such as 'X -> Y'? or initial window in rollingcausality.rpf?

" ...and the GC tests could be very different on one end of the data set from the other..."

[TomDoan]

Just generally. If you do any rolling sample calculation, it will change from window to window regardless of whether there is any change in the underlying process and can, in fact, change quite a bit from one part of the sample to another, again, even if there is no actual change in the process. That's the problem with pretty much this whole literature---authors interpret differences as being meaningful when they have done nothing to show that they really are.

[bok1234]

OK! I could summarize your tonight teaching as below.
1. Rolling regression output can shows whether their relationship are stable or not.
2. Each window generates different F values and some windows' F values comes in significant confidence interval. BUT THIS DOES NOT MEAN ALWAYS THAT THERE ARE SIGNIFICANT CHANGES IN THE PROCESS. So, in case of YOUR example, we cannot judge that the 2 circles in the graphs means true changes.

Please check my summary.
Let me ask you one more thing. Then, how should I interprete the graph of your example in the aspect of 'stability in relationships'? The number of tips (F values over critical value) of entire iceberg (total number of F values) are the key indicators? The more the number of tips, the less stable the relationship between X and Y

[TomDoan]

OK! I could summarize your tonight teaching as below.
1. Rolling regression output can shows whether their relationship are stable or not.
2. Each window generates different F values and some windows' F values comes in significant confidence interval. BUT THIS DOES NOT MEAN ALWAYS THAT THERE ARE SIGNIFICANT CHANGES IN THE PROCESS. So, in case of YOUR example, we cannot judge that the 2 circles in the graphs means true changes.

Please check my summary.
Let me ask you one more thing. Then, how should I interprete the graph of your example in the aspect of 'stability in relationships'? The number of tips (F values over critical value) of entire iceberg (total number of F values) are the key indicators? The more the number of tips, the less stable the relationship between X and Y

No. No. No. Again, please remember that in that example, the true value at every single entry is zero. Everything that you see in that graph, every peak, every valley, every zig, every zag is solely due to the effect of running the regression on a particular subsample. If you drill into the data you find that there are a few places where a big value of Y (randomly) precedes a big value of X---when a moving window includes those, it generates a high Y to X "causality" test, and when a window moves off of those, the causality test value drops (sometimes rather quickly). Even though there really is no actual difference.

[bok1234]

Then, your words seems to mean that the line in the example graph does not mean anything. If this is true, your example just warns that we should not misjudge the spurious causality relationship produced by the rolling causality test. Then, what is the purpose of doing rolling causality test? You said that "There's nothing wrong with testing for stability in relationships". So, I asked before how can I use the rolling causality test to judge whether the relationship between 2 variables are stable or not. Your ROLLINGCAUSALITY.RPF plays just a role for showing spurious causality example and does nothing else?

[TomDoan]

There is a literature on rolling causality tests. I've yet to see a published paper on it that (in my opinion) is proper statistical practice. Sometimes bad statistical practice makes it into the literature. We wrote this program to show people how to do it if they must, but to warn why it's a bad idea.

There are many different tests for stability---fluctuation tests, A-P tests, CUSUM tests, etc. All of those produce graphs that at least superficially look like the "rolling causality test" graphs. However, those have a specific global null hypothesis and the graphs for different time periods have a specific meaning relative to that. It's not clear what any changes in rolling causality test graphs actually mean.

[bok1234]

Finally, I think that I got to understand your explanation. When I got introduced ROLLINGCAUSALITY.RPF from you, I was happy because the code could be used for my test. But I was wrong. I misunderstood the real reason of your code introduction. After exchanging many Q&A with you, I decided to do traditional style - not rolling but using entire data at each test - Granger causality test to verify causal relationship between inflation expectations and real inflation rate. By the way, the Granger causality test, I think, shows only the causality direction. So, if I want to know how much varible X influence variable Y, I need to test simple regression, VAR, etc., right? Or please introduce me another appropriate test methods for my case besides traditional GC test. I think that you could do it because you already know my purpose of test, the model, and data.

[TomDoan]

No. The percentage reduction in forecast error in the causality test does have some meaning for the strength of the dynamic relationship. Now, the F statistic is a largely based upon that, but it judges statistical rather than economic significance---a 10% reduction in forecast error in a small data set may be statistically insignificant while a 2% reduction in a large one may statistically significant.