Autocorrelation. I guess I should throw in autoregression as well (special thanks to Abbie).
That is all you need to know….
But if you do want to know more, then I will proceed.
When academia say that the market, be it the Dow Jones Industrial Average or the S&P 500, is simply a “random walk”, what they mean is that the price of those indexes just wanders around seemingly aimlessly. That is, the price yesterday is highly correlated with the price today. Going further, the price yesterday has a great influence on today’s price. Hence the word autocorrelation – when what happens at time t is correlated with time t -1.
What does this look like? Below is a plot of the price of the S&P 500 since January 3, 1950 until December 16, 2015. That is 16,597 trading days! I believe any professor would say that is a large enough sample size…
What does this chart say? It may be hard to see, but it shows how close the price today is from yesterday’s price. However, let’s plot Y versus Y t -1 (Price today versus Price yesterday). This may be more helpful to visualize the correlation.
Wow…I do not even need to plot a regression line. The dots create one themselves!
Now, what does “autocorrelation” look like? Below is a plot:
If you look carefully there are two parallel blue dotted lines. When the black mass is above or below the dotted lines then there is statistical significance for the acf (autocorrelation) values. This means that Y (Price today) and the lag of Y (be it, yesterday’s price, last week’s price, last month’s price, etc.) are pretty correlated. You can see that the black is above the blue dotted line almost until Day 5000! Then it instantly drops below the bottom dotted blue line. We care about the absolute value so positive or negative makes no difference, and it’s a little beyond the scope of this article.
But let’s delve into it anyway! Day 5000 is about 20 years of trading, as there are around 250 trading days in a year (so 5000 / 250 = 20). Therefore, the price now is so large relative to the price 5000 (20 years) trading days ago that it would exhibit be negative correlation. Or perhaps another way to think about it is. You have two prices, say price 100 and price 6000. If you move forward to price 99 and backward to price 6001 then you would expect these prices to move inversely, meaning you would think price 99 would increase relative to price 100 (and price 6000 and price 6001 for that matter) and you would think price 6001 would decrease relative to the aforementioned prices. That is my best shot. Someone who is smarter than I am can confirm or deny this last paragraph.
If we create an autoregressive model, we can now intuitively estimate that the coefficient of Y t-1 (price of yesterday) will be extremely close to 1. This is because in a “random walk” acf values stay above/below the dotted lines for a long time. If the coefficient is below 1, then Y (price) will be pulled back to its mean and the autocorrelation will drop off rather quickly (fall within the two dotted lines). And if the coefficient is above 1, then the apocalypse occurs. So what is the coefficient (autoregressive term) of yesterday’s price??
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.0655848 0.0827029 0.793 0.428
yesterdaysprice 1.0001207 0.0001123 8905.497 <2e-16 ***
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 8.039 on 16594 degrees of freedom
Multiple R-squared: 0.9998, Adjusted R-squared: 0.9998
F-statistic: 7.931e+07 on 1 and 16594 DF, p-value: < 2.2e-16
And there you have it. The coefficient for yesterday’s price (the X variable in this equation) is exactly 1. All of the “statistical” measures hold up (i.e. the t value is greater than 2, p value is less than .05). Now you may have heard the phrase, random walk…with a drift. This means that the intercept, B0, is not equal to 0. However, from this data we cannot be sure.
Now when you are at your next cocktail party and your neighbor uses the phrase random walk when speaking about investments, you can keep him or her honest! Your welcome.
As always, this if for informational and educational purposes. Nothing contained herein constitutes tax, legal, insurance or investment advice, or the recommendation of or an offer to sell, or the solicitation of an offer to buy or invest in any investment product, vehicle, service or instrument. What has worked in the past may not work in the future. Past returns do not guarantee future results.