Why So Many Forecasts Are Useless: Ritholtz
June 27, 2014 by Barry Ritholtz
If you work in finance, you will invariably come across an example of single-variable analysis. Almost daily, we see terrible examples of this sort of analytic error, rife with logical weakness, yet offered with the highest degree of certainty.
The way this works is as follows: Some ominous data point will be shown, along with a chart that foretells some sort of outcome, often horrific. It usually is accompanied by a chart or table, along with text that warns “IF X OCCURS, THEN Y MUST FOLLOW.” Welcome to the mathematically ignorant, conceptually foolish, money-losing world of single-variable analysis.A closer look reveals this to be the worst sort of intellectual foppery.
The behavior of markets or economies simply can’t be explained by looking at just one thing. If single-variable analysis were remotely possible, the world’s forecasters would have a much, much better track record. They could simply pick the data point with the highest correlation to the target variable, and make much better predictions than we currently see. (Earnings versus Standard & Poor’s 500 Index performance, gross domestic product versus economic activity, interest rates versus bond returns.) Even a moderately accurate set of directional predictions should be easy.
But that doesn’t happen. Economies and markets are extremely complex. They have a variety of different inputs, including earnings, interest rates, psychology, economic activity, fund flows, taxes, sentiment, momentum, geopolitics, etc. The relevant significance of all of these inputs varies over time. There are periods when fund flow doesn’t seem to matter, like most of the past five years. Sometimes interest rate increases are really important to equity prices, other times they aren’t. Indeed, markets seem to embrace the idea of rotating memes. These different forces have different weights at different times.
In a lot of ways, markets exhibit behaviors quite similar to those predicted by chaos theory — dynamic, nonlinear, sensitive to initial conditions, etc. (For those who may be interested, James Gleick’s “Chaos: Making a New Science,” is a marvelous primer on the subject.). Some years ago, I noted that “quantum physics aside, I think we can all agree that the market is a terrifically complex mechanism for digesting an unholy spectrum of all too many data points. Merely controlling for one single variable over time is a surefire long-term money loser.”
What are a few examples of the single factors that have been making the rounds these days?
- GDP: “We have never had a negative 2.96 percent GDP report and not gone into recession…”
- Rising Rates: “The U.S. stock market doesn’t do well when interest rates are rising.”
- Earnings Surprises: “Earnings are good this quarter, better than expected, and therefore, the market’s going higher.”
- New Financial Products: “These new products are being adopted, therefore it means the bull market is coming to its peak.”
- Death Cross/Golden Cross: “When the 50 and 200 day moving average cross to the upside (downside), it bodes well (poorly) for any trading vehicle.”
You probably have read one or more of these things recently. They all share the same analytical flaw: Predicting the future actions of a complex system by tracking a single variable. Even when they end up identifying a correlation, we don’t know if it is anything more than a random coincidence.
Let’s just take one example of how foolish it is to reach a definitive conclusion based on a single data input. Look at the chart below: It shows a powerful relationship between per-capita consumption of sour cream with U.S. motorcycle riders killed in noncollision transport accidents. Correlation: 0.916391.
Source: Tyler Vigen
Given that Wall Street’s computing horsepower rivals that of NASA, if that single variable existed, we have to believe it would have already been found, and whatever informational advantages it offered would have been whittled away already.
This is before we even get to George Soros’s theory of reflexivity. Stated simply, the interaction in any complex system of unfolding events and participants’ reactions to those events creates a dynamic disequilibrium that leads to unpredictable, unexpected future results. In other words, our collective reaction to ever-changing data points ruins any predictability that might have otherwise come about from that data.
Theory aside, real world observations lead me to conclude that investors, analysts and economists aren’t going to successfully forecast a stock market or economy by simply looking at a single variable
