The Dangers of Putting Our Faith in Statistics


Most people try to avoid thinking about statistics, but they’re part of everything we do in investing. When other people get around to thinking about statistics, they often end up thinking about them the wrong way.

Most people assume the most important numbers of any data set are the average, median, and standard deviation, and they can safely ignore everything else. But it’s not that simple.

Statistics are about trying to figure out the unknown based on what we have seen in the past. It’s all about working with uncertainty. We can’t know what the true center of the stock market’s distribution of returns is, but we can make an estimate.

With investing, we always need to be careful about our level of certainty, even when the statistics seem rock solid. The financial markets are noisy places, and we need to appreciate that.

One of my favorite examples of this is to ask someone which they would expect to be larger next year – the size or value premium for US stocks (admittedly, there’s a pretty small group you can do this with). This seems like it should be really easy – from 1927 to 2016, the average annual size premium was 2.80 percent, and the average annual value premium was 4.98 percent*. Yes, they both have pretty big standard deviations, but look at the difference in the average – it’s pretty clear what the answer is, right?

Not so fast – that variance matters. We actually can’t really tell the difference between the two premiums. There’s no statistically significant difference between these two averages. To be precise, the p-value (a measure of how confident we are in a statistical result) for the difference in the annual average returns was 0.25.

In other words, there was a 1 in 4 chance that this difference is purely the result of random noise. To be considered statistically significant, we want there to be less than a 1 in 20 chance that it’s random.

This matters from an investment standpoint because it will affect how you divvy up your risk budget and what your asset allocation looks like. But the bigger and more important point is that you constantly need to be thinking in terms of uncertainty.

We talk a lot about thinking through risk, but it’s important to recognize that everything – including our descriptive statistics – is uncertain.

So what does this mean for you?

It’s one thing to play with numbers and point out interesting relationships. It’s another to actually do something useful with them.

But there’s a big takeaway from this uncertainty: we should be incredibly wary of overconfidence.

Overconfidence is one of the most destructive behavioral biases investors have. We all think we are better than we are – statistically speaking, almost everyone reading this article thinks they are a better than average driver.

In fact, there’s really no such thing as under-confidence. Even people who self-identify as pessimistic are still overconfident, just by slightly less than the average person.

Even when we are primed to think about the overconfidence of investors – or ourselves, specifically –pretty much everyone still acts overconfident. I am overconfident. You are overconfident. Your boss is really overconfident (and that goes double if you’re your own boss).

My mother-in-law used to call this America’s superpower. We just assume we know how to do everything – or at least we’ll figure it out as we go along. But overconfidence is universal – it may vary a little in degree across cultures, but it doesn’t go away.

So we need to be hypervigilant about minimizing the effects of overconfidence. Simply relying on a passive investment philosophy doesn’t cut it. It’s certainly better than trying to beat the markets, but it doesn’t eliminate the dangers of overconfidence, it just changes what we are overconfident about.

The biggest area where passive investors are overconfident (we’ll ignore just how overconfident this statement probably is) is probably our assessment of our own risk tolerance – especially men. We think we understand the risk and return relationships of the market (and we’ve already seen that’s not the case), so we think we can tough out those periods when the markets turn against us.

This is better than trying to outguess where the markets are going, but it’s still not good. Being overconfident about the level of investment risk that you can tolerate just straps you right back into the fear and greed roller coaster that we’re all trying to get off of.

Investing is about managing risk. And a huge part of doing this well is being humble. Humility allows us to recognize that however smart we are (or think we are), the markets are smarter.

Statistics allow us to understand certain relationships, and we become pretty comfortable making certain statements about why things happen. But we still regularly have to throw up our hands and say the markets are just weird sometimes.

It’s really easy to overextend and think we understand everything that is happening – especially when we have something as seemingly reliable as statistics on our side – but behavioral biases like overconfidence can prevent you from reaching the retirement you’ve always wanted.

*Information provided by Dimensional Fund Advisors LP. In US dollars. US size premium: Dimensional US Small Cap Index minus S&P 500 Index. US relative price premium: Fama/French US Value Research Index minus Fama/French US Growth Research Index. Indices are not available for direct investment. Their performance does not reflect the expenses associated with the management of an actual portfolio. Past performance is not a guarantee of future results. Index returns are not representative of actual portfolios and do not reflect costs and fees associated with an actual investment. Actual returns may be lower. 

McLean Asset Management Corporation (MAMC) is a SEC registered investment adviser. The content of this publication reflects the views of McLean Asset Management Corporation (MAMC) and sources deemed by MAMC to be reliable. There are many different interpretations of investment statistics and many different ideas about how to best use them. Past performance is not indicative of future performance. The information provided is for educational purposes only and does not constitute an offer to sell or a solicitation of an offer to buy or sell securities. There are no warranties, expressed or implied, as to accuracy, completeness, or results obtained from any information on this presentation. Indexes are not available for direct investment. All investments involve risk.

The information throughout this presentation, whether stock quotes, charts, articles, or any other statements regarding market or other financial information, is obtained from sources which we, and our suppliers believe to be reliable, but we do not warrant or guarantee the timeliness or accuracy of this information. Neither our information providers nor we shall be liable for any errors or inaccuracies, regardless of cause, or the lack of timeliness of, or for any delay or interruption in the transmission there of to the user. MAMC only transacts business in states where it is properly registered, or excluded or exempted from registration requirements. It does not provide tax, legal, or accounting advice. The information contained in this presentation does not take into account your particular investment objectives, financial situation, or needs, and you should, in considering this material, discuss your individual circumstances with professionals in those areas before making any decisions.

Bob French, CFA