Making big decisions without much information to go on
Quick game: I have just invented a device that will show me a “face” or a “tree” at random every time I use it. I’m going to use it 16 times, and you’re going to guess the probability that the device will show a “face” the next time I use it.
Ok, here are my results:
1. Tree
2. Face
3. Face
4. Face
5. Face
6. Face
7. Face
8. Face
9. Face
10. Tree
11. Face
12. Face
13. Face
14. Face
15. Face
16. Face
So in total, the device showed 14 faces and 2 trees. Ok, time to decide. Do you think it’s going to show a face or a tree next time? What are the chances?
If you’re like most people, you might have said “a face”! And maybe you would’ve put the probability at 88% or so.
That is, until I told you I was just flipping a quarter from Connecticut.
But now, you might make the opposite guess. The probability of a quarter landing heads or tails is 50%, and Pop just flipped a bunch of heads. That means a string of tails must be coming up, right?
Except it doesn’t mean that. Sure, after thousands of flips, you’ll start to see the balance of heads and tails get closer to 50%, but that next flip has the same 50% of landing on heads as the last flip had.
Thinking it through, we know this. But our “gut” tries to take a small sample size and turn it into a “trend”. With the quarter, it’s just fun and games, but where does this tendency threaten our money?
Starting a business
Around half of small businesses fail. Well, so says the governor of Texas, quoting the Small Business Administration. I can’t find that actual stat on the SBA website—it might be apocryphal. But anyway…
Start-ups are risky. And yet, entrepreneurs take the risk anyway. Is it because they tolerate risk better or because they’re not aware of the risk?
A few researchers from Georgia State University and Oakland University tried to figure that out by having 191 MBA students read a Harvard Business School case study and decide whether or not the subject of the study should quit his job and pursue a theoretical venture.
Granted, these students were “faking” it. They didn’t have to actually feel the pit in their stomach before making the decision to jump or not to jump. Somewhat hilariously, the proposed venture was for contact lenses for chickens. Better eyesight made the chickens fight less, which had financial implications for farmers.
Anyway, the students were asked to pick three important facets of the case that led them to suggest starting or abandoning the venture and explain why. If a student gave a reason that suggested he or she believed in the law of small numbers…that is, that he relied on feedback from a couple customers in the case in making a decision, they marked that down.
It turned out that the students who let small sample sizes guide their decisions were much more prone not to perceive the high risks associated with the venture.
Meta question of the post: Are 191 MBA students with a median age of 28 representative of the risk-attitudes of the entrepreneurial population? Eh.
Picking a mutual fund manager
Pick index funds. Low fees. Blah blah blah…but how do investors actually go about picking a fund? As you might have guessed, they don’t go through the empirical methods they think they do, and it’s true even among “sophisticated” investors.
Take hedge funds, for example. To invest in a hedge fund, you need a minimum net worth of $1,000,000 or an income of $200,000. So, these guys are supposed to know their way around a bank account.
But a couple researchers found that they chase returns like everybody else. Even though funds that perform well do tend to outperform funds that perform poorly (in their sample), investors piled even more money into the funds after a “winning streak” than predicted.
That less-sophisticated investors also chase returns is well documented.
But it’s not all strawberries and cream for index investors. Even our basic assumptions about stock returns are based on a tiny amount of market data.
We have 200 or so years of somewhat reliable market data upon which to estimate how well stocks perform over long periods of time. And let’s say you wanted to show how they tend to perform over 20-year periods, since you’re planning to hold onto your index fund until you retire.
That gives you only ten non-overlapping sets of data upon which to base your retirement decisions. It’s like wagering your life savings on info from the first 16 flips of a face-tree device. Maybe they were representative of the actual probability of stocks performing well in the future, but we don’t really know yet.
It’s going to be impossible to root out “small numbers” from every decision you make in life. I’m still going to go to restaurants and movies that a small sample of friends suggest.
But when it comes to making big decisions, take a little extra time to find as much data as you can, and if the data doesn’t exist, at least be aware that you don’t really know how things might turn out.
{ 5 comments… read them below or add one }
I agree with the point being made in the article, Pop. But I don’t agree that the only response is to find more data or to go ahead and make decisions in which you understand you cannot realistically possess much confidence (because there isn’t enough data).
What I try to do is to think through what makes sense and then see whether the data backs up the idea that should work according to the dictates of common sense. I question severely data that is not in accord with common sense. I question data that is in accord with common sense too, but not nearly as severely. I place much more confidence in data that is in accord with common sense.
You’re right that we just don’t have enough data to answer every question about investing that we would like to have answered. But we cannot take a pass. It’s not like we can say “Oh, I just won’t invest since there isn’t enough data.” There’s no neutral ground in this game.
Since you have to go with something, I think it makes sense to go with common sense. But I still like to check the data to make sure that my common-sense filter is functioning properly. If the data says that your common-sense impressions have never flown, maybe there is something counter-intuitive going on that you need to figure out.
I don’t go along with the people who look only at data and I don’t go along with the people who ignore data. I believe that we should give it weight but that all data-based insights need to be delivered with caveats for just the reasons you point out in this post.
Rob
I have always felt that the quarter flipping example like you just gave is kind of busted… (Assuming it is truly a 50/50 random chance) Yes we recognize that it is a 50/50 chance for the next flip, and past performances do not effect future performances… but like you said, after thousands of flips, you’ll start to see the balance of heads and tails get closer to 50%. So why wouldn’t you pick tree for the rest of the flips until the balance gets to 50% then you would have been correct for +12 flips and 50/50 on the rest.
I mean.. if you HAVE to pick heads or tails… it seems like why not pick the one that has the most potential for long term successes?
@Bryan – Thanks for the comment.
Let’s say I said I was going to flip a coin 50 times, and the first 16 flips ended up how I described above. Now, I ask you to guess what the ratio of heads to trees is going to be when all 50 flips are completed. The answer shouldn’t still be 25 heads and 25 trees. Now your guess should be something like 31/19.
Since the last 34 flips aren’t affected by the first 16, there’s no reason to think the quarter will “correct” the imbalance you started with. The same imbalance is even going to be there even when you have 10,000 or a billion flips. It just won’t look as big. At this point in the game, if you told me that you were going to flip a trillion more times, I would still have to guess that at the end of the trillion flips, heads would have a +12 edge. So you don’t get that +12 benefit you reference by guessing tree.
Does that make sense? Sorry for not explaining it well.
It seems to me that the price of a company’s stock depends a lot more on whether it is favored by the stock market rather than it being a sound company. This can make investing in it risky if management tries to make the stock price rise rather than growing the business.
@Pop
It does make sense… especially the bit about having 1 billion flips and the imbalance just not looking quite so big.
But if you had to pick heads or trees, I would still pick trees if the assumptions were really true (that it is 50/50) because either choice would be fine, might as well pick what makes me happy. If the assumptions were maybe not so great (for example, the probability of getting 2/16 is ~.2%, the probability of getting 8/16 is ~20%… thus the 2/16 result is 2 orders of magnitude lower than expected… is this difference significant enough to conclude that the model/experiment is not correct?) I would pick heads because perhaps this experiment isn’t really fair.
I’ve enjoyed reading the blog. Keep up the good work.