You Don't Experiment Enough

Error and apathy

The morning of February 5th, 2014 was an unpleasant one for London commuters. The sky was overcast and drizzling, with 7 mm of rain expected, and the Tube was crippled. The Rail Maritime Transport Union had called for a strike, and the staff had largely walked out.

The Tube was not totally shut down, however. The transit workers’ union does not totally command the allegiance of its members when it calls for industrial action, and so only 171 out of 270 stations were closed, determined entirely by who happened to show up to work. It would be possible for many commuters to get to their places of work, but they would have to experiment and try routes they’d never tried before.

It is important you know that the Tube, and London more generally, is difficult to geographically understand. A common sentiment is that everyone, upon moving to the city, knows a few locales around Tube stations, and has not a clue what lies between each of them, or how they relate to each other. It is not uncommon to learn with surprise that trips you might have thought required multiple trains can be avoided with a brief walk. This was before the ubiquity of apps for calculating out the optimal route. The maps of the Tube do not help here — they are highly stylized, and while extremely useful for knowing which station to change trains at, do not allow one to conveniently determine the shortest trip possible to them. Even if you did know the geography, the speeds of the train lines are different from each other.

So the shutdown of the Tube made people experiment who might never have experimented before. Not everyone found a better route — but 5% did. Larcom, Rauch, and Willems (2017) are able to use data on particular transport cards – the Oyster cards – to show how people, having experimented and found a new route, would then use it again and again to commute. As a basic accounting exercise, valuing the time spent traveling as costing half as much as the average hourly wages (£11.60) puts us at an extraordinary welfare gain from this. On average, each person would have to be willing to refuse £49 to ride a new train route once. The authors are cautious — this is a lower bound of the gains, as the new route could have advantages other than the time savings.

This ties into another mystery. Monopolies are less innovative than firms in more competitive markets. This is really weird, because the standard story of why innovation doesn’t happen is that firms will have their ideas stolen by their rivals, and research is not profitable to do in expectation. And we certainly do see this in very competitive markets — if competition is sufficiently cutthroat, nobody is able to pay the fixed costs to do research. And yet, surely as can be ascertained, innovation and competition follow an inverted U.

What’s more, there is considerable evidence that greater competition leads to higher efficiency. Perhaps I understate the case – there is overwhelming evidence, and it isn’t just due to selection. Take, for example, papers on the concrete industry by Backus and Syverson, Peru’s reform of permitting laws, Pavcnik on Chile’s manufacturing plants after trade liberalization, and many others. When firms don’t face much competition, they accept what they have, and don’t continue looking for new ways to maximize their profits.

I think this is a fundamental fact about us. We do not experiment enough. This is best not thought of as just us maximizing under the constraints of costly information, but a profound psychological bias. We evolved a set of heuristics well-suited for peasant life, and when times changed, we did not change with them.

There is something to be said for humans not making errors on average. It is a not unreasonable approximation of reality, and a useful corrective against those who impose their own preferences under the guise of helping others. On the other hand, to make too strong an assumption of rationality is to render the study of human behavior obsolete. We would be learning why things happen, but without any possibility of improving on them. The actions we take, and the cultural memes we spread, are part of humans maximizing too.

So, I think you would be happier if you experimented more. You would likely be better off for taking more risks, trying more new things, and taking fewer things for granted.