Do the Methods We Use Predict the Things We Care About?

A test of merger simulations

Someone has finally gone and written the paper I wanted more than anything. In industrial organization, one of the central questions is what would happen if a merger happens. This is not something which you can obtain with simple methods – finding the precise impact will require understanding the entire demand system for a set of goods. So, we have developed this vast theoretical apparatus to impose just enough structure to make reasonable predictions about the future.

Estimating these models takes a lot of time and work. For that reason, nobody has compared the predictions of the model to the ground truth of what happened after the merger was consummated at scale. What merger reviews we have had have been of only a few case studies at a time. So when I earlier considered writing essentially this same article, I found that there was really nothing to go off of – the new methods must surely be better, but nobody has evaluated at scale. What evaluations did exist were of a few case studies, too few for any serious statistical analysis to be run.

Until now, that is.