How Well Do Structural Demand Models Work? Counterfactual Predictions in School Choice

Working Paper

Parag A. Pathak and Peng Shi

November 2017

In 2013, Boston Public Schools considered several new choice plans that differ in where applicants can
apply. At the request of the mayor and district, we forecast the alternatives’ effects by estimating
discrete choice models. This work led to the adoption of a plan which significantly altered choice
sets for thousands of applicants. Pathak and Shi (2014) update forecasts prior to the policy
change and describe prediction targets involving access, travel, and unassigned students. Here,
we assess how well these ex ante counterfactual predictions compare to actual outcome under the
new choice sets.