Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis

Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, the researchers show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools.