Read on Twitter
Check out my new @AnnenbergInst working paper with @lina_anaya, Nagore Iriberri, and @Pedroreybiel just out! https://www.edworkingpapers.com/ai21-392
Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life. However, many design features of the tests themselves may lead to psychological reactions influencing performance
In particular, How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? From easiest to most difficult, from most difficult to easiest, something in between?
We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty.
We find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers
We confirm these results exploiting the random variation of difficulty across test booklets in PISA, for the years 2009, 2012, and 2015, providing additional external validity
We conclude that the order of the difficulty of the questions in tests should be considered carefully, in particular when comparing performance between test-takers who have faced different order of questions
