2503 | Measuring the unmeasurable? Systematic evidence on scale transformations in subjective survey data

Caspar Kaiser and Anthony Lepinteur

Economists routinely use survey measures of, for example, risk preferences, trust, political attitudes, or wellbeing. The literature generally treats numerical response categories as if they represent equal psychological intervals. We provide the first systematic test of this linearity assumption, developing a general framework to quantify how easily results can be overturned when it is relaxed. Using original experimental data, we show that respondents interpret survey scales in ways that do deviate from linearity, but only mildly. Focusing on wellbeing research, we then replicate 30,000+ coefficient estimates across more than 80 papers published in top economics journals. Replicated coefficient signs are remarkably robust to mild departures from linear scale use. However, statistical inference and estimates of relative effect magnitudes become unreliable, even under modest departures from linearity. This is especially problematic for policy applications. We show that these results generalise to many other widely used survey-based constructs.