Oh boy. In the last post, I discussed the importance of replicating research as a mechanism in the scientific process. Replication is especially important when the original research informs policy decisions and influences political actors. Well, it looks like a seminal paper (RandR) in economics by Reinhard-Rogoff (followed by their book (“This Time is Different: Eight Centuries of Financial Folly“) contains some outcome-changing errors.
You might be thinking…well, so what? Some nerdy economics paper turns out to be off the mark. Well…the problem here is – when it comes to debating economic policies for countries (think U.S. and E.U.) – the paper in question is the most influential in the aftermath of the global economic crisis. Since publication, it has been cited about 450 times in related work. (see here for a fuller explanation of the issues with the R&R work). The three main issues that distorted R&R’s results were: selective exclusions, unconventional weighting, and a coding error. The coding error is really quite unbelievable. Basically, it’s a spreadsheet error. Therefore, the key research that provided the basis for global moves toward austerity turns out to be partly based on someone messing up in Excel.
What’s interesting is how theories in social sciences go from paper to book to influencing policy with very little attempt to replicate the original results. Obviously, not the first time this has happened. Still, scholars need to wake up and smell the coffee.