Lies, Damned Lie and Evidence Based Policy

Lies by Leo Reynolds via Flickr

Talk to most policy makers – especially those who work directly with politicians – and they will roll their eyes and smirk at the very idea of evidence-based policy making.  Policy, as they well know, is based on politics. And politics is at best a concoction of evidence, opinion, anecdote, ideology, political nouse and whatever the weekend papers are saying. Groucho Marx had it right when he said that “politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies.”

Many of the world’s national statistical offices (NSOs) can count themselves lucky to be kept at arms length from such shenanigans. And I firmly support the idea that a statistical agency needs to be quasi-independent from the executive to ensure that the statistics remain – and appear to remain – impartial.   While such a “we’re just good friends” relationship has much to commend it, it comes at a cost: government statisticians are often naïve in their understanding of the world of the policy maker, and in particular in their understanding of how policy is made.

Ignorance is not bliss however. In a world where policy is based on evidence, the role of an NSO is relatively easy.  But it is much more difficult to have an impact  when evidence is only once voice among many competing for attention. 

In the former world, the statistical agency needs only to produce data. Their data – the evidence – is used rigorously and impartially by statistically-savvy policy makers to analyse and design, almost mechanistically, the policy solutions they require. Politicians are motivated by nothing more than finding the best solution to the correct problem.  How perfect.

In reality, though, the policy makers are incredibly busy, under immense - and often unreasonable - pressure to be seen to be “doing something” and at the beck and call of ministers with at least one eye on the next election. And so they need to be helped, tempted, persuaded, coerced and cajoled into making sure that evidence at least gets a look in when they are working on policy.

Neither statisticians - nor policy makers - may like the reality, but it is important that anyone with an interest in a better world recognizes the evidence about evidence-based policy.  And for statisticians this simple shift – shifting from a world view in which policy is based on evidence, to one in which policy is influenced by evidence – has profound implications. Because an NSO ought to judge its worth by the amount of evidence used in debate and decision making, not by the tonnage of statistical publications released.  And this means putting far more effort into designing the sorts of statistics that policy makers need, releasing them in the right way and at the right time, and then following through to make sure they are used. Although many statistical agencies have become much better at doing this in recent years, much much more could be done.  The budget on outreach in communication in most NSOs is, I’d wager, a fraction of that spent on any of the major collections.

It takes time to change mindsets and world views.  Perhaps we might start by recognizing that “evidence based policy” is to the statistical profession, much what the tooth fairy is to dentistry.

 Jon Hall