British Academy: The UK's National Academy for the Humanities and Social Sciences
Enquiry, Evidence and Facts: An Interdisciplinary Conference
Evidence-based Policy: So, What's Evidence?
Professor Nancy Cartwright
London School of Economics and Political Science, Houghton Street, London, WC2A 2AE
An abstract presented to the conference
‘Enquiry, Evidence and Facts: An Interdisciplinary conference’
at the British Academy, London, on 13 December 2007
Biography
Nancy Cartwright is Professor of Philosophy at the London School of Economics and at the University of California at San Diego. At the beginning of her career at Stanford University she specialized in philosophy of physics and in the join of history and philosophy of science. Since coming to LSE she has focussed her researches in the philosophy of the social and economic sciences. Her special interests are causality, modelling and evidence and one of her central concerns is to use philosophy to inform practice – technology and policy – and the reverse.
Abstract
There is increasing insistence world-wide that scientifically-based evidence play a central role in setting policy, a trend especially strong in the UK and USA. The demand for evidence-based policy has been accompanied by a large number of studies on how to implement this goal and new institutional rules and structures to bring it about. In all this laudable effort one crucial question receives insufficient attention: What is evidence for evidence-based policy? In this paper I aim to make vivid what kind of answer we need and lay the groundwork for supplying it.
There is a great deal of good work already available both in philosophy of science and in the evidence-based policy movement. But the work is on the wrong level to provide advice that is simultaneously reliable, efficient and usable.
Philosophical theories of evidence are not practicable. They tend to be too abstract to serve or they use notions like ‘likelihood ratios’ that presuppose that we already understand the very probabilistic relations between evidence and hypothesis that we are trying to learn.
More practical efforts, like those of SIGNs (used by the UK National Institute for Clinical Excellence to set ‘best practice’), tend to develop evidence-ranking schemes and advice about how to use the schemes. But these are not based on a general and principled theory about what counts as evidence. They tend to start in the middle, taking up issues local to specific methods, often with a pharmaceutical model in mind. The perspective is then too narrow and cannot accommodate vast amounts of genuinely useful and relevant scientific knowledge. For instance the stress on randomised controlled trials focuses on the need to eliminate bias, but this is not an issue for a large variety of other reasonably compelling kinds of evidence; for instance for derivation from theory (as in the support that theory can provide for the concrete design of a laser) or of a game-theory or econometric model. This is primarily because there is no well-articulated procedure of the kind I seek for how to process and apply these different kinds knowledge. The result is wasteful and must lead to decisions that are less reliable than ones based in a sensible way on the total evidence available.
This paper aims, then, to begin to develop a theory of evidence for evidence-based policy that is both practicable and principled. This should include
- A principled and practicable concept of evidence
- A principled and practicable account of what different pieces of evidence say about a hypothesis and with what strength they speak
- A principled and practicable account of how to evaluate a hypothesis in the light of all the candidate evidence.
To make headway with such a theory it will be important to overcome the gap between basic science and use. Scientifically established results do not come with labels: ‘I am relevant to proposals X, Y and Z‘. Indeed much relevant information – especially from indirect data or from abstract theory – will not even be couched in the same vocabulary as the policies under consideration. It is also important to learn how to benefit from the full range of our knowledge. Like SIGNs, most current guides admit a limited amount of the scientific evidence available and then either assign weights to different items or ignore much of what is admitted, following what Gerd Gigerenzer describes as a ‘choose-the-best’ strategy. But throwing away hard-won relevant information seems daft. As Anthony Atkinson, et al argue about EU indicators for social exclusion, “…in the present state of knowledge we cannot afford to neglect any source of knowledge.” This paper aims then for the beginnings of a theory that shows how to manage disparate knowledge, not ignore it.