British Academy: The UK's National Academy for the Humanities and Social Sciences
< SEE THE SHORT VERSION OF THIS ABSTRACT
Enquiry, Evidence and Facts: An Interdisciplinary Conference
Causal Models in Evidential Reasoning
Dr David Lagnado
Department of Pychology, University College London
Gower Street, London, WC1E 6BT
An abstract presented to the conference
‘Enquiry, Evidence and Facts: An Interdisciplinary conference’
at the British Academy, London, on 14 December 2007
Biography
David Lagnado is a lecturer in cognitive and decision sciences in the department of psychology at UCL. He also an ELSE Research Fellow at UCL, and was a post-doctoral researcher on the Evidence project. He has a PhD in philosophy, and an MSc in cognitive science. His main research is in human learning and inference, with particular focus on models of causal and probabilistic reasoning. His work on the Evidence project includes studies on how people use evidence to make probabilistic inferences, and how this fits with normative models of inference. Of particular interest is the role of causal knowledge: how do people acquire it, and how do they use it for prediction and explanation? He is the co-author of a recent book on the psychology of decision making: ‘Straight choices: the psychology of decision making’, 2007, Psychology press.
Abstract
How do people reason about legal evidence? What kinds of representations do they use, and how do they draw inferences from complex bodies of evidence? For example, when judging the guilt of a suspect, how do jurors assess and combine different items of evidence such as witness testimonies, alibis, confessions and forensic evidence? This task is challenging because people often need to consider a wide range of evidence, as well as complex interrelations between items of evidence. Existing psychological models of legal reasoning are either too restrictive or too vaguely specified. Thus on the belief-adjustment model (Hogarth & Einhorn, 1992) people are supposed to sequentially integrate evidence in line with a weighted additive rule. This model is well-defined but cannot accommodate interrelations between items of evidence. In contrast, the story model (Pennington & Hastie, 1992) maintains that people make holistic judgments on the basis of causal relations between items of the evidence. However, this model is formulated at a descriptive level, and does not formalize the representational or inference processes involved.
This paper reports several experimental studies that explore how mock jurors integrate different kinds of evidence. It also develops a novel psychological model that takes into account both people’s inferential capabilities and their limitations. This model is inspired by recent work in statistics on the formal modeling of evidence, in particular the use of Bayesian networks in legal contexts (Baio & Corradi, 2007; Dawid & Evett, 1997; Hepler et al., 2007; Taroni et al., 2006). Bayesian networks are a formal tool for representing the probabilistic relations between variables (e.g., hypotheses and items of evidence), and enable inferences to be drawn about unknown hypotheses on the basis of known observations or evidence. In particular, they can model the complex interrelations between different items of evidence that are typical in legal cases. Fully-fledged Bayesian networks are unlikely to provide a suitable model of human reasoning. The specification of precise conditional probabilities, and the complex quantitative computations required to draw inferences, seem beyond the capabilities of human reasoners (especially with numerous variables). Nevertheless, this does not rule out the applicability of key qualitative aspects of the Bayesian network formalism, in particular, the relations of relevance and causal dependency that are critical to legal analyses of evidence (Allen, 2004; Schum, 2001).
We propose that people represent and reason about legal evidence using small-scale qualitative networks. These focus on the qualitative relations between hypotheses and evidence statements, and require only comparative judgments of relevance and probability (cf. Wellman & Henrion, 1993). In addition, these networks are small-scale: only a limited number of variables can be entertained and reasoned over at any one time. This fits with well-established capacity limits on working memory (Cowan, 2001; Miller, 1956), and recent research showing that these are closely related to limits on reasoning (Halford et al., 2007; Juslin et al., 2007). This model serves as a guiding hypothesis for a range of experimental studies.
1) Discredited evidence
This set of studies (Lagnado & Harvey, 2007) investigates how people revise their beliefs when they discover that evidence is discredited; in particular, how the discredit of one item of evidence affects other items. For example, when the testimony of one witness is undermined, does this have a knock-on effect on other testimonies, or even unrelated forensic evidence? The discrediting of evidence involves a distinctive pattern of inference termed ‘explaining away’ (Pearl, 1988), which is naturally handled within the Bayesian network framework (but not so readily captured within most psychological models of inference). Thus these studies also seek to establish if people can ‘explain away’ discredited evidence in an appropriate manner.
These questions were explored by presenting mock jurors with simplified criminal cases, and asking them to judge the probability that a suspect was guilty on the basis of sequentially presented evidence. The main findings were that people explained away discredited evidence, and extended this discredit to other items of evidence. However, this extension was sensitive to the order in which information was presented. When the discredit was presented late, it was generalized from one item to another irrespective of the causal relations between these items. For example, when people discovered that a witness testimony was discredited they extended this not only to other statements by that witness, but also to unrelated forensic evidence. In contrast, when the discredit was presented early, it was only extended to causally related evidence.
This pattern of results is difficult to explain on current psychological models of belief revision. However, it can be explained on our proposal that people use limited-capacity networks and chunk information when it exceeds capacity constraints (Halford et al., 2007). In particular, the difference between early and late presentation of the discrediting information is accounted for if we assume that people group evidence as positive or negative relative to the guilt hypothesis. In the late condition two positive items of evidence are grouped together, and both items are affected by the discredit of one of them(irrespective of the appropriateness of this extension). In the early condition the discredit occurs before any grouping can occur, so this discredit is only extended to a new item if it is appropriately related.
2) Witness vs. alibi evidence
Alibi evidence is often a critical factor in legal cases, but there has been very little theoretical or empirical research on what makes an alibi convincing (or unconvincing), or how it differs from typical witness testimony. A formal model that captures the difference between impartial witness testimony and alibi testimony has been developed (Hepler, Lagnado & Baio, this volume). In conjunction with this work, experimental studies have been conducted to examine whether mock jurors conform to these models. People were presented with background crime cases, and then given either witness or alibi testimony. This testimony was subsequently discredited (either because of deception or error). The main findings were that when people discovered that an incriminating witness testimony was discredited, they reduced their judgments of guilt back to their original baseline judgments, irrespective of whether the discredit was due to deception or error. In contrast, when people discovered that an exonerating alibi testimony was discredited, they increased their judgments of guilt substantially above their original baseline judgments when the reason was deception, and increased them slightly above baseline when the reason was error. In short, people reacted differently to the discredit of witness or alibi evidence. Only in the case of discredited alibi evidence did the presence of deception supplement the guilt of the suspect. This pattern of results supports our proposed formal models and suggests that people are sensitive to the different dependency structures implied by typical witness or alibi evidence.
3) Confessions
Confession evidence also plays an important role in legal contexts, especially criminal cases such as murder. This type of evidence is controversial (Gudjonsson, 2003), because there are often alternative explanations for a confession, such as police coercion, fabrication or offers of leniency. This is well-modelled in Bayesian networks by ‘explaining away’, where the confession is potentially explained by several non-exclusive hypotheses: e.g., the suspect is guilty, or the police used force in interrogation etc. The situation becomes more complex when there are multiple confessions (e.g., when a gang are collectively accused of a crime). Indeed given certain assumptions it can be shown that although one confession raises the probability that the gang are guilty, any further confessions actually reduce this probability, while at the same time raising the probability that the police used force (Dawid & Lagnado, 2007).
This is a subtle consequence of Bayesian reasoning, and it is unclear how people will reason about such cases. In an experimental study mock jurors were presented with details of a crime, in which a gang had been arrested and separately interrogated. In contrast to the normative predictions, people’s assessments of the gang’s guilt steadily rose with each new confession. Follow-up studies are planned to explore the robustness of this finding. A tentative explanation, in terms of the limited-capacity model, is that additional confessions act as a grouped variable, rather being represented separately as required by a normative Bayesian representation.
Taken together these three sets of studies offer support for the proposed psychological model. People are sensitive to qualitative relations between items of evidence, and make appropriate inferences so long as the variables are limited in number, and the required computations are simplified. Future research will develop this model further, and apply it to other domains of reasoning.