splash_article6

Don’t miss our trial consultant responses at the end of this article: Jonathan Vallano & Steve Charman, and Jessica Boyle.

When a witness is asked to make an identification decision, the stakes are high. The correct identification of a guilty offender can lead to a successful conviction, whilst the incorrect ID of an innocent suspect can contribute to a miscarriage of justice. Given that the Innocence Project analysis of over 300 DNA-based exonerations has shown that around three-quarters of these wrongful convictions involved mistaken eyewitness evidence1, it is clear that we need to help witnesses to make the right decision. But how can we do this?

Here we discuss a solution that has been known to psychology research (and TV quiz-show producers) for many years. To illustrate, imagine you are asked a difficult question from memory (without using the internet!), such as Who was the guitarist on Elvis Presley’s “Sun Sessions” album?(*) If you answer the question and get it right you win a cash prize of $10,000, but if your answer is wrong you lose the same sum. Alternately you could choose not to answer, without reward or penalty. What would you do? For difficult questions like this, the common response is to opt out of answering. That is, for most people the risk of an error is too great to justify a guess, but for some (perhaps Elvis devotees) their knowledge means that the risk is lower, and they will be confident in their ability to win the cash prize. And most likely they would be right to take the risk with the question.

Exactly the same argument can be applied to the costs and benefits of making a lineup identification decision. The witness has the potential benefit of identifying the bad guy, at the potential cost of falsely accusing someone innocent. However, witnesses rarely opt out of this decision, even when they should. That is, even when their memory can’t support a good decision, they still make one. For some people, this is an avoidable mistake. Just as we would advocate that someone shouldn’t risk their cash trying to answer a question on a topic they know nothing about, so we argue that a witness who doesn’t have a good memory shouldn’t be committing to an identification decision. They need to be reminded that it is OK to say “don’t know”.

In the sections below we describe our recent experiments looking at the consequences of telling witnesses that it is OK to say don’t know. But we start by asking a very obvious question:

Why hasn’t anyone looked at don’t know responses before?

Actually they have – a study in 1980 by Warnick and Sanders2 demonstrated some of the potential benefits we describe below, but it was largely ignored. We believe that the reason for this inattention to don’t know options in identification decisions is the result of two powerful assumptions made by the legal community and by eyewitness scientists.

The first assumption is that an identification task already has an implied “don’t know” option. There is nothing to prevent a witness from saying that they “don’t know” when asked to make an identification. If this is the case, then adding a don’t know option can’t be of any use. We disagree with this assumption, because we know that people are notoriously bad at determining the options available to them. We also know that standard lineup instructions do not explicitly tell people that it is OK to say don’t know3. As our data show, unless the option to respond “don’t know” is explicitly brought to the witness’s attention, they are unlikely to use it. Warnick and Sanders found something similar over 30 years ago.

The second assumption is that an uncertain-witness, that is someone who finds it difficult to positively identify a single individual from a lineup, will choose not to identify anyone. This assumption is bolstered by the standard lineup instructions given to witnesses that emphasise that the offender may or may not be present, and that it is important to exculpate the innocent as well as incriminate the guilty. If witnesses are following these instructions, and identifying a single individual only when reasonably certain of the match to their memory of the perpetrator, then there would be no need to tell people that it is OK to say don’t know. Identification decisions would not then be made by uncertain witnesses.

A wealth of studies show that this assumption is false. Witnesses tend to pick, even when uncertain. Warning witnesses that the perpetrator may not be in the lineup does reduce this tendency4, but not to zero. Many people still pick when they shouldn’t.

Because we were sceptical about the assumptions about lineup choices, we ran two studies to look at the effectiveness of explicitly telling witnesses it is OK to say don’t know. We believed that this was likely to result in better decisions, and our studies were designed to measure just how much better they were. However, we knew that encouraging people to say “don’t know” only makes sense if people can judge when they do or do not know the correct answer. Previous research shows that providing a don’t know option does reduce errors quite a bit, but to a lesser extent it can also reduce the number of correct answers given5. This happens because people aren’t perfectly able to determine when they should answer or not. This is potentially a big concern for law enforcement. Whilst it is desirable to increase the accuracy of lineup identification decisions, this benefit cannot be outweighed by too great a reduction in the number of identification decisions. Thus, offering a don’t know option has the potential to reduce willingness to pick, with the danger that it may help free the guilty.

So, to summarise, our research programme had three aims. The first was to see whether witnesses are aware of the option to say “don’t know” if they want to. Then we were interested in the two outcomes of explicitly offering a don’t know option. How much did it improve the quality of decisions made, and how much did it reduce the quantity of decisions made?

Our Research

We conducted two large experiments funded by the Australian Research Council’s Discovery Project6 scheme. In both, our witnesses viewed a video clip of a crime and, after a delay, completed an identification procedure. Experiment 1 tested 420 witnesses using showups7 (i.e., presentation of a single photograph) and Experiment 2 tested 439 witnesses using 6-person simultaneous lineups8. Half of the identification procedures included the offender (i.e., they were “target present”) and the other half an innocent suspect (“target absent”). Witnesses were always warned that the offender may or may not be present and were told of the importance of responding appropriately.

In Experiment 1, participants were randomly allocated to one of three different showup conditions. In the standard showup condition participants were required to indicate whether or not the photograph depicted the offender by clicking a “Yes” or “No” button on the computer screen. In the “Don’t know” condition participants made the same “Yes” or “No” decision but also had the explicit option to click a button labelled “Don’t know”. Finally, participants in the “Own words” condition were asked to indicate whether or not the photograph was of the offender by typing their response into a text box using whatever words they wanted.

We included this own-words condition to see how often witnesses would spontaneously say that they don’t know. The answer is almost never. Only 2% (of 139) of those who answered in their own words wrote down “don’t know” or anything equivalent. In contrast, when explicitly provided with a “don’t know” button 19% (of the 140) participants chose to use it. In other words, telling people it is OK to say don’t know increases the likelihood that they will take up the option almost ten-fold. This answers our first question. Although a don’t know option may be implicit when people are asked to make an identification, witnesses tend not to use it.

Having established that witnesses don’t spontaneously use a “don’t know” option, we now turn our attention to the question of whether they should be made explicitly aware of this option. To do this, we compared the accuracy of decisions made and the number of correct decisions made when the don’t know option was or was not available.

How much does allowing people to say don’t know improve accuracy?

There are two ways a witness can give a correct answer: they can correctly identify the perpetrator if they are present, or they can correctly conclude the perpetrator is not-present if the suspect is innocent. Consequently, we looked at the accuracy of identification decisions and rejections separately for both experiments. Figure 1 displays the percentage of correct decisions of each type elicited following standard instructions or instructions that involved an explicit don’t know response as a valid option. The figure provides a striking and consistent answer to our question: For every type of lineup and every decision (identification or rejection), accuracy is improved by the inclusion of an explicit option to respond don’t know.

Chart

Figure 1. The percentage of correct decisions (identification or rejection) from showups (Experiment 1) and lineups (Experiment 2) under standard instructions and with an explicit don’t know option.

How much does allowing people to say don’t know reduce quantity?

Having established that a higher proportion of answers are correct if some witnesses opt out of a decision, the next question we asked was how many correct decisions were lost. These results are displayed in Figure 2. Again, the results are striking. For identification decisions where the suspect is guilty, there was no loss of correct decisions. Thus, for identification decisions, offering a don’t know option reduced the number of errors, but it didn’t reduce the number of correct responses at all.

The picture was more mixed for rejection decisions. For the show-up study (Experiment 1), once again, there was no reduction in the number of correct rejection decisions despite the increase in accuracy. However, for the lineup study (Experiment 2), offering a don’t know option reduced the number of correct rejections. We believe that this effect is of little importance: these are witness decisions to reject a lineup, and so are not decisions that are likely to end up in court. Moreover, the outcome is largely the same in both cases: the suspect is not picked either because the witness rejects the lineup, or says that they can’t decide. We do not believe that such errors would be regarded as dangerous in the courtroom.

So, we now have a clear answer to our final question: The don’t know option did not meaningfully reduce the quantity of correct identifications of a guilty suspect from either identification procedure, nor did it reduce the quantity of correct rejections of an innocent suspect presented in showups. However, the don’t know option did reduce the number of innocent suspects who were correctly rejected, but only by shifting a clear rejection to a don’t know decision.

How can it be that we have increased the quality of decisions without any meaningful impact upon the amount of useful evidence obtained? The answer is surprisingly simple. A subset of witnesses have correctly realised that they are unable to make a meaningful judgement. Normal procedures encourage them to make a decision when they shouldn’t, and they are generally wrong. Offering them a way of opting out enables them to avoid this error, leaving the field clear for those who are making better-informed decisions.

Isn’t this just like asking for confidence?

If the aim is to identify and exclude those who are not very sure, then you may be wondering what is new here. Aren’t witnesses who make decisions regularly asked how sure they are? This is the case, but we think that there are two advantages of offering a don’t know option over simply asking for a confidence judgement. The most important is that a confidence judgement occurs after the decision, and we know that a process of confirmation bias occurs once a decision is made. People tend to focus on evidence to support their decision and play down factors that contradict their decision9. As a result they can become increasingly convinced about their choice. For example, Jennifer Thompson who infamously misidentified Ronald Cotton as her rapist took between 4 and 5 minutes to make her first identification decision, and yet ultimately reported absolute certainty that she had picked the right person. Her subsequent confidence clearly didn’t reflect how difficult she had found the identification. Would the outcome have been different if she had been offered a don’t know option?

A second problem with a confidence judgement is that it is open to reinterpretation: What are the police or courts to do with the knowledge that a witness picked the suspect with “moderate” confidence, or with confidence rated at “50%” (or that it took 5 minutes)? In contrast, the selection or rejection of a don’t know option is unambiguous: The witness has declared that they can, or cannot, make a decision and this can’t be challenged by reinterpreting the meaning of “moderate”, or what “50% confident” means, or whether 5 minutes is a long time to make a decision.

Chart

Figure 2. The percentage of identification procedures eliciting correct decisions under standard instructions and with an explicit don’t know option.

Conclusions

Obviously, two studies, even with results as clear as these, don’t provide a final scientific answer to a question. As always, more real-world research is needed to establish the generality and usefulness of our findings. Nevertheless, we believe that there are two important take-home messages from our work. The first is that witnesses making an identification decision don’t know that it is OK to admit that they can’t make a decision. A consequence of this is that some witnesses are making avoidable errors. The second message is that in order to avoid such errors, all witnesses need to be explicitly told that it is OK to say “don’t know”. The result is better quality of evidence, at relatively little cost, which can only be good for justice.

(*) Answer: Scotty Moore
Nathan Weber, PhD (nathan.weber@flinders.edu.au) is a Senior Lecturer in Psychology at Flinders University, Adelaide, Australia. His research investigates memory and people’s understanding/knowledge of their own memories. A particular focus is on the application of this basic cognitive psychology to eyewitness memory. More information about his research can be found at flinders.edu.au/people/nathan.weber
Tim Perfect, PhD (tperfect@plymouth.ac.uk) is Professor of Experimental Psychology at Plymouth University in the UK. His research interests are broadly in the area of memory and its application, in particular to the area of eyewitness memory. He is co-editor of the SAGE Handbook of Applied Memory, which will be published late in 2013. Full details of his research activity can be found at www.plymouth.ac.uk/staff/tperfect
References
  1. The Innocence Project. http://www.innocenceproject.org/understand/Eyewitness-Misidentification.php
  2. Warnick, D. H., & Sanders, G. S. (1980). Why do eyewitnesses make so many mistakes? Journal of Applied Social Psychology, 10, 362-366. doi: 10.1111/j.1559-1816.1980.tb00716.x
  3. U.S. Department of Justice. (1999). Eyewitness evidence: A guide for law enforcement. Washington, DC: U.S. Department of Justice, Office of Justice Programs. https://www.ncjrs.gov/pdffiles1/nij/178240.pdf
  4. Malpass, R. S., & Devine, P. G. (1981). Eyewitness identification: Lineup instructions and the absence of the offender. Journal of Applied Psychology, 66, 482-489. doi: 10.1037/0021-9010.66.4.482
  5. Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103, 490-517. doi: 10.1037/0033-295X.103.3.490
  6. Australian Research Council’s Discovery Projects funding scheme project number DP0878901.
  7. Weber, N., & Perfect, T. J. (2012). Improving eyewitness identification accuracy by screening out those who say they don’t know. Law and Human Behavior, 36, 28-36. doi: 10.1037/h0093976
  8. Perfect, T. J., & Weber, N. (2012). How should witnesses regulate the accuracy of their identification decisions: One step forwards, two steps back? Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 1810-1818. doi: 10.1037/a0028461
  9. Wason, P. (1960). On the failure to eliminate hypotheses in a conceptual task. The Quarterly Journal of Experimental Psychology, 12, 129-140. doi: 10.1080/17470216008416717
We asked three trial consultants to respond to this paper. Jonathan Vallano & Steve Charman, and Jessica Boyle respond below.
Vallano and Charman respond:
Jonathan P. Vallano, Ph.D. (www.jpvallano.com) is an Assistant Professor of Psychology at the University of Pittsburgh at Greensburg in Pittsburgh, PA. He also works as a litigation consultant for both civil and criminal cases.
Steve Charman, Ph.D. (http://psychology.fiu.edu/faculty/stephen-charman)is an Associate Professor of Legal Psychology at Florida International University in Miami, FL. He studies eyewitness memory as it pertains to lineup identification performance, and provides expert testimony in criminal cases involving eyewitnesses.

Why telling a witness that it’s OK to say they don’t know is good for justice: A response

Weber and Perfect’s article provides initial support for the benefits of explicitly informing eyewitnesses that they can respond “I don’t know” when presented with a police lineup. Beyond enhancing eyewitness accuracy, the inclusion of a ‘don’t know’ option can reduce the inherent suggestibility in police lineups by not forcing witnesses to render a judgment regarding whether the perpetrator is in the police lineup. There is also an easily overlooked benefit of allowing witnesses to opt out of a decision with a ‘don’t know’ response: Witnesses not given that explicit option who tend to guess may become ‘spoiled’ for any future lineups if they identify a lineup filler. In contrast, the credibility of a witness who responds ‘don’t know’ is preserved, and the witness can be shown additional lineups. Thus, the addition of this simple and easy to implement option enhances the administration of justice.

At first glance, an “I don’t know” response may be interpreted by law enforcement as uninformative. Interestingly, however, Wells and Olson (2002) have shown that a ‘don’t know’ response actually has exonerating value: Witnesses are more likely to respond ‘don’t know’ when the suspect is innocent rather than guilty. In fact, this makes sense: A ‘don’t know’ response indicates that the witness lacked a strong enough recognition experience when viewing the lineup to make an identification, and thus suggests that the suspect is innocent.

The second author of this response has also recently collected data supporting this perspective (Kekessie & Charman, in preparation). In this study, we replicated Weber and Perfect’s results: Giving witnesses the explicit option of a ‘don’t know’ response when making a lineup decision decreased false identifications without a loss in correct identifications, thus increasing the overall diagnosticity of lineup identifications. Importantly, witnesses who responded ‘don’t know’ were more likely to have viewed a target absent, rather than target present, lineup, again demonstrating the exonerating value of a ‘don’t know’ response. Instead of thinking of a ‘don’t know’ response as uninformative then, we should regard it as evidence (albeit somewhat weak evidence) that the suspect is innocent.

Implications for Research and the Legal System

As noted by Weber and Perfect, few researchers have specifically examined this topic. Despite the preliminary findings of an explicit ‘don’t know’ option reducing false identifications with no concomitant reduction in correct identifications, we caution readers from drawing sweeping conclusions regarding the benefits of including a ‘don’t know’ option.

As a parallel, consider the early research exploring the benefits of simultaneous and sequential lineups, which initially concluded that sequential lineups were superior to simultaneous lineups (sequential lineups reduced false identifications without reducing correct identifications; Lindsay & Wells, 1985). However, later research demonstrated that sequential lineups may not be universally positive, as meta-analyses showed that these lineups also reduced the number of correct lineup identifications in target-present lineups (Steblay, Dysart, & Wells, 2012).

In fact, it is the rule, rather than the exception, for lineup manipulations that decrease false identifications to also decrease correct identifications (Clark, 2012). It is therefore highly important to replicate and extend these initial promising findings before concluding that an explicit ‘don’t know’ option comes at no cost. We recommend, for instance, that future research delve deeper into the motivational and cognitive mechanisms underlying witness’s identification decisions in the presence of this option. That is, how much implicit pressure does the lineup itself place on the witness to make a decision? If this pressure motivates witnesses to make some type of decision in a lineup, under what conditions does the ‘don’t know’ option effectively alleviate or fail to alleviate this pressure? Perhaps, for instance, an explicit ‘don’t know’ option fails to be beneficial under conditions in which there are strong cues to the witness that s/he should identify someone. (It should be noted, however, that Charman and Kekessie, in preparation, included a condition in which witnesses were given biased lineup instructions that strongly suggested the witness should identify someone; even in this condition, the ‘don’t know’ option decreased false identifications without affecting correct identifications, a finding that perhaps speaks to the robustness of the effect).

Although the discussed research provides strong evidence that including a “don’t know” option would reduce miscarriages of justice, it is unclear how the legal community would receive this option. It is possible that police officers would be resistant to employ this option, as they may be unsure of how to interpret a ‘don’t know’ response. It may be difficult to determine what steps to take upon obtaining a “don’t know” decision:

Do law enforcement subsequently administer the same or a different lineup, and most importantly, what happens to the status of the investigation?

Similarly, what will district attorneys make of a “don’t know” selection—will this selection frustrate prosecutors by discouraging the continued pursuit of the already identified suspect?

Moreover, little is known regarding the likelihood of a “don’t know” selection reaching the courtroom, and if so, how legal decision-makers (e.g., judges and jurors) will perceive this selection. It is highly likely that legal decision-makers may equate ‘don’t know’ with uncertainty and not appreciate the diagnostic value of this response.

Finally, we recommend that trial consultants use this information to inform litigators about how the absence of the ‘don’t know” option may affect an eyewitness’s lineup identification accuracy. Whenever possible, litigation consultants should also advocate for the inclusion of best practices such as these within the jurisdictions they practice.

References

Clark, S. E. (2012). Costs and benefits of eyewitness identification reform: Psychological science and public policy. Perspectives in Psychological Science, 7(3), 238-259. doi:10.1177/1745691612439584

Kekessie, S., & Charman, S. D. (manuscript in preparation). The impact of a “not sure” option on lineup identification decisions.

Lindsay, R. C., & Wells, G. L. (1985). Improving eyewitness identifications from lineups: Simultaneous versus sequential lineup presentation. Journal of Applied Psychology, 70(3), 556.

Steblay, N. K., Dysart, J. E., & Wells, G. L. (2011). Seventy-two tests of the sequential lineup superiority effect: A meta-analysis and policy discussion. Psychology, Public Policy, and Law, 17(1), 99. doi:10.1037/a0021650

Wells, G. L., & Olson, E. A. (2002). Eyewitness identification: Information gain from incriminating and exonerating behaviors. Journal of Experimental Psychology: Applied, 8(3), 155-167. doi:10.1037//1076-898X.8.3.155

Jessica Boyle responds:
Jessica Boyle, MA is a doctoral student studying Clinical Psychology with a concentration in Psychology and Law at the University of Alabama in Tuscaloosa, AL. Jessica conducts research within the University of Alabama’s Witness Research Lab under the supervision of Dr. Stanley Brodsky.

Solving Eyewitness Inaccuracy: Usefulness for the Jury Box

Mistaken eyewitness identification is a significant problem in the United States legal system. Wells and Quinlivan (2009) caution, “mistaken identification is the primary cause of conviction of the innocent” (p. 1). There is a large body of empirical research concentrating on the psychology of suspect misidentification. It is not uncommon for attorneys and expert witnesses to explain research findings surrounding this issue in the courtroom in order to help triers of fact evaluate evidence. A recent study by Weber and Perfect (2012) contribute to a smaller body of literature examining just how eyewitness identification procedures can be improved to prevent misidentification. Despite the limited data that exists surrounding the usefulness of the “I don’t know” option during lineup identifications, preliminary results are promising. Weber and Perfect (2012) assert that more studies are needed to corroborate the strength of the “I don’t know” option in investigative proceedings. Still, attorneys may be wise to utilize information gleaned from this study and others in certain case proceedings that rely heavily on witness identification evidence. Attorneys promote justice by equipping jurors with the most thorough picture of eyewitness identification evidence as possible.

Eyewitness identification accuracy was questioned in a systematic fashion starting in the 1970s (Wells and Quinlivan, 2009). This body of research tends to go in one of two directions. Researchers either attempt to uncover the mental processes involved in suspect identification and misidentification (e.g., Clark, Marshall & Rosenthal, 2009) or they seek to uncover tools to prevent the problem in the first place. Research examining the psychology of misidentification shows that witnesses are frequently compelled to identify suspects, even when they are less than confident in their ability to do so. A number of factors contribute to this problem. For instance, eyewitnesses may pick a suspect due to a desire to please law enforcement agents, whom they see in a position of authority (Greene & Heilbrun, 2011). Another frequently cited problem is the suggestive or improper administration of the identification task (Charman & Wells, 2008; Wells & Quinlivan, 2009). According to Weber and Perfect (2012) confirmation bias poses a significant threat to identification accuracy as well because once identifiers have chosen a subject, they will selectively concentrate on information that favors their decision, while rejecting the information that does not support their decision. These factors work alone or in conjunction and pose a risk to identification accuracy and the implementation of justice within our legal system.

While we do know a great deal about why suspect misidentification occurs, a much more limited number of studies have identified useful techniques that work to prevent the problem in the first place (e.g, Warnick & Sander, 1980; Sauerland, Sagana, & Sporer, 2011). Weber and Perfect’s (2012) research seeks to cancel out eyewitnesses that are ill equipped to make sound decisions during the identification process. In their study, mock eyewitnesses are given the explicit option to opt out of the identification task if they do not feel confident in their decision-making ability. Similar to research conducted over 30 years prior, it was found that giving an explicit “I don’t know” option tends to cancel out misidentifications and improve identification accuracy overall. Additionally, the opt-out option posed little threat to the quantity of reliable participant decisions.

Interestingly, Weber and Perfect (2008) are not the first to look at the potential benefit of giving witnesses an “I don’t know” option. Warnick and Sanders (1980) identified many of the same strengths of giving the “I don’t know” option, however their findings were dismissed due to the fact that people think eyewitnesses already know they have an “I don’t know” option. Previous research shows that time and again, multiple factors may compel an unsure witness to still identify a subject in a lineup. Through Weber and Perfect’s (2012) use of the “own words” experimental condition, they’ve shown that people will rarely, if ever, exercise their right to opt out of identifying a subject unless they are explicitly instructed that they may do so.

Weber and Perfect (2012) do a good job of identifying the current limitations of their research and acknowledge that more work can be done in laying out the benefits of utilizing an explicit “I don’t know” option. Although research thus far is limited, this does not mean that the information gleaned from the study by Weber and Perfect (2012) and others (e.g., Warnick & Sanders, 1980) should not be cited and explained in the courtroom, particularly in cases where unnecessarily heavy weight may be given to eyewitness identification evidence. It may be worth the investment in time and money to hire a jury consultant or other expert that can aid attorneys in developing the language to explain the limitations of eyewitness identification evidence to jurors. If someone is identified as the perpetrator of a crime, an attorney could explore the circumstances surrounding the lineup proceedings. If an “I don’t know” option was not given to the witness, this may weaken the reliability of the evidence in the jury’s mind. It may also be worthwhile to explore factors that could potentially compel a witness to identify a suspect when they are less than certain about their decision-making ability.

Weber and Perfect’s (2012) study not only contributes to a wealth of knowledge available to attorneys for litigation advocacy, it takes the issue of eyewitness identification accuracy one step further by offering a concrete solution and procedure to law enforcement in their enhancement of the criminal system. The more tools we have to improve the accuracy of evidence presented at trial, the better for justice. Empirical studies surrounding the limitations and potential improvements of eyewitness evidence provide more information surrounding complicated psychological issues that can be imparted to the jury. Jurors are then better equipped to handle such issues and can more thoughtfully engage in deliberation and decision-making processes.

References

Charman, S.D., & Wells, G.L. (2008). Can eyewitnesses correct for external influences on their lineup identifications? The actual/counterfactual assessment paradigm. Journal of Experimental Psychology: Applied, 14 (1), 5-20.

Clark, S.E., Marshall, T.E., & Rosenthal, R. (2009). Lineup administrator influences on eyewitness identification decisions. Journal of Experimental Psychology: Applied, 15 (1), 63-75.

Greene, E., & Heilbrun, K. (2011). Psychology and the legal system (7th ed.). Belmont, CA: Wadsworth/Thomson Learning.

Sauerland, M., Sagana, A., & Sporer, S.L. (2011). Assessing nonchoosers’ eyewitness identification accuracy from photographic showups by using confidence and response times. Law and Human Behavior, 36 (5), 394-403.

Warnick, D.H., Sanders, G.S. (1980). Why do eyewitnesses make so many mistakes? Journal of Applied Social Psychology, 10, 362-366.

Weber, N., & Perfect, T.J. (2011). Improving eyewitness identification accuracy by screening out those who say they don’t know. Law and Human Behavior, 36 (1), 28-36.

Wells, G.L., & Quinlivan, D.S. (2009). Suggestive eyewitness identification procedures and the Supreme Court’s reliability test in light of eyewitness science: 30 years later. Law and Human Behavior, 33, 1-24.