The Liberty Papers

Disclaimer: The views expressed here at The Liberty Papers either by the post authors or views found in the comments section do not necessarily reflect the views of The Innocence Project nor its affiliates.

In support of our fundraising efforts for The Innocence Project, I have decided to dedicate at least one post per week over the next four weeks to the cause of criminal justice reform – many of which are the very reforms The Innocence Project are working to bring about. As of this writing, you readers have already donated $310 – 62% of our $500 goal! Thanks to everyone who has donated so far or plans to donate. Remember: your donations are 100% tax deductible.

With that out of the way, now I will turn your attention to the topic at hand: Eyewitness Misidentification.

Back almost three years ago to the day, I wrote a post about Troy Davis who had his death row appeal denied despite seven eyewitnesses recanting their testimonies (this case is still winding its way through the courts; here is an update on where the case stands today). As is often the case whether here at The Liberty Papers or at other blogs, the discussion that followed my post was actually a great deal more interesting than the post itself IMHO. Jeff Molby, a person who comments on a somewhat regular basis, really got the discussion going with several Liberty Papers contributors and readers.

The part of the post that Jeff believed to be “misleading” was the following statement I took from The Innocence Project webpage that dealt with the role eyewitness misidentification plays in wrongful convictions:

Eyewitness misidentification is the single greatest cause of wrongful convictions nationwide, playing a role in more than 75% of convictions overturned through DNA testing.

While eyewitness testimony can be persuasive evidence before a judge or jury, 30 years of strong social science research has proven that eyewitness identification is often unreliable. Research shows that the human mind is not like a tape recorder; we neither record events exactly as we see them, nor recall them like a tape that has been rewound. Instead, witness memory is like any other evidence at a crime scene; it must be preserved carefully and retrieved methodically, or it can be contaminated.

This was Jeff’s response:

Eyewitness misidentification is the single greatest cause of wrongful convictions nationwide, playing a role in more than 75% of convictions overturned through DNA testing.

That’s a misleading stat. The relevant stat would be the percentage of convictions based on eyewitness identification that were later overturned due to DNA testing.

Comment by Jeff Molby — March 17, 2008 @ 12:51 pm

Perhaps the reason Jeff found the quote was misleading was my fault rather than The Innocence Project’s. The page that I took the quote from goes into greater detail complete with links for further reading. From my reading of their material, it seems to me that the statistics they are dealing with are from their now 266 exonerations. As the discussion unfolded, this forced me to do some additional research outside of The Innocence Project [Thanks a lot Jeff : ) ] to see if I could find more data to support –or refute The Innocence Project’s claim. Fellow contributor and lawyer by trade, Doug Mataconis also weighed in with his thought about the reliability of eyewitness testimony.

The highlights from this discussion are below the fold.

The relevant stat would be the percentage of convictions based on eyewitness identification that were later overturned due to DNA testing.

I don’t completely agree. A very small portion of convictions are amenable to DNA testing. The proper denominator would be the number of convictions that are subsequently tested where eyewitness identification played a role.

The problem with examining the cases tested by the innocence project is that there will be a sampling bias: people who are innocent are much more motivates to seek out avenues for exoneration, and the innocence project will select cases that where they have a higher probability of success to focus their limited resources.

Comment by tarran — March 17, 2008 @ 6:38 pm

The proper denominator would be the number of convictions that are subsequently tested where eyewitness identification played a role.

I would also include those convicts for whom DNA evidence was available, but never tested. If a convict isn’t pursuing all possible means for an acquital, it’s hard to assume he’s anything other than guilty.

Regardless, I think we can agree that the statistic cited is pure garbarge.

Comment by Jeff Molby — March 17, 2008 @ 8:05 pm

If you read a report analyzing aircraft crashes, and the report stated that cracks in the pressure hull had contributed to 75% of crashes, I doubt you would be claiming that that statistic was somehow misleading and that the important question was how many planes flew with similar cracks safely.

It depends on the point you’re trying to make.
Read that sentence again:

Eyewitness misidentification is the single greatest cause of wrongful convictions nationwide, playing a role in more than 75% of convictions overturned through DNA testing.

Notice that the assertion is about “wrongful convictions” (herein referred to as “apple”), yet their supporting statistic is about “convictions overturned” (herein referred to as “orange”).

We can agree that there are some unknown number of wrongful convictions that never get overturned, right? Isn’t it quite possible that there are other unmentioned factors that bias the sample?

And even if there aren’t, the author goes on to assert that eyewitness testimony is “often unreliable.” He didn’t offer a shred of evidence to support that statement. The eyewitness testimony could be right 99.999% of the time for all we know, because he only examined the convictions that were overturned.

In the context that it was presented, the stat is useless at best.

Comment by Jeff Molby — March 18, 2008 @ 9:46 am

The only wrongful convictions we know of are ones that are overturned. Well, that’s not completely correct; there are cases like the one that’s the focus of Stephen’s post here.

These aren’t apples and oranges. The convictions overturned is a subset of a larger, unknown and unknowable set, all the wrongful convictions. The use of a small sample to come to conclusions concerning larger sets is the heart of “hypothesis testing” which is one of the most highly developed branches of statistics.

You could argue that they are making conclusions about all apples from a sample of golden delicious apples, and I would agree with you. But to claim that this is apples and oranges is a bit much.

With that being said, they are absolutely correct that eyewitness testimony can be quite unreliable. In fact some branches of eyewitness testimony are known to be often unreliable, such as identification of a stranger. There was a series of psychological experiments that demonstrated this; the researchers could consistently get witnesses to finger the wrong man as a person who committed a a simulated robbery merely through having a patsy in the crowd loudly and incorrectly describe one aspect of the thief’s appearance.

Now, again, you could argue that the statistics they present, on their own do not back up this claim. However, the unreliability of witnesses when describing novel people or situations is well known enough that it needs no confirmation anymore than a guy talking about airplane design need to justify his assertion that air pressure can be used to calculate altitude.

Comment by tarran — March 18, 2008 @ 10:35 am

Jeff:

If you actually follow the link, you will see that TIP considers such factors as estimator variables, and system variables. TIP also has made observations about eyewitness misidentification based on the over 100 convictions they have helped overturn. If you look to the right side of the web page, you will find more outside information about the unreliability of the use of eyewitness identification: here’s an Iowa University Study and a website from one of the study’s authors Iowa State Psychology Professor Gary Wells .

If you are skeptical about TIP’s sources, do an independent Google Scholar Search on the topic from 2000 to 2008. You will find that Dr. Wells’ articles are among the top scholarly articles cited by other peer reviewed scholarly articles found by Google Scholar.

On Table 1 of this study which interviewed 64 psychologists who had courtroom experience found in the May 2001 issue of American Psychologist, the very first of thirty variables listed in determining the reliability of an eyewitness is the stress level at the time of the event. The statement next to this variable reads as follows: “Very high levels of stress impair the accuracy of eyewitness testimony.”

This would explain why I could not remember exactly what this man looked like even though he was looking right at me less than 2 feet away; I was scared shitless (to use a scientific term).

The very first page of the same study states the following:

In 1996, the National Institute of Justice reported on 28 wrongful convictions, cases in which convicted felons were exonerated by DNA evidence […] Remarkably, all of these cases contained one or more false identifications.

This is only the tip of the iceberg. The body of research being done concerning eyewitness mistakes is growing. Read these studies as well as a few of your own and then tell me that eyewitness testimony is generally reliable enough to take someone’s freedom or someone’s life.

Comment by Stephen Littau — March 18, 2008 @ 11:56 am

Stephen,

This case, and the issue of eyewitness testimony, reminds me of two classes I took, one in college and one in law school, where the professor engaged in an experiment that showed just how unreliable eyewitness testimony can be.

It should be familiar to anyone who’s taken a criminal justice class of any kind.

In both cases, we witnessed what looked like a crime (theft in one case, attempted assault in the other) but was actually a ruse between the professor and a graduate assistant. After it happened, the professor asked the class to recount details about what they saw such as:

– The height of the person who had run into the room
– What color hair they had.
– How tall they were
– Whether they were fat or thin
– What was said

Stuff like that.

Without exception, everyone got at least one crucial detail about the incident completely wrong. And that was mere minutes after it happened.

I don’t practice a lot of criminal law, but even on the civil side it became obvious to me quite early on that when someone gets up on the stand in a courtroom and is asked to recount the details of an event that happened years in the past, they usually aren’t getting everything completely right.

In a civil case, it usually isn’t crucial because documents usually control.

In a criminal case, where someone’s freedom or life is on the line, basing a conviction solely on the testimony of witnesses, without corroborating physical evidence seems foolish.

Comment by Doug Mataconis — March 18, 2008 @ 1:09 pm

You could argue that they are making conclusions about all apples from a sample of golden delicious apples, and I would agree with you. But to claim that this is apples and oranges is a bit much.

Ok, you can reduce the scale of my analogy if you like, but my point is still valid. The subset is non-random* and therefore it is not logically possible to draw conclusions about the whole from that subset; if you want to make a hypothesis from the subset and use that as the basis for further experiments on the whole, knock yourself out. You can’t, however, make conclusions.

*Not only is it not random, it isn’t even mixed up a little. The subset is narrowly defined to only include a single class of cases.

Now, again, you could argue that the statistics they present, on their own do not back up this claim.

That’s what I was doing.

With that being said, they are absolutely correct that eyewitness testimony can be quite unreliable

If you actually follow the link, you will see that TIP considers such factors as estimator variables, and system variables. TIP also has made observations about eyewitness misidentification based on the over 100 convictions they have helped overturn.

I don’t necessarily take issue with their conclusion. I just take issue with the way that stat was presented. It struck me as MSM-style sensationalism.

Even if the stat did speak directly toward their conclusion, they still should have given it some context. “Played a role in more than 75%”? Ok, where does that fall in comparison to other factors? Presumably there are multiple factors in most wrongful convictions, so 75% might not even be good enough for 3rd place for all I know.

Comment by Jeff Molby — March 19, 2008 @ 1:25 am

A Final Thought
Just how reliable are eyewitnesses and their testimonies? Whether or not The Innocence Project is being “sensational” in presenting their data to make their point, I think we can all agree that the courts rely entirely too much on eyewitness testimony – particularly when the eyewitness in question doesn’t know the person who they saw committing the crime. I think we can also agree that the techniques that the police use when interviewing eyewitnesses can be improved to minimize the possibility of convicting an innocent person.

The Innocence Project does have some great ideas for doing just that and some of their reforms have been adopted at the state and county level. To learn more about how The Innocence Project is working to reform eyewitness identification procedures, follow this link. If you believe these reforms and others will improve the system and you would like to help make these reforms possible, I would encourage you one more time to go to our “Innocence Partners Page” and make a tax deductible donation to help us meet our $500 goal (we’re only $190 away!).

Also, a special thanks to Jeff Molby, Tarran, and Doug Mataconis, for contributing so much to a discussion that was so valuable that it deserved to be revisited and reintroduced to readers a full three years later. This is what makes blogging such a rewarding hobby for me.