Thumbnail Image
Jed Rakoff

Episode 10: Science and the Gavel

With scientific evidence an intrinsic and complex part of today’s court cases, can judges keep up? Senior U.S. District Judge Jed Rakoff of the Southern District of New York discusses challenges and opportunities for jurists.

Show Notes: Science and the Gavel

Jed Rakoff

Jed Rakoff is a senior U.S. district judge for the Southern District of New York, as well as a lecturer at UVA Law, where he teaches Science and the Courts. Rakoff also sits by designation on the Second and Ninth Circuit Courts of Appeals. He has authored more than 150 published articles, 1,800 judicial opinions and five books.

Rakoff began his legal career clerking for the late Judge Abraham L. Freedman of the U.S. Court of Appeals for the Third Circuit, then joined Debevoise & Plimpton as an associate. He was later tapped to be an assistant U.S. attorney in the Southern District of New York, where he was chief of business and securities fraud prosecutions during his final two years. Before his appointment to the court in 1996, Rakoff was a litigation partner at two large law firms in New York.

He has served on the National Commission on Forensic Science, the National Academy of Science’s Committee on Eyewitness Identification, and the MacArthur Foundation’s Project on Law and Neuroscience, along with numerous additional board appointments. He participated in the development of the Third Edition of the federal judiciary’s “Manual on Scientific Evidence” and was co-editor of the “Judge’s Guide to Neuroscience.” Additionally, Rakoff has helped the U.S. government train foreign judges in nearly a dozen countries, and in 2014 he was included among Fortune Magazine’s list of the World’s 50 Greatest Leaders.

Listening to the Show

Transcript

[MUSIC PLAYING]

RISA GOLUBOFF: Hello, and welcome to Common Law, a podcast from the University of Virginia School of Law.

I'm Risa Goluboff, the dean.

LESLIE KENDRICK: And I'm Leslie Kendrick, the vice dean.

RISA GOLUBOFF: Leslie, this is it. This is the last show of season one.

LESLIE KENDRICK: I can't believe it.

RISA GOLUBOFF: I can't either, it's been a good season.

LESLIE KENDRICK: It really has.

RISA GOLUBOFF: I think there have been themes that have come out that we didn't necessarily expect.

LESLIE KENDRICK: Yeah, what themes?

RISA GOLUBOFF: You first, no. No, you first. Well, I was thinking, obviously the theme has been the future of law.

But that manifests itself in lots of different ways. Some of our episodes have really been

focused on technology and how the technology changes and enters into the law, and the law

tries to deal with the technology.

LESLIE KENDRICK: Yeah, like the autonomous vehicles and blockchain episodes.

RISA GOLUBOFF: Yeah, exactly what I'm thinking about. And then the other kind are really more social issues

and changes in society that are part and parcel of changes in the legal system.

LESLIE KENDRICK: Mm-hmm. I could think of the Innocence Project with Deirdre Enright and John Grisham. And

also our Dana Matthew episode about medical legal partnerships fitting that mold.

RISA GOLUBOFF: It seems to me, that underlying both of those trends and both types of episodes is the rapid

pace of change in the world, and especially science. I mean, science writ large. How do we

understand our world, how do we know our world? And then, of course, what does the law do

with that knowledge about the world?

LESLIE KENDRICK: We've talked to experts in so many different areas that have to do with that, and I think of this

last episode as being a capstone that brings many of these different strands to bear with one

last very special expert to talk about all of these different issues, and where the rubber meets

the road on them in the courtroom.

RISA GOLUBOFF: Exactly. Judge Jed Rakoff of the Southern District of New York is here with us today. He has

been on the federal bench for more than 20 years, and he teaches a course here at UVA

about the use of forensic and other scientific evidence in the courts. Judge Rakoff, thank you

so much for joining us.

JUDGE JED RAKOFF: My pleasure, it's an honor to be here.

RISA GOLUBOFF: To frame this conversation for our listeners, Judge Rakoff, could you start out by giving us an

example of a case in which scientific issues were an important part of what you had to do as a

judge?

JUDGE JED RAKOFF: So the first case that really put me to the test, so to speak, was in 2005, a case called In Re

Ephedra. And this arose from the fact that in the 1990s, there were sold over the counter a

natural substance which is now banned called ephedra, which was supposedly helpful in losing

weight and building muscles.

Unfortunately, about 1,000 people who took ephedra had within 24 hours, a heart attack or

stroke, and many of them died. And so about 800 cases were brought throughout the United

States claiming that ephedra had caused these deaths or injuries. And under something called

the multi-district litigation process, they were all consolidated before me.

And the main point of contention between the parties was that the plaintiffs had experts who

said ephedra can cause heart attacks and strokes, and then they would undertake to try to

show it at any individual case, and the defendants said ephedra does not have the capacity to

cause heart attacks or strokes, and it's just bad science.

So I held a 14-day hearing when I heard from numerous experts from both sides and got

deeply into the scientific literature on this. And of course, I knew nothing about ephedra before

the case, so it was a major challenge. But by the end, I think I at least had a adequate

understanding.

And I ruled finally that while ephedra could not cause heart attacks or strokes in perfectly

healthy people, people who were already predisposed towards heart attacks or strokes could

be put over the limit, so to speak, by consuming ephedra. And that somewhat down the middle

decision led to the parties then settling all 800 cases. So I felt it was a positive result.

LESLIE KENDRICK: Wow, so 800 cases. And a multi-district litigation like that raises issues about both the

shortcomings of scientific evidence and the shortcomings of maybe our tort system, in terms of

dealing with products that might create or heighten risks. But it might be very difficult to end up

saying that they caused any specific heart attack.

JUDGE JED RAKOFF: Yes. So the question for the judge is what's called general causation. Is the product capable of

causing heart attacks and strokes? And is that true across the board? Or only in people who

are predisposed because of other physical problems? If after my decision on that question, the

cases had then gone to trial, they would have gone to individual trials.

And in each individual trial, the jury would have had to determine, did ephedra cause the heart

attack or stroke of this particular individual? Was it a contributory factor? And if so, how much

did it contribute? That's called specific causation. But as I say, the cases settled before they

went to trial.

RISA GOLUBOFF: So there's not just, there's the general causation and the specific causation within the law, but

how does that relate to the way causation is thought of scientifically? There's not a perfect

match between what kinds of causation the law wants to find and the way scientists think

about causation.

JUDGE JED RAKOFF: That is totally true, and is a source of often, misunderstanding. So science is increasingly

probabilistic. And so causation is a function first of correlation, but it's correlation to such a

high degree, that causation seems at least plausible. And then you need a scientific theory.

And then most important, you need to test the theory.

In the case of ephedra, the test that would most normally be used, which is called an

epidemiological study, had not been done. Basically because it's a very expensive study. And

ephedra did not have to be approved in advance by the FDA, by the Food and Drug

Administration. So the manufacturer did not have a motive to spend that kind of money.

So in the absence of an epidemiological study, I had to look at other evidence of causation,

such as data collected by hospitals when people did have heart attacks and strokes, such as

animal studies. But those are always a little questionable, because the animals don't operate

quite the way we do. I had to look at studies of similar substances and what had been found

out. And in those cases, often epidemiological studies had been done, et cetera, et cetera. But

causation in science is at a higher degree of probability than causation in the law.

LESLIE KENDRICK: So how does that evidence come up within the court system?

JUDGE JED RAKOFF:So it can come in many ways. But increasingly, one side or the other will be presenting expert

testimony, or both sides will be presenting expert testimony. This raises a serious problem for

the so-called adversarial system, the Anglo-American style of presenting and trying cases,

where each side has its champion and the theory is that they will clash, but the truth will

emerge as the victor.

Experts in any scientific area will often have developed over a period of time, pretty strong

views favorable to one position or another. And while ultimately, the scientific community

frequently arrives at a consensus, it takes a while to get there. And until that happens, each

side can frankly, skew things to its advantage by hiring an expert who they know in advance

from the expert's prior writings and the like, is likely to be favorable to their position.

And a number of studies have shown that juries, when confronted with these clashing experts,

will sometimes simply cancel out the expert testimony and look to other stuff to make the

decision. But when the issue is science, you really can't do that. Now the Federal Rules of

Evidence provide another way to approach this, which is through court-appointed experts. A

great champion of this is Justice Breyer on the Supreme Court.

Judges have not made use of that as much as perhaps they ideally should. I think there's a

fear among judges that they will unwittingly in choosing the court's expert, choose someone

who has a bias in one direction or another, and therefore, they will skew the case, so to speak.

But the American Association for the Advancement of Science, which is the leading group of all

scientists, has made available to those judges who want it, a panel.

And they will, if a judge says I want a court-appointed expert on x, they will give the judge

three names of well-known established people who are willing to serve as experts for the

court. And the judge can then decide which one he wants or maybe run it by the parties to see

if they have objections to any expert or the like.

So I think the broad issue in some ways, presented by science in the courts, or one of the

broad issues, is that it may not lend itself to the pure adversarial system that we're so used to

in this country. And it may require some adjustments in the direction of more judicial

intervention.

RISA GOLUBOFF: Yeah, it strikes me in listening to you, that there's just, there's such a gap. I mean, earlier I

asked you about the gap in causation. But it seems like in order for the courts, and I think for

you as a judge, obviously, ephedra was a new thing, but you've gained expertise over the

kinds of issues that come up, the ways in which you're going to interact with experts. But then

it's yet a whole other step to get to juries, and to get to lay people who are going to be

interacting with these kinds of questions. And it just, it does bring to mind this question, is

there, what are we doing?

I mean, I don't know if that's a fair way to put it, but how do we think we're going to get to right

answers when scientists spend their lives doing this and they disagree about it and we have a

concept in our head that we can have lay people or legally trained judges who are not

scientifically trained, do you think that judges need more scientific training?

How do we, I mean, what you're saying about moving more toward a kind of inquisitorial

system a little bit is what I take you just now to be saying, is that the only way we're trying to fix

this problem? Or are there other ways?

JUDGE JED RAKOFF: Well, the Supreme Court I think, took a giant step in helping to solve this problem when it

decided in the early '90s a case called Daubert, D-A-U-B-E-R-T. Some people pronounce it

Daubert, but they're just snobs.

[LAUGHTER]

RISA GOLUBOFF: Daubert, it is.

LESLIE KENDRICK: You heard it here.

JUDGE JED And up to then, the standard for the admissibility of scientific evidence was whether it was

RAKOFF: generally accepted in the relevant community. And the problem with that was, the relevant

community was often defined fairly narrowly. So is ballistic testimony good? Well, all the

people who were ballistic experts said it was great.

RISA GOLUBOFF: Right.

JUDGE JED RAKOFF: So the Supreme Court in Daubert said, we want judges who make a more rigorous inquiry.

And we want them to look at least four factors. And the first and most important, is whether the

science has actually been independently tested. If it hasn't been tested, it's probably not

science.

The second factor is whether the methodology used by the particular expert has been peer

reviewed, reviewed by other independent scientists. And typically this would take the form of

articles in scientific journals, either critiquing or otherwise analyzing the particular

methodology. The third criterion was whether there was a known error rate for this

methodology for this particular scientific approach. And if so, if it was a very large error rate,

that might be a reason for excluding it.

Way back when, long before Daubert, the courts of the United States had excluded lie detector

evidence on largely that ground. That it has, and still has a very high error rate. And then the

final criterion was still to look at whether it's been generally accepted. But the court cautioned

that the community should be the scientific community generally, and not just a very highly

specialized portion of it.

So that has resulted in what are called the Daubert hearings. That's what my ephedra hearing

was, that occurred just before the judge, not before the jury. And 38 of the 50 states have

adopted Daubert now. And so you begin to see judges throughout the United States giving a

more rigorous look at the admissibility of any given scientific evidence. It's hard work.

Most judges have no scientific background. I was an English major. But on the other hand,

most judges or virtually all judges, were trial lawyers before they became judges. And trial

lawyers all tend towards being quick studies. It's an important aspect of being a trial lawyer.

That you can get into a complicated area. Someone comes to you with a case, you know

nothing about the case, by the time the case goes to trial, you're the world's leading expert,

although you'll forget it all as soon as the trial is over.

So judges have that quality. And I think many judges are now putting that use. And usually the

judge's decision is case dispositive. Much as we love our jury system, the sad fact is, that very

few cases go to trial anymore before a jury. They are mostly decided by these preliminary

determinations. So it really is the judge who in well over 90% of the cases is making the key

decision.

RISA GOLUBOFF: So often when we talk about things like science and the law or science and the courts, they

seem like these huge amorphous things that exist out there in the world without any people in

them. And you've been talking about scientific experts and different associations and the way

people interact.

But I've been thinking about how there are different people on the scientific side, and then

there are different people playing different roles on the law side. And you've actually played a

number of different roles. You're a judge now. You were a prosecutor. You were a lawyer in

private practice. Has your perspective on scientific evidence or the relationship between

technology and law or science and law, has changed for you in those different roles?

JUDGE JED RAKOFF: Definitely. When I was a prosecutor, I'd like to say in 1492, but a long time ago. None of the

forensic science was ever questioned. I would present it in a case. My colleagues would

present it in a case. The defense would usually stipulate to the results. Of a lot of what we now

know was very questionable forensic science that led to many innocent people being

wrongfully convicted, was taken for granted.

Oh, this is good science. It wasn't really until 2009 when the National Academy of Science

issued a groundbreaking report examining the science of the various forensic disciplines, that

people began to realize that a great deal of forensic science is very weak. Some of it may not

even deserve the term science at all.

The best of it is DNA. But if you take things like bite mark comparisons, tool mark

comparisons, much, much weaker. Probably not really deserving to be called science at all.

And more relevantly for our purposes, often inaccurate. So prosecutors today are at least a

little more sensitive to this issue.

I do think that there is a great need for greater training of prosecutors about these

shortcomings in forensic science. But it is beginning to percolate down. I would say this. I see

much better lawyering on scientific issues in civil cases than in criminal cases. And the reason

for that, is the parties in a civil case typically can afford very good experts. And those experts

not only can testify well, but can educate the lawyer as to the relevant issues.

In the criminal cases, the government usually relies on government experts who have never

been, in many cases, put to a serious challenge. And the defense, the criminal defense bar is

in many instances, woefully ignorant of the relevant science. And so you don't get the same

kind of sharp analysis.

And I'll give you just one example. I had a tool mark expert in a criminal case before me, and I

held a Daubert hearing. And going down the Daubert criteria, I finally got to what's your error

rate? What's the error rate of your methodology? And he said zero. I said, zero? He said, yes.

I said, how can that be? He said, well, every time I've testified, the guy's been convicted.

[LAUGHTER]

So he had a limited understanding.

RISA GOLUBOFF: Not quite so bad by error rate.

JUDGE JED RAKOFF: Exactly.

LESLIE KENDRICK: So, and you've done a lot of work on criminal justice and forensic evidence. And the point you

just raised this one about the legal system more than it is about the evidence. That there are

difficulties in getting good scientific evidence into the criminal justice process. Are there things

that our legal system should be doing differently to try to address that?

JUDGE JED RAKOFF: Well, on the criminals side first and foremost, money should be made available to the defense

to hire experts. A broader suggestion, which was the one made by the National Academy of

Science in 2009, was that there should be set up an independent national agency staffed by

scientists, who would make if not preliminary judgments, at least set preliminary standards for

what was good forensic science and not so good forensic science.

I think though, that a major problem in the way science is currently handled in the courts, is

that too many judges are intimidated by the notion that they would have to decide a scientific

issue. They're not scientists by training. And they feel a little discomfort. And the unfortunate

result, is that they will look to other ways to try to decide the issue before them rather than

getting deeply into the science and therefore, rather than advancing the collective judicial

knowledge of how to handle particular issues, because they will have ducked the issue.

I'm not quite sure how to change that. Judges by nature are not usually timid. But in this area,

I see too often the judges really avoiding getting into the science. I go back a little bit to the

notion of court-appointed experts might be helpful in that situation as well.

LESLIE KENDRICK: And while we're on the subject of how often these things get utilized, how often would you say

judges use court-appointed experts? Do you have a sense of that?

JUDGE JED RAKOFF: Very rarely. And I think it's unfortunate. But let me explain a little bit why. First, there's the fear

that unintentionally they will choose someone who they believe is neutral, but really is not

neutral. And because that person is the court-appointed expert and will be presented to the

parties and if the case does go to trial, to the jury as the court-appointed expert, that person

will carry a great deal of weight. So there's a little fear of unintentionally biasing the result.

The second problem is that there are no funds available to the court for this purpose. The

Federal Rule says that the parties have to pay for the expert. Well, that's fine and good if you

have well-heeled parties in a civil case or whatever. That's going to be a more substantial

problem in a criminal case and certain civil cases, where one party or another may be

relatively poor.

A third problem is that the experts who are on the panel put together by the American

Association for the Advancement of Science, they're thrilled to be talking with the judge.

They're not so thrilled to be going through a deposition. They're not so thrilled to be subject to

cross-examination. I had one court-appointed expert that I had appointed who complained to

me bitterly about how he had been treated during a deposition. And I said, well, it sounds to

me no different from the way you're treated at scientific meetings.

[LAUGHTER]

In any event, there have been a lot of those practical reservations that have made, I think,

judges less ready than they should be to use court-appointed experts. And then finally, there's

simply the fact that they're not used to doing it. Judges are very prone to fall into habits. It's,

oh yes, I know how to handle this, I've had this before, and they don't even think about the

possibility of court-appointed experts. So unfortunately, it's not been used nearly as much as it

should be.

RISA GOLUBOFF: So this may be a little far afield, but I'm just curious, when you're thinking about the resource

constraints, whether they're the resource constraints in criminal cases for defense lawyers to

use or the resource constraints for judges to be able to hire these court-appointed experts or

whether it's the resources that would be needed to create a whole federal agency that was

going to oversee somehow or provide resources for scientific evidence, what's the political

economy of getting those resources?

Who would want them? Who would push for them? Where would the opposition come? Tell

us, if you have a sense of that, what does that look like? Where are the interest groups on

scientific evidence and the courts?

JUDGE JED RAKOFF: Yeah. You may be asking the wrong person here. I'm a simple barefoot federal judge.

RISA GOLUBOFF: Oh, please.

[LAUGHTER]

LESLIE KENDRICK: You will not convince any of us--

[INTERPOSING VOICES]

RISA GOLUBOFF: --of that.

JUDGE JED RAKOFF: But I think one constituency for this would be the manufacturers of scientific products. The

pharmacological industry, the engineering industry. These are parties who are constantly

complaining, maybe rightly, maybe wrongly, that they are still being subjected to junk science.

That there are people bringing lawsuits, raising scientific claims that really are bogus and so

forth. Well, if they really believe that, then they might be a lobby for a genuinely independent

agency or the like to oversee this kind of stuff.

RISA GOLUBOFF: Wow, I feel like we haven't even begun to scratch the surface and yet, our time is mostly up.

So I'm curious, is there any one last thing you would want to say about science and the courts

or your experience or what to expect in the future?

JUDGE JED RAKOFF: Well, I guess I would say, if even an English major can deal with this stuff, you can too.

[LAUGHTER]

LESLIE KENDRICK: That's encouraging to all of us. I think that's been a theme that we've been talking about. The

fact that the law belongs to all of us. These new technologies also belong to all of us, and we

shouldn't be intimidated by them. We should learn about them and use them.

RISA GOLUBOFF: Yeah, I've been struck during this whole conversation. We started out with the premise that

law is everywhere. And now I'm going to add to that premise, science is everywhere in law. So

I feel like that's an important place to be.

LESLIE KENDRICK: Thank you so much, Judge, it's been a pleasure.

JUDGE JED RAKOFF: My pleasure, thank you.

RISA GOLUBOFF: It's been great to have you.

[MUSIC PLAYING]

So where we ended there, asking the question, should there be some kind of bureaucratic

answer to this, reminds me that we often talk about the law as if it's one thing or a common

law as if we all share one thing. But in fact, the law is a whole series of systems and tools and

powers and dynamics and relationships, and there are many different levers that one can pull.

And also, when we've been talking about the law all season, we've been talking about different

aspects of it, different institutions, different actors. And we should remember, it just makes me

want to remember that the law is incredibly varied and complex, even while we give it this one

definite article.

LESLIE KENDRICK: And it made me think of something that you're fond of saying, Risa, which is that the law is

made, not found. In hearing him talk about how he confronts these issues in his courtroom,

and judges and juries have to make decisions about legal questions and factual questions,

they're making law as they do that.

RISA GOLUBOFF: They are. And they don't, it's another thing I've been thinking about, the idea that the

technology is pushing us forward and the technology moves quickly and the law moves slowly,

something we've talked about before. The law might move slowly in some senses, but in

another sense, it doesn't have the luxury of time. You can't stop the machinery of the legal

system and say, well, we're not really ready to think about how to deal with liability for

autonomous vehicles yet.

LESLIE KENDRICK: Yeah, we're not sure. We're not sure yet how to do that.

RISA GOLUBOFF: So we'll just pause and we'll wait until we're ready. You can't do that, right? People have legal

needs. And legal cases come up, and the law has to answer the questions that are before it.

And the people in the law, people like Jed Rakoff and the lawyers and the juries and the

administrators, have to answer these questions even in the face of doubt and uncertainty, both

scientific doubt and technological doubt, and also societal doubt and legal doubt of how one

would best want to deal with these issues.

And one of things that's been challenging for me this series is, I'm a historian, right? I'm much

more comfortable in the past than I am in the future. And I find the questions that one has to

ask about the future much harder to answer. There's some security in looking backward. But I

think we've also found that often when you're looking forward, you have to be looking

backward. And the law and our law being a common law system, another riff on our title, is

backward looking, right? It is essentially backward looking as we go forward.

LESLIE KENDRICK: And so many of our guests have brought insights from past episodes in either technological

change or legal change, and how those might apply to what we're seeing now. And that's been

really educational, and gives me some hope that even though we can't know for sure what's

going to happen and there's lots and lots of new information and change that we have to try to

keep pace with, we do have some tools. We have legal tools and we have historical tools and

societal tools that we can use as we go forward.

RISA GOLUBOFF: I wonder if that's a hint about where we're headed next season, Leslie.

LESLIE KENDRICK: I don't know. I guess it could be. Do you want to do another season, Risa?

RISA GOLUBOFF: I'm in if you're in.

LESLIE

KENDRICK:

I'm totally in.

RISA GOLUBOFF: This has been really fun.

LESLIE KENDRICK: I agree. It's been a really, really good time.

[MUSIC PLAYING]

Well, that's it for this episode and for the first season of Common Law. If you're listening for

the first time, it's never too late to go back and check out our earlier episodes all about the

future of the law.

RISA GOLUBOFF: And if you're already a fan, let us know. You can leave us a review of the show wherever you

get your podcasts, or tweet us with ideas for new episodes @commonlawuva.

LESLIE KENDRICK: Our show's produced by Tyler Ambrose, Robert [? Amengaul, ?] Tony Field and Mary Wood.

Special thanks to the Virginia Quarterly Review, Virginia Humanities, and the Columbia

Journalism School. I'm Leslie Kendrick.

RISA GOLUBOFF: And I'm Risa Goluboff. Have a great summer. We'll be back next season with more Common

Law. Hope you'll join us then.

[MUSIC PLAYING]